S3, B2, and R2 Remotes
S3-compatible backends are common for production backups, but each provider has endpoint and auth differences.
Learning Focus
Configure object storage once, then validate read/write access with a repeatable smoke test before you point jobs at real backup prefixes.
Provider Snapshot
| Provider | Backend type | Endpoint requirement | Common auth |
|---|---|---|---|
| AWS S3 | s3 | Region-based | Access key + secret |
| Backblaze B2 | b2 or s3 | B2 endpoint | Key ID + app key |
| Cloudflare R2 | s3 | Custom account endpoint | Access key + secret |
Operational Defaults (Recommended)
| Default | Why it helps |
|---|---|
| Separate buckets/prefixes per environment | Prevents staging/prod mixups |
| Dedicated backup credentials | Reduces blast radius |
| Versioning (when available) | Makes deletes less catastrophic |
| Lifecycle policies | Controls costs automatically |
Recommended Setup Command
rclone config
Choose new remote, then set backend type and credentials exactly as provider docs specify.
note
For R2 (S3-compatible), you typically need a provider-specific endpoint. For AWS S3, the region is critical.
Endpoint Validation
remote-healthcheck.sh
rclone listremotes
rclone lsd s3-prod:
rclone mkdir s3-prod:healthcheck
rclone rmdir s3-prod:healthcheck
Permission Model (What You Actually Need)
Design permissions based on your operation:
| Operation | Typical needs |
|---|---|
copy to archive | list + write |
sync mirror | list + read + write + delete |
| verification-only | list + read |
warning
If your job uses sync, your credential likely needs delete rights. Treat that as a high-risk permission.
Example Remote Usage
rclone sync /backup/current s3-prod:infra/backups/current --progress
rclone sync /backup/current b2-prod:infra/backups/current --progress
rclone sync /backup/current r2-prod:infra/backups/current --progress
warning
Do not reuse root-account cloud credentials in automation. Create dedicated backup credentials with scoped permissions.
Verification Checklist
| Check | Command |
|---|---|
| List root | rclone lsd s3-prod: |
| Create prefix | rclone mkdir s3-prod:labs/rclone |
| Write object | rclone copy /tmp/test.txt s3-prod:labs/rclone/ |
| Delete object | rclone delete s3-prod:labs/rclone/test.txt |
| Remove empty dir | rclone rmdir s3-prod:labs/rclone |
Common Pitfalls
| Pitfall | Symptom | Resolution |
|---|---|---|
| Wrong region/endpoint | Signature mismatch, 403 | Reconfigure endpoint and region |
| Missing bucket permission | List works, write fails | Add object write/list/delete rights |
| Copying test config to prod blindly | Misrouted backups | Validate each remote with smoke tests |