Skip to main content

Check, Hash, and Dry Run

Data transfer is only half the job. Verification is what makes backup operations trustworthy.

Learning Focus

Learn a repeatable validation workflow: preview -> execute -> verify -> archive evidence (logs + check output).

Validation Workflow​

flowchart LR
  PLAN[Dry Run] --> EXEC[Execute Transfer]
  EXEC --> CHECK[Run rclone check]
  CHECK --> LOG[Store Logs + Exit Status]

Dry Run First​

rclone sync /srv/data s3-prod:backup/data --dry-run --log-level INFO
note

Dry-run shows intended changes but does not prove correctness. Verification (check) is a separate step.

Integrity Commands​

CommandPurpose
rclone check A BCompare source vs destination
rclone md5sum remote:pathGenerate MD5 list (backend-dependent)
rclone sha1sum remote:pathGenerate SHA1 list (backend-dependent)
rclone size remote:pathAudit object count and total bytes

Checks: What They Really Mean​

PatternMeaningUse when
check SRC DST --one-wayValidate destination has everything from sourceBackups where source is authoritative
check SRC DSTBidirectional mismatch reportComparing two replicas
check ... --size-onlyFast compare without hashesBackend lacks reliable hashes

Practical Example​

verify-transfer.sh
rclone sync /srv/data s3-prod:backup/data --log-file /var/log/rclone-sync.log
rclone check /srv/data s3-prod:backup/data --one-way --checkers 16 --log-file /var/log/rclone-check.log
note

Some providers do not expose the same hash algorithms for all objects. In those cases, use size/time validation plus spot checks.

Capture Evidence (Combined Output)​

When you want an audit trail you can attach to tickets/incidents:

rclone check /srv/data s3-prod:backup/data --one-way --combined /var/log/rclone-check.combined.txt

Common Pitfalls​

PitfallImpactPrevention
Skipping dry-run on syncUnexpected deletesAlways preview and inspect log output
Trusting only transfer exit codeDrift remains undetectedAdd rclone check after critical jobs
Assuming hash support is universalFalse mismatch expectationsConfirm provider hash capability

Hands-On Practice​

  1. Run a sync --dry-run.
  2. Run a real copy to a lab prefix.
  3. Run check --one-way.
practice-validation.sh
rclone sync /srv/data s3-prod:labs/validate --dry-run -vv
rclone copy /srv/data s3-prod:labs/validate --progress
rclone check /srv/data s3-prod:labs/validate --one-way

Quick Reference​

rclone sync SRC DST --dry-run
rclone check SRC DST --one-way
rclone size DST

What's Next​