Rclone Large File Patterns
Large-file pipelines need conservative parallelism and stronger resilience controls. A stable baseline avoids partial uploads and reduces expensive reruns.
Recommended Baseline
rclone sync /srv/large-data remote:archive/large-data \
--transfers 4 --checkers 8 \
--retries 10 --retries-sleep 15s --low-level-retries 20 \
--contimeout 30s --timeout 10m --dry-run -vv
Rule of Thumb
- Start conservative, then scale up after clean runs.
- Split large jobs by prefix/domain for easier recovery.
- Keep logs for post-failure root-cause analysis.
Examples with Output
1. Large-file preset example 1
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 2 --checkers 4 --retries 8 --timeout 5m --dry-run -vv
Expected output:
NOTICE: profile applied (2, 4, 8, 5m)
Would transfer 57 files
2. Large-file preset example 2
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 3 --checkers 6 --retries 8 --timeout 5m --dry-run -vv
Expected output:
NOTICE: profile applied (3, 6, 8, 5m)
Would transfer 64 files
3. Large-file preset example 3
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 4 --checkers 8 --retries 10 --timeout 6m --dry-run -vv
Expected output:
NOTICE: profile applied (4, 8, 10, 6m)
Would transfer 71 files
4. Large-file preset example 4
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 4 --checkers 10 --retries 10 --timeout 6m --dry-run -vv
Expected output:
NOTICE: profile applied (4, 10, 10, 6m)
Would transfer 78 files
5. Large-file preset example 5
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 4 --checkers 12 --retries 12 --timeout 7m --dry-run -vv
Expected output:
NOTICE: profile applied (4, 12, 12, 7m)
Would transfer 85 files
6. Large-file preset example 6
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 5 --checkers 10 --retries 12 --timeout 7m --dry-run -vv
Expected output:
NOTICE: profile applied (5, 10, 12, 7m)
Would transfer 92 files
7. Large-file preset example 7
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 5 --checkers 12 --retries 12 --timeout 8m --dry-run -vv
Expected output:
NOTICE: profile applied (5, 12, 12, 8m)
Would transfer 99 files
8. Large-file preset example 8
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 6 --checkers 12 --retries 14 --timeout 8m --dry-run -vv
Expected output:
NOTICE: profile applied (6, 12, 14, 8m)
Would transfer 106 files
9. Large-file preset example 9
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 6 --checkers 14 --retries 14 --timeout 9m --dry-run -vv
Expected output:
NOTICE: profile applied (6, 14, 14, 9m)
Would transfer 113 files
10. Large-file preset example 10
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 6 --checkers 16 --retries 16 --timeout 9m --dry-run -vv
Expected output:
NOTICE: profile applied (6, 16, 16, 9m)
Would transfer 120 files
11. Large-file preset example 11
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 7 --checkers 14 --retries 16 --timeout 10m --dry-run -vv
Expected output:
NOTICE: profile applied (7, 14, 16, 10m)
Would transfer 127 files
12. Large-file preset example 12
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 8 --checkers 16 --retries 18 --timeout 10m --dry-run -vv
Expected output:
NOTICE: profile applied (8, 16, 18, 10m)
Would transfer 134 files
13. Large-file preset example 13
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 8 --checkers 18 --retries 20 --timeout 12m --dry-run -vv
Expected output:
NOTICE: profile applied (8, 18, 20, 12m)
Would transfer 141 files
14. Large-file preset example 14
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 10 --checkers 20 --retries 20 --timeout 12m --dry-run -vv
Expected output:
NOTICE: profile applied (10, 20, 20, 12m)
Would transfer 148 files
15. Large-file preset example 15
Input command:
rclone sync /srv/large-data remote:archive/large-data --transfers 12 --checkers 24 --retries 24 --timeout 15m --dry-run -vv
Expected output:
NOTICE: profile applied (12, 24, 24, 15m)
Would transfer 155 files