-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider using rsync instead of cat to copy file #2
Comments
I tried using rsync but ran into a bunch of problems. Part of the issue is how very concurrent everything is-- you are opening up connections to tons of machines all at once. I found that basic SSH with cat was the most reliable by far. Same thing with using Paramiko-- I was able to use it for some stuff, but not for everything. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
rsync can do block level diffs and is more efficient is getting partial updates to a file.
https://linuxize.com/post/how-to-use-rsync-for-local-and-remote-data-transfer-and-synchronization/
Its an old school tool that just works ;-)
The text was updated successfully, but these errors were encountered: