I use rsync and a pruning script in crontab on my NFS mounts. I’ve tested it numerous times breaking containers and restoring them from backup. It works great for me at home because I don’t need anything older than 4 monthly, 4 weekly, and 7 daily backups.

However, in my job I prefer something like bacula. The extra features and granularity of restore options makes a world of difference when someone calls because they deleted prod files.

And I generally enjoy Veronica’s presentation. Knowledgable and simple.

It works fine if all you need is transfer, my issue with it it’s just not efficient. If you want a “time travel” feature, your only option is to duplicate data. Differential backups, compression, and encryption for off-site ones is where other tools shine.

I use rsync for many of the reasons covered in the video. It’s widely available and has a long history. To me that feels important because it’s had time to become stable and reliable. Using Linux is a hobby for me so my needs are quite low. It’s nice to have a tool that just works.

I use it for all my backups and moving my backups to off network locations as well as file/folder transfers on my own network.

I even made my own tool (https://codeberg.org/taters/rTransfer) to simplify all my rsync commands into readable files because rsync commands can get quite long and overwhelming. It’s especially useful chaining multiple rsync commands together to run under a single command.

I’ve tried other backup and syncing programs and I’ve had bad experiences with all of them. Other backup programs have failed to restore my system. Syncing programs constantly stop working and I got tired of always troubleshooting. Rsync when set up properly has given me a lot less headaches.

They linked blog post with the video: https://vkc.sh/everyday-rsync/

Thank you for putting into words what ive subconsciously been thinking for years. Every search result prioritizes videos at the top and I’m still annoyed every time. Or even worst I have to hunt through a 10 minute video for the 30 seconds of info I needed. Stoohhhhpppp internet of new! Make it good again!

Ad money.

Hear hear. Knowledge should be communicated in an easily shareable way that can also be archived as easily, in contrast to a video requiring hundreds of MB:s.

Especially for a command line tool

Yah, I really like this approach. Same reason I set up Timeshift and Mint Backup on all the user machines in my house. For others rsync + cron is aces.

It’s single thread, one file at a time.

That part threw me off. Last time i used it, I did incremental backups of a 500 gig disk once a week or so, and it took 20 seconds max.

Compared to something multi threaded, yes. But there are obviously a number of bottlenecks that might diminish the gains of a multi threaded program.

Hardlinks need to be on the same filesystem, don’t they? I don’t see how that would work with a remote backup…?

I couldn’t tell you if it’s the right way but I used it on my Rpi4 to sync 4tb of stuff from my Plex drive to a backup and set a script up to have it check/mirror daily. Took a day and a half to copy and now it syncs in minutes tops when there’s new data

yes, it’s the right way to go.

rsync over ssh is the best, and works as long as rsync is installed on both systems.

It depends

rsync is fine, but to clarify a little further…

If you think you’ll stop the transfer and want it to resume (and some data might have changed), then yep, rsync is best.

But, if you’re just doing a 1-off bulk transfer in a single run, then you could use other tools like xcopy / scp or - if you’ve mounted the remote NAS at a local mount point - just plain old cp

The reason for that is that rsync has to work out what’s at the other end for each file, so it’s doing some back & forwards communications each time which as someone else pointed out can load the CPU and reduce throughput.

(From memory, I think Raspberry Pi don’t handle large transfers over scp well… I seem to recall a buffer gets saturated and the throughput drops off after a minute or so)

Also, on a local network, there’s probably no point in using encryption or compression options - esp. for photos / videos / music… you’re just loading the CPU again to work out that it can’t compress any further.

I think Arch wiki recommends rsync for backups

@calliope It’s also great for local or remote backups over ssh, smb, etc.

It’s fine. But yes in the Linux space. We tend to want to host ourselves. Not have to trust some administrator of some cloud we don’t know/trust.

I tried rclone once because I wanted to sync a single folder from documents and freaked out when it looked like it was going to purge all documents except for my targeted folder.

Then I just did it via the portal…

I feel this too. I have a couple of “spells” that work wonders in a literal small notebook with other one liners over the years. Its my spell book lol.

One trick that one of my students taught me a decade or so ago is to actually make an alias to list the useful flags.

Yes, a lot of us think we are smart and set up aliases/functions and have a huge list of them that we never remember or, even worse, ONLY remember. What I noticed her doing was having something like goodman-rsync that would just echo out a list of the most useful flags and what they actually do.

So nine times out of 10 I just want rsync -azvh --progress ${SRC} ${DEST} but when I am doing something funky and am thinking “I vaguely recall how to do this”? dumbman rsync and I get a quick cheat sheet of what flags I have found REALLY useful in the past or even just explaining what azvh actually does without grepping past all the crap I don’t care about in the man page. And I just keep that in the repo of dotfiles I copy to machines I work on regularly.

This is why I still don’t know sed and awk syntax lol. I eventually get the data in the shape I need and then move on, and never imprint how they actually work. Still feel like a script kiddie every time I use them (so once every few years).

And not waste time copying duplicate data. And for the typical home user, it’s probably mo slower than other options.