You might not even like rsync. Yeah it's old. Yeah it's slow. But if you're working with Linux you're going to need to know it. In this video I walk through my favorite everyday flags for rsync. Su...

You might not even like rsync. Yeah it’s old. Yeah it’s slow. But if you’re working with Linux you’re going to need to know it.

In this video I walk through my favorite everyday flags for rsync.

Support the channel:
https://patreon.com/VeronicaExplains
https://ko-fi.com/VeronicaExplains
https://thestopbits.bandcamp.com/

Here’s a companion blog post, where I cover a bit more detail: https://vkc.sh/everyday-rsync

Also, @BreadOnPenguins made an awesome rsync video and you should check it out: https://www.youtube.com/watch?v=eifQI5uD6VQ

Lastly, I left out all of the ssh setup stuff because I made a video about that and the blog post goes into a smidge more detail. If you want to see a video covering the basics of using SSH, I made one a few years ago and it’s still pretty good: https://www.youtube.com/watch?v=3FKsdbjzBcc

Chapters:
1:18 Invoking rsync
4:05 The --delete flag for rsync
5:30 Compression flag: -z
6:02 Using tmux and rsync together
6:30 but Veronica… why not use (insert shiny object here)

  • state_electrician@discuss.tchncs.deEnglish
    211·
    6 hours ago

    Why videos? I feel like an old man yelling at clouds every time something that sounds interesting is presented in a fucking video. Videos are so damn awful. They take time, I need audio and I can’t copy&paste. Why have they become the default for things that should’ve been a blog post?

    • czardestructo@lemmy.worldEnglish
      3·
      1 hour ago

      Thank you for putting into words what ive subconsciously been thinking for years. Every search result prioritizes videos at the top and I’m still annoyed every time. Or even worst I have to hunt through a 10 minute video for the 30 seconds of info I needed. Stoohhhhpppp internet of new! Make it good again!

    • kchr@lemmy.sdf.orgEnglish
      31·
      3 hours ago

      Hear hear. Knowledge should be communicated in an easily shareable way that can also be archived as easily, in contrast to a video requiring hundreds of MB:s.

  • 1984@lemmy.todayEnglish
    4·
    6 hours ago

    I never thought of it as slow. More like very reliable. I dont need my data to move fast, I need it to be copied with 100% reliability.

  • clif@lemmy.worldEnglish
    15·
    21 hours ago

    I’ll never not upvote Veronica Explains. Excellent creator and excellent info on everything I’ve seen.

  • atk007@lemmy.worldEnglish
    23·
    1 day ago

    Rsnapshot. It uses rsync, but provides snapshot management and multiple backup versioning.

    • BonkTheAnnoyed@lemmy.blahaj.zoneEnglish
      4·
      23 hours ago

      Yah, I really like this approach. Same reason I set up Timeshift and Mint Backup on all the user machines in my house. For others rsync + cron is aces.

  • Appoxo@lemmy.dbzer0.comEnglish
    4·
    22 hours ago

    Veeam for image/block based backups of Windows, Linux and VMs.
    syncthing for syncing smaller files across devices.

    Thank you very much.

    • Encrypt-Keeper@lemmy.worldEnglish
      8·
      21 hours ago

      I’m not super familiar with Syncthing, but judging by the name I’d say Syncthing is not at all meant for backups.

    • conartistpanda@lemmy.worldEnglish
      6·
      22 hours ago

      Syncthing is technically to synchronize data across different devices in real time (which I do with my phone), but I also use it to transfer data weekly via wi-fi to my old 2013 laptop with a 500GB HDD and Linux Mint (I only boot it to transfer data, and even then I pause the transfers to this device when its done transferring stuff) so I can have larger data backups that wouldn’t fit in my phone, since LocalSend is unreliable for large amounts of data while Synchting can resume the transfer if anything goes wrong. On top of that Syncthing also works in Windows and Android out of the box.

    • HereIAm@lemmy.worldEnglish
      81·
      23 hours ago

      Compared to something multi threaded, yes. But there are obviously a number of bottlenecks that might diminish the gains of a multi threaded program.

    • okamiueru@lemmy.worldEnglish
      9·
      1 day ago

      That part threw me off. Last time i used it, I did incremental backups of a 500 gig disk once a week or so, and it took 20 seconds max.

  • NuXCOM_90Percent@lemmy.zipEnglish
    681·
    2 days ago

    I would generally argue that rsync is not a backup solution. But it is one of the best transfer/archiving solutions.

    Yes, it is INCREDIBLY powerful and is often 90% of what people actually want/need. But to be an actual backup solution you still need infrastructure around that. Bare minimum is a crontab. But if you are actually backing something up (not just copying it to a local directory) then you need some logging/retry logic on top of that.

    At which point you are building your own borg, as it were. Which, to be clear, is a great thing to do. But… backups are incredibly important and it is very much important to understand what a backup actually needs to be.

    • non_burglar@lemmy.worldEnglish
      21·
      2 days ago

      I use rsync and a pruning script in crontab on my NFS mounts. I’ve tested it numerous times breaking containers and restoring them from backup. It works great for me at home because I don’t need anything older than 4 monthly, 4 weekly, and 7 daily backups.

      However, in my job I prefer something like bacula. The extra features and granularity of restore options makes a world of difference when someone calls because they deleted prod files.

  • mesa@piefed.socialOPEnglish
    50·
    2 days ago

    Ive personally used rsync for backups for about…15 years or so? Its worked out great. An awesome video going over all the basics and what you can do with it.

    • Eager Eagle@lemmy.worldEnglish
      71·
      2 days ago

      It works fine if all you need is transfer, my issue with it it’s just not efficient. If you want a “time travel” feature, your only option is to duplicate data. Differential backups, compression, and encryption for off-site ones is where other tools shine.

      • suicidaleggroll@lemmy.worldEnglish
        1·
        22 hours ago

        If you want a “time travel” feature, your only option is to duplicate data.

        Not true. Look at the --link-dest flag. Encryption, sure, rsync can’t do that, but incremental backups work fine and compression is better handled at the filesystem level anyway IMO.

        • Eager Eagle@lemmy.worldEnglish
          1·
          21 hours ago

          Isn’t that creating hardlinks between source and dest? Hard links only work on the same drive. And I’m not sure how that gives you “time travel”, as in, browsing snapshots or file states at the different times you ran rsync.

          Edit: ah the hard link is between dest and the link-dest argument, makes more sense.

          I wouldn’t bundle fs and backup compression in the same bucket, because they have vastly different reqs. Backup compression doesn’t need to be optimized for fast decompression.

          • BCsven@lemmy.caEnglish
            1·
            19 hours ago

            Snapper and BTRFS. Its only adjusts changes in data, so time travel is just pointing to what blocks changed and when, and not building a duplicate of the entire file or filesystem. A snapshot is instant, and new block changes belong to the current default.

      • bandwidthcrisis@lemmy.worldEnglish
        2·
        1 day ago

        I have it add a backup suffix based on the date. It moves changed and deleted files to another directory adding the date to the filename.

        It can also do hard-link copied so that you can have multiple full directory trees to avoid all that duplication.

        No file deltas or compression, but it does mean that you can access the backups directly.

      • NekuSoul@lemmy.nekusoul.deEnglish
        1·
        2 days ago

        Agree. It’s neat for file transfers and simple one-shot backups, but if you’re looking for a proper backup solution then other tools/services have advanced virtually every aspect of backups so much it pretty much always makes sense to use one of those instead.

    • confusedpuppy@lemmy.dbzer0.comEnglish
      3·
      2 days ago

      I use rsync for many of the reasons covered in the video. It’s widely available and has a long history. To me that feels important because it’s had time to become stable and reliable. Using Linux is a hobby for me so my needs are quite low. It’s nice to have a tool that just works.

      I use it for all my backups and moving my backups to off network locations as well as file/folder transfers on my own network.

      I even made my own tool (https://codeberg.org/taters/rTransfer) to simplify all my rsync commands into readable files because rsync commands can get quite long and overwhelming. It’s especially useful chaining multiple rsync commands together to run under a single command.

      I’ve tried other backup and syncing programs and I’ve had bad experiences with all of them. Other backup programs have failed to restore my system. Syncing programs constantly stop working and I got tired of always troubleshooting. Rsync when set up properly has given me a lot less headaches.

      • sugar_in_your_tea@sh.itjust.worksEnglish
        3·
        23 hours ago

        That would only matter if it’s lots of small files, right? And after the initial sync, you’d have very few files, no?

        Rsync is designed for incremental syncs, which is exactly what you want in a backup solution. If your multithreaded alternative doesn’t do a diff, rsync will win on larger data sets that don’t have rapid changes.

  • vext01@lemmy.sdf.orgEnglish
    4·
    1 day ago

    I used to use rsnapshot, which is a thin wrapper around rsync to make it incremental, but moved to restic and never looked back. Much easier and encrypted by default.

  • Mio@feddit.nuEnglish
    51·
    1 day ago

    I think the there are better alternatives for backup like kopia and restic. Even seafile. Want protection against ransomware, storage compression, encryption, versioning, sync upon write and block deduplication.

    • Toribor@corndog.socialEnglish
      11·
      23 hours ago

      This exactly. I’d use rsync to sync a directory to a location to then be backed up by kopia, but I wouldn’t use rsync exclusively for backups.