At the time of writing, Lemmyworld has the second highest number of active users (compared to all lemmy instances)

Also at the time of writing, Lemmyworld has >99% uptime.

By comparison, other lemmy instances with as many users as Lemmyworld keep going down.

What optimizations has Lemmyworld made to their hosting configuration that has made it more resilient than other instances’ hosting configurations?

See also Does Lemmy cache the frontpage by default (read-only)? on !lemmy_support@lemmy.ml

  • Skull giver@popplesburger.hilciferous.nl
    2·
    2 years ago

    Lemmy.world is using some Big Boy Hardware while most other servers are hosted on more hobbyist-grade hardware.

    Lemmy is quite efficient software. It has its challenges and weirdness, but I wouldn’t bat an eye if someone told me a server with 200 people was running on a Raspberry Pi attached so an SSD in someone’s basement. It barely consumes any CPU and for small communities the entire thing runs on less memory than a copy of Chrome loading the homepage.

    Needless to say, a network of servers that has been serving hundreds of people every day struggled to adapt to hundreds of thousands of people coming in at once.

    This isn’t unique to Lemmy either. We saw this happen to Mastodon when Twitter first started fucking up, then we saw it on Mastodon alternatives when the main Mastodon servers closed registration while they desperately scaled up.

    kbin, which is written in PHP, has also been seriously struggling. I’m pretty sure they still have their Cloudflare anti-DDoS-firewall enabled, meaning federating with kbin users is basically impossible until they can get that fixed.

    There are all sorts of things that could be improved (the frontend can use more aggressive caching, the backend might be written in a way that you can spawn more to take the load, the database can be sharded, you name it) but for Lemmy’s current size, throwing more cores and more RAM at the problem will quickly and easily fix the issue.

    If anything, this shows how inefficient some easier web application frameworks are. Good luck getting Mastodon to run well with less than half a GB reserved for Mastodon itself. Maybe there’s a point where Mastodon’s architecture starts to become more efficient than Lemmy’s, but from what I can tell that’s not all that likely.

      • Skull giver@popplesburger.hilciferous.nl
        1·
        2 years ago

        For super cheap, there’s Oracle’s free* servers. Restricted to about 50mbps in bandwidth but doesn’t cost you a penny. Get the ARM server if you can, those 4 cores and 24GB of RAM will serve you well.

        For cheap, there are hosting providers like Contabo and Hetzner that will offer you servers for a fair price. You’ll need to do your own backups though!

        OVH has some very cheap dedicated servers if you need more performance. Your uplink will be limited to 100mbps but you get dedicated CPU cores, dedicated hard drives/SSDs, dedicated RAM, everything. This comes with the risk that you’ll need to do your own backups because those dedicated components can die and there’s no failover (by default).

        It’s hard to say what hardware you’ll actually need, but I’ve read somewhere that lemmy.ml used to run on a machine with 8 CPU cores to serve thousands of users before everyone migrated over. The admin of lemmy.world posts regular updates on how the server is doing, I think the 21k users and all of its federated communities are running on a 32 core CPU with 128GB of RAM. Based on this, 6 cores and 8GB of RAM should be more than enough to serve ±100 users with the responsiveness of the lemmy.world instance (probably better because you’ll have comparatively more bandwidth).

        *=free forever but you do need to answer a call from a marketeer