In the sign up process, it discourages using the primary srever. In the docs it talks about performance limitations. In contast Mastadon does not do this. Worse yet, most people think of a single centralized server. And this server only has some 500+ active users a month. Maybe 10 times that many lurkers?
While I love the Lemmy design, and the community looks great, I fear that it does not scale. Worse yet I suspect that the Activity Pub protocol is grossly inefficient for Reddit style applications. Maybe a publish subscribe message broker, like MQTT would be much more efficient.
So what is gong on here. Is this an issue? Am I correct to be scared about Lemmy scalability.
And just for context, I think you are totally on the right track, way ahead of all ofther reddit clones, but maybe just a little off course. Can you crank up performance?
So far there are no performance problems at all. Lemmy.ml runs on a dual core vps and only uses like 5-10% of each core (load average around 0.5). The reason we want to discourage use of lemmy.ml is that it makes for an unhealthy ecosystem if the majority of users are all on a single instance. I also dont think that federation would be a performance problem unless you have a really huge instance. If we run into any performance issues, then Rust will give us a lot of ways to optimize that.
Is the programming language really the bottleneck when dealing with web servers like that? The last time Lemmy became slow it was because of the way the database was used and not because of CPU/RAM bottleneck with Rust if I remember correctly. Good job by the way on fixing it, Lemmy is really snappy and I like it!
I guess Lemmy will struggle when there will be posts with thousands of comments but we are not there yet. In this case Rust wont help much and only a better API with pagination will (if it’s not already implemented), though I guess Rust certainly helps a bit.
Rust is just a really nice language (no null pointers or anything similar). We arent using it for performance reasons, thats just a nice bonus.
Once this is implemented, large threads can be handled much more efficiently.
Yep, I find rust amazing too. I just think that the ecosystem has some maturation to do, there are still many major crates not having reached version 1.0, but we’re getting there, and some compiler features (GAT, const generics) will soon make some of my grievances go away.
Glad to see you’re already working on thread pagination. Pagination for something with a non trivial sorting order doesn’t seem easy to solve. Lemmy is amazing thanks for everything!