Should be interesting to see how the fediverse in general handles more traffic, as we’ve seen with kbin and lemmy over the last month or so there are certainly some growing pains
at least we are making the most of our new space here, we all seem to be building something fun here
Federation in general seems to be pretty buggy, I’m running my own instance and I can see a ton of failed jobs happening in the logs. Seems like some of the issues are short comings of the ActivityPub protocol
@jeff@federated.fun i don’t know if there is much we can do about that though. I have read through the activity pub spec as I was considering writing a fediverse web app…I won’t disagree, it is glorified json. Unfortunately it’s the “standard” that has been loosely agreed upon. I might have the willpower to write a passion project FOSS fediverse web app but I know I couldn’t remotely begin architecting a new federation protocol as well.
The one that’s bouncing around in the back of my mind is a ActivityPub <-> NNTP gateway. Have a server that federates to things on ActivityPub and then exposes then via NNTP so that Gnus or trn would be able to speak to the server, read messages and post new messages.
That doesn’t seem impossible… I’m not sure what business logic would be needed to make them easily interoperable. Honestly the biggest complaints I’ve read about the fediverse isn’t the UIs available to view content. The issue is the bugginess of federation and the lack of content recommendation algorithms on the platforms.
I’ve been mulling over the idea of a fediverse content crawler to allow instances to mass federate content to their instance…but then like I said you also need a good recommendation algo as well.
We have a ton of dev work going into making new UIs for Lemmy but personally I think what I said above should get some love too.
For Reddit, you’ve got The Algorithm which keeps suggesting things. There’s /r/all and /r/popular which has a bunch of stuff… and sometimes /r/popular has something interesting that isn’t part of the groups you subscribe that you’d want to subscribe to. There’s also people suggesting other groups to crosspost something too - which adds to the discovery for the person with content and everyone reading with a “oh, that sub also has things… another day, another cat sub.”
Twitter had its algorithm - but one of the important things was the hashtags. For example, there is #energytwitter - https://twitter.com/search?q=%23energytwitter which has lots of material about the electric grid. If you’re on twitter, subscribe to that hash tag and that interesting stuff goes past your feed.
However, with the twitter exodus, people formed instances around a hashtag… mastodon.social was “full” and not accepting new accounts, and “taking over” another instance with all that community’s content was impolite… so there’s https://mastodon.energy/explore which is the people who left twitter and went to mastodon to be active there. There’s non energy stuff, but the local hashtags and activity is good.
Now, if you’ve got an account on techhub.social but want the energy twitter experience… you can’t really get it. You’d have to subscribe to everyone on mastodon.energy or browse it in read only mode… and to get a general “question to the experts”, you’d need someone there to boost it rather than just tagging it with #energytwitter. Or you get two accounts… which is one of the things that federation tries to not have. Unfortunately, identity doesn’t have a good federation solution yet so that one identity could log into two different mastodon instances.
The other one is the multiple instances of the same thing. !java@lemmy.ml vs !java@programming.dev - subscribe to both? Just one? Got content - post to both? or just one?
Anyways, this is where we are now. ActivityPub isn’t awful. Mastodon instances seem alright. Lemmy seems a bit “by itself” with poor federation to others. I can subscribe to events on https://events.nixnet.services in Mastodon without any seeming problem… or things on Wordpress.
I believe Lemmy needs to work on its federation. It’s ok as a reddit clone, but without a really good reason to be federated other than to be federated. Being implemented as a reddit clone, it also brought with it the social problems of Reddit without too much consideration about the design of social software and trying to correct those problems. A Group Is Its Own Worst Enemy is a good read - https://gwern.net/doc/technology/2005-shirky-agroupisitsownworstenemy.pdf - all the things that Reddit did wrong (and right) are attempting to be repeated here.
Wow I appreciate the lengthy response! So where do we go from here? Do we need to add new features that make content more discoverable? Do we need some great sorting algorithm here? Do we need a 3rd party tool that scrapes the fediverse and tells us what we might like?
I’m trying to answer a bunch of questions like this. I’ve been messing with ActivityPub in my spare time to see what my implementation of it would look like. I think I have some ideas that could solve some of the issues we are seeing pop up. Honestly I’d love to have some people help me brainstorm features and architect out a system.
At the end of the day… I’m on the fediverse because I don’t like Facebook, twitter, instagram, and (now) Reddit. It’s not just the people and the content on those platforms I didn’t enjoy, it’s also how they wanted you to interact with the content. I’m not looking to build a platform clone, I’m looking to build something that can fully utilize ActivityPub and can provide a feature rich experience to the user.
Maybe I have too lofty of goals…idk, let me know!
Should be interesting to see how the fediverse in general handles more traffic, as we’ve seen with kbin and lemmy over the last month or so there are certainly some growing pains
at least we are making the most of our new space here, we all seem to be building something fun here
@lohrun @snoopa lemmy’s federation is buggy af and is causing excessive congestion from failed jobs
Federation in general seems to be pretty buggy, I’m running my own instance and I can see a ton of failed jobs happening in the logs. Seems like some of the issues are short comings of the ActivityPub protocol
@lohrun they are shortcomings in lemmy’s flavor of apub. activity pub is more akin to json than it is an actual social networking protocol.
@jeff@federated.fun i don’t know if there is much we can do about that though. I have read through the activity pub spec as I was considering writing a fediverse web app…I won’t disagree, it is glorified json. Unfortunately it’s the “standard” that has been loosely agreed upon. I might have the willpower to write a passion project FOSS fediverse web app but I know I couldn’t remotely begin architecting a new federation protocol as well.
The one that’s bouncing around in the back of my mind is a ActivityPub <-> NNTP gateway. Have a server that federates to things on ActivityPub and then exposes then via NNTP so that Gnus or trn would be able to speak to the server, read messages and post new messages.
That doesn’t seem impossible… I’m not sure what business logic would be needed to make them easily interoperable. Honestly the biggest complaints I’ve read about the fediverse isn’t the UIs available to view content. The issue is the bugginess of federation and the lack of content recommendation algorithms on the platforms.
I’ve been mulling over the idea of a fediverse content crawler to allow instances to mass federate content to their instance…but then like I said you also need a good recommendation algo as well.
We have a ton of dev work going into making new UIs for Lemmy but personally I think what I said above should get some love too.
Discovery is one of the great problems of the fediverse.
Old school, nntp - you got a list of all the groups and grep’ed for whatever was interesting. https://www.eternal-september.org/groups.php?hierarchy=alt - anything interesting? Subscribe. Discovery wasn’t too hard.
For Reddit, you’ve got The Algorithm which keeps suggesting things. There’s /r/all and /r/popular which has a bunch of stuff… and sometimes /r/popular has something interesting that isn’t part of the groups you subscribe that you’d want to subscribe to. There’s also people suggesting other groups to crosspost something too - which adds to the discovery for the person with content and everyone reading with a “oh, that sub also has things… another day, another cat sub.”
Twitter had its algorithm - but one of the important things was the hashtags. For example, there is #energytwitter - https://twitter.com/search?q=%23energytwitter which has lots of material about the electric grid. If you’re on twitter, subscribe to that hash tag and that interesting stuff goes past your feed.
However, with the twitter exodus, people formed instances around a hashtag… mastodon.social was “full” and not accepting new accounts, and “taking over” another instance with all that community’s content was impolite… so there’s https://mastodon.energy/explore which is the people who left twitter and went to mastodon to be active there. There’s non energy stuff, but the local hashtags and activity is good.
Now, if you’ve got an account on techhub.social but want the energy twitter experience… you can’t really get it. You’d have to subscribe to everyone on mastodon.energy or browse it in read only mode… and to get a general “question to the experts”, you’d need someone there to boost it rather than just tagging it with #energytwitter. Or you get two accounts… which is one of the things that federation tries to not have. Unfortunately, identity doesn’t have a good federation solution yet so that one identity could log into two different mastodon instances.
And so - the great federation discovery problem. And I’m not referring to !startrek@startrek.website
The other one is the multiple instances of the same thing. !java@lemmy.ml vs !java@programming.dev - subscribe to both? Just one? Got content - post to both? or just one?
Anyways, this is where we are now. ActivityPub isn’t awful. Mastodon instances seem alright. Lemmy seems a bit “by itself” with poor federation to others. I can subscribe to events on https://events.nixnet.services in Mastodon without any seeming problem… or things on Wordpress.
I believe Lemmy needs to work on its federation. It’s ok as a reddit clone, but without a really good reason to be federated other than to be federated. Being implemented as a reddit clone, it also brought with it the social problems of Reddit without too much consideration about the design of social software and trying to correct those problems. A Group Is Its Own Worst Enemy is a good read - https://gwern.net/doc/technology/2005-shirky-agroupisitsownworstenemy.pdf - all the things that Reddit did wrong (and right) are attempting to be repeated here.
Wow I appreciate the lengthy response! So where do we go from here? Do we need to add new features that make content more discoverable? Do we need some great sorting algorithm here? Do we need a 3rd party tool that scrapes the fediverse and tells us what we might like? I’m trying to answer a bunch of questions like this. I’ve been messing with ActivityPub in my spare time to see what my implementation of it would look like. I think I have some ideas that could solve some of the issues we are seeing pop up. Honestly I’d love to have some people help me brainstorm features and architect out a system. At the end of the day… I’m on the fediverse because I don’t like Facebook, twitter, instagram, and (now) Reddit. It’s not just the people and the content on those platforms I didn’t enjoy, it’s also how they wanted you to interact with the content. I’m not looking to build a platform clone, I’m looking to build something that can fully utilize ActivityPub and can provide a feature rich experience to the user. Maybe I have too lofty of goals…idk, let me know!