Note: this seems like it has something to do with the database, and something getting royally messed up post upgrade.

After trying all sorts of network hacks and updates, I eventually just decided to backup my Postgres container, and nuke it.

With a fresh Postgres DB running along with 0.18.0, my self hosted site is back online. Of course, my local post history and all my subs are gone… but at least my site is operational again.

I’d advise anyone self-hosting to not upgrade to 0.18.0 yet.

Yes. Absolutely does happen on other instances that have thousands of users.

Have you tried enabling the slow query logs @ruud@lemmy.world? I went through that exercise yesterday to try to find the root cause but my instance doesn’t have enough load to reproduce the conditions, and my day job prevents me from devoting much time to writing a load test to simulate the load.

I did see several queries taking longer than 500ms (up to 2000ms) but they did not appear related to saving posts or comments.

Oh, Big-O notation? I never thought I’d see someone else mention big O notation out in the wild!

:high-five:

This sounds like a bug…