I setup a new instance a week or two ago, and have some subscriptions to communities on lemmy.world. The logs for the Lemmy process are currently showing a constant flood of warnings for what look like ordinary activity (likes, creates, undos, etc). Here are two recent entries:

lemmy-1  | 2024-05-13T21:41:48.243217Z  WARN lemmy_server::root_span_builder: Unknown: 
lemmy-1  |    0: lemmy_apub::insert_received_activity
lemmy-1  |            with ap_id=Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/activities/announce/like/3a96d6df-3229-4e5a-a6d1-ce2f6f3ca3d5", query: None, fragment: None }
lemmy-1  |              at crates/apub/src/lib.rs:198
lemmy-1  |    1: lemmy_apub::activities::community::announce::receive
lemmy-1  |              at crates/apub/src/activities/community/announce.rs:153
lemmy-1  |    2: lemmy_server::root_span_builder::HTTP request
lemmy-1  |            with http.method=POST http.scheme="http" http.host=leftopia.org http.target=/inbox otel.kind="server" request_id=43212a7e-1c3c-4b6a-84d6-4cd6082af392
lemmy-1  |              at src/root_span_builder.rs:16
lemmy-1  | 2024-05-13T21:41:48.356594Z  WARN lemmy_server::root_span_builder: Unknown: 
lemmy-1  |    0: lemmy_apub::insert_received_activity
lemmy-1  |            with ap_id=Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/activities/announce/like/9f70cd5b-175a-4ac3-a852-322494f01981", query: None, fragment: None }
lemmy-1  |              at crates/apub/src/lib.rs:198
lemmy-1  |    1: lemmy_apub::activities::community::announce::receive
lemmy-1  |              at crates/apub/src/activities/community/announce.rs:153
lemmy-1  |    2: lemmy_server::root_span_builder::HTTP request
lemmy-1  |            with http.method=POST http.scheme="http" http.host=leftopia.org http.target=/inbox otel.kind="server" request_id=050271b4-2081-4264-936a-329d70374239
lemmy-1  |              at src/root_span_builder.rs:16

Tailing the Docker logs is like watching a torrent of these warnings, and I’m worried about the stress its adding to my fairly meager VM. Any tips on how to debug this? I’m proficient with Rust, so I’m willing and able to jump into the code if it’ll help.

  • Andrew@piefed.social
    link
    fedilink
    English
    arrow-up
    5
    ·
    7 months ago

    Might just be 'cos LW is currently sending every activity out twice. Video from earlier today: https://imgur.com/a/lX0LPZk

    You could probably tail your nginx log in another terminal to see if the journalctl errors match up with 400 errors for duplicates there…

    Might not be this, of course (could just be that I saw this earlier, and it’s influencing my thinking)

    • insaan@leftopia.orgOP
      link
      fedilink
      arrow-up
      3
      ·
      7 months ago

      It’s definitely happening when I’m getting updates from lemmy.world, and while I don’t know how to get at the HTTP details you’re showing in your video, I do see a lot of 400’s in the nginx log from Docker:

      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 400 62 "-" "Lemmy/0.19.3; +https://lemmy.world"
      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 400 62 "-" "Lemmy/0.19.3; +https://lemmy.world"
      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 400 62 "-" "Lemmy/0.19.3; +https://lemmy.world"
      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
      proxy-1  | 135.181.143.221 - - [13/May/2024:23:03:43 +0000] "POST /inbox HTTP/1.1" 400 62 "-" "Lemmy/0.19.3; +https://lemmy.world"