Edit2: Thanks all for your responses! I have checked the logs, https://lemmy.nz/comment/6192604, and based on that removed tracker-miner-fs as it’s a search/index tool which I don’t need. No idea why it took over all memory. I’ll also get a WiFi Smartplug as a kill switch. Hopefully that solves it. Thanks again heaps!
I’ve got a HP ProDesk G3 which I’m using as home server, I’ve installed Ubuntu on it. Earlier this week the services I host on it stopped (Immich & Frigate). I tried to SSH, but it just hung after asking for a password. I could ping it, but it was just unresponsive.
I had to force reboot it manually. This is fine, but I’m not always at home.
The chip has Intel vPro as far as I know, which could be an option, but I have no idea how this works. The documentation on the Intel site seems focused on enterprises. I tried to connect with RealVNC which does not work, so I think I’ve got to install/configure something on the server first.
I also asked Bing Chat but it came up with non existing packages & commands. Welcome your thoughts!
/edit: I just found this, which seems to be exactly what I need: https://manpages.ubuntu.com/manpages/focal/en/man7/amt-howto.7.html
Maybe investigate why it hung?
That could be a sign of something bigger about to kill it altogether
Yes, thanks for that. Good point. I checked the logs, and minutes before it crashed I can see below in the logs. Seems like either a GPU error or out of memory error. No idea what tracker-miner-f is by the way. It also shows a massive list of processes with their memory usage.
This goes beyond my knowledge :(
Feb 21 17:27:49 hppd600-g3 kernel: i915 0000:00:02.0: [drm] GPU HANG: ecode 9:0:00000000 Feb 21 17:32:43 hppd600-g3 kernel: 1305621 total pagecache pages Feb 21 17:32:43 hppd600-g3 kernel: 16258 pages in swap cache Feb 21 17:32:43 hppd600-g3 kernel: Free swap = 0kB Feb 21 17:32:43 hppd600-g3 kernel: Total swap = 1000444kB Feb 21 17:32:43 hppd600-g3 kernel: 2065206 pages RAM Feb 21 17:32:43 hppd600-g3 kernel: 0 pages HighMem/MovableOnly Feb 21 17:32:43 hppd600-g3 kernel: 64196 pages reserved Feb 21 17:32:43 hppd600-g3 kernel: 0 pages hwpoisoned Feb 21 17:32:43 hppd600-g3 kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-113.slice/user@113.service/background.slice/tracker-miner-fs-3.service,task=t> Feb 21 17:32:43 hppd600-g3 kernel: Out of memory: Killed process 833 (tracker-miner-f) total-vm:625676kB, anon-rss:3144kB, file-rss:4816kB, shmem-rss:4kB, UID:113 pgtables:280kB oom_score_adj:200 Feb 21 17:32:43 hppd600-g3 kernel: i915 0000:00:02.0: [drm] Resetting rcs0 for stopped heartbeat on rcs0Tracker miner fs generates thumbnails for files iirc. There was a recent vulnerability where malicious files could crash it and execute code just by being on disk. Make sure you haven’t been hit by malware
I’ve uninstalled it, it’s an index/search tool. Don’t need it :D
Usually comes with your DE, sometimes removing it breaks your DE
Yeah tracker miner sounds dodgy. I’ve only installed Immich & Frigate on the box, and no dodgy repositories. It’s also auto updating. Will do research how to check for malware, thought that was a Windows only thing :D
I’ve previously had a problem with my server becoming unresponsive when running immich. It’s been a while, but I remember there being some kind of memory leak having to do with immich. It was in their GitHub issues and everything. On my system it would take about a day and a half and then ssh, along with everything else, would become unresponsive. Rebooting would fix it for a day and a half. I stopped running immich and it hasn’t happened since. I suppose you could try using a cron job to restart immich periodically and see if that resolves your problem.
That is good to know! Will keep an eye on memory usage of immich. I really like it, so I’m reluctant to let it go.