Tuesday, May 7, 2024
 Popular · Latest · Hot · Upcoming
1
rated 0 times [  1] [ 0]  / answers: 1 / hits: 4952  / 2 Years ago, mon, march 28, 2022, 1:28:49

I'm struggling to find the reason why my Ubuntu server hangs/crashes.


I have installed a barebone Ubuntu Server 20.04.2 LTS on my old computer to try and use it as a personal server to host some projects of mine.


My issue is after a few hours, sometimes as low as 2 to 3 hours or as long as 8 to 12, my server will eventually hang.


I get a visual output, but it is completely unresponsive to either keyboard, mouse or ssh. Sometimes it will log CPU #X stuck for Xs! but never seems to be coming from the same place. Sometimes kworkers, sometimes md-udev, sometimes systemd-networkd.
Other times I get just no visual output at all and just have to force a reboot.


So currently running on my server are:



  • UFW

  • OpenVPN

  • Cockpit

  • Portainer

  • Docker container with nginx reverse proxy

  • Let's encrypt docker container

  • 2 Nextcloud docker containers

  • around 5 other docker containers running small apps/web apps


What's puzzling me the most is I get not logs at all, I've checked my kernel and system logs and usually the last line is an unrelated UFW block command of some spammer ip address but nothing more substantial. I've also tried looking at /var/crash/ but same thing, either empty or the log of my reboot.


I've looked around this website and people usually seem to indicate this might be a hardware issue but I've never had an issue running this system overnight on Windows 10 when it was my main setup.


What I've tried:



  • Multiple clean reinstall of Ubuntu Server 20.04.2

  • Updated my BIOS and removed any overclock I might have applied

  • Checked my temperatures and system load, both OK

  • Checked my RAM with Memtest86 for 3x4 runs, totalling over 12 hours and got no errors


My system specifications are:


Ryzen 1700 @stock
16 GB CORSAIR GDDR4 @2133MHz but rated for 3200Mhz
MSI B350 Gaming PLUS MOBO
Gigabyte GTX 1080
Samsung 960 EVO 512GB nVme SSD
650W corsair power supply
Plenty of cooling

If any of you has any lead or advice on how to diagnose my setup, or some more steps I could try I would be grateful.


Thanks for your help




EDIT 2021-03-03


After some more digging about ubuntu ryzen and freezes, I came across this thread on AMD forums. This suggests that there are issues with the C6 state on ryzen 1st gen. Their advice is to switch the power state in BIOS to Typical Current Idle which apparently has an impact on C6 state. I'll try that and see if anything changes.


More From » server

 Answers
7

Alright, I think I'll answer my own question.


My problem might have been linked to Ryzen CPUs not playing nice with Linux.
According to this AMD forum tread, a lot a people have encountered similar issues with Ryzen CPUs and Linux while idle and a Reddit thread with OP reporting similar issues. There seem to be a connection between C6 CPU state and hangs.


How to fix it


BIOS


Look into your BIOS and try to find Power Supply Idle Control or Global C-state Control and change that value from Auto or Low Current Idle to Typical Current Idle.


Disable C6 state


If that didn't fix it or you didn't have the option, another way to disable it can be found at this github repo.




I personnaly had the option in my MSI motherboard BIOS, applied the correct value and been running 48h without any problems, albeit maybe with a slightly higher power consumption.


This sure has been a frustrating issue to debug, but in the end I ended up learning some new stuff and make my server somewhat stable again.


Thanks to everyone who participated and shared my frustation, I hope this thread might be useful to someone else.


[#1861] Wednesday, March 30, 2022, 2 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
rieency

Total Points: 299
Total Questions: 116
Total Answers: 111

Location: Wales
Member since Tue, Dec 14, 2021
2 Years ago
rieency questions
Wed, Aug 25, 21, 15:17, 3 Years ago
Wed, Jun 23, 21, 04:37, 3 Years ago
Fri, Sep 30, 22, 12:07, 2 Years ago
Thu, Feb 24, 22, 00:50, 2 Years ago
Tue, Sep 20, 22, 15:25, 2 Years ago
;