What's new
PGBlitz.com

Register Now! Find useful tips, Interact /w Community Members and join the part the Best Community on the Internet!

Hetzner with PGBlitz keeps crashing

Status
Not open for further replies.

antigravity

Blitz 1st Class
Hey everyone

I've recently set up PGBlitz Encrypted on a Hetzner server, and after a few hours it seems to completely crash - can't even SSH into it. I have to order a reboot from the Hetzner side to get it back up and running.

My library is quite large - with approx 1500 TV Shows and 6000 Movies.

Server has 16GB of RAM, so I can't imagine RAM is the issue.

How can I find out what is causing the system to crash?

Thank you
 

antigravity

Blitz 1st Class
Couple of weird things occuring. RuTorrent stops labelling torrents correctly, and stops unpacking automatically.

Tautulli which monitors a remote plex server keeps dropping out.

But when I look at system load in NetData, it never goes over 8 (running an 8 threaded CPU)
 

antigravity

Blitz 1st Class
I've reduced the bandwidth available for PGBlitz, Sabnzbd and RuTorrent - based on the idea that maybe the network interface was getting flooded.

How can you turn off all the cron backups? I'm wondering if that has something to do with it too
 

antigravity

Blitz 1st Class
Thanks i"ll see if I can run a RAM test of some sort..

Anyone know how to disable the CRON backups? Can't see a way to access PG Cron using the script
 

MrDoob

Administrator
Project Manager
Donor
login as root

typ >> crontab -e
 

antigravity

Blitz 1st Class
Thank you.

Reducing speeds on PGBlitz, Sabnbzd and RuTorrent seems to have stopped the crashing. Thinking maybe IO or network load was too high causing all kinds of issues.
 

antigravity

Blitz 1st Class
Yeah I am running an SSD for PGBlitz and all the apps.

Torrents are saved and seeded from a RAID0 2x 3TB 7200rpm

So in theory that shouldn't be a problem.
 

MrDoob

Administrator
Project Manager
Donor
raid0 2 x 3 TB the drive is IO/ ERROR #software raid are pretty slow !!! alias fakeraid !!

and pretty sure you dont mounte the /opt/ on your ssd


Bash:
df -h /opt/  && df -h  /mnt/downloads/ && df -h | grep "/mnt/" &&  df -h | grep "/dev/"

df -h / --total --local

lscpu | grep "CPU(s):" | tail +1 | head -1 | awk  '{print $2}'

free -m | grep Mem | awk 'NR=1 {print $2}'

post this output please of all codes
 

antigravity

Blitz 1st Class
Server seems to crash every two days or so. At random times. Mostly when I'm not actively using it (Sonarr/Radarr/RuTorrent still running)

Here is a pastebin of the syslog before it crashed last night. Lots of errors, particularly with the network interface, some python errors, SMART temp numbers towards the end look a little worrying too


I also ran RAM and SMART tests. Everything passed ok. SMART results looked ok.
 

antigravity

Blitz 1st Class
load netdata container and see where's the bottle neck. could be slow disk io, cpu or ram.
Checked netdata and everything seems ok. Load is normally around 2-3 at most.

I get some HDD backlog activity, but nothing overly concerning.

raid0 2 x 3 TB the drive is IO/ ERROR #software raid are pretty slow !!! alias fakeraid !!
RAID0 is quicker than non raid though? Even software raid. I've tested the HDD speeds and they are good.

and pretty sure you dont mounte the /opt/ on your ssd
Why's that? SSD is much faster than HDD, both for sequential and random read/writes.

Bash:
df -h /opt/  && df -h  /mnt/downloads/ && df -h | grep "/mnt/" &&  df -h | grep "/dev/"

Filesystem Size Used Avail Use% Mounted on
/dev/sdc3 442G 51G 369G 12% /
Filesystem Size Used Avail Use% Mounted on
/dev/sdc3 442G 51G 369G 12% /
gdrive: 1.0P 154T 1.0P 14% /mnt/gdrive
tdrive: 1.0P 0 1.0P 0% /mnt/tdrive
pgunion 4.1P 307T 4.1P 7% /mnt/unionfs
gcrypt: 1.0P 154T 1.0P 14% /mnt/gcrypt
tcrypt: 1.0P 0 1.0P 0% /mnt/tcrypt
/dev/md0 5.5T 4.0T 1.2T 78% /mnt/downloads/rutorrent
/dev/sdc3 442G 51G 369G 12% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
/dev/sdc2 488M 112M 352M 25% /boot
/dev/md0 5.5T 4.0T 1.2T 78% /mnt/downloads/rutorrent


Bash:
df -h / --total --local
Filesystem Size Used Avail Use% Mounted on
/dev/sdc3 442G 51G 369G 12% /
total 442G 51G 369G 12% -


Bash:
lscpu | grep "CPU(s):" | tail +1 | head -1 | awk  '{print $2}'
8

Bash:
free -m | grep Mem | awk 'NR=1 {print $2}'
15951
 

MrDoob

Administrator
Project Manager
Donor
Okay you know it better I leave the thread.

Have fun to find the fail .
 

MrDoob

Administrator
Project Manager
Donor
FYI

For software RAID, data is configured in the host server CPU. Hardware RAID uses the internal motherboard or a separate RAID card.

Because it processes on the internal server, software RAID is slower than hardware RAID However, because hardware RAID requires purchasing additional hardware, software RAID costs less.

Here an other post

Software RAID
Advantages:

Unlike hardware RAID, software RAID uses the processing power of the operating system in which the RAID disks are installed. The cost is lower because no additional hardware RAID controller is required. It also permits users to reconfigure arrays without being restricted by the hardware RAID controller.

Disadvantages:

Software RAID tends to be slower than hardware RAID. Since some processing power is taken by the software, read and write speeds of your RAID configuration, along with other operations carried out on the server can be slowed down by it. Software RAID is often specific to the operating system being used, so it cannot generally be used for partitions that are shared between operating systems.

Replacing failed disk in the software RAID is a bit more complex. You have to firstly tell your system to stop using the disk and then replace the disk.



Or here

 

antigravity

Blitz 1st Class
Software RAID is slower than Hardware RAID. Completely agree.

But software RAID0 (striping across two disks) isn't slower than running a single disk on its own.

Okay you know it better I leave the thread.

Have fun to find the fail .
Geez thanks! Why are you upset? I'm curious to know why /opt shouldn't be run on an SSD.

And you asked.me to provide you with info to help, and I did?
 
Status
Not open for further replies.

Similar threads


Development Donations

 

Trending

Top