Heavy RuTorrent activity crashing server

antigravity

Blitz 1st Class
Original poster
85
7
Hi

My Hetzner server crashes regularly - so I've moved it to another machine in hope that it would fix the issue.

However once again, even with a new machine -the server crashed again last night. Completely shut down. Required a physical restart.

It seems that RuTorrent could be the cause - when a bunch of new files are added at once - particularly if unpacking is required also.

How much RAM is allocated to the RuTorrent docker? Any idea how to alleviate this issue?

Thanks

Dave
 

antigravity

Blitz 1st Class
Original poster
85
7
Are there any heavy users of RuTorrent here? I have 100s of torrents loaded at any one time, and can have a heap added at any one time, which I'm thinking is causing PGBlitz and my server to crash.

Curious to know if there are any other heavy users of ruTorrent that don't have any issues.

I'm on my second server now- crashes daily.
 

bloodray

Blitz 2nd Class
36
11
Rutorrent is very CPU hungry from what I have seen. Monitor your netdata when adding multiple torrents. You are most probably overworking the server so either tone down your use or upgrade to much more powerful one. I understand sticking with rutorrent and I am guessing you are using RSS Feed Manager but you could be better off with a different torrent client if RSS Manager is no biggie. I know some people do not use RuTorrent due to the resource issue with too many torrents and success has been had in other torrent clients.
 

antigravity

Blitz 1st Class
Original poster
85
7
Interesting... Im currently running an i7 with 9000 cpumark, so I should have the necessary CPU.

I've also ran a similar setup (not PGBlitz) and had no issues for years.

I'm wondering if it has something to do with the fact the RuTorrent download folder is replicated under MergerFS, so any disk activity is duplicated...

Makes me wonder why the download folders use MergerFS at all. Can't see the point?
 

bloodray

Blitz 2nd Class
36
11
Could be. May be the developers can comment on that. I am running the same CPU so shared what I saw. If there is a fix, that could be great.
 

MrDoob

Administrator
Project Manager
Donor
1,459
420
I run 7400 torrents in the PG Server
38.500 on 7 instances on my other server and nothing crashed !!!

And I am a really heavy torrent user
 

antigravity

Blitz 1st Class
Original poster
85
7
I run 7400 torrents in the PG Server
38.500 on 7 instances on my other server and nothing crashed !!!

And I am a really heavy torrent user
Thanks -have you adjusted the docker image at all?

Do you mind sharing your rtorrent.conf file... Maybe my config is incorrect?

I find the server crashes completely when I add 5+ files at once.
 
Last edited:

antigravity

Blitz 1st Class
Original poster
85
7
You have to check with rutorrent. It doesn't matter how strong your cpu is, it's how taxing the program is.
Admin9705 - On my previous Seedbox (non-Docker), I ran RuTorrent on Ubuntu 16.04 for years with no issues. Same activity, same CPU.

Could it be that not enough resources are being allocated to the RuTorrent docker (excuse my ignorance, I don't know Docker very well).

Or could the fact that the RuTorrent download folder is replicated in the /mnt/unionfs folder - meaning that read/writes are hitting the BUS twice?

If I'm wrong - great :) Just asking some questions to try and figure out why I can't run RuTorrent with PGBlitz whereas I had no issues previously.
 

Admin9705

Administrator
Project Manager
5,819
2,008
Admin9705 - On my previous Seedbox (non-Docker), I ran RuTorrent on Ubuntu 16.04 for years with no issues. Same activity, same CPU.

Could it be that not enough resources are being allocated to the RuTorrent docker (excuse my ignorance, I don't know Docker very well).

Or could the fact that the RuTorrent download folder is replicated in the /mnt/unionfs folder - meaning that read/writes are hitting the BUS twice?

If I'm wrong - great :) Just asking some questions to try and figure out why I can't run RuTorrent with PGBlitz whereas I had no issues previously.
Docker will not limit resources unless we tell it to. Unless u have a test box, there an insane amount of variables causing your situation
 

antigravity

Blitz 1st Class
Original poster
85
7
Ok thanks. I've migrated to another new server, with new disks this time.

Will see how we go.
 

antigravity

Blitz 1st Class
Original poster
85
7
Just an FYI if anyone else comes across this issue. I know I'm not crazy, and after 3 seperate servers all experiencing issues with RuTorrent, I knew something was going on.

Anyone who uses PGBlitz and RuTorrent on a normal dedicated server (not a GCE instance) - may experience these issues.

People saying "it works fine on my machine" doesn't really help anyone experiencing issues. Hopefully this post will! :)

I migrated to a new server, i7 7700, 64GB RAM, 2x 512GB NVME's and an additional 6TB 7200rpm disk to seed torrents from.

This removed the following potential issues from the equation:
1. Hardware issues
2. Software RAID0

I even set it up so that torrents are originally written to one of the NVMe drives to remove any potential IO issues due to high levels of read/write access while downloading a torrent.

After all this - I no longer experienced complete crashes of my system. I believe the original crashes were caused by faulty hardware, or excessive IO load caused by RuTorrent on the software RAID0.

However - I still experienced random, regular crashes of RuTorrent, and in particular, it's plugins (autotools, auto unpack and ratio assigning).

After some investigation I found that RTorrent has a couple of major issues when handling large quantities of torrents.

1. Session Save which occurs every 20 minutes.
This creates a massive amount of disk thrashing and can cause the UI to become unresponsive, and for plugins to not work.

The only fix here is to extend the time between session saves.

See more info here:

2. DNS Timeout.
RTorrent can occassionally hang on hostname lookups. Once again, this causes an unresponsive UI, and stops plugins from working correctly

The fix here is to setup local DNS caching with dnsmasq.

More info here:
and
 
  • Like
Reactions: bloodray
Assists Greatly with Development Costs

Create an account or login to comment

You must be a member in order to leave a comment

Create account

Create an account on our community. It's easy!

Log in

Already have an account? Log in here.

Similar threads