What's new
PGBlitz.com

Register Now! Find useful tips, Interact /w Community Members and join the part the Best Community on the Internet!

[Problem] Sonarr/Radarr don't import automatically

Assists Greatly with Development Costs

recse

Junior Member
Hi

I have PGblitz installed in my Server and I have some problems with Sonarr and Radarr.

The problems are that both don't import the TV/movies into the mnt/unionfs/{tv/movies} field.

I have the "Completed download handling" enabled and the "Drone factory option" disabled, and the "Remote paths" correctly.

I checked the log, and it showns the following:

"Unknown serie" or "Unable to parse file".

Furthermore, I tried the manual import and it works succesfully so, I don't know where it is the problem.

thanks
 

fr0sty

Senior Member
Staff
Donor
I have nothing in my "remote path mappings" and "hardlinking" off. Ensure you have a root path configured under media management.
 

recse

Junior Member
Hi,

If you have nothing in your Remote Path, then how is your Usenet/Torrents moving from mnt/Downloads/USENETTORRENT to mnt/unionfs/TV?

As per the guide, "Hardlinking" must be on, right?
 

fr0sty

Senior Member
Staff
Donor
Torrent is different. I found this works well for usenet as i struggled when i first changed over.
 

recse

Junior Member
Well, in my case, i'm using torrent so, do you know how could I do in order to it works?

thanks
 

fr0sty

Senior Member
Staff
Donor
When i was using it, exactly as instructed in wiki. Sorry i couldn't be of more help
 

artiume

Full Member
Remote mapping is required if the two programs see mapped locations differently. With the way PG sets everything up, all the containers see the same mapped paths. I'm not sure what you put exactly though. If you want to post your config that would help :)
 

Hitsville

Junior Member
Donor
Guys Im preety new to PGBlitz and having read the wiki Im even less clear.
Firstly Im running PG Move unencrypted on a Hetzner dedi with GDrive mounted.
Im guessing I mount mty existing shows on Google via /mnt/unionfs/

With the Shows mounted doesnt it just directly write to the unionfs mount as soon as Sonarr detects the download is finished?
It seems to work fine.

The whole thing that throws me in the wiki is all the stuff about setting remote path mappings within Sonarr/Radarr and the whole PGMove workflow. Is that just redundant info when Gdrive is mounted?

I actually had everything set up but conflicting info and an issue where my torrents seemed to get deleted randomly (since figured out the Clone cleaner stuff, so that part is solved).

In the wiki (see below) but everything seems to work fine without it? So please just for sanity's sake can someone confirm that its irrelevant for me and that I can simply have the ARR aps pointing to both the torrent client and Gdrive via /mnt/unionfs/


5. How it works

  1. When radarr/lidarr/sonarr imports your downloads from /mnt/downloads/_downloader_/_category_/... it reads the download mappings and instead of looking for /mnt/downloads/_downloader_/_category_/... it looks for /mnt/unionfs/_downloader_/_category_/.... So at this point, it's accessing the files inside the merger.
  2. Now Arr will do a hardlink/mv operation to rename and organize the media. Arr will think it's putting the files in /mnt/unionfs/_category_/..., but it's actually putting them into /mnt/downloads/_category_/...
  3. At this point your files are imported and renamed, they are accessible at /mnt/unionfs/_category_/...
  4. The move script runs and ignores any "known downloader" folders inside /mnt/downloads, and moves everything it doesn't ignore (like /mnt/downloads/_category_) to /mnt/move for uploading.
  5. Now your files are inside /mnt/move for uploading, run pgmove to see the status.

-- Arr has to organize the files, To hardlink inside mergerfs, it requires everything being inside the merger (aka /mnt/unionfs) for rapid importing to work. The downloader mappers tell arr to look in /mnt/unionfs/nzbget so then it does a mv. But underlying, mergerfs will do a rename in the real location, which is /mnt/downloads. There isn't a way to force that rename operation to occur in /mnt/move instead or we would just do that, this is a behavior we cannot change.


Files that exist in /mnt/downloads/_category_ will be moved for uploading.


Move runs in a batch, it grabs everything ready at that time and uploads it. It will not move other files in the background while actively uploading!


Arr however will continue to import and organize the next set of files. So typically files sit in /mnt/downloads/_category_ for 0-45m (depending on active upload).


During this entire process, once arr does the import, the file stays at /mnt/unionfs/_category_/[...] this is so plex can scan it and have it available without first waiting for it to be uploaded.
 

UncleBuck

Senior Member
Staff
I am thinking someone can explain this better than me but there is a reason for having the remote path mappings.

Basically, when your download client finishes a download it will report the location of the files to Arr as /mnt/downloads/category/. If Arr tries to import from /mnt/downloads/category/ into /mnt/unionfs/move/ it will do a copy which will take some time depending on the file size.

By adding a remote mapping Arr will translate the path reported by your download client from /mnt/downloads/category/ to /mnt/unionfs/category/. So now it sees both locations as being part of the mergerfs mount. When importing it will do a hardlink/mv operation from /mnt/unionfs/category/ to /mnt/move/ which is pretty much instantaneous.
 

Hitsville

Junior Member
Donor
I am thinking someone can explain this better than me but there is a reason for having the remote path mappings.

Basically, when your download client finishes a download it will report the location of the files to Arr as /mnt/downloads/category/. If Arr tries to import from /mnt/downloads/category/ into /mnt/unionfs/move/ it will do a copy which will take some time depending on the file size.

By adding a remote mapping Arr will translate the path reported by your download client from /mnt/downloads/category/ to /mnt/unionfs/category/. So now it sees both locations as being part of the mergerfs mount. When importing it will do a hardlink/mv operation from /mnt/unionfs/category/ to /mnt/move/ which is pretty much instantaneous.
Yeah thanks for the reply.
I finally managed to spend sometime with it and sort of figures the reason was essentially speed of availability of the file for viewing.
It does make sense.
I think the wiki sort of confused me because it implied it was an essential part of setup.
Always scary for a newbie.

Anyway as I upload straight to Gdrive it’s non essential for my use case and everything works fine.

Thanks for clarifying though. PGblitz just rocks.
 
Assists Greatly with Development Costs
T

TheShadow

Guest
Yeah thanks for the reply.
I finally managed to spend sometime with it and sort of figures the reason was essentially speed of availability of the file for viewing.
It does make sense.
I think the wiki sort of confused me because it implied it was an essential part of setup.
Always scary for a newbie.

Anyway as I upload straight to Gdrive it’s non essential for my use case and everything works fine.

Thanks for clarifying though. PGblitz just rocks.
It is essential, it's not just speed. When using mergerfs, you must stay within the union for certain operations. The mappings make it so arr stays within the union.

Several reasons for this:

mergerfs performance suffers when downloading nzb and torrent files within the union. So that's why the downloaders do not download in the union.

The mappings allow the benefit without the drawback.

Without the mappings your system will copy files, not move/rename. It's unneeded disk IO to copy a file that could otherwise be renamed instead. This will cause IO wait and double the disk activity.

The performance difference is noticable. It can have an impact on streams during high disk activity.

Without the mappings, you need 2x the disk space for normal operations, while getting 1/2 of the disk perf, double the wear and tear on the drive, etc,... all for nothing. You have nothing to gain not configuring the mappings, that's why we stress it's required.

Keep in mind, torrents are still copied depending on torrent status, client, and version of sonarr. Torrents are usually copied on import so not to interrupt seeding.

In 8.6.6 will add a configuration notice banner, about these mappings during arr installation.

Also the cleaner will be updated so it does not touch torrent downloads. It will only remove 10gb or less files in the nzb folders. Most of the time, arr fails to import for good reasons. This is the best compromise between varying user wants and needs.
 

Hitsville

Junior Member
Donor
OK. So just to be clear.
Even though my movies/shows are moved directly up to to Gdrive and the fact that I only ever use torrents you feel I still need the mappings?
Regards the cleaner that would be great although right now I have it set to 11 days which is a day beyond the longest seedtime required by any of my trackers.
 
T

TheShadow

Guest
OK. So just to be clear.
Even though my movies/shows are moved directly up to to Gdrive and the fact that I only ever use torrents you feel I still need the mappings?
Regards the cleaner that would be great although right now I have it set to 11 days which is a day beyond the longest seedtime required by any of my trackers.
Yes
 

Create an account or login to comment

You must be a member in order to leave a comment

Create account

Create an account on our community. It's easy!

Log in

Already have an account? Log in here.

Similar threads


Development Donations

 

Top NZB NewsGroups!

Members - Up To a 58% Discount!

Trending

Online statistics

Members online
14
Guests online
104
Total visitors
118
Top