What's new
PGBlitz.com

Register Now! Find useful tips, Interact /w Community Members and join the part the Best Community on the Internet!

Understanding the Plexguide Ecosystem

flicker-rate

Respected Member
There's a ton of moving parts involved in a automated media server system. It's a bit like building legos, and what Admin has done is source all the required pieces into a box with assembly instructions and a brilliant menu system. Many of the more complex pieces are packaged and preconfigured to work together in a way to make for a more automated install. I won't get into the technical details (like docker/portainer/ansible), but just the bare minimum you'll need to know to operate and maintain your system.

View attachment 337

Whew, ok, there's a lot going on here, but don't worry, I'll break it down starting left from right.

The journey begins on the high seas of BitTorrent and Usenet. To sail these waters, one requires the guidance of Torrent Tracker sites and Usenet Indexers. You can think of these as a digital treasure map (but more like a phonebook directory). They help you find content for Plex by pointing out the location of the files you request. Trackers and indexers can be public⁶, semi-public (as in pay-to-play) and private (invite required). Private trackers and indexers will get net the best content, the fastest download speeds and the best selection. Note that if you go with Torrents, you'll need a bigger hard drive to seed files. Also note there is a special place in hell for people who don't seed. If you go with Usenet, know that many older files (and many new ones) will be blown out of the water by the DMCA Royal Navy before you can download the file, so gaining access to private indexers or having multiple indexers and block accounts will be necessary. You can either pick using torrents or usenet, but I recommend both— Torrents for older and rare files, and Usenet for fast access to recent files.

These are helper programs that aggregate all your Usenet indexers (NZBhydra) and Torrent trackers (Jackett) together in one place. This is optional, as Radarr/Sonarr/Lidarr can (usually) be configured to access your indexers and torrent trackers directly. If you have many indexers/trackers or private ones, I recommend going with this route.

Radarr⁽ᵃ⁾ is for Movies, Sonarr⁽ᶜ⁾ is for TV and Lidarr⁴ is for Music. These programs are your vigilant crewmates on lookout for new movies, shows and albums. They automatically check the RSS feeds from your indexers and trackers (directly, or indirectly via nzbhydra/jackett) and tell NZBget/Deluge to snatch the file immediately if it matches what you request it to. They can also search for older releases. You can configure these to look for content based on lists, so you don't have to import content manually. Note that these programs can also see what's in UnionFS (a filesystem) and will not download media you've already plundered). More on that later.

These are your download clients. NZBget⁽ᵃ⁾ handles Usenet downloads, Deluge⁽ᵇ⁾ handles torrent downloads. They assemble the file together in temporary directories, perform a integrity check based on file hashes and repairs them if necessary. See notes on ruTorrent¹ .

Once the files are finished downloading, they are moved (or copied for torrent files) into "completed" directories, which Radarr/Sonarr/Lidarr then processes by stripping out .nfo's, samples and extras and renaming the file in a way that Plex can find the correct metadata. After processing, the files are then packaged into an appropriately named folder and placed onto a loading dock (/mnt/move*) where it awaits shipment.

Rclone is a utility program that syncs files to the cloud. You can think of rclone as a truck that arrives every 8 minutes, driven by a systemd daemon called move.sh that delivers your files to the Google Drive warehouse for long-term storage. The workers at Google don't exactly have the best reputation when it comes to respecting your privacy, so you may wish to obfuscate your booty (get it?) by encrypting your files before it's whisked away to the great cloud in the sky.

Google doesn't like it when you spam it with requests to see what's in your cloud storage every time Plex issues a library rescan, and will issue a 24 hour API ban if you hit their servers too hard. To mitigate this issue, you'll need to mount your Google Drive with PlexDrive. PlexDrive caches the metadata of your gdrive (filenames, locations) into a database, then presents that information as a pseudo directory called a fuse mount. Now when Plex scans this directory, it only issues new api requests to google for files that have been recently changed, instead of every single file of your entire library.

There's a caveat though, Plex doesn't actually scan the pseudo-directory. Plexdrive is actually mounted onto a union filesystem called UnionFS which combines the contents of what's in the loading dock awaiting upload (/mnt/move/*) and the contents of PlexDrive into one cohesive directory (/mnt/unionfs). Note that Radarr/Sonarr/Lidarr can read from this directory so it won't download duplicates. The advantage to having UnionFS combine these two directories is the ability to watch files right after they are done downloading, instead of waiting for it to be uploaded.

From there, Plex⁽ᵉ⁾ scans the library from UnionFS where it's indexed and presented with metadata like IMDB ratings, movie posters and descriptions in a nice way on port 32400.

That's an ugly port number though, so we'll present it in a nicer way on port 443 (https) with your own domain name with Traefik. Traefik is a reverse proxy that routes traffic from various TCP ports into port 443 and applies SSL encryption so traffic can't be sniffed en route to your users. Note that all the other programs also have their traffic routed through Traefik, I just haven't shown it on the diagram for simplicity's sake.

Finally, users can then browse to your website and watch. If they'd like to make a request for another movie or show, they can access Ombi, which is basically a front-end middleman that passes on media requests to Radarr or Sonarr. The advantage is that you won't have to give your users admin access to the rest of your configs and you can deny/allow requests if you want. It can also send all your users an email with what shows have been recently added to your Plex library.

Programs you can substitute:
  • a. NZBget > SABNZBD
  • b. Deluge > ruTorrent
  • c. Sonarr > Couchpotato > Medusa
  • d. Radarr > Sickrage
  • e. Plex == Emby (Plex has better UX, Emby is open source, both are semi-free)
1. ruTorrent has many security flaws! It is very difficult to protect the RPC and HTML ports unless they are completely behind a firewall on LAN (default: no). Exploits have already been spotted in the wild already and are being used to install crypto miners onto your machine. (at least plexguide asks for your consent before installing one)

2. NZBget is faster and more memory-efficient than SABNZBD because it's written in C rather than python. SABNZBD does have more bells and whistles though.

3. I didn't include Ubooquity (a comic book server) or Mylar (equivalent of radarr/sonarr, but for books)

4. Lidarr is still in it's alpha stage and may be buggy. Use at your own risk.

5. Other bells and whistles I recommend: Tautulli (to monitor your plex users and notify you of recent downloads), and Netdata (for monitoring your server performance under load).

6. Sailing public tracker waters is dangerous! Protect your broadside with the DelugeVPN container option and a PIA VPN subscription.

7. For extra security, you may want to install fail2ban, disable root logins with password, denyhosts and disable port access to your applications.
 
Last edited by a moderator:

Admin9705

Administrator
Project Manager
Wait, wow?! This is amazing. You have know idea how helpful this is. Been caught in so many things! I'll make sure to link this in the FAQ and Newbie Area. @flicker-rate outstanding!
 

barryclamsworth

Legendary Member
Donor
Fantastic, thank you so much for this.

Was considering trying to write up my understanding of this as means of asking these very questions.
 

Admin9705

Administrator
Project Manager

Escondido

Senior Member
That's really cool man. This will definitely help make things clearer for people setting things up. I've been using SAB since the get go, but I might have to around with NzbGet.
 

EasternPA

Respected Member
Staff
This is excellent work. Any chance you can put together something that helps us understand the paths as they appear within the docker containers? For instance, your diagram shows NZBGet dumping files in /mnt/nzbget/incomplete (or complete) but those paths appear as /incomplete and /complete within the NZBGet UI.

I'm particularly interested in what paths we're supposed to use in Plex itself. I know it is supposed to be based on /mnt/unionfs but I'm not sure if that is masked in the Plex container.

Thanks again for sharing, this is so helpful to newcomers!

EDIT: Now that I got Plexdrive and Rclone working (in another thread), I finally went about configuring the paths for the Plex server. I saw how easy it is to browse the folders and assign them to each type of media. It wasn't clear if there was any harm in using /yourserver/mnt/unionfs or just /unionfs (the path as shown inside the container) so I just used /unionfs.
 
Last edited:

Admin9705

Administrator
Project Manager
This is excellent work. Any chance you can put together something that helps us understand the paths as they appear within the docker containers? For instance, your diagram shows NZBGet dumping files in /mnt/nzbget/incomplete (or complete) but those paths appear as /incomplete and /complete within the NZBGet UI.

I'm particularly interested in what paths we're supposed to use in Plex itself. I know it is supposed to be based on /mnt/unionfs but I'm not sure if that is masked in the Plex container.

Thanks again for sharing, this is so helpful to newcomers!

EDIT: Now that I got Plexdrive and Rclone working (in another thread), I finally went about configuring the paths for the Plex server. I saw how easy it is to browse the folders and assign them to each type of media. It wasn't clear if there was any harm in using /yourserver/mnt/unionfs or just /unionfs (the path as shown inside the container) so I just used /unionfs.
ya that yourserver is suppose to be deleted.
 

flicker-rate

Respected Member
This is excellent work. Any chance you can put together something that helps us understand the paths as they appear within the docker containers? For instance, your diagram shows NZBGet dumping files in /mnt/nzbget/incomplete (or complete) but those paths appear as /incomplete and /complete within the NZBGet UI.
.
View attachment 376Container --> Host mappings can be found in the portainer configs for each individual docker container.

To find this, you can go to https://portainer.<your_domain>.com or <host ip>:9000, click on the application you want the mappings for, click on "Volumes" under the advance container settings box. Here you'll be able to map custom directories if you want. As a sidenote, Plex has the root directory of the host / mapped to /yourcomputer inside of the plex container, so you'll be able access preexisting content without too much fuss.
 

mixedvadude

Junior Member
View attachment 376Container --> Host mappings can be found in the portainer configs for each individual docker container.

To find this, you can go to https://portainer.<your_domain>.com or <host ip>:9000, click on the application you want the mappings for, click on "Volumes" under the advance container settings box. Here you'll be able to map custom directories if you want. As a sidenote, Plex has the root directory of the host / mapped to /yourcomputer inside of the plex container, so you'll be able access preexisting content without too much fuss.
I'm not seeing the Advanced container settings anywhere on my end...
 

flicker-rate

Respected Member
I saw all those, but I'm talking about being able to actually edit the volume mapping
To edit the volume mappings, first stop the container, then click on edit/duplicate. Down at the bottom there's a tab called volumes and you can add new mappings or edit existing ones.
 
Assists Greatly with Development Costs

Create an account or login to comment

You must be a member in order to leave a comment

Create account

Create an account on our community. It's easy!

Log in

Already have an account? Log in here.

Development Donations

 

Top NZB NewsGroups!

Members - Up To a 58% Discount!

Trending

Online statistics

Members online
13
Guests online
91
Total visitors
104
Top