What's new
PGBlitz.com

Register Now! Find useful tips, Interact /w Community Members and join the part the Best Community on the Internet!

Solved Using PGBlitz but unable to upload more then 750G per day

Status
Not open for further replies.

itsmylife10173

Blitz 3rd Class
Donor
Im using PGBlitz with 20 keys deployed but my uploads stop after 750G per day. Here is what I have tried to fix this.
  1. reboot
  2. deploy PGClone again
  3. made new keys and deploy (when I do this it will upload 750G and then stop till the next day)

I have around 40T more to upload, suggestions on what to look at would be helpful.
Thanks.

3100

Stops at 750G
3101
 

itsmylife10173

Blitz 3rd Class
Donor
I removed all of the keys, created 10 and deployed. now the upload stops after 1T of media is uploaded. Any suggestion on how to increase that? Thanks
 

itsmylife10173

Blitz 3rd Class
Donor
This is what it shows when i type "pgblitz" in the comad line. I dont see any specific error is there a different command I should run?

2019/03/10 16:49:25 INFO :
Transferred: 1.000T / 7.836 TBytes, 13%, 21.838 MBytes/s, ETA 91h10m9s
Errors: 92 (retrying may help)
Checks: 3375 / 3375, 100%
Transferred: 434 / 3334, 13%
Elapsed time: 13h20m36.2s
Transferring:
......List of 16 movies 0% uploaded........
 
Assists Greatly with Development Costs
M

MrDoob

Guest
--> Errors: 92 (retrying may help)

i think thats a problem .
-------

did you change the keys during the upload?
 

itsmylife10173

Blitz 3rd Class
Donor
Yesterday the upload had stopped afer 750G I changed the keys and then moved everything from /mnt/pgblitz/upload to a temp dir, rebooted the server then moved the media from the temp dir to /mnt/move/. PGBlitz uploaded 1T of the media and then stopped.
 
M

MrDoob

Guest
but i uploaded each day more then 2 - 4 TB ..

service pgunion status
service pgblitz status
 

itsmylife10173

Blitz 3rd Class
Donor
[email protected]:~$ service pgunion status
● pgunion.service - PGUnion
Loaded: loaded (/etc/systemd/system/pgunion.service; enabled; vendor preset: enabled)
Active: active (exited) since Sat 2019-03-09 21:31:23 UTC; 20h ago
Process: 2011 ExecStart=/bin/bash /opt/appdata/plexguide/pgunion.sh (code=exited, status=0/SUCCESS)
Main PID: 2011 (code=exited, status=0/SUCCESS)
Tasks: 33 (limit: 7372)
CGroup: /system.slice/pgunion.service
└─2421 mergerfs -o sync_read,allow_other,direct_io,use_ino,func.getattr=newest,nonempty,secur

Mar 09 21:31:23 googolplex systemd[1]: Started PGUnion.
lines 1-10/10 (END)

[email protected]:~$ service pgblitz status
● pgblitz.service - 'PGBlitz Loaded Balanced Multi SA GDrive Uploader'
Loaded: loaded (/etc/systemd/system/pgblitz.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2019-03-09 21:31:23 UTC; 20h ago
Main PID: 2021 (bash)
Tasks: 77 (limit: 7372)
CGroup: /system.slice/pgblitz.service
├─ 2021 /bin/bash /opt/appdata/pgblitz/pgblitz.sh
└─40364 rclone moveto /mnt/largevolume/pgworking/pgblitz/upload gdsa03C:/ --config /opt/appda

Mar 10 03:22:50 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:23:27 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:24:03 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:24:40 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:25:16 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:25:52 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:26:29 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:27:06 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:27:42 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:28:18 googolplex bash[2021]: cat: write error: Broken pipe
 

itsmylife10173

Blitz 3rd Class
Donor
Thank you
looks likes its uploading again hopfully it doesnt stop. :)


[email protected]:~$ service pgblitz status
● pgblitz.service - 'PGBlitz Loaded Balanced Multi SA GDrive Uploader'
Loaded: loaded (/etc/systemd/system/pgblitz.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2019-03-09 21:31:23 UTC; 20h ago
Main PID: 2021 (bash)
Tasks: 77 (limit: 7372)
CGroup: /system.slice/pgblitz.service
├─ 2021 /bin/bash /opt/appdata/pgblitz/pgblitz.sh
└─40364 rclone moveto /mnt/largevolume/pgworking/pgblitz/upload gdsa03C:/ --config /opt/appda

Mar 10 03:22:50 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:23:27 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:24:03 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:24:40 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:25:16 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:25:52 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:26:29 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:27:06 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:27:42 googolplex bash[2021]: cat: write error: Broken pipe
Mar 10 03:28:18 googolplex bash[2021]: cat: write error: Broken pipe

[email protected]:~$ kill 2021
-bash: kill: (2021) - Operation not permitted
[email protected]:~$ sudo kill 2021
[sudo] password for plexadmin:
[email protected]:~$ service pgblitz restart
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to restart 'pgblitz.service'.
Authenticating as: plexadmin
Password:
==== AUTHENTICATION COMPLETE ===

[email protected]:~$pgblitz

2019/03/10 18:08:55 INFO :
Transferred: 23.710G / 6.997 TBytes, 0%, 78.024 MBytes/s, ETA 26h2m2s
Errors: 0
Checks: 2547 / 2547, 100%
Transferred: 33 / 2992, 1%
Elapsed time: 5m11.1s
Transferring:
* movies/some movie.mp4: 40% /3.541G, 6.361M/s, 5m40s
* movies/some movie.mp4: 30% /5.299G, 5.875M/s, 10m44s
* movies/some movie.m2ts: 37% /3.995G, 5.857M/s, 7m18s
* movies/some movie.mkv: 35% /4.375G, 5.292M/s, 9m5s
* movies/some movie.mkv: 18% /7.868G, 5.290M/s, 20m34s
ect...

2019/03/10 18:08:55 INFO :
Transferred: 1.000T / 7.812 TBytes, 13%, 19.866 MBytes/s, ETA 99h51m58s
Errors: 98 (retrying may help)
Checks: 3375 / 3375, 100%
Transferred: 434 / 3328, 13%
Elapsed time: 14h40m6.2s
Transferring:
* movies/some movie.avi: 0% /1.094G, 0/s, -
* movies/some movie.mp4: 0% /1.993G, 0/s, -
* movies/some movie.m4v: 0% /855.519M, 0/s, -
ect.....
 

itsmylife10173

Blitz 3rd Class
Donor
After lots of scrolling.....
2019/03/11 02:16:49 ERROR : movies/SomeMovieTitle.mp4: Failed to copy: googleapi: Error 403: User rate limit exceeded., userRateLimitExceed$

after I restart it will run for a few minuets then stop again.
creating more/different keys seems not to help.

Currently I have no errors but it is not uploading.
3107
:/
 

itsmylife10173

Blitz 3rd Class
Donor
It seems you are able to upload multiple terabytes per day with PGBlitz but after a certain point the 750g limit is enforced. This ban is to the account not the API key. looking though the rclone forums I have found this to be the case with other users. Seems like there is not really anything that can be done PGBlitz is working as it should google is the limiting factor. luckily for me I only have about 6T of 32T left to upload.
Thanks for the help MrDoob
 
Status
Not open for further replies.

Similar threads


Development Donations

 

Top NZB NewsGroups!

Members - Up To a 58% Discount!

Trending

Top