I did not know how many theads to put to max out a tesla p4, I found various numbers and started with the highest, 3,000 something. shortly after it starts trying to transcode, unraid locks up completely. I lowered it to 1k, same, now it's at 16 and seems to run fine, but maybe putting 0 could have it use as much as possible if you don't know how many threads to put as the max?

using an i7-10700k and the p4, how many threads should I use from the cpu, all 16? will it cause everything else to slow to a crawl or will it share the cpu fairly? what kind of speed gains do you get from eg 2 threads vs 16 threads?

Any chance of a docker release?

I have a video file with dv profile 5 Dolby Vision, Version 1.0, dvhe.05.06, BL+RPU and it fails with this script.
Do you know what parameters should I use for such file ?

Just curious about your thoughts on building and releasing this as a docker container to simplify deployment and use of the script.

Thank you.

If the "Indexes" directory does not exist inside Contents, the script throws an error. In reviewing the code, I see a mkdir for the tmp folder but I didn't see one for the bundle_path. However I am not a programmer in anyway.

[Errno 2] No such file or directory:
'/var/lib/plexmediaserver/Library/Application Support/Plex Media 
Server/Media/localhost/8/06faad73082ea7937e85af3f2ac98f7136baad0.bundle/Contents/Indexes/index-sd.bif'

Running on windows, i keep getting the same error when its trying to process my movies, "ERROR | mp_main:process_item:221" it will go threw all my movies with that same error, then stall out when it hits the TV section, i already saw the other issue about running on windows that has been closed, but my problem seems not related to that.

When I run the script, it failed with hundreds of:

ERROR | main:process_item:204 - Failed to load library from
libmediainfo.so.0 - libmediainfo.so.0: cannot open shared object file: No such file or directory

MediaInfo and all other dependencies are installed.

My plex server is in docker on the same machine as the script. I’m running with the Ubuntu server.

FYI, I tried your script because my video thumbnails do not seem to be generating normally anymore. Other than that, however, my server is working fine.

Hi

I just setup your script on serveral systems and tried to tweak it to get faster thumbnails generated.
I noticed its pretty slow to generate the thumbnails.

How fast does it run for you? My media is located on a smb share, maybe I have some bottleneck somewhere.

I read, that the ffmpeg fps setting for vf_parameters is pretty slow and a faster equivalent should exist by using the frame select function. I did not figured out yet how to use the frame select instead of fps to get the "thumbnails every 10 seconds" output.

Any ideas to speed this up would be great.

Thanks for the script.

Although somewhat intuitive, docs may want to reflect that ffmpeg is required.

I am testing this out and it looks awesome. When I first tested, gpustat was not available. I recommend adding this to the list of recommended pip3 installs in the docs.

I know this script was most likely designed to function in a linux environment and I know asking for windows support would be a long shot so I'm reviewing the code to try to see if I can get it running on a Windows box. I've made some changes to manually specify the location of FFMPEG and MediaInfo and I can see it start the FFMPEG process before erroring out with " ERROR | mp_main:process_item:204 - NVML Shared Library Not Found" . If you can think of anything that might get it going forward let me know, I'll keep trying to move it forward.

Getting this error when trying to run the script.

``concurrent.futures.process._RemoteTraceback:`
"""
Traceback (most recent call last):
File "/usr/lib/python3.10/concurrent/futures/process.py", line 246, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/home/administrator/./plex_generate_previews.py", line 212, in process_item
os.mkdir(indexes_path)
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Media/9/50aba4dd1dc622d128431d7f21feca8c2b31cfa.bundle/Contents/Indexes'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/administrator/./plex_generate_previews.py", line 285, in
run()
File "/home/administrator/./plex_generate_previews.py", line 247, in run
future.result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Media/9/50aba4dd1dc622d128431d7f21feca8c2b31cfa.bundle/Contents/Indexes'

I was getting this error on a few of my files. So I did some digging and read that if you add 'ignore' onto the decode part, it fixes this problem: Zulko/moviepy#959

So if anyone comes across this problem, just change err.decode('utf-8') to err.decode('utf-8', 'ignore') on both instances. I'm still not sure if this breaks anything, but apparently, it just tells it to ignore the extra byte.

    if proc.returncode != 0:
        err_lines = err.decode('utf-8', 'ignore').split('\n')[-5:]
        logger.error(err_lines)
        raise Exception('Problem trying to ffmpeg images for {}'.format(video_file))

    # Speed
    end = time.time()
    seconds = round(end - start, 1)
    speed = re.findall('speed= ?([0-9]+\.?[0-9]*|\.[0-9]+)x', err.decode('utf-8', 'ignore'))
    if speed:
        speed = speed[-1]
    logger.info('Generated Video Preview for {} HW={} TIME={}seconds SPEED={}x '.format(video_file, hw, seconds, speed))

There is a known issue where AV1 codes are extremely slow to process when using the docker container.

For example, it runs at 0.8x speed in the container; on the native machine, the exact same FFmpeg command runs at 10x. The only difference I can see is that the av1 lib in the container uses [libaom-av1 @ 0x559ad8f192c0] 3.8.2, while the native host uses [libdav1d @ 0x55e42e5f4640] libdav1d 1.2.1.

I am unsure how to change that as I use the linuxserver/ffmpeg base image.

Leaving this ticket open, I hope a ffmpeg expert can help as i've run out of ideas.

Any episode where the file is a multi-part (ie S01E01-E02) is processed as many times as it appears in Plex. This is logical since Plex treats it as Part 1 and Part 2 with reference to the same file and so it would pull from the database sequentially. The problem is that the same bundle identifier is used. When using multi-threading, the processing occurs simultaneously with multiple instances of ffmpeg overwriting each other. The script then tries to pull the files from the same folder into a .bif. Whichever instance occurs later in time throws errors for missing jpg frame files. This wouldn't occur if the .bif existed prior to analysis since the code looks for an existing .bif. It only occurs because the Plex db tells the script to process the file twice and the second (or third, etc) ffmpeg instance begins prior to the final creation of the .bif by the first instance.

I see two solutions. One obvious work-around is to limit the script to only one CPU or GPU thread. That worked for debugging and narrowing down the cause of the errors. The better solution is to change the setup and teardown of temp bundle folders to allow detection of an existing process and skip multiple processing for the same bundle. I will attempt a commit after testing.

my compose file:

version: '3'
services:
previews:
image: stevezzau/plex_generate_vid_previews:latest
environment:
- PLEX_URL=http://192.168.2.90:32400
- PLEX_TOKEN=zMQQrtertertasNx
- PLEX_BIF_FRAME_INTERVAL=3
- THUMBNAIL_QUALITY=2
- PLEX_LOCAL_MEDIA_PATH=/mnt/thumbs/
- TMP_FOLDER=/tmp/thumbs/
- PLEX_TIMEOUT=120
- GPU_THREADS=8
- CPU_THREADS=6
volumes:
- /mnt/user/Plex/Plex Media Server/Media/localhost:/mnt/thumbs
- /mnt/user/data/media/anime/movies:/mnt/anime/movies
- /mnt/user/data/media/tv:/mmt/tv
- /mnt/user/data/media/movies:/mnt/movies
- /mnt/user/data/media/anime/series:/mnt/anime/series
deploy:
resources:
reservations:
devices:
- capabilities: [gpu]
runtime: nvidia

it actually ran and did anime movies and regular movies fine. but when it got to the shows, I get this:

my /data/media/ dir is r/w for all, and it was able to do the movies fine, what exactly does the error 'a red x'? mean. doubt it's permissions when it can do movies fine (just added a new one and it generated that fine.)

This is a fresh plex install/db so I could generate thumbs for it using my tesla p4. (anyone know how many threads max for that I should put to max it out??)
The plex drive is at about 90gb now from generating for the movies folders.

Thanks in advance.

It seems as though my install is working correctly, but finds an issue with the files in the database folder. Terminal output below:

2023-05-29 18:29:32.072 | INFO | main:run:221 - Getting Movies from Plex

2023-05-29 18:29:36.855 | INFO | main:run:223 - Got 3535 Movies from Plex
⠋ Working... 0% -:--:-- 0/3535
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.11/concurrent/futures/process.py", line 256, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ragslinux/Downloads/plex_generate_previews.py", line 196, in process_item
os.mkdir(indexes_path)
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/PlexDB/mnt/plexdatabase/Database/Library/Application Support/Plex Media Server/Media/1/1acb721be80391c103803a043e573d02ea73c03.bundle/Contents/Indexes'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/ragslinux/Downloads/plex_generate_previews.py", line 269, in
run()
File "/home/ragslinux/Downloads/plex_generate_previews.py", line 231, in run
future.result()
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/PlexDB/mnt/plexdatabase/Database/Library/Application Support/Plex Media Server/Media/1/1acb721be80391c103803a043e573d02ea73c03.bundle/Contents/Indexes'

Hi. Nice work on this script.

I'm running this on Windows Server 2019 and seeing " 'NoneType' object is not iterable" at line 214 appear for every bit of media that the built-in Plex Thumbnail Generator hasn't yet procesed. Annoyingly this isn't the line where the error seems to be occurring, just where the script is at when it prints that line to stdout, so knowing where the fault actually lies is a bit elusive. I've had a look at the code, but it isnt immediately obvious what the 'NoneType' object is referring to, because the script is correctly pulling the number of episodes (27k+) from Plex.

'NoneType' would normaly mean that something further up the chain is returning a 'None' object and passing is down the chain whereby your script is expecting a list or dict of some kind, and isn't yet handling this potential failure mode or anticipating receiving a None object.

Any ideas what is delivering a 'None' object to your 'process_item' function?

Hi.

I see that your script works by offloading some stuffs on the GPU, whick I have to guess is Nvidia only at the time being.
I do wonder if the script would, one day, be compatible with VAAPI / QSV for Intel based iGPUs ?

Hi! Thanks for the script. It's amazing. I used the script to generate previews for my library a while back. It however took quite a while owing to my library's size. Since then, I've added some new media to my library. If I run the script again, is it going to regenerate previews for the movies/shows for which I'd already generated the previews before? Or is it only going to process the new movies?

Hello there,
I ended up using your script with a few modifications so just wanted to say thank you but also point out it a few things I noticed.

I was attempting to using the cuda acceleration on my nvidia 3060 and noticed that if I set the GPU processes to say 4, and CPU to 4 I will actually get all 8 using GPU, as the logic in the script will continue to enable hardware transcode if the nvidia-smi utility does not show those processes as active on the card. This had me interested and doing a bit deeper dive but it seems NONE of the processes are actually being performed on the GPU.

Using top i got the full command line ffmpeg that was being spawned and attempted to run it manually and saw the same thing. What i noticed was it was using the standard cpu mjpeg transcoder. Further researched showed that there is currently no support for any nvidia cards to perform mjpeg transcoding on gpu. Additionally even if any part was accelerated, because of the filters used (cuda filters have to be used to perform the resizing,etc on gpu) the frames would need to be copied back and forth (https://ffmpeg.org/ffmpeg-filters.html#toc-scale_005fnpp-1).

I ended up removing the cuda parameter and increased the thread count to “auto” and saw significant speed increases.

35% 0:00:06 198/5612023-09-23 17:07:21.176 | ERROR | mp_main:process_item:207 - list index out of range

The Plex Direct URL works great, but 127.0.0.1 also works if the plex install is running on the same instance as the script. Using localhost loopback is faster, avoids unnecessary DNS traffic, and more reliable.

Thought this should have a better title.

https://www.reddit.com/r/linux/comments/xs8zff/to_my_fellow_linux_nvidia_users_use/

Could this help at all?

I emailed you my DBs and other info @stevezau

Hi there,

Big fan of your work.
I am facing the following issue:

2022-08-16 23:52:29.529 | INFO | main:run:214 - Getting Movies from Plex

2022-08-16 23:52:31.373 | INFO | main:run:216 - Got 913 Movies from Plex
⠼ Working... 0% -:--:-- 0/913
2022-08-16 23:52:39.804 | ERROR | main:process_item:198 - NVML Shared Library Not Found
⠇ Working... 0% -:--:-- 0/913
2022-08-16 23:52:46.598 | ERROR | main:process_item:198 - NVML Shared Library Not Found
⠋ Working... 0% -:--:-- 0/913
2022-08-16 23:52:53.870 | ERROR | main:process_item:198 - NVML Shared Library Not Found
⠧ Working... 0% -:--:-- 0/913

My config:

EDIT These Vars

PLEX_URL = 'https://127.0.0.1:32400/' # If running locally, can also enter IP directly "https://127.0.0.1:32400/"
PLEX_TOKEN = 'PLEXTOKEN'
PLEX_BIF_FRAME_INTERVAL = 5
THUMBNAIL_QUALITY = 4 # Allowed range is 2 - 6 with 2 being highest quality and largest file size and 6 being lowest quality and smallest file size. #
PLEX_LOCAL_MEDIA_PATH = '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Media/localhost'
TMP_FOLDER = '/dev/shm/plex_generate_previews'
GPU_THREADS = 0
CPU_THREADS = 2

It should not use any GPUs, but only CPU, am I missing something ?

I have a large library and am starting to see a timeout error as below:

requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='127.0.0.1', port=32400): Read timed out. (read timeout=30)

Any ideas how to increase the timeout?

docker run -it --rm
--name=plex_generate_vid_previews
--runtime=nvidia
-e NVIDIA_VISIBLE_DEVICES=all
-e PUID=1000
-e PGID=1000
-e PLEX_URL='http://192.168.2.1:32400'
-e PLEX_TOKEN='wX99567567567xYxXMr'
-e PLEX_BIF_FRAME_INTERVAL=2
-e THUMBNAIL_QUALITY=2
-e PLEX_LOCAL_MEDIA_PATH='/Media/localhost/'
-e GPU_THREADS=5
-e CPU_THREADS=5
-v /mnt/user/data/media/tv:/your/media/files
-v /mnt/user/Plex/:/plex/folder
stevezzau/plex_generate_vid_previews:latest

running unraid using hotios plexpass container.

someone on irc says it could be because the script mentions linuxservers container and that uses a different dir structure (which is what you have as default).

/Plex/ here is the root;

and then /Media/localhost/ is where your default location would point to. Am I setting it up correctly or?

Thanks in advance.

When a movie only contains DV profile 5, the creation of previews fails.

Any way to exclude certain libraries? Thanks!

I have them set to exclude from home screen and global search, is that the reason why?

How should I set the following paths:

PLEX_LOCAL_VIDEOS_PATH_MAPPING = '/path/this/script/sees/to/video/library'
PLEX_VIDEOS_PATH_MAPPING = '/path/plex/sees/to/video/library'

Both my libraries (Movies, TV Shows), consist of multiple folders, so I am not sure about how to set these paths in order to run the script. I am running the script on Windows (my Plex server is on the same Windows 11 machine), and all my media is on Synology NAS, connected via SMB.