theporndatabase / namer Goto Github PK
View Code? Open in Web Editor NEWRenames adult video files so that the plex/jellyfin plugins and stash script will match without user input. Runs server or command mode.
Home Page: https://theporndb.net
Renames adult video files so that the plex/jellyfin plugins and stash script will match without user input. Runs server or command mode.
Home Page: https://theporndb.net
After last couple updates, Namer stopped creating .nfo files.
I have a file that got renamed as follows:
French Maid to Hire 5 (2021) - NathanBlake - 2021-07-07 - 2158
This should be rounded off to nearest right resolution
French Maid to Hire 5 (2021) - NathanBlake - 2021-07-07 - 2160p
Pull request 123 added an arch check for videophash.py
On arm64, it expects the arch type arm64, but distributions know arm64 as aarch64.
Running namer directly results in a Unsupport architecture error aarch64
Adding the dict link 'aarch64': 'arm64'
runs, but videohashes-arm64-linux fails to produce output, thus an error is raised
Traceback (most recent call last):
File "/usr/lib/python3.11/threading.py", line 995, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
self.run()
File "/usr/lib/python3.11/threading.py", line 975, in run
self._target(*self._args, **self._kwargs)
> File "/home/andy/.local/lib/python3.11/site-packages/namer/watchdog.py", line 137, in __processing_thread
handle(command)
File "/home/andy/.local/lib/python3.11/site-packages/namer/watchdog.py", line 54, in handle
process_file(command)
File "/home/andy/.local/lib/python3.11/site-packages/namer/namer.py", line 176, in process_file
phash = VideoPerceptualHash().get_stash_phash(command.target_movie_file) if command.config.search_phash else None
File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 91, in get_stash_phash
return self._get_stash_phash(file, stat.st_size, stat.st_mtime)
File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 96, in _get_stash_phash
return self.__execute_stash_phash(file)
File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 134, in __execute_stash_phash
data = json.loads(stdout, object_hook=lambda d: SimpleNamespace(**d))
File "/usr/lib/python3.11/json/__init__.py", line 359, in loads
return cls(**kw).decode(s)
File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Running videohashes-arm64-linux manually with a link to a video file generates the error
ffmpeg/ffprobe executables not found
Scanning through the compiled code for videohashes-arm64-linux shows several matches for arm64, but none for aarch64.
I assume it's looking for an arm64 ffmpeg, and cannot find an aarch64 ffmpeg.
Not sure if this is an upstream error from https://github.com/peolic/videohashes or where the videohashes executable comes from
The latest version to be installed via pip on python 3.11 is 1.5.17
All later version require python < 3.11
ERROR: Ignored the following versions that require a different python version: 1.6.0 Requires-Python >=3.8,<3.11; 1.6.1 Requires-Python >=3.8,<3.11; 1.6.2 Requires-Python >=3.8,<3.11; 1.6.3 Requires-Python >=3.8,<3.11; 1.6.4 Requires-Python >=3.8,<3.11; 1.6.5 Requires-Python >=3.8,<3.11; 1.6.6 Requires-Python >=3.8,<3.11; 1.6.7 Requires-Python >=3.8,<3.11; 1.6.8 Requires-Python >=3.8,<3.11; 1.6.9 Requires-Python >=3.8,<3.11; 1.7.0 Requires-Python >=3.8,<3.11; 1.7.1 Requires-Python >=3.8,<3.11; 1.7.10 Requires-Python >=3.8,<3.11; 1.7.11 Requires-Python >=3.8,<3.11; 1.7.12 Requires-Python >=3.8,<3.11; 1.7.13 Requires-Python >=3.8,<3.11; 1.7.14 Requires-Python >=3.8,<3.11; 1.7.15 Requires-Python >=3.8,<3.11; 1.7.2 Requires-Python >=3.8,<3.11; 1.7.3 Requires-Python >=3.8,<3.11; 1.7.4 Requires-Python >=3.8,<3.11; 1.7.5 Requires-Python >=3.8,<3.11; 1.7.8 Requires-Python >=3.8,<3.11; 1.7.9 Requires-Python >=3.8,<3.11; 1.8.0 Requires-Python >=3.8,<3.11; 1.8.1 Requires-Python >=3.8,<3.11; 1.8.2 Requires-Python >=3.8,<3.11; 1.8.3 Requires-Python >=3.8,<3.11; 1.8.4 Requires-Python >=3.8,<3.11; 1.8.5 Requires-Python >=3.8,<3.11; 1.8.6 Requires-Python >=3.8,<3.11; 1.9.0 Requires-Python >=3.8,<3.11; 1.9.1 Requires-Python >=3.8,<3.11; 1.9.2 Requires-Python >=3.8,<3.11; 1.9.3 Requires-Python >=3.8,<3.11; 1.9.4 Requires-Python >=3.8,<3.11; 1.9.5 Requires-Python >=3.8,<3.11; 1.9.6 Requires-Python >=3.8,<3.11; 1.9.7 Requires-Python >=3.8,<3.11
ERROR: Could not find a version that satisfies the requirement namer==1.9.7 (from versions: 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.19, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.9, 1.5.10, 1.5.11, 1.5.12, 1.5.13, 1.5.14, 1.5.15, 1.5.16, 1.5.17)
ERROR: No matching distribution found for namer==1.9.7
I think only pyproject.toml needs to be updated
Enhancement to namer so that a flag is set in config file and all the queries go to TPDB movies vs trying to also search Scenes
A flag to set automated vs manual tagging, if set as manual, scenes/movie will required to be approved in the Web UI before processing the scene/movie.
The image paths are static instead of relative
/media/dest/BrownBunnies/BrownBunnies - 2023-01-05 - Fucking in the Rain-background.png
Random performer image is used as Poster instead of the actual poster image
/media/dest/BrownBunnies/BrownBunnies - 2023-01-05 - Fucking in the RainSarai-Minx-image.png
No break or separator between original filename and performer name in performer images
/media/dest/BrownBunnies/BrownBunnies - 2023-01-05 - Fucking in the RainJonathan-Jordan-image.png
theporndbid displays page path instead of id. Should be split into theporndbid with id only, and theporndburl with full url instead
scenes/2620378
Since Plex ignores 'Movie' items that have TV Show style filenames, it will skip over scenes that have S##:E## style titles.
Need an option to ignore that part of a scene title if it's in the metadata (Such as Nubiles)
Something along the lines of removal of r'[sS]\d{1,3}:?[eE]\d{1,3}'
My failed folder is flooded with .gz files from failed matches.
After a successful match should the .gz file linked not be deleted as well?
Looking to possible have Namer store the phash that is generated in the file itself and/or in the NFO that it generates.
Thanks!
I have been playing with namer, watchdog, and every other fun solution out there over the last few years. This thing is amazing!!! There is one thing I would love to see, however.
I manage my own posters and backgrounds. In the scenes folder, there will be 3 files -- poster.jpg, background.jpg, and the video file itself. I would love to be able to put those folders through namer and have those pictures travel along with the file.
Again, thanks so much!!!
Hi there
I have the following in my docker compose
namer: container_name: namer image: ghcr.io/theporndatabase/namer:latest environment: - PUID=${PGID} - PGID=${PUID} - TZ=${TZ} - NAMER_CONFIG=/namer.cfg volumes: - ${DOCKERCONFDIR}/namer:/config - ${DOCKERSTORAGEDIR}:/data restart: always
and I have edited the namer.cfg with the correct directories, but when i run compose it is as though it is not reading the config file. and actually, the namer directory gets created, but it is empty except for the namer.cfg.
It is possibly installing somewhere I just don't see?
thanks
app
My current folder structure goes as:
Mofos/Bangbus/SceneFolder/scene.mp4
With Namer I can only do:
Bangbus/Scenefolder/scene.mp4
Parent information is stored on TPDB so would it be possible to add a {network}, {network_full} option for the file renaming? As of now I can only go to site level then have to manually move it into it's parent folder. Would be awesome if it could do it by it's self, thanks!
I think this cloud be handled via a list, I am envisioning it like
Keyword = BTS, Bonus, Extras
and the {keyword} should be used if its found in the filename string
I've noticed that i have a number of scenes with incorrect dates. However, the site and scene name is correct, and easy to search in the web UI via the Scene search box. Could namer possibly try site+scene name if the initial date lookup doesn't work? Not sure how the query API works.
My desired usecase is using watchdog to automatically see which files the Plex Agent would otherwise fail to match on -- configuring watchdog to rename files ONLY if they required a manual match in the watchdog webui.
Two different cases:
sitename.22.03.05.scenename.blahblah.1080p.mp4
to watch_dir gets automatically matched by watchdog successfully -> move it to dest_dir without changing anything. (This being my new requested functionality)not.in.correct.naming.format.etc.etc.mp4
doesn't get matched by watchdog -> I manually match it in webui -> it gets renamed using the format specified in new_relative_path_name
. (This being the current/normal functionality)Why?
Thanks for the great work otherwise, I found this project earlier today and love the effort put into it :)
Over the past week, I have noticed that posters retrieved via namer no longer contain an overlay of the site logo/name. Previously, namer would retrieve a poster that was in portrait layout with logo overlaid. Now, it just retrieves the same poster as the background which is in landscape. Has something changed with the PornDB site?
If Namer tries to place a matched scene in a DEST folder that it doesn't have permissions to access then the file will disappear. I looked in all my namer work folders and in the surrounding folders and couldn't find it
This seems like a pretty unique bug as most people are not as dumb as I and have fucked up perms.
Good afternoon,
I am having issues with automatic matching. I am having a 100% fail in matching, and having to manually match each file, so it is likely something on my end not configured correctly. Here are some of the details for my setup and config file:
Setup:
Here is the config file variables as they currently stand:
[namer]
porndb_token = [the key I generated with my account I have used here]
name_parser = {_site}{_sep}{_optional_date}{_ts}{_name}{_dot}{_ext}
inplace_name={site} - {date} - {name}.{ext}
prefer_dir_name_if_available = True
min_file_size = 300
write_namer_log = False
write_namer_failed_log = True
target_extensions = mp4,mkv,avi,mov,flv
update_permissions_ownership = True
set_dir_permissions = 775
set_file_permissions = 664
set_uid =
set_gid =
trailer_location =
sites_with_no_date_info =
movie_data_preferred = missax
vr_studios = 18 VR,Babe VR,Badoink VR,Dorm Room,Kink VR,Real VR,RealJamVR,Sex Like Real,SexBabesVR,SinsVR,SLR Originals,Swallowbay,Virtual Taboo,VirtualRealPorn,VR Bangers,VR Cosplay X,VR Hush,VRConk,VRedging,Wankz VR
vr_tags = virtual reality, vr porn
site_abbreviations = {"bex" : "BrazzersExxtra", "ps": "PropertySex"}
max_performer_names = 6
use_requests_cache = True
requests_cache_expire_minutes = 10
override_tpdb_address =
[Phash]
search_phash = True
require_match_phash_top = 3
send_phash_of_matches_to_tpdb = False
[metadata]
write_nfo = False
enabled_tagging = True
enabled_poster = True
enable_metadataapi_genres = False
default_genre = Adult
language = eng
[duplicates]
preserve_duplicates = True
max_desired_resolutions = -1
desired_codec = hevc, h264
[watchdog]
ignored_dir_regex = .UNPACK.
del_other_files = False
extra_sleep_time = 30
new_relative_path_name={site} - {date} - {name}/{site} - {date} - {name}.{ext}
watch_dir = H:\Namer Folders\New Files
work_dir = H:\Namer Folders\Processing
failed_dir = H:\Namer Folders\Failed
dest_dir = H:\Namer Folders\Completed
retry_time =
web = True
port = 6980
host = 0.0.0.0
web_root =
allow_delete_files = False
add_max_percent_column = True
debug = True
diagnose_errors = True
Thanks in advance for your insights on getting this up and running without doing a manual match for each file!
Would be very handy to have Namer perform a ffmpeg file validation and toss any failing items into something like a "/badmedia" folder for later re-download.
FFMPEG can do a quick recod to null to check for errors, a sample command line is 'ffmpeg -v error -i file.avi -f null - 2>error.log'. For more explanation please refer to https://superuser.com/questions/100288/how-can-i-check-the-integrity-of-a-video-file-avi-mpeg-mp4 (as an example)
Afternoon, All,
Apologies if this one is blatently simple, I'm trying to get to grips with Python.
Just playing with the configuration and am getting a curious outcome:
_File "C:\Users[USER PATH]\AppData\Roaming\Python\Python310\site-packages\namer\configuration.py", line 412, in to_dict
"watch_dir": str(self.watch_dir),
AttributeError: 'NamerConfig' object has no attribute 'watch_dir'
Yet when I go to line 271 (the object it refers to) there is indeed a directory. I've tried as many formats as I can but the error remains the same. I've tried the following:
watch_dir: P:\Test
watch_dir: "P:\Test"
watch_dir: r"P:\Test"
watch_dir: rf"P:\Test"
Am I missing something insanely obvious?
Thanks guys.
Ether a setting in the config for site abbreviations i.e. (abbreviations=["abr":"A Bare Rump", ....] )
or as a separate file for them
As discussed in discord https://discord.com/channels/957516730727563285/957516731654488091/1015037937206956155
When making changes via namer.cfg, under the option "enabled_poster", we get an all-or-nothing situation. When set to true, in addition to download the poster image, namer also downloads a background inage and all performers for said scene / movie.
If possible, it would be great to add an option on namer.cfg so we can choose which images we want (example, only poster + background, and nothing else)
When using .nfo files, both Plex and Jellyfin are very good at getting the performers' images without having namer downloading them. This, plus the added feature would help on keeping the folders clean and organized.
Thanks in advance, good day.
It moves the file from watch dir to work dir and then starts calculating phash and throws this:
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/dist-packages/namer/main.py", line 87, in
main()
File "/usr/local/lib/python3.10/dist-packages/namer/main.py", line 74, in main
namer.watchdog.create_watcher(config).run()
File "/usr/local/lib/python3.10/dist-packages/namer/watchdog.py", line 177, in run
self.start()
File "/usr/local/lib/python3.10/dist-packages/namer/watchdog.py", line 242, in start
for file in self.__namer_config.watch_dir.rglob("**/."):
File "/usr/lib/python3.10/pathlib.py", line 1047, in rglob
for p in selector.select_from(self):
File "/usr/lib/python3.10/pathlib.py", line 493, in _select_from
for p in successor_select(starting_point, is_dir, exists, scandir):
File "/usr/lib/python3.10/pathlib.py", line 492, in _select_from
for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
File "/usr/lib/python3.10/pathlib.py", line 482, in _iterate_directories
for p in self._iterate_directories(path, is_dir, scandir):
File "/usr/lib/python3.10/pathlib.py", line 471, in _iterate_directories
with scandir(parent_path) as scandir_it:
FileNotFoundError: [Errno 2] No such file or directory: 'XXX'
Weirdly enough, XXX is the scene folder inside the "watch" dir. And as it has already moved the file to work dir, it won't be there.
Installed today on Ubuntu using pip command. Files are mp4.
I'm trying to let some movies auto tag and fail, example https://metadataapi.net/movies/dorcel-betrayal
Filename is Dorcel.2021-04-14.Betrayal.mp4 in line with the metadata.
Namer always checked the scene's url only, never switching to movie when checking with the verbose flag.
Changes to namer.cfg to the movie_data_prefered, adding dorcel, still starts namer with
INFO Querying: https://api.metadataapi.net/scenes?parse=dorcel.2021-04-14.Betrayal&limit=25
I checked the metadataapi code and tried to follow the movie boolean.
I understood __metadata_api_lookup is started with scene reference due to the Movies=False, switching to Movies afterwards with the not movies.
def __metadata_api_lookup(name_parts: FileNameParts, namer_config: NamerConfig) -> List[ComparisonResult]:
movies = False
if name_parts.site:
if name_parts.site.strip().upper() in namer_config.movie_data_prefered:
movies = True
results = []
results = __metadata_api_lookup_type(results, name_parts, namer_config, movies)
if not results or not results[0].is_match():
results = __metadata_api_lookup_type(results, name_parts, namer_config, not movies)
return results
The challenge here is now that Dorcel Betrayal is also available as split scenes https://metadataapi.net/scenes/dorcel-betrayal-english-scene-1
Thus, metadataapi will never trigger the movie check, as the scene check returns a result already.
I don't know how to solve the issue directly, maybe adding an option into namer to start with Movie or Scene first.
Alternatively, changing the filename to include Movie so it triggers the function.
I also tried to rename the file to the id, but the scraping fails.
Don't know if anyone else would use it, but it would be nice to be able to provide a list of sites that are not currently indexed to auto-move to a "/ManualProcess" folder so that they aren't retried at night and time isn't spent hashing them
I have
ignored_dir_regex = .*_unpack.*
for the regex match but
watchdog process called for RoccoSiffredi.22.10.13.Tiffany.Tatum.XXX.2160p.MP4-WRB/_unpack/RoccoSiffredi.22.10.13.Tiffany.Tatum.XXX.2160p.MP4-WRB.mp4
isn't being matched by the regex and it keeps trying to process the currently unpacking file.
Thoughts?
In namer.cfg
, in the Phash section, comments say that send_phash = True
is ignored if search_phash = False
.
[Phash]
# Calculate and use phashes in search for matches
search_phash = False
# Should phashes be sent to tpdb if no phash match was found but a name match was.
# Requires search_phash be true or is ignored.
send_phash = True
However, in L204, search_phash
is not checked if new_metadata
is found, probably through manually matching a failed file.
send_phash
and search_phash
to be true
2022-10-04T16:48:21.432824+0000 ERROR An error has been caught in function 'on_any_event', process 'MainProcess' (3266684), thread 'Thread-1' (140080837351168):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/opt/namer.vrnzb/namer/__main__.py", line 62, in <module>
main(sys.argv[1:])
File "/opt/namer.vrnzb/namer/__main__.py", line 52, in main
namer.watchdog.create_watcher(default_config()).run()
File "/opt/namer.vrnzb/namer/watchdog.py", line 161, in run
self.start()
> File "/opt/namer.vrnzb/namer/watchdog.py", line 231, in start
self.__event_handler.prepare_file_for_processing(file)
File "/opt/namer.vrnzb/namer/watchdog.py", line 118, in prepare_file_for_processing
working_command = move_command_files(command, self.__namer_config.work_dir)
File "/opt/namer.vrnzb/namer/command.py", line 70, in move_command_files
shutil.move(target.target_directory, working_dir)
File "/usr/lib/python3.8/shutil.py", line 787, in move
real_dst = os.path.join(dst, _basename(src))
File "/usr/lib/python3.8/shutil.py", line 750, in _basename
return os.path.basename(path.rstrip(sep))
AttributeError: 'PosixPath' object has no attribute 'rstrip'```
Namer stops processing files with this error.
I have been using namer to put processed scenes in this format
new_relative_path_name={site}/{site} - {date} - {name} [tpdb-{uuid}]/{site} - {date} - {name}.{ext}
However I have only just noticed that it is no longer working and saying {uuid} is not a valid key. Have you dropped support for this? I see nothing is change logs that that is the case
Hi,
I'm trying to have sabnzbd write downloads to a directory that namer watches. Part of the process is sabnzbd uses a temp UNPACK directory, with the dir name. namer sees that directory, waits 30 seconds, and then copies it to the work dir.
This is before sabnzbd is done unpacking, as you can see from this sabnzbd error:
Error renaming "/namer_watch/_UNPACK_PervMom.22.02.27.Jenna.Noelle.Taste.Of.Jenna.XXX.MP4-P0RNL0V3RSD" to "/namer_watch/PervMom.22.02.27.Jenna.Noelle.Taste.Of.Jenna.XXX.MP4-P0RNL0V3RSD"
Can namer somehow be configured to ignore UNPACK directories and only process when they've been renamed?
Thanks!
Hello,
I recently observed an issue when I would run namer. I'll submit new video files into my watch
directory and observe the application fail to make a match to the database. I've confirmed that a match should indeed be made, as the hash values equal those in the DB. Enabling debug, I see that it accurately identified the correct Studio, Date, Performers, and Title (| DEBUG | Phash match with ...(Correct Information)...
), but doesn't rename the file accordingly. It then moves them into the failed
directory. I've used namer successfully and this issue has only started to happen recently. I haven't edited my config
file (besides enabling debugging) in a way that would cause this to happen. I do also see this entry in the debug view: | DEBUG | Could not parse target name which may be a file (or directory) name depending on settings and input: ...
. I'm not sure if this is a significant error or not. The file name is not conducive to being accurately parsed, but I usually aim for a match via hash values.
I then tested namer on a file which was successfully processed a month ago, and it failed to identify it. Similar behavior as stated above. Where it found the right DB entry, but didn't rename it. I'm not sure what the issue might be for this behavior. Are there any other logs which I can view or share that might help pinpoint the issue?
Edit: If I rename the file with the name given when namer
finds a match based on Phash (line in the debug window) and move it back into my watch
directory, the program will correctly name it and move into the dest
folder. In this manner, I need to run the program twice. One time to have fail but give the name. A second time once I manually edit the name for it to pull the movie details and succeed.
I'm using the standard watchdog command: python -m namer watchdog
.config file (token removed): namer.txt
Snippet from the terminal with the pertinent parts:
namer | 2023-03-24 20:43:22 | ℹ️ INFO | Calculating phash for file "/data/work/movie.mp4"
namer | 2023-03-24 20:43:30 | ℹ️ INFO | Calculated hashes: {'duration': NUMBER, 'phash': 'NUMBER', 'oshash': 'NUMBER'}
namer | 2023-03-24 20:43:30 | ℹ️ INFO | Requesting GET "https://api.metadataapi.net/scenes?hash=NUMBER&hashType=PHASH"
namer | 2023-03-24 20:43:33 | ℹ️ INFO | Requesting GET "https://api.metadataapi.net/sites/number"
namer | 2023-03-24 20:43:33 | ℹ️ INFO | Requesting GET "https://api.metadataapi.net/scenes"
namer | 2023-03-24 20:43:33 | 🐞 DEBUG | Phash match with 'SITE- DATE- TITLE'
namer | 2023-03-24 20:43:33 | 🐞 DEBUG | Match was 100.00 for
namer | 2023-03-24 20:43:33 | 🐞 DEBUG | Match was 0.00 for
I've checked that the hash values are all legitimate and can be found in the DB. The snippet makes it seem as though the hash had a 100% match to an entry in the DB. Do I understand the scoring system, or am I making a misunderstanding? It seems when a name match is made, the 'score' value can exceed 100. Is the video hash bound to 0-100?
I have namer running in a Docker container on a synology NAS. There's files in the watch_dir
, and files in the work_dir
but namer is just sitting there not doing anything. It was processing files before that. I restarted the container a couple of times, but that does not seem to help.
Here are the logs, along with a dump of the config:
2023-05-11 19:30:29 | ℹ️ INFO | Namer Config:
porndb_token: ****************************************
inplace_name: {full_site} - {date} - {name} [WEBDL-{resolution}].{ext}
prefer_dir_name_if_available: True
target_extensions: ['mp4', 'mkv', 'avi', 'mov', 'flv']
write_namer_log: False
write_namer_failed_log: True
trailer_location:
sites_with_no_date_info: []
movie_data_preferred: ['missax']
vr_studios: [redacted for brevity]
vr_tags: ['virtual reality', 'vr porn']
site_abbreviations: {redacted for brevity}
update_permissions_ownership: False
set_dir_permissions: 775
set_file_permissions: 664
set_uid: 1026
set_gid: 100
max_performer_names: 6
enabled_requests_cache: True
requests_cache_expire_minutes: 10
override_tpdb_address: https://api.metadataapi.net
plex_hack: False
Phash:
search_phash: True
send_phash: True
use_alt_phash_tool: False
max_ffmpeg_workers: None
use_gpu: True
Duplicate Config:
preserve_duplicates: False
max_desired_resolutions: -1
desired_codec: ['hevc', 'h264']
Tagging Config:
write_nfo: True
enabled_tagging: True
enabled_poster: True
download_type: ['poster', 'background', 'performer']
enable_metadataapi_genres: True
default_genre: Adult
language: eng
mark_collected: True
Watchdog Config:
ignored_dir_regex: .*_UNPACK_.*
min_file_size: 100
del_other_files: False
new_relative_path_name: {full_site}/{full_site} - {date} - {name} [WEBDL-{resolution}].{ext}
watch_dir: /config/test
work_dir: /config/work_dir
failed_dir: /media/unmatched
dest_dir: /media/matched
retry_time:
extra_sleep_time: 30
web: True
port: 6980
host: 0.0.0.0
web_root:
allow_delete_files: True
add_max_percent_column: True
debug: False
manual_mode: False
diagnose_errors: False
2023-05-11 19:30:29 | ℹ️ INFO | ffmpeg version "4.4.2-0ubuntu0.22.04.1" found
2023-05-11 19:30:29 | ℹ️ INFO | ffprobe version "4.4.2-0ubuntu0.22.04.1" found
2023-05-11 19:30:29 | ℹ️ INFO | Start porndb scene watcher.... watching: /config/test
Namer version: v1.13.2
Built on: 2023-02-23T16:16:54Z
Git Hash: 57b1234a5e3788c7aa3c67c01dd98df9677061eb
2023-05-11 19:30:29 | ℹ️ INFO | Starting server: http://0.0.0.0:6980/
2023-05-11 19:32:17 | ℹ️ INFO | Scanning dir /media/unmatched for sub-dirs/files to process
2023-05-11 19:51:47 | ℹ️ INFO | Scanning dir /media/unmatched for sub-dirs/files to process
2023-05-11 20:03:45 | ℹ️ INFO | Scanning dir /media/unmatched for sub-dirs/files to process
I added several files to the watch folder of Namer. The files failed to find a match automatically so they went to the failed folder. I went into the web GUI and manually matched 4 files. The 4 files were moved to the working folder and got stuck for around 5 minutes. After 5 minutes 3 of the 4 files were moved to the destination folder. One file remained in the working folder and got stuck for another 10 minutes until I was forced to close Namer. While this was going on my ram usage continued to steadily rise and my HDD was maxed out at 100%. When I closed Namer it was using approximately 16GB of ram and nearly had my computers 32GB Of ram maxed out. The second I closed our Namer, my ram utilization went from 28.5GB to 12.7GB. Here is a copy of the command prompt prior to me closing Namer.
message.txt
5262/1108589413899972689/image.png)
Start by figuring out the command line tools to edit a video:
https://mkvtoolnix.download/doc/mkvpropedit.html
You should test adding the video to a plex library to see if metadata is read - not sure if plex/jellyfin/emby support this or not.
If they don't, not sure you should be doing this work, but carry on if you want.
Once you have the commands - maybe post them here if as a comment - incase you decide to give up.
Read some code, look at two files in namer:
1: https://github.com/ThePornDatabase/namer/blob/main/namer/ffmpeg.py
To see how to properly call external commands.
2: https://github.com/ThePornDatabase/namer/blob/main/namer/mutagen.py
To see what needs to be built for mkv's. The mutagen file supports mp4s.
Write your code with a test similar to mutagen:
https://github.com/ThePornDatabase/namer/blob/main/test/namer_mutagen.py
Update namer.py to call your new file for mkvs (around here: https://github.com/ThePornDatabase/namer/blob/main/namer/namer.py#L121)
Profit.
Further to earlier post re files not getting processed it appears all the files that were picked-up from watch_dir processed and moved and moved to failed_dir or dest_dir have somehow got corrupted.
The filenames in failed_dir are the same as original, the files open in VLC but can't be played. Upon scanning with ffmpeg on Windows, I get 100's of errors as below. The original file does not get errors, so somewhere namer has done something to the files. There are many files that seem to be corrupted. Any idea how to fix them?
missing picture in access unit with size 68
Invalid NAL unit size (0 > 64).
This is the simplest ui we could possible have. It would show what files are currently unmatched.
From there we could build many more features (retry options, select from one of the current items in the namer_log, etc).
Should be done in python.
I'd like to have the option to toggle this. For large libraries these can add up.
whenever i run watchdog i get this error even after changing api keys updating, uninstalling and reinstalling the namer:
2023-08-09 20:11:23 | ❌ ERROR | An error has been caught in function '__processing_thread', process 'MainProcess' (11116), thread 'Thread-2 (__processing_thread)' (23204):
Traceback (most recent call last):
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
self.run()
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\watchdog.py", line 146, in __processing_thread
handle(command)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\watchdog.py", line 54, in handle
process_file(command)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\namer.py", line 179, in process_file
search_results = match(command.parsed_file, command.config, phash=phash)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 505, in match
results: List[ComparisonResult] = __metadata_api_lookup(file_name_parts, namer_config, phash)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 179, in __metadata_api_lookup
results: List[ComparisonResult] = __metadata_api_lookup_type(results, name_parts, namer_config, scene_type, phash)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 158, in __metadata_api_lookup_type
results = __update_results(results, name_parts, namer_config, scene_type=scene_type, phash=phash)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 147, in __update_results
for match_attempt in __get_metadataapi_net_fileinfo(name_parts, namer_config, skip_date, skip_name, scene_type=scene_type):
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 468, in __get_metadataapi_net_fileinfo
file_infos = __get_metadataapi_net_info(url, name_parts, namer_config)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 455, in __get_metadataapi_net_info
file_infos = __metadataapi_response_to_data(json_obj, url, formatted, name_parts, namer_config)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 401, in __metadataapi_response_to_data
found_file_info = __json_to_fileinfo(data, url, json_response, name_parts, config)
File "C:\Users\kings\AppData\Local\Programs\Python\Python310\lib\site-packages\namer\metadataapi.py", line 331, in __json_to_fileinfo
file_info.site = data.site.name
AttributeError: 'types.SimpleNamespace' object has no attribute 'site'
Hi,
I've been using the latest docker container, to process a big backlog of unmatched scenes. Generally, things work great. However, sometimes I notice from the docker logs, that namer appears stuck, with the last log line being a query to TPDB.
Is there a timeout on this query? I resolve this by restarting the container.
Thanks.
Please forgive my novice question. How can I properly configure namer to utilize the webUI aspect? I'm new to web interfaced Python. I've set the webUI flag in the config to True and have played around with IP and Port values, but nothing works.
I either receive a refused to connect
error in the browser, or the following (if change the target host IP):
2023-04-08 09:07:19 namer | File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
2023-04-08 09:07:19 namer | return _run_code(code, main_globals, None,
2023-04-08 09:07:19 namer | File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
2023-04-08 09:07:19 namer | exec(code, run_globals)
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/namer/__main__.py", line 69, in <module>
2023-04-08 09:07:19 namer | main(sys.argv[1:])
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/namer/__main__.py", line 59, in main
2023-04-08 09:07:19 namer | namer.watchdog.create_watcher(config).run()
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/namer/watchdog.py", line 176, in run
2023-04-08 09:07:19 namer | self.__webserver = NamerWebServer(self.__namer_config, self.__command_queue)
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/namer/web/server.py", line 166, in __init__
2023-04-08 09:07:19 namer | super().__init__(self.__namer_config.host, self.__namer_config.port, webroot, blueprints)
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/namer/web/server.py", line 55, in __init__
2023-04-08 09:07:19 namer | self.__make_server()
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/namer/web/server.py", line 60, in __make_server
2023-04-08 09:07:19 namer | self.__server = create_server(self.__app, host=self.__host, port=self.__port, clear_untrusted_proxy_headers=True)
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/waitress/server.py", line 78, in create_server
2023-04-08 09:07:19 namer | last_serv = TcpWSGIServer(
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/waitress/server.py", line 244, in __init__
2023-04-08 09:07:19 namer | self.bind_server_socket()
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/waitress/server.py", line 361, in bind_server_socket
2023-04-08 09:07:19 namer | self.bind(sockaddr)
2023-04-08 09:07:19 namer | File "/usr/local/lib/python3.10/dist-packages/waitress/wasyncore.py", line 396, in bind
2023-04-08 09:07:19 namer | return self.socket.bind(addr)
2023-04-08 09:07:19 namer | OSError: [Errno 99] Cannot assign requested address
1969-12-31 17:00:00 namer exited with code 1
I've tried the default IP address as well as others (local host, etc). I haven't altered the web_root
entry as I'm not sure about its behavior.
I'm running namer is a Docker container.
The majority of files placed in my watch directory are processed fine but I also am seeing about 10% of files completely ignored. They are not moving to failed. These all have mp4 extensions. I have tried renaming the files to the same name as a working file and nothing. Restarted and reinstalled namer with no success. Any suggestions? These do not even move to failed.
Hello,
I've set up namer on docker, using the docker_compose. I'm passing the 4 folders as per the compose file below. Namer moved the files from 'watch' to 'work', but then seems to have stopped. I've added the porndb_token, added uid / gid as per the compose file.
I have also enabled the web-server (web = True), but not sure how to access this to view if working.
Any ideas how to move forward with this?
DOCKER COMPOSE:
version: "3"
services:
namer:
container_name: namer
image: ghcr.io/theporndatabase/namer:latest
environment:
- PUID=1001
- PGID=1000
- TZ=Europe/London
- NAMER_CONFIG=/config/namer.cfg
volumes:
#- /apps/namer/:/config <- this will store the namer.cfg file copied by you from the git repo ( namer/namer.cfg.default )
- /srv/dev-disk-by-uuid-xxxxxxxxxxxx/Config/namer:/config
- /media:/data #<- this will have the four folders namer needs to work, referenced in the namer.cfg file you create.
- /srv/dev-disk-by-uuid-xxxxxxxxxxxx/downloads:/data/watch_dir
- /srv/dev-disk-by-uuid-xxxxxxxxxxxx/_work:/data/work_dir
- /srv/dev-disk-by-uuid-xxxxxxxxxxxx/_failed:/data/failed_dir
- /srv/dev-disk-by-uuid-xxxxxxxxxxxx/z_:/data/dest_dir
restart: always
The config is thus:
new_relative_path_name={full_site}/{full_site} - {date} - {name} [WEBDL-{resolution}].{ext}
# directory where new downloads go.
watch_dir = data/watch_dir
# temporary directory where work is done.
# a log file shows attempted matches and match closeness.
work_dir = data/work_dir
# Should processing fail the file or directory is moved here.
# retries occur every 12 hours. Files can be manually moved to watch-dir
# to force reprocessing.
failed_dir = data/failed_dir
# dir where finalized files get written.
dest_dir = data/dest_dir
Can we have namer ether not yolo all files into the work dir, and only move a few at a time (idk only have 10 files in there at any one time), or keep a log of what has been moved into the work folder so that in case of a restart it knows "Oh I still have this to do"
Otherwise I have to keep moving the unprocessed files back into the watch folder every time the docker container freaks out and restarts
Is there a way to set the frequency of failed job retries to something like weekly or monthly instead of daily?
Looking to have the possibility of Namer:
TPDB can do the management of what has been collected but namer adding it in would be cool.
Thanks
It is giving this error when installing the namer in Windows 10, do you know how to solve it? thanks for the work
C:\Windows\system32>pip install namer
Collecting namer
Using cached namer-1.13.2-py3-none-any.whl (38.0 MB)
Collecting ConfigUpdater<4.0.0,>=3.1.1
Using cached ConfigUpdater-3.1.1-py2.py3-none-any.whl (34 kB)
Collecting Flask-Compress<2.0,>=1.12
Using cached Flask_Compress-1.13-py3-none-any.whl (7.9 kB)
Collecting Pillow<10.0.0,>=9.1.1
Using cached Pillow-9.4.0-cp311-cp311-win_amd64.whl (2.5 MB)
Collecting Unidecode<2.0.0,>=1.3.4
Using cached Unidecode-1.3.6-py3-none-any.whl (235 kB)
Collecting ffmpeg-python<0.3.0,>=0.2.0
Using cached ffmpeg_python-0.2.0-py3-none-any.whl (25 kB)
Collecting flask<3.0.0,>=2.2.0
Using cached Flask-2.2.3-py3-none-any.whl (101 kB)
Collecting imagehash<5.0.0,>=4.3.1
Using cached ImageHash-4.3.1-py2.py3-none-any.whl (296 kB)
Collecting jinja2<4.0.0,>=3.1.2
Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting jsonpickle<4.0.0,>=3.0.1
Using cached jsonpickle-3.0.1-py2.py3-none-any.whl (40 kB)
Collecting loguru<0.7.0,>=0.6.0
Using cached loguru-0.6.0-py3-none-any.whl (58 kB)
Collecting mutagen<2.0.0,>=1.45.1
Using cached mutagen-1.46.0-py3-none-any.whl (193 kB)
Collecting oshash<0.2.0,>=0.1.1
Using cached oshash-0.1.1.tar.gz (3.4 kB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [19 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 14, in
File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools_init_.py", line 268, in
monkey.patch_all()
File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools\monkey.py", line 97, in patch_all
patch_for_msvc_specialized_compiler()
File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools\monkey.py", line 157, in patch_for_msvc_specialized_compiler
patch_func(*msvc14('get_vc_env'))
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools\monkey.py", line 147, in patch_params
mod = import_module(mod_name)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools_distutils_msvccompiler.py", line 20, in
import unittest.mock as mock
ValueError: source code string cannot contain null bytes
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
First, I would like to state that I know very little about coding. I have been searching everything to find an answer to this. Everything was working fine until this morning when it wasn't. No matter what file I toss into my 'watch' folder, this same error keeps coming up. I made sure to update python and I also updated namer to the latest, and updated the config file to the latest. Still get this error:
2023-07-29 08:54:05 | ℹ️ INFO | ffmpeg version "6.0-essentials_build-www.gyan.dev" found
2023-07-29 08:54:05 | ℹ️ INFO | ffprobe version "6.0-essentials_build-www.gyan.dev" found
2023-07-29 08:54:05 | ℹ️ INFO | Start porndb scene watcher.... watching: I:\Namer\watch
2023-07-29 08:54:05 | ℹ️ INFO | Starting server: http://0.0.0.0:6980/
2023-07-29 08:54:24 | ℹ️ INFO | watchdog process called for BrattySis.23.07.07.Dani.Diaz.Stepsis.Stop.Staring.At.My.Dick..mkv
2023-07-29 08:54:24 | ℹ️ INFO | Moving I:\Namer\watch\BrattySis.23.07.07.Dani.Diaz.Stepsis.Stop.Staring.At.My.Dick..mkv to I:\Namer\work\BrattySis.23.07.07.Dani.Diaz.Stepsis.Stop.Staring.At.My.Dick..mkv for processing
2023-07-29 08:54:24 | ℹ️ INFO | Processing: I:\Namer\work\BrattySis.23.07.07.Dani.Diaz.Stepsis.Stop.Staring.At.My.Dick..mkv
2023-07-29 08:54:24 | ℹ️ INFO | Calculating phash for file "I:\Namer\work\BrattySis.23.07.07.Dani.Diaz.Stepsis.Stop.Staring.At.My.Dick..mkv"
2023-07-29 08:54:32 | ℹ️ INFO | Calculated hashes: {'duration': 2179, 'phash': 'd82a78de3e8bc780', 'oshash': '2bada661f9491b62'}
2023-07-29 08:54:32 | ℹ️ INFO | Requesting GET "https://api.metadataapi.net/scenes?hash=d82a78de3e8bc780&hashType=PHASH"
2023-07-29 08:54:32 | ℹ️ INFO | Requesting GET "https://api.metadataapi.net/scenes?parse=brattysis.2023-07-07.Dani%20Diaz%20Stepsis%20Stop%20Staring%20At%20My%20Dick&limit=25"
2023-07-29 08:54:32 | ❌ ERROR | An error has been caught in function '__processing_thread', process 'MainProcess' (16764), thread 'Thread-2 (__processing_thread)' (23296):
Traceback (most recent call last):
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 995, in _bootstrap
self._bootstrap_inner()
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 1038, in _bootstrap_inner
self.run()
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 975, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\watchdog.py", line 146, in __processing_thread
handle(command)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\watchdog.py", line 54, in handle
process_file(command)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\namer.py", line 179, in process_file
search_results = match(command.parsed_file, command.config, phash=phash)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 505, in match
results: List[ComparisonResult] = __metadata_api_lookup(file_name_parts, namer_config, phash)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 179, in __metadata_api_lookup
results: List[ComparisonResult] = __metadata_api_lookup_type(results, name_parts, namer_config, scene_type, phash)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 158, in __metadata_api_lookup_type
results = __update_results(results, name_parts, namer_config, scene_type=scene_type, phash=phash)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 147, in __update_results
for match_attempt in __get_metadataapi_net_fileinfo(name_parts, namer_config, skip_date, skip_name, scene_type=scene_type):
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 468, in __get_metadataapi_net_fileinfo
file_infos = __get_metadataapi_net_info(url, name_parts, namer_config)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 455, in __get_metadataapi_net_info
file_infos = __metadataapi_response_to_data(json_obj, url, formatted, name_parts, namer_config)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 401, in __metadataapi_response_to_data
found_file_info = __json_to_fileinfo(data, url, json_response, name_parts, config)
File "C:\Users\jobu5\AppData\Local\Programs\Python\Python311\Lib\site-packages\namer\metadataapi.py", line 369, in __json_to_fileinfo
if tag.name not in tags:
AttributeError: 'types.SimpleNamespace' object has no attribute 'name'
Any help would be really appreciated. Let me know if you need what's in my config file or anything. I will do my best to learn and understand as I want to correct this if it's something I did wrong. Thank you!
Do not download the poster of the scene
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.