Giter Club home page Giter Club logo

pachinko's Introduction

             _   _     _       
 ___ ___ ___| |_|_|___| |_ ___ 
| . | .'|  _|   | |   | '_| . |
|  _|__,|___|_|_|_|_|_|_,_|___|
|_|

modular, plugin based media sorter  
created by @rbtr  

Build Status Go Report Card Release Docker License

Updates!

pachinko is now part of the GitHub Sponsors program! If you enjoy it and want to support my work, please check out my sponsor page or click on the "๐Ÿ’– Sponsor" button up above!

what it is

pachinko is a media sorter. it integrates with the tvdb and the moviedb to, given a directory of reasonably named mix media, organize that media into a clean hierarchal directory structure ideal for use in media servers like plex, kodi/xbmc, etc.

unlike some of the prior implementations of this idea, pachinko was designed from inception to be automation and container-friendly.
it has no heavy gui - configure it through the config file or via flags, then execute it and walk away.

it is written in go so that it is approachable for anyone interested in contributing without sacrificing performance.
the plugin-style architecture keeps the core codebase clear and efficient.

design

plugins

pachinko has a plugin based pipeline design. the base plugin types are:

  • input - add data from a datasource to the stream
  • processor - modify the datastream in-flight
  • output - deal with the processed data by e.g. moving files from their original dir to their sorted path.

these base plugin types make pachinko flexible. composing a pipeline of many combination of plugins is possible.

additionally there are subtypes of processor plugins:

  • preprocessor - parse data already present in the datastream to classify, clean, or add information to the data before main processor plugins run. the preprocessors make modifications to the datastream based only on the data already present in the objects in the datastream.
  • (intra)processor - the main working processors, where external datasources may be queried to enrich the datastream and significant modifications made.
  • postprocessor - last chance to modify the datastream before it is sent to outputs, but after the rich data has been added. the postprocessors make final modifications that shouldn't be the responsibility of the intraprocessors but may depend on the data enrichments that those have added.

the subtypes exist mainly to allow ordering of plugin flow.

datatypes

pachinko currently supports these data types:

  • tv and
  • movie video files

other datatypes planned include: images (and whatever you would like to contribute!)

inputs

pachinko currently supports these inputs:

  • local filesystem (path).

other datastore types planned include : s3 (and whatever you would like to contribute!)

outputs

pachinko currently supports these outputs:

processors

pachinko has the following optional processors:

  • tv identifier (pre-tv)
  • movie identifier (pre-movie)
  • tvdb (intra-tvdb)
  • tmdb (intra-tmdb)
  • tv path solver (post-tv_path_solver)
  • movie path solver (post-movie_path_solver)
  • file deleter (deleter)

how to run it

pachinko is distributed as a container and as a cross-platform binary.

the container is recommended:

$ docker run -v /path/to/source:/src:z -v /path/to/dest:/dest:z -v /path/to/cfg:/cfg rbtr/pachinko:latest --config /cfg

to run the binary:

$ ./pachinko sort --config /path/to/config

options

pachinko is configurable via file (yaml, toml), cli flags, or env vars.

the config file is recommended:

dry-run: true
log-level: debug
inputs: []
outputs: []
processors: {}

the full, current list of options is available by running ./pachinko genconf on the commandline.
the core pachinko options are:

option inputs usage
conf string full path to config file - ignored in the config file
dry-run bool dry-runs print only, pachkino will not make changes
log-level string one of (trace,debug,info,warn,error) for logging verbosity
log-format string one of (json,text)

inputs, outputs, and processors are lists of plugins objects and look generally like:

inputs:
- name: path
  src-dir: /path/to/source
outputs:
- name: stdout

note that each plugin may have its own independent config options; refer to that plugin's docs for details on configuring that specific plugin. here, the path input plugin has a src-dir parameter that we configure in the plugin list item.

the plugin list is processed in the written order and repeats are allowed. all loaded plugins are guaranteed to see each of the items in the datastream at least once. if the order that your datastream is processed by each plugin matters, make sure to load your plugins in the correct order!

testimonials

here's what users had to say when asked what they thought about pachinko:

Ew. Pachinko? Why would you name it pachinko? Pachinko just makes me think of flashing lights and cigarette smoke. - a Japanese user

license

pachinko is licensed under MPL-2 which generally means you can use it however you like with few exceptions.

read the full license terms here.


created by @rbtr
inspired by the functionality and frustrating user experience of: sorttv by cliffe, filebot, tinymediamanager, and others
and the excellent architecture patterns of telegraf, caddy, coredns, and others

pachinko's People

Contributors

jvstein avatar rbtr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

gwylanscheeren

pachinko's Issues

extracting metadata from filename should start with the file and proceed up the tree, not the reverse

for example,
/src/My Cool Show Season 1-2-3-4-5-6-7/My Cool Show S01E01.mkv
will be incorrectly matched as "My Cool Show Season 1-2-3-4-5" Season 6 Episode 7, since the path parser extracts metadata from the entire path of the file, lazily, left to right.
it would be better to parse the leaf filename initially, then proceed up the dir tree if the full metadata is not present in just the filename. the extraction from the filename should be prioritized over subsequent extractions from the containing dirs, if in conflict.

Config command fails if ~/.pachinko.yaml exists

Found this one when trying to write up issue #6. The following error happens if the ~/.pachinko.yaml file exists and you then run pachinko config.

FATA[0000] 8 error(s) decoding:

* 'inputs[0]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'
* 'outputs[0]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'
* 'outputs[1]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'
* 'processors[0]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'
* 'processors[1]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'
* 'processors[2]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'
* 'processors[3]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'
* 'processors[4]' expected type 'string', got unconvertible type 'map[interface {}]interface {}'

[plugin] add file deletion plugin

something to clean up the extra trash - text files, images, subs - in the src directories
bonus delete the empty src directories when sorted

[plugin] add trakt plugin

plugin ideas:
trakt identifier - trakt version of the tvdb plugin for adding metadata
trakt collector - sink/output which "adds to collection" on trakt

Generated config sometimes unusable

The config generated by pachinko config is not always usable. This is because the order of the processors list is non-deterministic and potentially reordered each time it is written.

The particular issue I ran into is that the video-id processor must be executed before the tv-id processor (because of the check in video.go#L42. If the processors are out of order, it will not work.

Example bad config created by the command:

dry-run: false
inputs:
- name: path
  src-dir: ./src
log-format: text
log-level: info
outputs:
- create-dirs: true
  dry-run: true
  name: move
  overwrite: false
- name: stdout
pipeline:
  buffer: 0
processors:
- api-key: ""
  name: tvdb
  request-limit: 10
- dest-dir: dest
  episode-names: false
  format: not-implemented
  name: tv-dest
  season-dirs: true
  tv-prefix: tv
- matchers:
  - (?i)(?:s+\d+)(?:\.|\s|-|_|x)*(e+\d+)
  - (?i)(?:season)+(?:(?:\.|\s|-|_|x|\/)\d+)
  - (?i)(?:[^[:alpha:]\d])\d{1,3}[x-]\d{1,3}
  name: tv-id
- matchers:
  - (?i)\b([\s\w.-]*)[\s.\/-]+(?:\((\d+)\))?[\s.\/-]?(\d{1,3})[x-](\d{1,3})
  - (?i)\b([\s\w.-]*?)?(?:[\s\(.\/-](\d{4})[\s\).\/-])?[\s\w.-]?(?:s+(\d+))(?:\.|\s|-|_|x)*(?:e+(\d+))
  - (?i)\b([\s\w.-]*)[\s.\/-]+(?:\((\d+)\))?[\s.\/-]?(?:season|series).?(\d+).?(?:episode)?[^\d(]?(\d+)
  name: tv-meta
  sanitize-name: true
- file-types:
  - .avi
  - .divx
  - .m4v
  - .mkv
  - .mov
  - .mp4
  - .xvid
  name: video-id

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.