roru / video-packager Goto Github PK

Clients (services) who are in need of various kinds of video transformation, packaging and publishing do not care about underlying execution framework used. They operate witр 3 entities only - input data, transformation, output data. They do not care how particular transformation is being implemented under the hood.

That means a client service have to have the way to make a request for a particular transformation for the given set of inputs expecting to get the set of results eventually.

There are several tasks to solve here:

1. Collect user input
2. Locate user context (YML file)
3. Merge user inputs to user context
4. Merge user context with system context(s)
5. Locate transformation by name (CWL script)

Having these 5 steps resolved a transformation becomes possible to execute.

Le't try to implement simple but real scenario - 4K UHD SDR video 2-pass transcoding.

Source video for development and testing:

tearsofsteel_4k.mp4

Prerequisites:

ffmpeg with x265 support

Hints:

• In order to make local ffmpeg run faster cut video to first N seconds.
  Example: ffmpeg -i video_full_duration.mp4 -t 10 -c copy video_10_seconds.mp4
• Use preset=ultrafast to speed up transcoding

Documentation:

https://www.ffmpeg.org/ffmpeg-codecs.html#libx265

Workflow steps:

1. Analyze video

Use ffprobe to get high level info about an MP4 file (tracks and codec details):

ffprobe -v quiet -print_format json -show_format -show_streams video_4k.mp4

or

ffprobe -v quiet -print_format json -show_format -show_streams video_4k.mp4 > video_4k.mp4.json

2. Ensure video is 4K UHD SDR

Ensure there is a video track - there is stream with "codec_type": "video"
Width >= 3840
Height >= 2160

3. Transcoding

Parameters:

• input_file
• output_file
• profile=main|main10|mainstillpicture
• preset=ultrafast|superfast|veryfast|faster|fast|medium(default)|slow|slower|veryslow|placebo
• fps (frames per second)

Commands:

2-pass transcoding

Having the following input and analyzed parameters input_file =video_4k.mp4, output_file =transcoded_video_4k.mp4, profile=main, preset=ultrafast, fps=24:

keyint=fps*2
min-keyin=fps*2

# pass 1
ffmpeg -y -r 24 -i 'video_4k.mp4' -vsync 1 -pix_fmt yuv420p -c:v libx265 -tune zerolatency -profile:v main -x265-params 'input-res=3840x2160:fps=24:level-idc=5.0:high-tier=1:sar=1:no-open-gop=1:keyint=48:min-keyint=48:scenecut=0:no-scenecut=1:colorprim=bt709:bitrate=20000:vbv-maxrate=30000:vbv-bufsize=40000:pass=1' -preset ultrafast -an -map 0:v:0 -an -f mp4 /dev/null 

So the template of the command is:

ffmpeg -y -r {fps} -i 'video_4k.mp4' -vsync 1 -pix_fmt yuv420p -c:v libx265 -tune zerolatency -profile:v {profile} -x265-params 'input-res=3840x2160:fps={fps}:level-idc=5.0:high-tier=1:sar=1:no-open-gop=1:keyint={fps*2}:min-keyint={fps*2}:scenecut=0:no-scenecut=1:colorprim=bt709:bitrate=20000:vbv-maxrate=30000:vbv-bufsize=40000:pass=1' -preset {preset} -an -map 0:v:0 -an -f mp4 /dev/null 

The command above produces the following files:

x265_2pass.log
x265_2pass.log.cutree

It is required to preserve these files for pass 2:

# pass 2
ffmpeg -y -r 24 -i 'video_4k.mp4' -vsync 1 -pix_fmt yuv420p -c:v libx265 -tune zerolatency -profile:v main -x265-params 'input-res=3840x2160:fps=24:level-idc=5.0:high-tier=1:sar=1:no-open-gop=1:keyint=48:min-keyint=48:scenecut=0:no-scenecut=1:colorprim=bt709:bitrate=20000:vbv-maxrate=30000:vbv-bufsize=40000:pass=2' -preset ultrafast -an -map 0:v:0 -an -f mp4 transcoded_video_4k.mp4

4. Expecting the transcoded_video_4k.mp4 have been created

In practical scenarios there are two types of input context at least:

• user context (e.g. input/output file path)
• system/script level context (e.g. passlogfile path)

The user context is used to solve practical user (better to say - clients, in relation to available workflows) tasks having a certain properties of a workflow, while the system context is used by video engineers who are responsible to manage quality of a workflow.

In other words there is a need to merge multiple input context into composite one before particular workflow run. Merge should not be limited by the number of 2 context.

Complex workflows require many command line tools to be provisioned. They have to be preinstalled or downloaded before workflow proceed. Preinstalled tools are fine for local development environment. In production environment the commandline tools should be somehow provided at runtime. Tools management and provisioning are good to be inverted of control. This is there where the Docker is very good for. Let's pack all required tools into Docker image and run our workflows with it providing all required tools in Docker way.

As the first step let's dockerize Ubuntu Linux and FFMPEG 4.0.x with libx265 support.