Maybe use cadence as base?

It looks like tdarr is doing the same and is working already.

Free as in freedom, yet another distributed media converter

This is a simple system to queue in media conversion, e.g. video to mkv, in a network where neither server nor client are always online. You just put in a task to convert and ask if the job is done. At the beginning, we will juse php with the build in webserver and a basic framework. We will put it to `/srv/webapps/net_bazzline/distributed_media_converter".

It looks like it is not possible to resume an interrupted ffmpeg-job. We could run the stuff in a vm and pause the vm or split a video into segments and glue the converted segments together. To get things moving, we won't use a database. The broker just moves files into dedicated paths. We also won't use a database, the broker is single threaded so it will use a basic json file to keep things up to date. Server can also reject an item (disk full, not supported conversion)

What about using rqlite as database? Or etcd

{
  [
    <string: file_path: {
      "server_id": <string>,
      "item_id": <string>,
      "created_at": <date>,
      "status": <string>,
      "last_checked_at": <date>
    }
  ]
}

{
  "item_id": <string: uuid>,
  "created_at": <date>,
  "client_id": <string>,
  "file_name": <string>,
  "sha512_checksum": <string>
  "file_size": <string>,
  "status": <string>
}

{
  [
    <uuid: item_id>: {
      "client_id": <string>,
      "status": <string>
    }
  ]
}

{
  "to_do": [
    <uuid: item_id>: {
      "created_at": <date>,
      "client_id": <string>,
      "conversion_type": <string>,
      "original_file_name": <string>,
      "current_file_name": <string>,
      "sha512_checksum": <string>
      "file_size": <string>
  },
  "in_progress": [
    <uuid: item_id>: {
      "created_at": <date>,
      "client_id": <string>,
      "conversion_type": <string>,
      "original_file_name": <string>,
      "current_file_name": <string>,
      "sha512_checksum": <string>
      "file_size": <string>
  },
  "finished_successful": [
    <uuid: item_id>: {
      "created_at": <date>,
      "client_id": <string>,
      "original_file_name": <string>,
      "current_file_name": <string>,
      "sha512_checksum": <string>
      "file_size": <string>
  },
  "finished_with_errors": [
    <uuid: item_id>: {
      "created_at": <date>,
      "client_id": <string>,
      "original_file_name": <string>,
      "sha512_checksum": <string>
      "file_size": <string>,
      "errors": [
        <string>
      ]
    },
  ]
}

{
  <int: process_id>: {
    "pid": <int>,
    "started_at": <date>,
    "item_id": <string>
  }
}

  "list_of_paths_to_observer": [
    "path_one": {
      "observed_media_types": [
        "audio",
        "image",
        "video"
      ]
    },
    "path_two": {
      "observed_file_types": [
        "audio",
        "image",
        "video"
      ]
    }
  ],
  "list_of_servers": [
    "hostname_or_ip_address": {
      "port": <int>,
      "list_of_supported_media_types": [
        "audio",
        "image",
        "video"
      ]
    }
  ]

"list_of_supported_media_types": [
  "audio",
  "image",
  "video"
],
"listen_on_port": <int>,
"number_of_conversion_threads": <int>

#return all existing entries per client id
GET /api/v1/item/{client_id}

#return existing entry status
GET /api/v1/item/{client_id}/{item_id}

#create new entry
POST /api/v1/item

#delete existing entry
DELETE /api/v1/item/{client_id}/{item_id}

• create some basic examples

• basic server api with endpoints to
  • put a new item to the processing queue
  • get status of one item
  • get status of all items per client
  • fetch converted item
• basic cli client to interact with the api
  • upload new item
  • download converted item
  • fetch status of item

• implement server cli broker
  • handles status of each item
    • to do
    • in progress
    • finished successful
    • finished with error
  • commands
    • add to queue
    • list queue

• implement conversion thred observer
• implement one conversion thread

• implement real conversion logic with support for libx265 and libx264 videos

• implement gracefull shutdown of conversion threads
  • will stop at current position
  • will remove in progress file
• implement start up
  • check if there is stuff to clean up

• create systemd service
• create install and uninstall scripts for server and client side

• implement client to automatically check if server is available

• add support for images [png|jpg] to webp
• implement multi threaded
• create interfaces to ease up change of broker or broker database
• add option to deal with multiple servers
• add option to organize server per task
• replace json "databases" with sqlite or more
• extend client code to observe file paths to automate requesting conversion
• add support for apache webserver instead of interla php server
• add authentication for client and server
• think about rewriting it in RUST

• Should we use Symfony 5?
  • Rest API init?
  • Documentation

• Build a Simple REST API in PHP - 20190308
• How To Create A Simple REST API in PHP? Step By Step Guide! - 20200629