Add metadata sink for a file about meteor HOT 7 CLOSED

and filename could be called path imo?

from meteor.

@mabdh makes sense, so should we have a config like this:

config:
    path: ./dir/some-dir/postgres_food_app_data.json
    format: json

Where format is for defining data format and we write the data in the expected format in the whatever file extension user provides??

from meteor.

What about we just use filename field and directory could be inferred from that field ?
e.g.

filename: ./dir/some-dir/postgres_food_app_data

from meteor.

What about we just use filename field and directory could be inferred from that field?

In this case we can also include the format then?? Cause just having the directory and filename patched together can be confusing if there is no extension at the end?

filename: ./dir/some-dir/postgres_food_app_data.json

from meteor.

sinks:
    name: file
    config:
        format: "yaml/json"
        filename: "postgres_food_app_data"
        output-dir: path/to/folder

I thought this should be a fine format for config with all the necessary info required for the file sink.
Also, supporting the json and yaml format is the expected behavior and not some other format, right?? @StewartJingga

Feel free to add any suggestions/feedback or additional expectations from this sink.
cc: @ravisuhag @mabdh @scortier

from meteor.

Also, have thought to keep the format and output-dir fields in config optional, and add some default values ourselves.
For eg:
format: json
output-dir = current

from meteor.

@GrayFlash but on the other hand, json and yaml are just a data format. they don't have to be coupled to the file extension. in this case, i don't think we should restrict file extension to the user but the best practice should be following what the data format is (we let user decide).

from meteor.