Deserialise arbitrary types in python about dag-factory HOT 3 OPEN

This sounds like a great idea to me. 👍

from dag-factory.

@dinigo have you ever started writing code for this? This would be very useful indeed.

I'm facing similar situation with kubernetes pod operator where all arguments are kubernetes python api objects

from dag-factory.

@vitoravancini, We were using our own coupled config-to-DAG library. For standardization purposes we decided to go with a FOSS solution such as dag-factory one (thanks @ajbosco !).

I suspect it only requires changing where config is loaded

            config: Dict[str, Any] = yaml.load(
                stream=open(config_filepath, "r", encoding="utf-8"),
-               Loader=yaml.FullLoader,
+               Loader=yaml.UnsafeLoader,
            )

This way you can make use of types other than the mandatory ones in yaml.

Mi strong opinion is that, since you are the owner of the config, you can use the said "UnsafeLoader". In my opinion this name is not accurate, since most part of people managing yaml files are not doing so in an unsupervised way from the network. So this reference to a possible injection attack is out of context.

If you where so kind as to create a PR and write a couple of tests following the above examples that will create a working env for us to collaborate

from dag-factory.