[flow framework] Need more type info in DataConverter interface about aws-sdk-java HOT 5 CLOSED

+1 for this feature. Or at least enable overriding DataConverter using @Activity instead of only possible to define the DataConverter in @Activities.

from aws-sdk-java.

Hi, I have asked the Simple Workflow team to look at your question.

from aws-sdk-java.

Flow Framework calls fromData passing it the type information of the top level type that it is trying to deserialize, which in the case of activity inputs is Object[]. The serialization format is expected to contain the necessary information to properly handle the round trip serialization/deserialization of types including generic types and collections. Even if Flow Framework passed in an array of types to fromData, your DataConverter needs to be able to handle the object graph within it, for example you may have a field of type Object[].

Which serialization format are you using in your custom DataConverter? The JsonDataConverter included in Flow Framework produces JSON with type information (it uses Jackson JSON Processor under the covers).

from aws-sdk-java.

Handling the object graph once you know the top-level type is not a problem. yes, it's true one could define their activity as something like myActivity(Object[] arg1, Object[] arg2, etc.) but that's up to individual users if they want to structure their code that way. At least by passing in an array of types to fromData, I have the choice to preserve type information or not.

It's really limiting and requires pretty tight coupling in a distributed system like this for the producer of the output to need to know how the consumer will deserialize the data.

E.g. If I have workers written in Ruby, and a decider in Java, the Ruby workers are are expected to include Java type information for deserialization in their JSON output?

The Flow Framework has the actual type information for the activity method arguments available to it. It would be really helpful to pass it along.

from aws-sdk-java.

You are correct in pointing out that Flow Framework has additional type information about the parameters that it can pass to the DataConverter. I was trying to clarify that this would not completely solve the problem in the general case and your serialization format would need some type hints for things like collections, derived types etc. Type hints don't have to be language specific, for example in XML you could use xsi:type. I think to properly implement a cross-language use case you would also need some mechanism to express types in a language neutral way that you can then map to types in each of the languages -- protocol buffers and XSD come to mind.

That said, we will consider exposing the declared parameter types to the fromData method to make it easier to implement custom DataConverters.

from aws-sdk-java.