[Core feature] Support for Native JSON types in flyte about flyte HOT 6 OPEN

To support native JSON types in Flyte, you propose introducing a new LiteralType.JSON in FlyteIDL, using binary/bytearray for transport, and allowing custom JSON serializers/deserializers. This approach aims to address issues with current JSON handling via protobuf Struct, particularly with integer representation.

Would you like more detail on how to implement this feature or any potential challenges?

Sources:

• GitHub Issue #4505
• GitHub Issue #1282
• Discuss Flyte

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

from flyte.

Related to #4505

from flyte.

#take

from flyte.

From #4505 (comment), I see the follow Python types that can use Literal.JSON:

1. dicts -> JsonPickle
2. dataclasses -> Use Mashumuro's encoders/decorders or the object's to_json + from_json (These methods are defined with DataClassJSONMixin)

@kumare3 Are you considering other Python types that would use Literal.JSON?

from flyte.

I am saying wholesale migration to json for these unsupported types from struct

from flyte.

#5337

from flyte.