Comments (10)
Hi @gshank.
If str
is on the first place of Union
args, then it will be on the first place in Union
inside of InvalidFieldValue
exception. There is no reordering under the hood. I suggest you double check the dataclass field types.
Anyway, if you put List[str]
on the first place, then input data might be coerced to List[str]
from a str
value because str
supports iterator protocol and there is no strict validation with isinstance
at the moment. However, in this case you can avoid this problem by coercing a value to a list of values:
@dataclass
class NodeConfig(DataClassDictMixin):
unique_key: Optional[Union[List[str], str]] = None
@classmethod
def __pre_deserialize__(cls, d):
unique_key = d.get('unique_key')
if isinstance(unique_key, str):
d['unique_key'] = [unique_key]
return d
from mashumaro.
'str' is in the first place of the Union args, but somehow it's still ending up in second place in the InvalidFieldValue exception, so something is reordering it. Thanks for the example, but we'd prefer it the other way, where if it's a single str it stays that way. I wonder if this could be a Python bug? I'll try it with some other Python versions.
from mashumaro.
You can see the definition here, in the NodeConfig class: https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/contracts/graph/model_config.py
from mashumaro.
I wonder if this could be a Python bug?
I don't think so.
I see that you make some non-trivial changes to the input data with hooks. If the problem occurs in the tests only, I think logging the data
value at the beginning and at the end of __pre_deserialize__
method can help to figure it out. I'd like to know what's going on with the initial dictionary after pre_hook and post_hook applied.
from mashumaro.
It looks like it's a bug in typing.get_type_hints. In that particular case it returns: 'unique_key': typing.Union[typing.List[str], str, NoneType]. I'm going to change the definition from Optional[Union[str, List[str]] to Union[str, List[str], None], which so far seems to work. It's intermittent (since in most other case it doesn't do that), and I'm not up for debugging Python.
Thanks for taking a look.
from mashumaro.
@gshank I think you didn't try mashumaro 2.11 at least? There were some fixes for Union
and None
: https://github.com/Fatal1ty/mashumaro/releases/tag/v2.11.
Release 2.9.1 also contains a fix for deserialization override for Union
types: https://github.com/Fatal1ty/mashumaro/releases/tag/v2.9.1
P.S. I should definitely create a template for the issues :)
from mashumaro.
Anyway, if the problem is solved, we can close this issue.
from mashumaro.
We're on mashumaro 2.9 right now. I did a typing.get_type_hints(NodeConfig) (which leaves mashumaro out of it entirely), and the order came out different than it was in the code. I believe that for Unions mashumaro constructs code that tries the various types in order?
It's not totally clear to me that this is actually a typing.get_type_hintsbug. In theory, I guess that elements in the Union should be transposable. But this doesn't work very well for some types when trying to serialize by trying them out. Perhaps the solution is some kind of combined str/List[str] type which would handle this properly.
from mashumaro.
I believe that for Unions mashumaro constructs code that tries the various types in order?
Yes, it does it in the initial order. Union arguments are stored in __args__
attribute.
In theory, I guess that elements in the Union should be transposable.
Ideally, yes. But without strict validation of input data types it's not possible. I'm working on a validation feature that would help with this.
Perhaps the solution is some kind of combined str/List[str] type which would handle this properly.
Or just create a list of one element during deserialization in case of str
value since List[str]
is a valid type here :)
from mashumaro.
That works too :)
from mashumaro.
Related Issues (20)
- Generic dataclasses with Hashable typevar bound do not work HOT 3
- Query string support HOT 5
- Benchmark is not working in master branch HOT 2
- omit_default=True evaluates default_factory value at declaration time which can cause a circular reference HOT 3
- Multiple aliases for field HOT 6
- "`dict[str, T]` as a field type is not supported" HOT 2
- InitVar with no default value HOT 6
- The types defined inside the function result in syntactically invalid generated code
- Allow for more complex logic for subclass Discrimnator field matching HOT 3
- TypedDicts not working with `from __future__ import annotations` HOT 2
- omit_default breaks IntFlag serialization HOT 1
- Using Union with int/float casts to whichever appears first HOT 3
- Not parsing Generics correctly HOT 2
- Unserializable field in 3.12 if defined as a Generic TypeVar with mixin bounds HOT 6
- Allow propagation of class based discriminator settings to subclasses HOT 3
- Reject extra keys on deserialization HOT 7
- Investigate support for recursive Union types HOT 2
- Supports `numpy.ndarray` type for `orjson` HOT 2
- Add support for PEP 695
- Union type of [int | float] not serialized correctly HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mashumaro.