Comments (11)
Oh, I first thought this had to do with oneof
fields not working well, hence the reference from commit, but I see you need something else.
from python-betterproto.
👉 https://stackoverflow.com/questions/42622015/how-to-define-an-optional-field-in-protobuf-3
Actually, turns out in Protobuf 3 all fields are optional, and primitive values such as integers, bools will always contain their default value if they were not explicitly set.
In other words, there is no way to check which field was set.
Please let us know if that answers your question~
from python-betterproto.
Sure, but would it be possible to track which fields are set? Maybe an optional feature of it significantly impacts performance. Or another option would be to set fields to None by default?
from python-betterproto.
@nyurik
Concerning having each instance track which fields have been set (unrelated to what is serialized in the end), this isn't something dataclasses are very well suited to, but it's possbile.
I think this would require modifying the Message base class to:
- include set of names of fields that have been set per instance (logically this just needs to be a bitvector with one bit per field, would have to see how efficiently this can be done in python with reasonable complexity)
- add a setattr method thus set adds the target field to the updated set whenever it is set.
There would certainly be some performance cost to this, the cost/benefit isn't obvious to be. Neither is how this could be made optional.
from python-betterproto.
Sure, but would it be possible to track which fields are set? Maybe an optional feature of it significantly impacts performance. Or another option would be to set fields to None by default?
Just to be clear, do you mean, whether they were set by the python client (user of this library)? or whether they were set in the message that came in over the wire
from python-betterproto.
@nyurik
Concerning having each instance track which fields have been set (unrelated to what is serialized in the end), this isn't something dataclasses are very well suited to, but it's possbile.
Technically, it would certainly be possible to keep track of set-actions. However, if it means we introduce a method like .is_set()
, it will only work when clients have set it, and does not tell whether it was set on the wire.
I'm concerned that this will confuse consumers of the library, who will expect to be able to inspect set/unset fields on all messages.
Alternatively, having None values by default might solve that issue, which sounds pretty intuitive to me. Although this also ties in with the question, what do we do with incoming messages with oneof
fields. Do we also ignore the default values for unused fields and set them all to None? Normally, within oneof, unused int32 fields will be 0, and not None.
Out of curiosity, why do you need to keep track of the set fields, if they are set in the client code?
from python-betterproto.
For future reference, the above request is not satisfied by the feature betterproto.serialized_on_wire
, which only detects fields that are messages, whereas the example concerns messages with primitive fields.
https://github.com/danielgtaylor/python-betterproto#determining-if-a-message-was-sent
python-betterproto/betterproto/__init__.py
Lines 927 to 933 in eec24e4
from python-betterproto.
Technically, this proto file is in proto 2 format. Proto 3 does not have optionals.
Superficially, that would mean we would not support this case, however, it is the case that clients/servers may send messages across the wire and omit some fields.
This could mean that:
- they didn't know about the field
- or that they wish to transmit the default value (which clients are required to return when a field is not present)
- or that it was a proto2 message that was optional and unset
- or that the server has updated their version of the message so that the field was wrapped with
oneof
- or that only part of the message was sent. protobuf is designed so that messages can be split up and re-assembled.
We can probably keep track of the information you want, but we must be careful interpreting it.
from python-betterproto.
If anyone else needs a quick fix, you can simply change the defaults for primitive types to None
this way:
def init_default_gen_patch(self):
default_gen = {}
for field in dataclasses.fields(self.cls):
meta = FieldMetadata.get(field)
if meta.proto_type == TYPE_MESSAGE:
default_gen[field.name] = self.cls._get_field_default_gen(field, meta)
else:
default_gen[field.name] = lambda: None
self.default_gen = default_gen
betterproto.ProtoClassMetadata.init_default_gen = init_default_gen_patch
from python-betterproto.
This is fixed in 2.0.0b5 with Message.is_set()
from python-betterproto.
If anyone else needs a quick fix, you can simply change the defaults for primitive types to
None
this way:def init_default_gen_patch(self): default_gen = {} for field in dataclasses.fields(self.cls): meta = FieldMetadata.get(field) if meta.proto_type == TYPE_MESSAGE: default_gen[field.name] = self.cls._get_field_default_gen(field, meta) else: default_gen[field.name] = lambda: None self.default_gen = default_gen betterproto.ProtoClassMetadata.init_default_gen = init_default_gen_patch
@vidklopcic Do you have a guide on where to place that snippet when using plain betterprot, e.g.
python -m grpc_tools.protoc -I . --python_betterproto_out=. echo.proto
?
from python-betterproto.
Related Issues (20)
- `Struct().from_dict(...)` has no effect HOT 5
- Missing import in generated code when using well known types HOT 1
- Generated code has non-existent relative import HOT 3
- Error parsing proto with map field HOT 3
- Type error on server-side unary-unary handler code
- Invalid code generated when package name contains uppercase letter HOT 2
- Support proto3 optional fields HOT 1
- Warning for optional fields HOT 1
- Make `grpclib` dependency optional HOT 1
- deepcopy with betterproto.Enum fields fails HOT 2
- Enums are set as integers instead of enums when parsing betterproto.Message HOT 4
- why I use from_json method Missing field? HOT 1
- Bug: Empty string in object is not serialized HOT 3
- Optional fields in proto3 HOT 6
- module 'google.protobuf' has no attribute 'Any' HOT 1
- `mypy --strict` fails with `Name "betterproto.ServiceStub" is not defined` HOT 1
- Paths are not normalized to posix format on Windows
- Add option to turn off adding of empty __init__.py files
- Imports can conflity with typing types HOT 3
- Deepcopy incorrectly sets the 'active' oneof field to last field in structure (breaking which_one_of)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-betterproto.