Property names that conflict with dict built-ins are problematic. about typesystem HOT 13 CLOSED

I couldn't replicate the issue like this...

import typesystem


class Node(typesystem.Schema):
    name = typesystem.String()
    cams = typesystem.Array(items=typesystem.String(), min_items=0, max_items=4)


node = Node.validate({'name': "abc", 'cams': ["foo"]})
print(dict(node))

Could you show me exactly how to replicate the issue?

from typesystem.

I use TypeSystem 0.2.2 with Sanic 19.3.1 and Python 3.7

here is my schema code:

class GatewayConfig(Schema):
    sn = String(min_length=16, max_length=16)
    cam_describs = Array(items=String(), min_items=0, max_items=4)
    ch = Integer()
    sf = Integer()
    nodes = Array(items=String(), min_items=0, max_items=64)
    keys = Array(items=String())

here is my sanic request code:

@gateway_bp.route('/gateway_config/<name_id>', methods=['PUT'])
@authorized()
async def import_data(request, name_id):
    data_type, error = GatewayConfig.validate_or_error(request.json)
    if error:
        log.error(error)
        raise exceptions.InvalidUsage("data valid error")
    await request.app.db.gateway_config.update_one({'_id': ObjectId(name_id)}, {'$set': dict(data_type)})
    msg = "update success"
    log.info(msg)
    return response.json({"message": msg})

then it raised attribute of type 'list' is not callable

from typesystem.

Okay, so reduce that down to a reproducible example case that's just using typesystem. In particular, what is the value of request.json`?
At that point I'd be able to get involved.

from typesystem.

I print request.json at the head

{'sn': '1812010009000100', 'cam_describs': ['拍天摄像头', '60度角正前方', '60度角拍草地', '60度角拍风塔'], 'ch': 0, 'sf': 7, 'nodes': [], 'keys': ['Temprature', 'Humidity', 'WindSpeed', 'WindDir', 'Ap', 'Voltage', 'CSQ', 'GPSLongti', 'GPSLati', 'BT']}

from typesystem.

Ah, the problem here is keys. It's conflicting with the dict built-in keys().

Here's a minimal reproducible example:

import typesystem


class Example(typesystem.Schema):
    keys = typesystem.Array(items=typesystem.String())


data = {'keys': ['Broken']}
node = Example.validate(data)
print(dict(node))

from typesystem.

I'm not sure right away the best way to go about fixing this. You can avoid it by not using the name "keys" although that's not ideal.

from typesystem.

Thanks a lot.

from typesystem.

I renamed "keys" to "fields", but it raised an another problem

Error while handling error: 'AttributeError' object has no attribute 'status_code'
Stack: Traceback (most recent call last):
  File "/Users/quentin/PycharmProjects/sensormanager/venv/lib/python3.7/site-packages/sanic/app.py", line 917, in handle_request
    response = await response
  File "/Users/quentin/PycharmProjects/sensormanager/authorize.py", line 33, in decorated_function
    response = await f(request, *args, **kwargs)
  File "/Users/quentin/PycharmProjects/sensormanager/api/gateway.py", line 87, in post_data
    print(data)
  File "/Users/quentin/PycharmProjects/sensormanager/venv/lib/python3.7/site-packages/typesystem/schemas.py", line 193, in __repr__
    key: getattr(self, key) for key in self.fields.keys() if hasattr(self, key)
AttributeError: 'list' object has no attribute 'keys'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/quentin/PycharmProjects/sensormanager/venv/lib/python3.7/site-packages/sanic/app.py", line 933, in handle_request
    response = await response
  File "/Users/quentin/PycharmProjects/sensormanager/exception.py", line 6, in error_handler
    return json({"err": exception.args[0]}, status=exception.status_code)
AttributeError: 'AttributeError' object has no attribute 'status_code'

It takes me long time to rename the keys in my project, but it still need rename again.
I am tired. Could you give a solution, thanks a lot.

from typesystem.

Well, you’ve done well there. The two sets of things it can’t be named are:

• The dict built-in attribute names
• And “fields” since that’s existing API on Schema classes

from typesystem.

could you help me to name a key which would not raise exception? how about "names"?

from typesystem.

How about "meta"?

See English definition of meta.

The 2. variation applies to your case.

from typesystem.

I'm not sure right away the best way to go about fixing this. You can avoid it by not using the name "keys" although that's not ideal.

Indeed. Also, not everybody will use typesystem for data they control. For example, the first case I want to use this for is to replace marshmallow in my sans-I/O Google Music library where I'd need items. And for another project I would need fields. Names like items, keys, values, fields are far too valuable in data that people would be validating/marshaling

Let me preface this by saying that I love doing this for many things (e.g. in my audio-metadata library). But, IMHO, a library like typesystem has to do everything within reason to prevent conflicting with the data it's meant to handle. We only have to look at some prior art to see that the answer is to not have Schema inherit from MutableMapping, rename fields to something less likely to clash (schema_fields?), and add an explicit way to convert to a dict. attrs has an asdict helper, marshmallow has dump/dumps methods, schematics has to_native/to_primitive methods.

As of now, I simply couldn't begin using typesystem for anything due to the field name limitations it incurs. The OP had to forgo their top 2 choices for field names due to those limitations. I'm quite certain this will cause one or both of those things to happen to others in the future unless something is changed.

PS Thank you for all the wonderful work you and the people of Encode have been doing. I'm planning (and hoping) to use your projects in the near future.

from typesystem.

And for another project I would need fields. Names like items, keys, values, fields are far too valuable in data that people would be validating/marshaling

Yeah. I'm strongly in favour of any future version taking a different tack where we don't create Schema instances at all. (Or at least not as the default). Instead have validation default to returning plain ol' dict instances, and allow the user to override that behavior.

from typesystem.