I'm Paul (he/him). I'm a long-standing techie :-)
Always happy to talk about DevOps, DevSecOps, Python and much more...
Pythonic library to aid with serialisation and deserialisation to/from JSON and XML.
Home Page: https://py-serializable.readthedocs.io/
License: Apache License 2.0
currently supporting python >- 3.7
3.7 is end of life -- see https://devguide.python.org/versions/
lets drop PY3.7 support
checklist:
sys.version_info
.to_xml()
currently returns like
<?xml version='1.0' encoding='utf-8'?>
<my:book xmlns:my="http://the.phoenix.project/testing/defaultNS" my:isbn_number="978-1942788294">
<my:id>f3758bf0-0ff7-4366-a5e5-c209d4352b2d</my:id>
</my:book>
the defaultNS can be just set properly, and hen the NS-id can be omitted.
like
<?xml version='1.0' encoding='utf-8'?>
<book xmlns="http://the.phoenix.project/testing/defaultNS" isbn_number="978-1942788294">
<id>f3758bf0-0ff7-4366-a5e5-c209d4352b2d</id>
</book>
using a defaultNS would make the resulting documents smaller, but still equal.
that the f...
eval
of List[FooBar]( ... )
instead of native list(...)
...
the typing has no value here, this is pure runtime! and it is not only unnecessary overhead.
is this really needed?
and the FooBar
is not even imported in this context, the typing just makes no sense !!!1elf
_ARRAY_TYPES = {'List': List, 'Set': Set, 'SortedSet': Set}
_DEFAULT_XML_SEQUENCE = 100
_SORTED_CONTAINERS_TYPES = {'SortedList': List, 'SortedSet': Set}
# ...
# Will load any class already loaded assuming fully qualified name
self._type_ = eval(f'{mapped_array_type}[{results.get("array_of")}]')
self._concrete_type = eval(str(results.get("array_of")))
this is a library, expected to be usable in various environments.
therefore, it is needed to run the lib in various test envs, too.
the poetry lock file is making this hard.
therefore: lets remove the poetry lock file, if possible.
for QA reasons, the ci/tests should run periodically, to check if all dependencies still match
the dev-dependnecies should be pinned to exact versions, to make testing reliable, despite the absence if a lock file.
and for some reasons, the current lock file appears to cause problems on CI:
the current implementation of custom serialization in JSON is actually a custom normalization.
Whatever is returned, it will be JSON-encdeded when needed.
unfortunately, this functionality is not there when custom-serializing for XML.
it allows strings only or casts the returns to strings. this prevents needed functionality.
Feature request: if serialize
returns a list instead of a string, then the list is treated as child-items, that will then be serialized.
proposed changes
elif prop_info.custom_type:
if prop_info.is_helper_type():
- SubElement(this_e, new_key).text = str(prop_info.custom_type.serialize(v))
+ new_cont = prop_info.custom_type.serialize(v)
else:
- SubElement(this_e, new_key).text = str(prop_info.custom_type(v))
+ new_cont = prop_info.custom_type(v)
+ if isinstance(custom_val, list): # content = list of children
+ SubElement(this_e, new_key).extend(
+ c.as_xml(view_=view_, as_string=False, element_name=None, xmlns=xmlns) for c in new_cont
+ )
+ else: # content = text
+ SubElement(this_e, new_key).text = str(new_cont)
Example implementation here:
serializable.patch.zip
Inadvertantly opened this ticket half way through writing the subject.
PyLance thinks that serializable.serializable_class decorated classes are untyped. I'm unsure if there is a proper way to resolve type based issues. The "fix" seems to be to have one undecorated class and one decorated class that references the former: i.e. "MyThing" and "SerializableMyThing" in order to allow for casting in individual programs.
Looking forward to some insight on this. CycloneDX-Python-Lib makes use of this library and it can be a bit cumbersome to work with classes provided by that library. Attempting to cast falls into the same trap.
serializer does not add namespaces to XML-attributes, so setting a defaultNS did not work.
the following will fail:
ElementTree.tostring(
foo.as_xml(view_=_view, as_string=False, xmlns='myNS')
method='xml',
encoding='unicode', xml_declaration=True,
default_namespace='myNS'
)
currently, xml attributes are not compltely controllable when serialized.
the view, context and other info are not accessible.
proposal.
use xml_normalize()
, instead of xml_serialize()
If some user of py-serializable calls serializable.serializable_class
before configuring the root logger (which is common during import blocks), the root logger ends up configured with the default basic configuration.
This happens because we call logging.debug
in ObjectMetadataLibrary.register_klass
. Calling logging.debug
(or similar) will automatically call logging.basicConfig
if no handlers are configured for the root logger.
We should probably do what most modules do and set up our own logger and NullHandler
rather than using the root logger.
import logging
import serializable
@serializable.serializable_class
class Foo:
pass
_logger = logging.getLogger()
_logger.addHandler(logging.NullHandler())
_logger.error("foo")
Outputs ERROR:root:foo
, which shouldn't happen if I only configure a NullHandler
.
I'm currently working around this by wrapping the import of anything that uses py-serializable
(in this case, CycloneDX) in a logging configuration hack:
_root_logger = logging.getLogger()
_root_null_handler = logging.NullHandler()
_root_logger.addHandler(_root_null_handler)
from cyclonedx.model.bom import Bom
_root_logger.removeHandler(_root_null_handler)
and this one was completely unable to detect the defaultNS, and assumed it was '' -- which is the identifier for the defaultNS not the defaultNS itself ...
and then the NameSpace-detection on XML-attributes fails ...
examples:
# is string already! why the cast?
if type(type_) == str:
type_to_parse = str(type_)
# is in list already! why the cast?
if results.get('array_type', None) in self._SORTED_CONTAINERS_TYPES:
mapped_array_type = self._SORTED_CONTAINERS_TYPES.get(str(results.get("array_type")))
serializing set
, list
works.
but support for tuple
is missing. it must have been forgotten, so i would rate this as a bug.
the implementation claims to support all basic collections.
current implementation for custom (de)serialize does not allow to have a differrent value returned for JSON or XML.
the custom (de)serializer has no change to tell the difference.
in some cases it is needed to retun a value for the target.
solution: inject the SerializationType.{JSON,XML}
as an argument into BaseHelper.{de,}serialize()
during _{as,from}_{json,xml}()
proposed changes
class BaseHelper(ABC):
def __init__(self, *args: Any, **kwargs: Any) -> None:
pass
@classmethod
@abstractmethod
- def serialize(cls, o: object) -> Any:
+ def serialize(cls, o: object, t: 'SerializationType') -> Any:
raise NotImplementedError
@classmethod
@abstractmethod
- def deserialize(cls, o: str) -> Any:
+ def deserialize(cls, o: str, t: 'SerializationType') -> Any:
raise NotImplementedError
example implementation here:
serializable.patch.zip
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.