bswck / configzen Goto Github PK
View Code? Open in Web Editor NEWManage configuration with pydantic.
Home Page: https://bswck.github.io/configzen/
License: GNU General Public License v3.0
Manage configuration with pydantic.
Home Page: https://bswck.github.io/configzen/
License: GNU General Public License v3.0
Something like https://docs.hikari-py.dev/en/latest/_modules/hikari/files/#AsyncReader.
poetry remove aiofiles
:)
No response
Allow Python modules to serve as configuration resources.
^extend: programmatic_config.py
...
No response
A config route that supports type-safety and runtime validation.
If we have
class ConfItem(BaseConfiguration):
pair: tuple[int, int]
class Conf(BaseConfiguration):
item: ConfItem
then instead of writing
Conf.at("item.pair[0]")
write
Conf.at(Conf.item.pair[0])
where
Conf.item.pair[0]
is in fact
LinkedRoute(Conf, Route([GetAttr("item"), GetAttr("pair"), GetItem(0)]))
at runtime.
No response
No response
Let's go. I'll put a plan here, once I have one.
The initial idea is to keep the initial configuration structure in some kind of graph and then use it when exporting the configuration.
Model.export(preserve_comments=True)
(after migration to pydantic v2)
Model.export_model(preserve_comments=True)
No response
Software without unit tests is like a child without shoes.
Prone to pain.
(Does not apply)
No response
I wish I really didn't have to.
Does not apply
I guess save config on every assignment/update(). Not sure how that could look like yet, especially in the asynchronous environment where that would be blocking.
(Does not apply)
No response
importlib.import_module
requires the package
argument to be non-null in the case of a relative import (e.g. importlib.import_module("..abc")
is not allowed). This causes problems when trying to import through a directory in the wrap_module
method. Take the following directory tree as an example:
.
├── module
│ └── conf.py
└── my_config.py
This will cause a TypeError
when trying to call wrap_module
to import my_config
in configzen, because the top level module
directory is not a python package, causing the passed value to import_module
to be None
.
# the error is relative to the filesystem, so heres a quick script that would generate a repro
import os
os.mkdir("./repro")
os.mkdir("./repro/module")
with open("./repro/module/conf.py", "w") as f:
f.write(
"""from configzen import ConfigModel
class Test(ConfigModel):
host: str
port: int
Test.wrap_module(".my_config")"""
)
with open("./repro/my_config.py", "w") as f:
f.write(
"""host = '0.0.0.0'
port = 5000"""
)
print("now try python3 repro/module/conf.py")
Traceback (most recent call last):
File "/home/zero/Desktop/Projects/Python/repro/module/conf.py", line 8, in <module>
Test.wrap_module(".my_config")
File "/home/zero/.local/lib/python3.10/site-packages/configzen/model.py", line 2430, in wrap_module
importlib.import_module(module_name, package=package)
File "/usr/lib/python3.10/importlib/__init__.py", line 121, in import_module
raise TypeError(msg.format(name))
TypeError: the 'package' argument is required to perform a relative import for '.my_config'
No response
Does not apply.
Does not apply.
No response
End of support: 2024-10 (ref).
Does not apply.
Credit to @ZeroIntensity for the idea.
[config.yml
]
foo: # a comment
bar: 5
biz:
baz: 10
> configzen foo.bar # automatically detect it's about config.yml`
5
> configzen foo.bar=6 # set foo.bar to 6, preserve comments`
[config.yml] Set foo.bar to 6
No response
No response
Easily change settings just from process arguments. There's probably something like that in Hydra, unsure.
from configzen import ConfigModel, ConfigField, cli_param
class MyModel(ConfigModel):
debug: bool = ConfigField(alias="debug_mode", cli_param=cli_param("-d", "--debug"))
verbose: bool = ConfigField.cli_param("-v", "--verbose")
def _cli_callback(self):
...
Integrate with argparse
:
parser = argparse.ArgumentParser()
...
arguments = MyModel.cli.get_parser_arguments()
for nf_args, options in arguments.items():
parser.add_argument(*nf_args, **options)
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
...
subparsers.add_parser(MyModel.cli.as_parser("name"))
Integrate with click
:
if __name__ == "__main__":
MyModel.cli()
Integrate with typer
:
app = typer.Typer()
app.add_typer(MyModel.cli.typer())
No response
JSON keys tend to be in camelCase, as it originates from JavaScript, while Python attributes are expected to be in snake_case, per PEP 8. It would be nice for configzen to translate keys in camelCase in a JSON file to their corresponding snake_case attributes.
{
"fooBar": "hello world"
}
from configzen import ConfigModel
class Something(ConfigModel):
foo_bar: str
conf = Something.load("something.json")
print(conf.foo_bar) # hello world
No response
No serious package doesn't use coverage.
Does not apply
No response
One of the reasons for the creation of configzen was its scope management usefulness.
For example when modifying the settings through the application that uses configzen, one might want to make a nice UI for modifying particular configuration. When configuration gets really big, it might be really hard to fit it one form.
New functionality: abstract framework for configzen UIs and pagination.
(Does not apply)
No response
Structuring complex configuration without repeating yourself.
In detail, the functionality should bring new query language for loading and managing the configuration.
This would definitely help to maintain a neat and tidy, yet complex configuration system.
Having
class MyI18n(ConfigModel):
locale: str
timezone: str
we could configure it with a single file
# path/to/i18n.yaml
locale: pl_PL
timezone: Europe/Warsaw
and then load it with
MyI18n.load("path/to/path/to/i18n.yaml")
However, if we want to maintain multiple configuration files that should actually do something similar to 'transcluding' this configuration internally, we need to either have identical copies of the common configuration in them (and then if one changes, the other may be inconsistent with it), or have something that performs the transclusion from path/to/i18n.yaml
in those configuration files. That transclusion would happen without changing the documents, since the configzen library is built on abstraction over file formats – thus, in a YAML configuration we could then even import a JSON config.
Assuming that both for production and development mode of an example considered application we use the following code to load configuration:
class MyConfig(ConfigModel):
i18n: MyI18n
# other independent config variables...
config = MyConfig.load("path/to/development.yaml" if __debug__ else "path/to/production.yaml")
then instead of maintaing separately i18n
section in file
# path/to/development.yaml
i18n: # as in path/to/i18n.yaml
locale: pl_PL
timezone: Europe/Warsaw
# other independent config variables...
and separately maintaing i18n
section in file
# path/to/production.yaml
i18n: # as in path/to/i18n.yaml
locale: pl_PL
timezone: Europe/Warsaw
# other independent config variables...
and wasting time on worrying whether they are the same as they should,
we could have
# path/to/development.yaml
i18n:
.import: path/to/i18n.yaml
# other independent config variables...
and
# path/to/production.yaml
i18n:
.import: path/to/i18n.yaml
# other independent config variables...
Please note that I could rewrite these examples into JSON, TOML or anything else supported by anyconfig and it would still work identically.
These import()
statements could be even part of a rich query language, that could merge imported configurations, such as
options:
.merge:
.import: path/to/base.yaml
.import: path/to/overrides.yaml
to merge dictionaries out of imported base.yaml
and overrides.yaml
.
Namely, that would result in
AppropriateConfigModel(options={**anyconfig.load("base.yaml"), **anyconfig.load("overrides.yaml")})
Moreover, accessing particular scopes of the imported configurations would be considered useful.
We could make a default-settings.yaml
with default configs dedicated for other models.
# path/to/default-settings.yaml
i18n:
locale: pl_PL
timezone: Europe/Warsaw
database:
name: postgres
password: myapp
port: 5432
and then, in the application development configuration:
# path/to/development.yaml
i18n:
.import(i18n): path/to/default-settings.yaml
database:
name: postgres_dev
password:
.import(database.password): path/to/default-settings.yaml
port:
.import(database.port): path/to/default-settings.yaml
and in the application production configuration:
# path/to/production.yaml
i18n:
.import(i18n): path/to/default-settings.yaml
database:
.import: path/to/default-settings.yaml
# path/to/production.yaml
i18n:
locale: pl_PL
timezone: Europe/Warsaw
database:
.import(database): path/to/default-settings.yaml
name: postgres_dev
And this way, we would create inheritance of configurations:
# path/to/production.yaml
.import: path/to/default-settings.yaml # base configuration
database:
# inform that we inherit from path/to/default-settings.yaml database section,
# because now we just overwrite that imported section and YAML will
# otherwise forget about it (it does not merge)
.import(database): .
# overwrite desired option with our custom value
name: postgres_dev
which would be equivalent to:
# path/to/production.yaml
# start import(path/to/default-settings.yaml)
i18n:
locale: pl_PL
timezone: Europe/Warsaw
database:
name: postgres
password: myapp
port: 5432
# end .import: path/to/default-settings.yaml
database:
# start .import(database): path/to/default-settings.yaml
name: postgres
password: myapp
# end .import(database): path/to/default-settings.yaml
name: postgres_dev
that eventually evaluates to
# path/to/production.yaml
i18n:
locale: pl_PL
timezone: Europe/Warsaw
database:
name: postgres
password: myapp
name: postgres_dev
No response
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.