Settings management using pydantic
Home Page: https://docs.pydantic.dev/latest/usage/pydantic_settings/
License: MIT License
Settings management using Pydantic, this is the new official home of Pydantic's BaseSettings.
This package was kindly donated to the Pydantic organisation by Daniel Daniels, see pydantic/pydantic#4492 for discussion.
For the old "Hipster-orgazmic tool to manage application settings" package, see version 0.2.5.
See documentation for more details.
@samuelcolvin do you plan to add support for configuration file loading? Such as being able to load settings from toml, json, ini, etc.. files? I would be interested in contributing to this effort if possible.
First of all, thanks for all the work on pydantic!
I recently started with it and it is great.
I identified the following behaviour and expected it to be different given the docs and the behaviour on individual cases
Priority when generating a settings class:
This applies when parameters are fully set or lef out , however,
Looking forward to a solution and/or an reply on whether this is intended behaviour after all.
class NestedConfig(BaseSettings):
nested_var1 : str = 'Default nested var1 is not set, overwrrite by dict/yaml '
nested_var2 : str = 'Default nested var2, SHOULD NOT have been set? but via config in PartlyOverwriteByYaml class?'
class PartlyOverwriteByYamlConfig(BaseSettings):
var1 : str
nested : NestedConfig = NestedConfig(
nested_var1 = "should will not be set, overruled by dict/yaml",
nested_var2 = "THIS PARAM SHOULD HAVE BEEN SET, BUG?"
)
partly_nested_yaml= {
"var1" : "dummy",
"nested" : {
"nested_var1" : "nested var1 set via yaml/dict config",
# "nested_var2" : "nested var 2 set via params"
}
}
print("===============")
print("4 : Unexpected")
cfg_partly_nested_yaml_fails= PartlyOverwriteByYamlConfig(**partly_nested_yaml)
print(cfg_partly_nested_yaml_fails)pydantic version: 1.10.2
pydantic compiled: True
install path: C:\Users\%user%\AppData\Local\pypoetry\Cache\virtualenvs\scrappaper-Qd2gr6XB-py3.8\Lib\site-packages\pydantic
python version: 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]
platform: Windows-10-10.0.22000-SP0
optional deps. installed: ['devtools', 'dotenv', 'typing-extensions']
.dict() and .json()construct(), pickling, private attributes, ORM modeHi,
Sorry if this is reported just before you're intending to do a release. If so please just close π
I've just used Kludex's codemod to upgrade my project. As a consequence I noticed I need to install pydantic-settings as a new dependency. However, as the latest pydantic-settings release sets a requirement on Pydantic v2.0a3 (fixed 4 days ago, but not release), and since the FastAPI v0.100 alpha release uses a feature introduced in v2.0a4, the two are currently incompatible.
Not sure how clear that was, but just wanted to give a heads up that loosening the requirement on pydantic version seems necessary to make this work with the FastAPI pre-release π Thanks!
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.8.2
pydantic compiled: True
install path: /home/user/anaconda3/envs/myenv/lib/python3.9/site-packages/pydantic
python version: 3.9.2 (default, Mar 3 2021, 20:02:32) [GCC 7.3.0]
platform: Linux-5.10.19-1-MANJARO-x86_64-with-glibc2.33
optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']
When passing _env_file to my Settings class i would expect the value settings.Config.env_file to be overwritten with the keyword argument. Instead the original value is kept:
from pydantic import BaseSettings
class Settings(BaseSettings):
data: str = None
class Config:
env_file = '.env'
settings = Settings(_env_file='prod.env')
print(settings.Config.env_file) # Prints '.env'i am not sure if this is intended behavior or a bug.
If it is intended, is there any way to get the value from the _env_file argument after instantiation?
Thanks for creating this project, btw. It is really one of the most pleasuring libraries to work with!
The nested BaseSettings raise a validation error when its attribute is uppercase.
But it is OK when its attribute is lowercase.
You can reproduce the error with the example code below.
I didn't find any docs about how the upper case affects the behavior of pydantic.
Did I miss anything?
Thanks
# The following code will raise error...
from pydantic import BaseSettings
class Nested(BaseSettings):
GOOD_BOY: str = "james"
class MyConfig(BaseSettings):
nested: Nested = Nested()
class Config:
# env_prefix = "MY_" # the prefix can't be mixed with the `env_nested_delimiter`
env_nested_delimiter = '__'
os.environ["NESTED__GOOD_BOY"] = "env_boy"
config = MyConfig()
print(config)
# The following code will run correctly.
from pydantic import BaseSettings
class Nested(BaseSettings):
good_boy: str = "james"
class MyConfig(BaseSettings):
nested: Nested = Nested()
class Config:
# env_prefix = "MY_" # the prefix can't be mixed with the `env_nested_delimiter`
env_nested_delimiter = '__'
os.environ["NESTED__GOOD_BOY"] = "env_boy"
config = MyConfig()
print(config)
# The only difference is that its attribute is uppercase.pydantic version: 1.9.2
pydantic compiled: True
install path: /sdc/home/xiaoyang/miniconda3/lib/python3.9/site-packages/pydantic
python version: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0]
platform: Linux-5.4.0-126-generic-x86_64-with-glibc2.27
optional deps. installed: ['dotenv', 'typing-extensions']
.dict() and .json()construct(), pickling, private attributes, ORM modeHi π
I just saw the FastAPI 0.100.0 prerelease announcement, the Pydantic v2 migration guide and wanted to give this all a try with one of my project's test suites with the latest and greatest.
I ended up getting this error:
self = EnvSettingsSource(env_nested_delimiter=None, env_prefix_len=0)
def __call__(self) -> dict[str, Any]:
data: dict[str, Any] = {}
for field_name, field in self.settings_cls.model_fields.items():
try:
field_value, field_key, value_is_complex = self.get_field_value(field, field_name)
except Exception as e:
raise SettingsError(
f'error getting value for field "{field_name}" from source "{self.__class__.__name__}"'
) from e
try:
field_value = self.prepare_field_value(field_name, field, field_value, value_is_complex)
except ValueError as e:
> raise SettingsError(
f'error parsing value for field "{field_name}" from source "{self.__class__.__name__}"'
) from e
E pydantic_settings.sources.SettingsError: error parsing value for field "ui_languages" from source "EnvSettingsSource"And this is my code that triggered the error when calling LanguageSettings().ui_languages:
from pathlib import Path
from pydantic import Field
from pydantic_settings import BaseSettings
def comma_separated_string_to_set(raw_value: str) -> set[str]:
if not raw_value:
msg = f"{raw_value} must be a comma separated string"
raise ValueError(msg)
return {item.strip() for item in raw_value.split(",")}
class LanguageSettings(BaseSettings):
ui_languages: set[str] = Field(env="UI_LANGUAGES")
class Config:
env_file = Path("envs/test.env")
@classmethod
def parse_env_var(cls, field_name: str, raw_val: str) -> Any:
if field_name == "ui_languages":
return comma_separated_string_to_set(raw_val)Here's my poetry config:
[tool.poetry.dependencies]
fastapi = {extras = ["all"], version = "^0.100.0b1", allow-prereleases = true}
pydantic = {version = "^2.0b3", allow-prereleases = true}
pydantic-settings = {version = "^2.0b1", allow-prereleases = true}Is all this expected when moving from old/current Pydantic, or is there a bug in pydantic-settings?
Hey,
In this issue, I'd like to propose a general improvement to the pydantic-settings, especially the handling of the sources. If it's agreeable, will create a PR.
The current implementation of the source settings consists of the following three sources:
Adding more sources and changing the order of sources requires patching the customise_sources method of Config class. Additionally, most of the logic in the EnvSettingSource and SecretSettingSource is transferrable to any kind of KeyValue source and Directory&File based source respectively.
Therefore, I'd propose that we:
os.environ exposes this already, for the local secret directory we would implement one. For other exotic sources like remote password managers, the user can implement it for themselves.Config class itself)An example of how it could look:
import inspect
import os
from typing import ClassVar, Dict, List, Mapping, Optional, Type, Union
from pydantic import BaseModel
def parse_sources(sources: List[Mapping], setting: 'BaseSettings'):
# Simplistic view on how parsing could look like.
result = {}
for field in setting.__fields__.values():
for source in sources:
value = source.get(field.name)
if value:
result[field.alias] = value
break
return result
get_env_source = lambda: os.environ
def get_env_file_source(file_path: Union[str, os.PathLike], encoding:str, case_sensitive:bool = False):
try:
from dotenv import dotenv_values
except ImportError as e:
raise ImportError('python-dotenv is not installed, run `pip install pydantic[dotenv]`') from e
file_vars: Dict[str, Optional[str]] = dotenv_values(file_path, encoding=encoding or 'utf8')
if not case_sensitive:
return {k.lower(): v for k, v in file_vars.items()}
else:
return file_vars
class BaseSettings(BaseModel):
def _build_values(self, init_kwargs: Dict[str, Any], source_kwargs: Dict[str, Any]):
sources = [init_kwargs]
for source_provier in self.__config__.source_providers:
signature = inspect.signature(source_provier)
kwargs = {}
for parameter in signature.parameters.values():
# Checks if the required parameter is provided to either of
# build_values or Config class. In config class attribute name
# is same as the parameter name while in the build_values
# function it expects to be prefixed with _
value = source_kwargs.get(f"_{parameter.name}", getattr(self.__config__, parameter.name))
if not value and isinstance(parameter.default, inspect._empty):
raise ValueError("Function `{source_provider}`: Missing value for required parameter `{parameter.name}`")
kwargs[parameter.name] = value
sources.append(source_provier(**kwargs))
return parse_sources(sources, self)
class Config:
prefix: str = ''
env_file: Optional[str]
env_file_encoding: Optional[str]
nested_delimiter: Optional[str]
source_providers = [
get_env_source,
get_env_file_source,
...
]
__config__: ClassVar[Type[Config]]Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.4
pydantic compiled: False
install path: C:\Users\aida\AppData\Local\Programs\Python\Python37\Lib\site-packages\pydantic
python version: 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
platform: Windows-10-10.0.19041-SP0
optional deps. installed: ['typing-extensions']
I'm using pydantic to load a configuration from a .env file. It works with no problem, but now I would like to set the path to an SSL certificate within the env file and generate variables containing encryption and decryption keys accordingly.
I've tried to set the values for encryption and decryption keys with validators, but it's no use. It seems like the config class does not expose the values present in the env file to the validators.
Reason for this request, is that I can't find any solution nor similar question on the internet and it seems a feature that could be interesting. Libraries like https://docs.authlib.org/en/stable/jose/jwe.html require a key in the form of a certificate.
Below part of the code I've been using.
from pydantic import BaseSettings, validator
class Settings(BaseSettings):
"""
Class representing the configuration of the service.
Reads the configuration from environment files.
"""
# JWT settings
EXPIRE_DELTA: int = 7200
ALGORITHM: str = "RS512"
SSL_LOCATION: str
ENCRYPTION_KEY: str = ""
DECRYPTION_KEY: str = ""
class Config:
env_file = "config/production.env"
@validator("ENCRYPTION_KEY", always=True)
def load_encryption_key(cls, v, values, **kwargs):
print(values)
print(kwargs["field"])
# Doesn't work
# print(values["SSL_LOCATION"])
# Doesn't work either
# print(type(values["SSL_LOCATION"]))
return ""
@validator("DECRYPTION_KEY", always=True)
def load_decryption_key(cls, v, values):
print(v)
return ""
...Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.8.2
pydantic compiled: False
install path: /usr/local/lib/python3.8/site-packages/pydantic
python version: 3.8.6 (default, Nov 20 2020, 23:57:10) [Clang 12.0.0 (clang-1200.0.32.27)]
platform: macOS-11.2.1-x86_64-i386-64bit
optional deps. installed: ['typing-extensions']
When the following are true
Config.allow_population_by_field_name is Truean extra fields not permitted error will be raised, however the same error is not raised if the field alias is passed to init instead of the field name
import os
from pydantic import BaseSettings, Field
class Config(BaseSettings):
user_name: str = Field(alias='userName', env='CONFIG_USER_NAME')
class Config:
allow_population_by_field_name = True
os.environ.pop('CONFIG_USER_NAME', None)
c = Config(userName='alice')
print(c) # alice
c = Config(user_name='mary')
print(c) # mary
os.environ['CONFIG_USER_NAME'] = 'bob'
c = Config()
print(c) # bob
c = Config(userName='lisa')
print(c) # lisa
c = Config(user_name='chris') # error, extra fields not permitted
print(c) # should be "chris"There is an issue with environment variables using json values for sub modules.
class SubModel(BaseModel):
v1: str = None
l1 : list[str] = None
class Settings(BaseSettings):
sub_model: SubModel = None
class Config:
env_nested_delimiter = '__'
os.environ['sub_model'] = '{"v1": "aa", "l1": ["a","b","c"]}'
works
os.environ['sub_model__l1'] = '[1,2,3]'
does not work
After debugging the code i have found the issue:
The method explode_env_vars, extracts the value but does not call parse_env_var for it
from pydantic import (
BaseModel,
BaseSettings,
Field,
)
import os
class SubModel(BaseModel):
v1: str = None
l1 : list[str] = None
class Settings(BaseSettings):
sub_model: SubModel = None
class Config:
env_nested_delimiter = '__'
os.environ['sub_model__l1'] = '[1,2,3]'
global_config = Settings()
print(global_config.dict())pydantic version: 1.10.4
pydantic compiled: False
python version: 3.9.7 (default, Dec 22 2022, 21:29:13) [Clang 14.0.0 (clang-1400.0.29.202)]
platform: macOS-13.1-x86_64-i386-64bit
.model_dump() and .model_dump_json()model_construct(), pickling, private attributes, ORM modeHey, parse_env_var does not work with nested settings. Reading the docs one would think that this should be possible as parsing nested environment variables is a feature.
The given example code prints the following error:
pydantic.error_wrappers.ValidationError: 1 validation error for Settings
submodel -> other_numbers
value is not a valid list (type=type_error.list)
import os
from typing import Any
from pydantic import BaseModel, BaseSettings
class SubModel(BaseModel):
other_numbers: list[int]
class Settings(BaseSettings):
numbers: list[int]
submodel: SubModel
class Config:
env_nested_delimiter = '__'
@classmethod
def parse_env_var(cls, field_name: str, raw_val: str) -> Any:
if field_name in ('numbers', 'submodel__other_numbers'):
return [int(x) for x in raw_val.split(',')]
return cls.json_loads(raw_val)
os.environ['numbers'] = '1,2,3'
os.environ['submodel__other_numbers'] = '4,5,6'
print(Settings().dict())pydantic version: 1.10.7
pydantic compiled: True
install path: ../.venv/lib/python3.11/site-packages/pydantic
python version: 3.11.3 (main, Apr 5 2023, 15:52:25) [GCC 12.2.1 20230201]
platform: Linux-6.3.2-arch1-1-x86_64-with-glibc2.37
optional deps. installed: ['typing-extensions']
.dict() and .json()construct(), pickling, private attributes, ORM modeAs mentioned in pydantic/pydantic#4492, I have been using https://github.com/danields761/pydantic-settings/ mostly for its with_attrs_docs decorator. (That being said, I wanted to link to its documentation here, but cannot find itβ¦)
It greatly reduces the visual noise / clutter when documenting fields that otherwise do not even need a Field().
Hence, I would love to see this be supported in this new project as well (not caring so much about the actual syntax / import, in case the name should change).
How will pydantic-settings interact with pydantic-next?
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
$ python -c "import pydantic.utils; print(pydantic.utils.version_info())"
pydantic version: 1.8.2
pydantic compiled: True
install path: /home/lucieng/.local/share/virtualenvs/data-infrastructure-NYWW-n8C/lib/python3.9/site-packages/pydantic
python version: 3.9.10 (main, Jun 15 2022, 16:44:29) [GCC 10.2.1 20210110]
platform: Linux-5.10.0-16-cloud-amd64-x86_64-with-glibc2.31
optional deps. installed: ['dotenv', 'typing-extensions']
from pydantic import BaseSettings, Field
class ExampleSettings(BaseSettings):
schema: str = Field("public")Raises the error here (1) as schema shadows an existing type.
However, using alias='schema' will soon no longer work for BaseSettings as described by this warning (2).
Error 1 above could check if the type is BaseSettings to build the error message accordingly and suggest the correct fix. An alternative would be linking to an external page about the issue to describe in more detail or just printing more information in place.
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.9.0
pydantic compiled: True
install path: /Users/irsath/workspace/pydantic_bug/venv/lib/python3.9/site-packages/pydantic
python version: 3.9.9 (main, Dec 12 2021, 10:56:00) [Clang 13.0.0 (clang-1300.0.29.3)]
platform: macOS-12.2.1-arm64-arm-64bit
optional deps. installed: ['dotenv', 'typing-extensions']
Given the code below
from typing import Set
from pydantic import (
BaseModel,
BaseSettings
)
class SubConfig(BaseModel):
subset: Set[str]
class Settings(BaseSettings):
topset: Set[str]
sub: SubConfig
class Config:
env_nested_delimiter = "__"
env_file = ".env"
env_file_encoding = "utf-8"
print(Settings().dict())Given the .env file below
topset='["foo.com"]'
sub__subset='["bar.com"]'According to the documentation
Complex types like list, set, dict, and sub-models are populated from the environment by treating the environment variable's value as a JSON-encoded string.
We expect as an output:
{'topset': {'foo.com'}, 'sub': {'subset': {'bar.com'}}}Actual ouput:
Traceback (most recent call last):
File "/Users/irsath/workspace/pydantic_bug/main.py", line 21, in <module>
print(Settings().dict())
File "pydantic/env_settings.py", line 38, in pydantic.env_settings.BaseSettings.__init__
File "pydantic/main.py", line 331, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Settings
sub -> subset
value is not a valid set (type=type_error.set)
It seems that while root level fields are JSON parsed when they are complex, nothing is done for nested fields loaded using env delimiters.
env_settings.py#188
# field is complex but no value found so far, try explode_env_vars
env_val_built = self.explode_env_vars(field, env_vars)
if env_val_built:
d[field.alias] = env_val_built ===> No complexity checks on subfieldsCurrently, there is field value priority (from the docs):
In the case where a value is specified for the same Settings field in multiple ways, the selected value is determined as follows (in descending order of priority):
- Arguments passed to the Settings class initialiser.
- Environment variables, e.g. my_prefix_special_function as described above.
- Variables loaded from a dotenv (.env) file.
- The default field values for the Settings model.
However, in some cases it would be nice to swap 1 and (2,3) priorities. For example, I would like to create the configuration system based on config files (YAML, TOML, INI, etc) and env vars where envvars/dotenv have highest priority (env vars override values from config files that looks reasonable in some cases just like now env vars (2) override dotenv file (3)).
It might look something like this:
class Settings(BaseSettings):
...
s = Settings(**load_from_config_file(config_path))In this case values from config file will override values from env vars.
Currently, the default behavior cannot be changed without subclassing BaseSettings class and overriding _build_values method. This is not mentioned in the current docs but previously it was mentioned (see pydantic/pydantic#340, pydantic/pydantic#341, pydantic/pydantic#343).
Would you mind considering the customization of this behavior via Config of BaseSettings model?
Hi π
I just saw the FastAPI 0.100.0 prerelease announcement, the Pydantic v2 migration guide and wanted to give this all a try with a little sample project.
from pydantic import PostgresDsn
from pydantic_settings import BaseSettings
from pydantic_settings.main import SettingsConfigDict
class Settings(BaseSettings):
ENVIRONMENT: str = "development"
SQLALCHEMY_DATABASE_DSN: PostgresDsn = (
# Specifying the psycopg dialect since it is different from the default
# psycopg2, using 127.0.0.1 instead of localhost to disambiguate IPv4 vs IPv6
"postgresql+psycopg://test:[email protected]/test"
)
model_config = SettingsConfigDict(case_sensitive=True, env_file=".env")The line with model_config = SettingsConfigDict(case_sensitive=True, env_file=".env") is currently giving me the following warnings: "Parameter 'env_file_encoding' unfilled", "Parameter 'env_nested_delimiter' unfilled", "Parameter 'env_prefix' unfilled" and "Parameter 'secrets_dir' unfilled", which does not happen with this equivalent normal Pydantic model:
from pydantic import BaseModel, ConfigDict
class SomeModel(BaseModel):
id: int
model_config = ConfigDict(from_attributes=True)The only difference I found is that pydantic.ConfigDict is declared as class ConfigDict(TypedDict, total=False), while pydantic_settings.main.SettingsConfigDict is lacking that optional parameter. Adding it seems to do the trick. I'm happy to open a PR with the change if you guys consider it is the right approach.
As per #27
Hi,
I am following the example for nesting a BaseSettings class with BaseModels and I am using env_nested_delimiter = '_'.
Running the example code I get:
pydantic.error_wrappers.ValidationError: 4 validation errors for Settings
sub_model -> v1
field required (type=value_error.missing)
sub_model -> v2
field required (type=value_error.missing)
sub_model -> v3
field required (type=value_error.missing)
sub_model -> deep
field required (type=value_error.missing)
While it looks possible to use the single underscore delimiter, the values are not picked up and I am not sure which ones the settings object is looking for in this example.
import os
from unittest import TestCase
from pydantic import BaseModel, BaseSettings
class DeepSubModel(BaseModel):
v4: str
class SubModel(BaseModel):
v1: str
v2: bytes
v3: int
deep: DeepSubModel
class Settings(BaseSettings):
v0: str
sub_model: SubModel
class Config:
env_nested_delimiter = '_'
env_prefix = 'TEST_'
class TestConfig(TestCase):
def setUp(self) -> None:
self.original_env = os.environ.copy()
def tearDown(self) -> None:
os.environ.clear()
os.environ.update(self.original_env)
def test_nested_delimiter(self):
os.environ['TEST_V0'] = 'v0'
os.environ['TEST_SUB_MODEL_V1'] = 'v1'
os.environ['TEST_SUB_MODEL_V2'] = 'v2'
os.environ['TEST_SUB_MODEL_V3'] = '3'
os.environ['TEST_SUB_MODEL_DEEP_V4'] = 'v4'
config = Settings()
self.assertEqual(config.v0, 'v0')
self.assertEqual(config.sub_model.v1, 'v1')
self.assertEqual(config.sub_model.v2, b'v2')
self.assertEqual(config.sub_model.v3, 3)
self.assertEqual(config.sub_model.deep.v4, 'v4')pydantic version: 1.10.2
pydantic compiled: True
install path: /Users/petroslabropoulos/Projects/workable/ml-utils/.venv/lib/python3.10/site-packages/pydantic
python version: 3.10.5 (main, Jul 20 2022, 15:20:07) [Clang 13.1.6 (clang-1316.0.21.2.5)]
platform: macOS-13.3.1-x86_64-i386-64bit
optional deps. installed: ['dotenv', 'typing-extensions']
.dict() and .json()construct(), pickling, private attributes, ORM modeCurrently BaseSettings can take the following config values from its __init__ method:
pydantic-settings/pydantic_settings/main.py
Lines 47 to 54 in c05410f
Then these values are used if provided, else the ones from the SettingsConfigDict, e.g.:
pydantic-settings/pydantic_settings/main.py
Lines 109 to 114 in c05410f
Is there a reason why env_prefix isn't part of this scheme? If this is possible, I can make a PR implementing this.
When pydantic loads the configuration, it records what data source the configuration is read from, and provides debug-level log output.i think if the feature is added, this library will work better.
By reading the source code, I found that it is currently unable to support recording the configuration source. This can be achieved by encapsulating the chainmap, or by adding an additional dict to record. Due to my limited level, I am afraid that direct modification of the source code will not completely cover it. testing, thus introducing more bugs, so I hope the development team can add this feature.
The env_prefix is used for both env vars and secrets, which makes it impossible to combine env vars (with prefix) and secrets (without prefix) into a single settings model.
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.7.2
pydantic compiled: False
install path: /usr/local/src/pydantic/pydantic
python version: 3.7.7 (default, Mar 11 2020, 00:27:03) [GCC 8.3.0]
platform: Linux-5.4.0-54-generic-x86_64-with-debian-10.3
optional deps. installed: []
Environment vars:
printenv
RESTAPI_USE_PRODUCTION_DB=1
RESTAPI_DB_READ_ONLY=0
Secrets in container:
ls /run/secrets/
db_read_only_password db_read_only_user db_read_write_password db_read_write_user
from pydantic import BaseSettings
class DBSettings(BaseSettings):
use_production_db: bool = False # env var
db_read_only: bool = False # env var
db_read_only_user: str = None # secret
db_read_only_password: str = None # secret
db_read_write_user: str = None # secret
db_read_write_password: str = None # secret
class Config:
env_prefix = 'RESTAPI_'
secrets_dir = '/run/secrets'
db_sett = DBSettings()Tested without the env_prefix = 'RESTAPI_' line, which filled the settings from the secrets correctly, env vars were not read as expected.
env_prefix is also used for secretsenv_prefix for the secretssecrets_prefix to be used as secrets prefixIn V1, given an OS that has case sensitive environment variable names: If you have a settings-object like the below (simplified from the pydantic docs):
class Settings(BaseSettings):
foo: str
then: If only "FOO" is set, "FOO" is obviously used. Likewise if only "foo" is set, "foo" is used. But if both "foo" and "FOO" is set, "foo" is preferred. This is a bit surprising!
I'm used to environment variable names always being upper case, regardless. For that reason, I would like the ability to only read from upper case environment variable names regardless of what it looks like in the rest of the code. This could look like:
class Config:
force_uppercase_names = True # Default: unset, no change from current system
or
class Config:
environment_name_casing = "upper" # Default: None/unset, no change from current system
At minimum, the docs should be updated with a warning that on an OS with case sensitive environment variable names and case_sensitive set to False, this will occur, to simplify tracking down the obscure bugs I can see this leading to.
.model_dump() and .model_dump_json()model_construct(), pickling, private attributes, ORM modeOutput of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.7.3
pydantic compiled: True
install path: <...>/.venv/lib/python3.9/site-packages/pydantic
python version: 3.9.1 (default, Dec 19 2020, 12:43:07) [Clang 12.0.0 (clang-1200.0.32.21)]
platform: macOS-10.16-x86_64-i386-64bit
optional deps. installed: ['typing-extensions']
With the following code:
from pydantic import BaseModel, BaseSettings
class FooModel(BaseModel):
value: str
class Config:
title = 'foo'
class FooSettings(BaseSettings):
value: str
class Config:
env_prefix = 'FOO_'
class FooSettingsFixed(BaseSettings):
value: str
class Config(BaseSettings.Config):
env_prefix = 'FOO_'Pylance complains about FooSettings:
It looks like there may be something missing in the definition for BaseSettings, since BaseModel config works without subclassing BaseModel.Config.
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.9.0
pydantic compiled: True
install path: /private/tmp/settings/.venv/lib/python3.10/site-packages/pydantic
python version: 3.10.1 (main, Jan 6 2022, 17:54:04) [Clang 13.0.0 (clang-1300.0.29.30)]
platform: macOS-11.6.4-arm64-arm-64bit
optional deps. installed: ['dotenv', 'typing-extensions']
When nesting a model in a Settings class and populating it with an environment file, pydantic does not match environment variables to variables of nested models if the latter is upper cased. Example (adapted from the tutorial).
I'm thinking this is a bug because capitalization of non-nested models is handled correctly. However, if it's somehow expected behaviour, perhaps a mention in the documentation could help people out who tend to use upper case variable names for settings.
Assume the following .env file:
export V0=0
export V1=0
export SUB_MODEL__V2=informationThen the following code runs fine:
from pydantic import BaseModel, BaseSettings
class SubModel(BaseModel):
v2: str # <-- lower case
class Settings(BaseSettings):
v0: str
V1: str
sub_model: SubModel
class Config:
env_nested_delimiter = "__"
s = Settings(_env_file='.env')
print(s.dict())
"""
{'v0': '0', 'V1': '0', 'sub_model': {'v2': 'information'}}
"""but the following crashes (the only change is the capitalization of SubModel.V2):
from pydantic import BaseModel, BaseSettings
class SubModel(BaseModel):
V2: str # <-- upper case
class Settings(BaseSettings):
v0: str
V1: str
sub_model: SubModel
class Config:
env_nested_delimiter = "__"
s = Settings(_env_file='.env')
print(s.dict())
"""
Traceback (most recent call last):
File "/private/tmp/settings/pyds.py", line 15, in <module>
s = Settings(_env_file='.env')
File "pydantic/env_settings.py", line 38, in pydantic.env_settings.BaseSettings.__init__
File "pydantic/main.py", line 331, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Settings
sub_model -> V2
field required (type=value_error.missing)
"""I'm guessing that pydantic/pydantic#3115 is affected by the same bug (if it is a bug), but since capitalization isn't mentioned I thought it would be best to create a new issue.
PS: Thanks for a great library π₯
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.8.1
pydantic compiled: True
install path: /home/.../.cache/pypoetry/virtualenvs/...-Mqcvcv6--py3.9/lib/python3.9/site-packages/pydantic
python version: 3.9.4 (default, Apr 14 2021, 17:44:30) [GCC 10.2.0]
platform: Linux-5.11.16-arch1-1-x86_64-with-glibc2.33
optional deps. installed: ['typing-extensions']
import os
import pydantic
os.environ['nested_field'] = 'nested_field_value'
class Nested(pydantic.BaseSettings):
field: str = pydantic.Field('', env='nested_field')
class Settings(pydantic.BaseSettings):
root: str = ''
nested: Nested = Nested()
s = Settings()
print('s.dict() ', s.dict())
print('s.dict(exclude_defaults=True) ', s.dict(exclude_defaults=True))
print('s.nested.dict(exclude_defaults=True)', s.nested.dict(exclude_defaults=True)).output:
s.dict() {'root': '', 'nested': {'field': 'nested_field_value'}}
s.dict(exclude_defaults=True) {}
s.nested.dict(exclude_defaults=True) {'field': 'nested_field_value'}
expected:
s.dict() {'root': '', 'nested': {'field': 'nested_field_value'}}
s.dict(exclude_defaults=True) {'nested': {'field': 'nested_field_value'}}
s.nested.dict(exclude_defaults=True) {'field': 'nested_field_value'}
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.8.2
pydantic compiled: True
install path: /Users/jduckworth/.pyenv/versions/3.8.9/envs/mlhub-backend-dev/lib/python3.8/site-packages/pydantic
python version: 3.8.9 (default, Apr 23 2021, 14:38:08) [Clang 12.0.0 (clang-1200.0.32.28)]
platform: macOS-10.15.7-x86_64-i386-64bit
optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']
Running mypy --show-error-codes ./pydantic_mypy_error.py with the files shown below gives the following error:
pydantic_mypy_error.py:9: error: Unexpected keyword argument "_env_file" for "EnvFileBaseSettings" [call-arg]
Found 1 error in 1 file (checked 1 source file)
mypy.ini[mypy]
plugins = pydantic.mypy
pydantic_mypy_error.pyfrom pydantic import BaseSettings
class EnvFileBaseSettings(BaseSettings):
my_var: str
if __name__ == "__main__":
_ = EnvFileBaseSettings(_env_file=".test.env").test.env# .test.env
MY_VAR=thing
Selected Assignee: @davidhewitt
Selected Assignee: @samuelcolvin
Description
I'm using BaseSettings to parse an environment variable that's a json string with a newline character. The variable is being parsed differently when using VSCode's debugger.
The json is invalid because of the way the string is being parsed differently.
Example Code
# main.py
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
foo: dict
class Config:
env_file = ".env"
settings = Settings()
print(settings.foo)# .env
FOO='{"foo":"bar\nbaz"}'Python, Pydantic & OS Version
pydantic version: 2.0a3
pydantic-core version: 0.25.0 release build profile
python version: 3.11.4 (main, Jun 15 2023, 07:55:38) [Clang 14.0.3 (clang-1403.0.22.14.1)]
platform: macOS-13.4-arm64-arm-64bit
I initially experienced this with Pydantic v1.10.9.
As per comment on #18 (review)
Thank you for adding settings management to Pydantic, it's a pleasure to use a single well-done library to manage both models and settings from .env files and environment variables! π
I'm not sure about its future in Pydantic V2 (π please don't remove it π ) since I can't find references in the main branch anymore. Still I'd like to mention a small improvement that could be done when reading .env files.
Currently, if the .env file does not exist it is simply ignored. I don't know if this is done by design but I would actually prefer a strict behaviour that raises an error asap.
What happened to me instead is that I provided a wrong path in the env_file and pydantic later raised with a ValidationError for missing fields that were specified in the .env file. I still consider this a lucky case because an error was eventually raised. What would have been worse is if my settings had defaults for every field, which would have resulted in no ValidationError being raised at all!
Let me know what do you think about it. If you have a strong preference for the current behaviour I guess I could always add some file existence check before parsing my settings. Still, it would be nice to have this done by Pydantic IMO.
Could you also share an update on the planned support for settings in V2? In the online v2 plan there isn't a final answer, but I imagine you have already taken actions on it.
Thank you so much and keep making Pydantic one of the best libraries of the Python ecosystem π
.dict() and .json()construct(), pickling, private attributes, ORM mode$ python -c "import pydantic.utils; print(pydantic.utils.version_info())"
pydantic version: 1.3
pydantic compiled: False
install path: /Users/andrew/.virtualenvs/project_name/lib/python3.7/site-packages/pydantic
python version: 3.7.5 (default, Oct 19 2019, 01:20:12) [Clang 10.0.1 (clang-1001.0.46.4)]
platform: Darwin-18.7.0-x86_64-i386-64bit
optional deps. installed: ['devtools']
Thanks for a great library and for taking the time to read this!
When pydantic raises validation errors about settings, it would be nice if it would include the settings prefix, if specified.
Let's assume a basic set of settings where variables are meant to be specified in the process environment.
from pydantic import BaseSettings
class Settings(BaseSettings):
"""Define basic settings to be overridden by the environment"""
var_name: str
class Config:
env_prefix = "app_prefix"If this class is instantiated with no changes to the environment, a Python traceback is printed with the following at the end
var_name
field required (type=value_error.missing)
The actual Validation object appears to be:
ValidationError(
model='Settings',
errors=[
{'loc': ('var_name',), 'msg': 'field required', 'type': 'value_error.missing'},
]
)It would be nice if instead of specifying only the variable name it also included the env_prefix. What is most helpful is something like:
ERROR: Set Environment Variable APP_PREFIX_VAR_NAME
To print a nicer error message, I am currently doing the following:
from colorama import Fore, Style
from pydantic.error_wrappers import ValidationError
from .config import Settings
def main():
try:
settings = Settings()
except ValidationError as e:
prefix = Settings.Config.env_prefix
for error in e.errors():
if error["type"] == "value_error.missing":
for var in error["loc"]:
env_var = f"{prefix}{var}".upper()
message = f"ERROR: Set Environment Variable {env_var}"
print(Fore.RED + message + Style.RESET_ALL)
else:
print(f"{Fore.RED}{error}{Style.RESET_ALL}")
exit(-1)
if __name__ == "__main__":
main()very similar to pydantic/pydantic#2284
I would love to be able to leverage built-in settings configuration, but in order to do that, I need support for both ConfigMaps and Secrets
i don't know if a file per setting is absolutely necessary, but it would be great if instead of a secrets_dir: str|None you had a directories: List[str]|None or search_dirs: List[str]|None argument, where settings were loaded by iterating over 1..N directories using much the same logic as today.
beyond just allowing use of both ConfigMaps and Secrets, this could then produce conflicts, which instead of a defect, would actually be a very handy a feature, it allows for overrides (such as when running locally) this can be accomplished by having the expectation that the last found match wins.
class Settings(BaseSettings):
my_secret_data: str
my_not_secret_data: str
my_overridden_data: str
class Config:
search_dirs = [ '/run/config' , '/run/secrets', ]
with files
/run/config/my_not_secret_data # foo
/run/config/my_overridden_data # bar
/run/secrets/my_overridden_data # baz
/run/secrets/my_secret_data # fu
pragmatic expectation:
all 4 files get opened/read/parsed/validated
resulting object is
s = Settings() # s.my_secret_data == 'fu', s.my_not_secret_data == 'foo', s.my_overridden_data == 'baz',
Reading the documentation and as BaseSettings inherits from BaseModel it'd be expected to have Config.extra parameters working when Config.env_file is defined.
It appears that without Config.env files it works as expected, but when it comes from env_files the extra config is ignored.
If this is expected I can make a PR to make the documentation more explicit on this specific point.
from pydantic import BaseSettings, Extra
""""
# .env.test
title="cool stuff"
other=42
"""
class Plop(BaseSettings):
title: str
class Config:
extra: Extra = Extra.forbid
env_file = ".env.test"
Plop()pydantic version: 1.10.2
pydantic compiled: True
install path: .venv/lib/python3.11/site-packages/pydantic
python version: 3.11.0 (main, Oct 24 2022, 19:55:51) [GCC 9.4.0]
platform: Linux-5.17.0-051700-generic-x86_64-with-glibc2.31
optional deps. installed: ['dotenv', 'typing-extensions']
.dict() and .json()construct(), pickling, private attributes, ORM modeI've implemented a recursive env variables parser based on the models json schema (only used for the schema structure and field names). It isn't complete and only support the cases I needed, but I propose to upstream it or maybe package it as plugin if the pydantic maintainers have some interest.
I wasn't happy with the json based env values, so I created this.
The code is here:
It has some specific (non exhaustive) features:
MY_STR_LIST=one,two or MY_STR_LIST_0=one)MY_OBJ_LIST_2_FIELD=changed should update an element at the index 2 in the array of objects.And limitations:
one_two.three and one.two_three fields have overlapping env names if the delimiter is _, work around is to use a __ (double _) delimiter.I've seem some work towards a similar feature in pydantic/pydantic#4216, and maybe they both overlap in features, but both implementation could live next to each other (probably as plugin ?) and let users decide what is best for there use case.
I am happy to provide a Pull request, but I prefer to talk a bit about this addition before I try anything. Maybe this has a better place in a community package (if one is planned ?) that people can install if they need tooling around pydantic ?
.dict() and .json()construct(), pickling, private attributes, ORM modeOutput of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.9.1
pydantic compiled: True
install path: /home/vscode/.local/lib/python3.10/site-packages/pydantic
python version: 3.10.4 (main, Apr 7 2022, 03:15:56) [GCC 10.2.1 20210110]
platform: Linux-5.4.0-1072-gcp-x86_64-with-glibc2.31
optional deps. installed: ['typing-extensions']
I don't know if it's really a bug or not, but I've found that when using BaseSettings with nested BaseModel as described in the documentation, the priority of sources is not propagated to the nested fields.
In the following example, I've set the config so env_settings takes precedence over init_settings in the BaseSettings Configuration class. It works well for fields defined at the root level, but it does not work for fields in the Foo sub model.
I've tried to switch Foo to heritate from BaseSettings instead of BaseModel and update manually the priority there too, but this leads to an unattended behavior: the field sub_bar can be set via the env var sub_bar instead of only alias_sub_bar and foo__sub_bar (and I don't want that).
import os
from pydantic import BaseModel, BaseSettings, Field
class Foo (BaseModel):
sub_bar: str = Field(..., env = 'alias_sub_bar')
class Configuration (BaseSettings):
root_bar: str = Field(..., env = 'alias_root_bar')
foo: Foo
class Config:
env_nested_delimiter: str = '__'
@classmethod
def customise_sources (cls,
init_settings,
env_settings,
file_secret_settings):
return (
env_settings,
init_settings,
)
os.environ['alias_root_bar'] = 'from_env_alias'
os.environ['alias_sub_bar'] = 'from_env_alias'
configuration = Configuration(
root_bar = 'from_init',
foo = {
'sub_bar': 'from_init',
},
)
print(f'{configuration.root_bar=} [should be from_env_alias]')
print(f'{configuration.foo.sub_bar=} [should be from_env_alias]')Output:
configuration.root_bar='from_env_alias' [should be from_env_alias]
configuration.foo.sub_bar='from_init' [should be from_env_alias]
Thank you for your help!
The following code raise an error
Traceback (most recent call last):
File "/Users/alexey.petrunnik/PycharmProjects/temp/temp6.py", line 12, in <module>
s = Settings()
^^^^^^^^^^
File "pydantic/env_settings.py", line 39, in pydantic.env_settings.BaseSettings.__init__
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Settings
data
JSON object must be str, bytes or bytearray (type=type_error.json)
.env file
DATA='{"key":"value"}'
from pydantic import BaseSettings, Json
class Settings(BaseSettings):
data: Json[dict[str, str]]
class Config:
env_file = '.env'
env_file_encoding = 'utf-8'
s = Settings()
print(s.data)pydantic version: 1.10.9
pydantic compiled: True
install path: /Users/alexey.petrunnik/Library/Application Support/pdm/venv/lib/python3.11/site-packages/pydantic
python version: 3.11.2 (main, Feb 16 2023, 02:55:59) [Clang 14.0.0 (clang-1400.0.29.202)]
platform: macOS-13.4-arm64-arm-64bit
optional deps. installed: ['dotenv', 'typing-extensions']
.dict() and .json()construct(), pickling, private attributes, ORM modeHello,
Everything is in the title.
The documentation for model_config displays the defaults used for the BaseModel's Config inner class, but nothing is said about BaseSettings's Config.
One could reasonably assume that the defaults would be the same, yet they are not as some are modified here, like extra and validate_all.
So I think this should be mentioned somewhere in the settings management page.
Implement protected namespace in BaseSettings. we need to:
settings_settings_ a protected namespace.Output of my python -c "import pydantic.utils; print(pydantic.utils.version_info())"
pydantic version: 1.8.2
pydantic compiled: True
install path: /home/xxx/.cache/pypoetry/virtualenvs/app-YZGTusa8-py3.9/lib/python3.9/site-packages/pydantic
python version: 3.9.5 (default, May 19 2021, 11:32:47) [GCC 9.3.0]
platform: Linux-5.4.0-73-generic-x86_64-with-glibc2.31
optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']Hello,
I have the problem that I want to store values in my .env file in the style "TEST_PASSWORD", "TEST_USERNAME" etc. and then use them in my settings class as "test_password", "test_username". As a normal environment variable via export TEST_PASSWORD="XXX" this works.
If I now set this value via a dotenv file it only works if the variable is really written the same way (uppercase).
Also setting the Field(..., env="TEST_USERNAME") does not work.
Is this by design or is it really a bug?
class TestSettings(BaseSettings):
test_username: str = Field(..., env="TEST_USERNAME")
test_password: str = Field(..., env="TEST_PASSWORD")
class Config:
env_file = '.env'
case_sensitive = FalseCurrently only one .env file is supported according to the source code, but sometimes the submodules of the project will have some custom variables that need to be set, and all these variables are not very well managed in one .env file, and it is easy to make changes to the wrong situation. For this reason, I think it should be managed in separate .env files
Consider multiple .env source reads, or dynamic .ini reads (this part of the dynamic allows the user to stitch the path to the ini file based on the module path)
In this example
class Settings(pydantic.BaseSettings):
VAR: str = pydantic.Field(env=("NEW_VAR", "OLD_VAR"))
class Config:
env_prefix = "MY_"
I would expected MY_NEW_VAR and MY_OLD_VAR to work, but it doesn't.
ππ»
Does it look like MRO is getting changed somehow when inheriting from BaseSettings? (I'm probably missing something)?
from pydantic import BaseSettings
class ConfigBase(BaseSettings):
p1: str = None
class AppConfig(ConfigBase):
p2: str = "AppConfig"
class AWSConfigBase(ConfigBase):
p1: str = "AwsConfigBase"
class Config(AppConfig, AWSConfigBase):
pass
print(Config().p1)
> None
where as without BaseSettings
class ConfigBase:
p1: str = None
class AppConfig(ConfigBase):
p2: str = "AppConfig"
class AWSConfigBase(ConfigBase):
p1: str = "AwsConfigBase"
class Config(AppConfig, AWSConfigBase):
pass
print(Config().p1)
> AwsConfigBasepydantic version: 1.10.2
pydantic compiled: True
install path: ........./python3.10/site-packages/pydantic
python version: 3.10.8 (main, Oct 13 2022, 10:17:43) [Clang 14.0.0 (clang-1400.0.29.102)]
platform: macOS-13.0.1-x86_64-i386-64bit
optional deps. installed: ['devtools', 'dotenv', 'email-validator', 'typing-extensions']
.dict() and .json()construct(), pickling, private attributes, ORM modeI have a class BaseSettings with a nested BaseModel. Within the nested model (named nested), I defined a variable with sub_var1: str = Field(..., env="SUB_VAR") (as mentioned in the docs).
I would expect that the environment variable name specified in the nested Field is loaded from my .env file. But it's not (see the example below). My .env file looks as follows:
MAIN_VAR=main
SUB_VAR=sub
It only works with env_nested_delimiter='__' and an environment variable NESTED__SUB_VAR1 but I would expect it to work with SUB_VAR (as it's set in the Field).
I'm not sure if this is a bug or expected behavior. If it's the latter I'm doing a feature request, I guess.
from pydantic import BaseSettings, BaseModel, Field
class SubModel(BaseModel):
sub_var1: str = Field(..., env="SUB_VAR")
sub_var2: str = "bar"
class Settings(BaseSettings):
main_var: str = Field(..., env="MAIN_VAR")
nested: SubModel
class Config:
env_file = '.env'
env_nested_delimiter = '__'
print(Settings().dict())pydantic version: 1.10.4
pydantic compiled: True
install path: /home/rafael/miniconda3/envs/ds/lib/python3.8/site-packages/pydantic
python version: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0]
platform: Linux-5.15.0-58-generic-x86_64-with-glibc2.17
optional deps. installed: ['dotenv', 'typing-extensions']
.model_dump() and .model_dump_json()model_construct(), pickling, private attributes, ORM modeMy use case is I'm reading a toml config file and passing what is read directly to the model constructor. However, I have no way to allow users env var or .env overrides.
Essentially, configs persisted to disk cannot benefit from BaseSettings.
Currently, there is a logic for parsing env variable values by json.loads whenever a field is complex.
We may can move the parsing to Pydantic by using some hook to automatically insert the JSON validator in the core schema if field type is dict, list, set, ...
Here is the suggested code by @samuelcolvin:
@classmethod
def __get_pydantic_core_schema__(cls, source: Type[Any], handler: Callable[[Any], core_schema.CoreSchema]) -> core_schema.CoreSchema:
class WalkCoreSchemaIgnoreJson(WalkCoreSchema):
def handle_json_schema(self, schema: core_schema.JsonSchema) -> core_schema.CoreSchema:
return schema
def insert_json_decoding(s: core_schema.CoreSchema) -> core_schema.CoreSchema:
if s['type'] in ('list', 'dict'):
return core_schema.json_schema(s)
return s
schema = handler(source)
return WalkCoreSchemaIgnoreJson(insert_json_decoding, apply_before_recurse=False).walk(schema)main branch, or equivalentBug1οΌ
There is a Settings class globally. This parent class inherits from BaseSettings. This class does almost only one thing, loading configurations according to the custom level. However, in subclasses that inherit this class, instantiation will report the following error
global_settings = GlobalSettings()
File "pydantic\env_settings.py", line 39, in pydantic.env_settings.BaseSettings.__init__
File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GlobalSettings
__root__
__main__.__spec__ is None (type=value_error)
I don't use main.spec in the class, and main.spec is also illegal in python member variables
Bug2οΌ
According to the guidance of the official website, the top-level configuration class in the project overloaded the customize_sources method and modified it with classmethod. I need to load the configuration from apollo, so I wrote an apollo parsing function to read the configuration of apollo and return it in the form of a dictionary, but if I donβt set the default value, an error of missing variables will be reported, which means that the apollo configuration is not not in effect
Involving business code, it is not convenient to copypython 3.8
pydantic 1.10.7
os window10
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.9.1
pydantic compiled: True
install path: /home/nameerpk/.local/share/virtualenvs/pydantic-BGFCE35f/lib/python3.10/site-packages/pydantic
python version: 3.10.1 (main, Apr 24 2022, 17:03:48) [GCC 9.3.0]
platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
optional deps. installed: ['typing-extensions']
Consider the yaml file with the keys Greeting and User. I'm trying to create my settings from this source.yml file:
Greeting: Hello
Person: npkThe settings has keys greeting and user. This is the file I wrote for settings:
import yaml
from pydantic import BaseSettings, Field
class Settings(BaseSettings):
greeting: str = Field("hello", env="Greeting")
person: str = Field(..., env="Person")
class Config:
@classmethod
def customise_sources(cls, init_settings, env_settings, file_secret_settings):
return (yaml_settings,)
def yaml_settings(settings):
features_file = "./source.yml"
with open(features_file, "r") as stream:
features = yaml.safe_load(stream)
return features
print(Settings())Since the keys in yaml file and the model are different, I have passed the env argument into Field. But running this breaks with this output:
File "pydantic/env_settings.py", line 38, in pydantic.env_settings.BaseSettings.__init__
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 3 validation errors for Settings
person
field required (type=value_error.missing)
Greeting
extra fields not permitted (type=value_error.extra)
Person
extra fields not permitted (type=value_error.extra)
But it works when I use alias instead of env, and it gives this output with a warning:
FutureWarning: aliases are no longer used by BaseSettings to define which environment variables to read. Instead use the "env" field setting. See https://pydantic-docs.helpmanual.io/usage/settings/#environment-variable-names
class Settings(BaseSettings):
greeting='Hello' person='npk'
Even though it says to use env instead of alias, only alias is working here.
Hi,
experience a strange behavior when using the 'prefix'-option in settings. I cannot get it working with inheritance.
With the code below, this .env-file works while the one further down does not work, although it uses the prefix in the variable names. Do I miss anything? From the docs, it doesn't seem too complex to use.
example.env (working):
PARENT_SETTING="prefix_parent"
CHILD_SETTING=42
example.env (not working):
PREFIX_PARENT_SETTING="prefix_parent"
PREFIX_CHILD_SETTING=42
results in:
pydantic.error_wrappers.ValidationError: 2 validation errors for ChildSettings
parent_setting
field required (type=value_error.missing)
child_setting
field required (type=value_error.missing)
from pydantic import BaseSettings, Field
class ParentSettings(BaseSettings):
parent_setting: str = Field(env="PARENT_SETTING")
class ChildSettings(ParentSettings):
child_setting: int = Field(env="CHILD_SETTING")
class Config:
env_prefix = "PREFIX_"
case_sensitive = False
env_file = "example.env"
allow_reuse = True
if __name__ == '__main__':
settings = ChildSettings(_env_file="example.env")
print(settings.json(indent=2))Windows 10
pydantic-1.10.9
.dict() and .json()construct(), pickling, private attributes, ORM modeSelected Assignee: @hramezani
Selected Assignee: @samuelcolvin
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google β€οΈ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.