sergiobenitez / figment Goto Github PK
View Code? Open in Web Editor NEWA hierarchical configuration library so con-free, it's unreal.
License: Apache License 2.0
A hierarchical configuration library so con-free, it's unreal.
License: Apache License 2.0
Hello, I noticed that environment variables like these are converted to an integer:
SECURE_PASSWORD="123456"
As I'm expecting for a string, I get an error in the config parsing.
I thought that putting "
around the number would be enough to make it be considered as a string.
# Parsed as number. Shouldn't it be considered as string?
SECURE_PASSWORD="123456"
# This one is considered as a number, which makes sense
SECURE_NUMBER=123456
Currently Figment requires a couple of lines of code to get started. This includes the config structure (which is reasonable) as well as setting up the extractors. I suggest an alternative interface to generate the configuration by writing a one liner, based on cargo features
A basic developer writing a binary crate called foo
writes the following code to extract the Config
struct from a toml and environment (import statements elided)
// main.rs
// Impl Serialize and deserialize
struct Config { /**/ }
let config: Config = Figment::new()
.merge(Toml::file("Foo.toml"))
.merge(Env::prefixed("FOO_"))
.extract()?;
# Cargo.toml
[dependencies]
figment = { version = "*", features = ["toml", "env"] }
If the developer later decides to enable yaml
support, the following changes are required
// main.rs
// Impl Serialize and deserialize
struct Config { /**/ }
let config: Config = Figment::new()
.merge(Toml::file("Foo.toml"))
+ .merge(Yaml::file("Foo.yaml")
+ .merge(Yaml::file("Foo.yml")
.merge(Env::prefixed("FOO_"))
.extract()?;
# Cargo.toml
[dependencies]
- figment = { version = "*", features = ["toml", "env"] }
+ figment = { version = "*", features = ["toml", "yaml", "env"] }
Notice how the developer is required to add both .yaml
and yml
files, because the yaml spec allows for both file extensions. This could be improved using a little more rusty magic.
Provide an auto
method on Figment
that automatically sets up all Figment extractors based on currently enabled features
// figment.rs, inside the impl block
pub fn auto(name:&str) -> Figment {
let names = [name, name.to_upper(), name.to_lower(), name.to_upper_first()];
let mut figment = Figment::new();
for name in names {
#[cfg(feature = json)
figment = figment.merge(Json::file(format!("{name}.json")));
#[cfg(feature = toml)
figment = figment.merge(Toml::file(format!("{name}.toml")));
#[cfg(feature = yaml)
figment = figment.merge(Yaml::file(format!("{name}.yml"))).merge(Yaml::file(format!("{name}.yaml")));
// ...
}
// env is the last one always
#[cfg(feature = env)
for name in names {
figment.merge(Env::prefixed(format!("{}_", name.to_upper()));
}
figment
}
// main.rs
// Impl Serialize and deserialize
struct Config { /**/ }
// Still allows for extra customisation, if wanted
let config: Config = Figment::auto("foo").extract()?;
# Cargo.toml
[dependencies]
figment = { version = "*", features = ["toml", "env"] }
Adding yaml support is as easy as editing the Cargo.toml
. Just include the yaml
feature and it will automatically add all variants without modifying main.rs
# Cargo.toml
[dependencies]
- figment = { version = "*", features = ["toml", "env"] }
+ figment = { version = "*", features = ["toml", "env", "yaml"] }
This could be further enhanced by providing a macro that automatically uses the crate name as the config name, as well as offer some degree of configuration with varargs.
// Use crate name as `name` parameter
let config: Config = figment::auto!().extract()?;
// Use custom name
let config: Config = figment::auto!(name="foo").extract()?;
// Override priority order
let config: Config = figment::auto!(order=[env,toml]).extract()?;
json
files over yaml
.cargo add
, write the scheme and let config: Config = figment::auto()
.yaml
, .yml
, uppercase, lowercase, title...) reduces chances of debugging typosFigment::new()
can still be usedFigment::auto()
returns a Figment
struct, which can be further chained with more extractorsWhen reading from environment variables, for secrets, it might be beneficial to have variables of type PASSWORD_FILE
that points to a file that contains the value for PASSWORD
.
E.g.:
$ ./my_app --password=abc
Got password: abc
$ PASSWORD=abc ./my_app
Got password: abc
$ PASSWORD_FILE=/tmp/password ./my_app
Got password: abc
$ cat /tmp/password
abc
I think this could be implemented as a separate Provider
. WDYT?
Related: clap-rs/clap#4013
I'm curious if there's any support or way built in to write out values to a config file?
I'd like to write something similar to how git config
works, where you can do
`git config --global A.B=C"
and it writes that value out to the global git config. Does anything close to that exist, or will I have to roll my own?
Hi,
First of all: I just recently started using Figment and really like it - thanks for your work :)
I did not really know where the best place to ask the following question would be, therefore I decided to create an issue.
However of course feel free to point me in another direction and close the issue, if this is the wrong place to ask questions ;)
Now to my question:
I am currently struggling with merging multiple configuration sources for a nested configuration.
Here's a simplified example, to illustrate my setup:
I have a toml file structured like this:
[config section A]
sectionAOption1 = 'abc'
sectionAOption2 = 'xyz'
[config section B]
sectionBOption1 = 'foo'
#[derive(Deserialize, Serialize, Clone, Default, Debug)]
pub struct Config {
pub sectionA: SectionAConfig,
pub sectionB: SectionBConfig,
}
#[derive(Deserialize, Serialize, Clone, Default, Debug)]
struct SectionAConfig {
section_a_option_1: String
section_b_option_2: String
}
#[derive(Deserialize, Serialize, Clone, Default, Debug)]
struct SectionBConfig {
section_b_option_1: String
}
#[derive(Parser)]
#[command(author, version, about, long_about = None)]
struct Cli {
// Other args shared with some sub commands
#[command(flatten)]
common_args: CommonArgs,
// Some configuration options which are also available as arguments
#[command(flatten)]
some_config_options: SomeConfigOptions,
#[command(subcommand)]
command: Option<Commands>,
}
#[derive(clap::Args)]
struct SomeConfigOptions {
#[arg(long, short = 's', default_value = "default")]
section_a_option_1: String
}
My struct provides my configuration defaults. Then the toml file takes priority and clap arguments have the highest priority.
What would be the best/simplest way to merge these CLI-arguments with my other sources? As shown above I can't just use my whole CLI-configuration as a source, because a) it has other arguments and commands as well and b) its structure is different to my actual configuration structure. I thought of implementing From<SomeConfigOptions>
on my Config
, however then I would need to provide default values to the fields missing from my clap args and these would then override parts of my configuration from e.g. the toml file. Another way I thought of would be to create some kind of DTO struct which imitates the structure of my config, so that the nesting would be correct. But that would mean I have to write such a struct for every optional argument and that would feel really hacky. Or do I need to write a provider? But even if so, how would that look like so that it solves my problem?
I feel like I am missing something here and am probably overthinking the whole thing. But I just can't figure out what a good way to do this would be.
Thanks in advance :)
EDIT: Hm ๐ค one way I could think of, is to manually put the values from the arguments into nested HashMap
s - that should probably work, but isn't really pretty either...
It would be nice for debugging if there was a way to dump the final unified confguration to a text file or json blob.
I use INI.
(If you're wondering why, I prefer to spare my (often non-technical) users from having to worry about significant whitespace (YAML), syntax errors and escape characters (JSON, TOML, XML, YAML), nesting errors (XML, YAML) โ and I want to provide in-line documentation via comments (not in JSON).)
Is this abandoned?
Consider my main .rs file:
use figment::{Figment, providers::{Serialized}};
use structopt::StructOpt;
mod config;
use config::Config;
fn main() {
let cli_config = Config::from_args();
println!("{:#?}", cli_config);
let app_config: Config = Figment::from(Serialized::defaults(Config::default())) /// theme is "glorious"
.merge(Serialized::from(cli_config, "default"))
.extract().unwrap();
println!("{:#?}", app_config); /// theme becomes None if --theme is not set
}
Now consider my default implementation in the config.rs file:
use serde::{Deserialize, Serialize};
use structopt::StructOpt;
#[derive(Debug, Deserialize, Serialize, StructOpt)]
#[structopt(name = "xwebgreet")]
pub struct Config {
#[structopt(short, long)]
theme: Option<String>,
}
impl Default for Config {
fn default() -> Config {
Config {
theme: Some("glorious".to_string()),
}
}
}
The issue is when I run the program and do not pass --theme
as argument then cli_config has theme as None
, and when merging it changes the value of theme to None
even though a default value exists.
Can I somehow prevent this behavior ? When I have an Option type variable and if it's value is None
, then while merging I don't want the value to be changed to None
if it already exists.
I have a little problem with deserializing an enum. If you use the following code
use figment::{Figment, providers::{Format, Toml, Serialized, Env}};
use serde::{Deserialize, Serialize};
fn main() {
let figment = Figment::from(Serialized::defaults(Test::default()))
.merge(Toml::file("Test.toml"))
.merge(Env::prefixed("TEST_"));
println!("{:#?}", figment.extract::<Test>().unwrap());
}
#[derive(Debug, Deserialize, Serialize)]
pub struct Test {
service: Option<Foo>
}
impl Default for Test {
fn default() -> Self {
Test {
service: None
}
}
}
#[derive(Debug, Deserialize, Serialize)]
pub enum Foo {
Mega,
Supa
}
with this as content for Test.toml
service = "Mega"
i get the following error.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { tag: Tag(Default, 2), profile: Some(Profile(Uncased { string: "default" })), metadata: Some(Metadata { name: "TOML file", source: Some(File(".../Test.toml")), interpolater: }), path: ["service"], kind: InvalidType(Str("Mega"), "enum Foo"), prev: None }', src\main.rs:10:49
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
It looks like the Mega
from the toml file is recognized as a Str
instead of the expected enum.
Directly using the toml
crate with toml::from_str::<Test>(r#"service = "Mega""#)
results in a successful deserialization.
Hey,
first of all, many thanks for maintaining this library. It is truly of great help for developing configurable applications.
My issue is as follows. In my application I prefer to use nested structs for my configuration, for example:
struct Config {
log: Log,
}
struct Log {
format: LogFormat,
level: LogLevel,
}
Since I want stuff to be configurable via environment variables, I use Env::prefixed
along with split
to handle nestings correctly. So APP_LOG_FORMAT
works.
However, things start to get messy when I add snake_case fields to the mix:
struct Config {
node_identifier: String
}
In this case, Figment is unable to correctly resolve APP_NODE_IDENTIFIER
as it assumes a node.identifier
nesting when employing split
. Thus, I have to choose between "nesting with no multi-word/snake_case field names" and "snake_case but no nesting".
Is there any way to use environment variables, nested structs and snake_case field names at the same time, out-of-the-box?
Bests,
Attila
Hey!
Wondering if it would be possible to add support for ambiguous paths. Let's say I want to split by _
but I also have a nested variable that happens to also include _
.
struct Config {
foo: Foo
}
struct Foo {
a_b_c: String
}
An environment variable of FOO_A_B_C=hi
doesn't resolve correctly, even though I would expect it to, with the following config:
Figment::new()
.merge(Env::split("_"))
.extract()
Is this something that is within scope for this project?
Currently when you have a vector in your config the merge option always replaces the vectors.
I would love an option to append the vectorsfrom the different sources (including default) into one big vector.
Is it possible to do this with figment?
If so, where should I be looking?
Hello.
I am using Rocket which depends on this crate and it fails to compile with the armv5te-unknown-linux-musleabi
target, because AtomicU64 is not supported and unconditionally used in value/tag.rs.
The error is:
2 | use std::sync::atomic::{Ordering, AtomicU64};
| ^^^^^^^^^
| |
| no `AtomicU64` in `sync::atomic`
| help: a similar name exists in the module: `AtomicU8`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0432`.
error: could not compile `figment`
Small feature request. We are using YAML as our config format, and we would like to use camel cased field names instead of snake case, but still use snake case in our Rust structs.
So this:
fooBar: 'abc'
Would automatically map to this:
pub struct Test {
pub foo_bar: String,
}
Ideally this just happens automatically, and is transparent to both the user and developer. This also seems like a nicer approach to littering #[allow(non_camel_case_types)]
everywhere.
It looks like you bumped the serde_yaml
dependency in 9001673 with no further changes. Would it be possible to widen the range in the Cargo.toml
to allow 0.8
and 0.9
. The problem with version 0.9 is, it is tricky to use with Kubernetes and other resources (dtolnay/serde-yaml#298). Which means quite a few other libs are still stuck with 0.8 for now.
I would like to avoid having to compile two versions of serde_yaml
.
A version range like >= 0.8 < 0.10
should work.
Is there a way for Figment to read in a .env file and populate the config that way? The Env provider seems to only read in environment variables already loaded into the environment. Do I need to use another crate like dot_env to do so? Thanks.
I have a test db on my local machine with a password 111111
. I set up environment variables. This is my Figment config:
Figment::new()
.merge(providers::Env::prefixed("PLOG_"))
.merge(providers::Toml::file("plog.config.toml"))
.extract()
This is Config
:
#[derive(Deserialize)]
pub struct Config {
log_level: Option<String>,
pub host: Option<String>,
pub port: Option<u16>,
pub db_host: Option<String>,
pub db_port: Option<u16>,
pub db_user: String,
pub db_password: String,
}
As you can see, db_password
is type String
, but I have an error saying this:
Error: invalid type: found unsigned int `111111`, expected a string for key "DB_PASSWORD" in `PLOG_` environment variable(s)
Thanks for writing figment!
I'm investigating it as a replacement to an existing application-specific module which implements nesting on top of the config
crate (and is a bit unwieldy). In this particular scenario, I'd like to be able to detect when a user has typo'ed a setting or profile name.
As far as I can see, I can reasonably easily catch typos and similar within field names of user-supplied tomls etc. by setting annotating the target structs with #[serde(deny_unknown_fields)]
, e.g.
I'd also be able to tell the user something like:
"You seem to have chosen a profile debugging
, which has not been defined. Currently defined profiles are default
, debug
, remote-debug
."
but looking at the figment the example code:
// Selecting non-existent profiles is okay as long as we have defaults.
let config: Config = Config::figment().select("undefined").extract()?;
... and browsing around the source, I can't really see a way to differentiate select()
ed profile which has wholley inherited from defaults vs. one which actually exists in the config sources?
I think the best I can do, is warn the user if they've manually selected a profile which is equal to the default profile.
I'd be interested to hear any thoughts on this...
Thanks!
As the title states. If I have this struct that implements Default
:
#[derive(Clone, Debug, Deserialize, PartialEq, Serialize, Validate)]
struct Foo {
field: Option<String>
}
impl Default for Foo {
fn default() -> Self {
Foo {
field: Some(String::from("default"))
}
}
}
The default()
isn't called for Foo
struct when it's contained within a HashMap
, like so:
#[derive(Clone, Debug, Deserialize, PartialEq, Serialize, Validate)]
struct Config {
#[validate]
foos: Option<HashMap<String, Foo>>,
}
This results in all my Foo
s having a None
value for field
, when I want a String no matter what.
First of all, thank you very much for the library. It has already saved me a lot of time. Now my questions: I want to list the sources of all settings, similar to the output of git config --list --show-origin
. Is there a way to iterate over all values in the figment? Also I'd like to be able to set config settings on different levels, much like --global
and --local
in git. What is the easiest way to do this, keeping the figment up-to-date at the same time? Ideally I'd want to be able to modify the value on the figment and serialize back to the source files depending on the location metadata.
I'm not sure if this is a bug or intentional, but serde and figment behave differently when deserializing single value structs:
use figment::{Figment, providers::{Toml, Format}};
use serde::Deserialize;
#[derive(Copy, Clone, Deserialize, Debug)]
struct Foo(pub u64);
#[derive(Deserialize, Debug)]
struct Config {
foo: Foo
}
const CONFIG: &str = "foo = 42";
fn main() {
// this works
let config: Result<Config,_> = toml::from_str(CONFIG);
println!("Serde: {config:?}");
// this fails
let config: Result<Config, _> = Figment::from(Toml::string(CONFIG)).extract();
println!("Figment: {config:?}");
}
There are two ways to make this example work with figment:
Foo
with #[serde(transparent)]
, but afaik that is only possible if you own the type"foo = [42]"
, but this causes the serde example to fail.Note that this is not limited to TOML, but also happens with other providers, e.g. JSON.
Hi,
I really appreciate the work you've put into Figment! It has saved me so much time already when building out config files for applications.
I'm having a small issue with a list of nested objects picking up their defaults. I'm reading values from a TOML file and providing defaults via the Default impl. However when I provide partial values for my nested objects, I'm getting a parsing error indicating fields are missing.
Config structure here:
#[derive(Debug, Deserialize, Serialize)]
pub struct ListenerConfig {
pub address: String,
pub timeout: u32,
}
impl Default for ListenerConfig {
...
}
#[derive(Debug, Deserialize, Serialize)]
pub struct Config {
pub listeners: Vec<ListenerConfig>,
pub storage_path: String,
}
impl Default for Config {
...
}
Parsing logic:
impl Config {
pub fn parse(path: Option<String>) -> Result<Self> {
let mut config = Figment::new().merge(Serialized::defaults(Config::default()));
config = config.merge(Serialized::defaults(ListenerConfig::default()));
if let Some(path) = path {
config = config.merge(Toml::file(path));
}
Ok(config.extract()?)
}
}
If I parse a TOML file with an incomplete ListenerConfig
, such as
[[listeners]]
address = "127.0.0.1:8088"
then I'll get a parsing error that the timeout
field is missing. I suspect I'm missing something about how to correctly merge/join nested values, but I'm not sure what to try next. Any help or pointers would be appreciated!
Repo with all code for reproducing error is here https://github.com/plauche/figment-nested-defaults-repro!
I have the following config:
use figment::{providers::Env, Figment};
use serde::Deserialize;
#[derive(Deserialize, Debug, Clone)]
pub struct DatabaseConfig {
pub url: String,
}
#[derive(Deserialize, Debug, Clone)]
pub struct OpenlibraryConfig {
#[serde(default = "OpenlibraryConfig::url")]
pub url: String,
}
impl OpenlibraryConfig {
pub fn url() -> String {
"https://openlibrary.org".to_owned()
}
}
#[derive(Deserialize, Debug, Clone)]
pub struct BookConfig {
pub openlibrary: OpenlibraryConfig,
}
#[derive(Deserialize, Debug, Clone)]
pub struct AppConfig {
pub book: BookConfig,
}
/// Get the figment configuration that is used across the apps.
pub fn get_figment_config() -> Figment {
Figment::new().merge(Env::prefixed("APP_").split("__"))
}
let config: AppConfig = get_figment_config().extract()?;
For the above code, I get the error:
Error: Error { tag: Tag::Default, profile: Some(Profile(Uncased { string: "default" })), metadata: None, path: [], kind: MissingField("book"), prev: None }
However, if I have APP_BOOK__OPENLIBRARY__URL="https://openlibrary.org"
in my environment, it works fine.
I have this in Cargo.toml
:
figment = { version = "0.10.8", features = ["env"] }
I feel kinda dumb for having to ask, but if I extract a struct with a RelativePathBuf
, how do I convert that into an absolute path?
It would be nice to be able to streamline clap integration so that a figment can produce ArgMatches so the Clap-required fields in a Command can be satisfied.
Figment works great for loading, parsing, and assigning the values to Rust structs. Currently, it also does some validation (as part of serde) for stuff like invalid type, invalid value, etc (see errors::Kind
).
It would be nice if Figment supported custom validation rules that we could easily apply to our fields, which in turn would be handled by Figment's built in error. Maybe a new enum variant like Kind::InvalidField(fieldName, errorMessage)
? Could be applied using field attributes:
struct Foo {
#[figment(validate = "validate_range")]
field: i64;
}
fn validate_range(value: i64) -> Result<(), Error> {}
The reason I'm asking for this, is that it's currently a lot of overhead to integrate something like validator
or another serde validation lib, because of the difference in error's that are used. I'm currently having to do a lot of mapping between errors, and for the errors to be returned as a collection (instead of 1-by-1). It would also be nice for the validation to be ran in the same flow that figment uses, instead of being done in a secondary flow manually.
Also open to any other solutions out there.
I am using Figment with Rocket and have encountered some rather odd behavior with join
. Join is not supposed to replace a value if it already exists in that key, but I have found a case where it does.
#[database("rockpass")]
pub struct RockpassDatabase(diesel::SqliteConnection);
async fn database_migrations(rocket: Rocket<Build>) -> Rocket<Build> {
embed_migrations!();
let connection = RockpassDatabase::get_one(&rocket).await.expect("database connection");
connection.run(|c| embedded_migrations::run(c)).await.expect("diesel migrations");
rocket
}
#[derive(Deserialize, Serialize)]
#[serde(crate = "rocket::serde")]
pub struct RockpassConfig {
registration_enabled: bool,
access_token_lifetime: i64,
refresh_token_lifetime: i64
}
impl Default for RockpassConfig {
fn default() -> RockpassConfig {
RockpassConfig {
registration_enabled: true,
access_token_lifetime: 3600,
refresh_token_lifetime: 2592000
}
}
}
#[launch]
fn rocket() -> _ {
let figment = Figment::from(rocket::Config::default())
.merge(Serialized::defaults(RockpassConfig::default()))
.merge(Toml::file("/etc/rockpass.toml").nested())
.merge(Toml::file("rockpass.toml").nested())
.merge(Env::prefixed("ROCKPASS_").global())
.select(Profile::from_env_or("ROCKPASS_PROFILE", "release"));
println!("{:?}", figment.extract_inner::<String>("databases.rockpass.url"));
let figment = figment.join(("databases.rockpass.url", ":memory:"));
println!("{:?}", figment.extract_inner::<String>("databases.rockpass.url"));
[release]
[release.databases.rockpass]
url = "/tmp/rockpass_release.sqlite"
The first print prints: Ok("/tmp/rockpass_devel.sqlite")
But the second print prints: Ok(":memory:")
If figment initially contained in that key Ok("/tmp/rockpass_devel.sqlite")
I understand that figment.join
should not overwrite the value.
But the strangest thing is that if when I launch the application I pass the value through an environment variable ROCKPASS_DATABASES='{rockpass = { url = "/tmp/test.sql" }}' cargo run
then figment.join
works as it should:
Ok("/tmp/test.sql")
Ok("/tmp/test.sql")
Is it just me doing (or guessing) something wrong or is there really a bug here?
I have the following configuration:
pub fn get_figment_config() -> Figment {
Figment::new().merge(Env::prefixed("APP_").split("__"))
}
#[derive(Deserialize, Debug, Clone)]
struct Config {
pub name: Vector<String>
}
And I have the environment variable APP_NAME="name1,name2,name3"
, but this does not work and I get this error:
Error: Error { tag: Tag(Default, 1), profile: Some(Profile(Uncased { string: "default" })), metadata: Some(Metadata { name: "`APP_` environment variable(s)", source: None, provide_location: Some(Location { file: "crates/utilities/src/lib.rs", line: 54, col: 20 }), interpolater: }), path: ["background", "mailer", "data"], kind: InvalidType(Str("Dew1,Deb2"), "a sequence"), prev: None }
Note: The paths may be a bit different, I just put a random struct in the example.
The figment::coalesce::Coalescible
trait seems very helpful when writing providers, any chance to make that pub
?
When trying to configure rocket through environment variables, I am currently running into a bug with respect to toml string parsing within environment variables.
When setting an environment variable to
{key="value1\nvalue2"}
per the toml spec, the \n should be interpreted as an escape sequence. However, rocket does not, instead keeping the \n as two separate characters without change.
create new rocket project with
use rocket::{get, launch, routes, State, fairing::AdHoc};
use serde::Deserialize;
#[derive(Debug, Deserialize)]
struct SubConfig {
key: String,
}
#[derive(Debug, Deserialize)]
struct Config {
sub: SubConfig,
}
#[get("/")]
fn index(config: State<Config>) -> &'static str {
println!("{:?}", &config.sub);
"Hello, world!"
}
#[launch]
fn rocket() -> _ {
rocket::build().mount("/", routes![index]).attach(AdHoc::config::<Config>())
}
as main source file, adding serde to the dependencies with the derive feature enabled.
then start the project from bash with
ROCKET_SUB={key=\"value1\\nvalue2\"} cargo run
and query localhost:8000 to see printed in the console
SubConfig { key: "value1\\nvalue2" }
instead of the expected output:
SubConfig { key: "value1\nvalue2" }
Note: the extra slash in the commandline is correct, as can be seen by running
ROCKET_SUB={key=\"value1\\nvalue2\"}
echo "${ROCKET_SUB}"
Rocket version: e1307ddf48dac14e6a37e526098732327bcb86f0
OS: Ubuntu 20.04
I'm losing my mind with this issue, please check out and tell if I did something dumb. This code should work:
pub fn load_data(r: &Rocket) {
let f = r.figment();
let profile = f.profile().to_string();
let db_url: String = f.extract_inner("databases.kittybox.url").unwrap();
println!("{:#?}: {:#?}", profile, db_url);
println!("{:#?}", f);
}
Output looks like this:
"release": "postgres://kitty:hackme@localhost:5432/kittybox"
Figment {
profile: Profile(
Uncased {
string: "release",
},
),
metadata: {
Tag(Default, 1): Metadata {
name: "Rocket Config",
source: Some(
Code(
Location {
file: "/home/kittyandrew/.cargo/registry/src/github.com-1ecc6299db9ec823/figment-0.9.4/src/figment.rs",
line: 112,
col: 24,
},
),
),
interpolater: ,
},
Tag(Default, 2): Metadata {
name: "TOML file",
source: Some(
File(
"/home/kittyandrew/dev/Kitty-API/Rocket.toml",
),
),
interpolater: ,
},
Tag(Default, 3): Metadata {
name: "environment variable(s)",
source: Some(
Code(
Location {
file: "/home/kittyandrew/.cargo/git/checkouts/rocket-8bf16d9ca7e90bdc/1f1f44f/core/lib/src/config/config.rs",
line: 194,
col: 14,
},
),
),
interpolater: ,
},
Tag(Default, 5): Metadata {
name: "Default",
source: Some(
Code(
Location {
file: "/home/kittyandrew/.cargo/registry/src/github.com-1ecc6299db9ec823/figment-0.9.4/src/figment.rs",
line: 112,
col: 24,
},
),
),
interpolater: ,
},
},
value: Ok(
{
Profile(
Uncased {
string: "default",
},
): {
"address": String(
Tag(Default, 1),
"127.0.0.1",
),
"cli_colors": Bool(
Tag(Default, 1),
true,
),
"ctrlc": Bool(
Tag(Default, 1),
true,
),
"keep_alive": Num(
Tag(Default, 1),
U32(
5,
),
),
"limits": Dict(
Tag(Default, 1),
{
"forms": Num(
Tag(Default, 1),
U64(
32768,
),
),
},
),
"log_level": String(
Tag(Default, 1),
"critical",
),
"port": Num(
Tag(Default, 1),
U16(
8000,
),
),
"secret_key": Array(
Tag(Default, 1),
[
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
Num(
Tag(Default, 1),
U8(
0,
),
),
],
),
"tls": Empty(
Tag(Default, 1),
None,
),
"workers": Num(
Tag(Default, 1),
U16(
16,
),
),
},
Profile(
Uncased {
string: "global",
},
): {
"address": String(
Tag(Global, 2),
"0.0.0.0",
),
"databases": Dict(
Tag(Global, 2),
{
"kittybox": Dict(
Tag(Global, 2),
{
"url": String(
Tag(Global, 2),
"postgres://kitty:hackme@localhost:5432/kittybox",
),
},
),
},
),
"template_dir": String(
Tag(Global, 2),
"templates/",
),
},
Profile(
Uncased {
string: "release",
},
): {
"databases": Dict(
Tag(Custom, 2),
{
"kittybox": Dict(
Tag(Custom, 2),
{
"url": String(
Tag(Custom, 2),
"postgres://kitty:hackme@kitty-api-db:5432/kittybox",
),
},
),
},
),
"port": Num(
Tag(Custom, 2),
I64(
8080,
),
),
},
},
),
}
When running cargo run --release
, if you look at the value (with "release"
profile), it's different from returned one. I've tried to access .port
value, and it worked fine.
When using multiple providers in which different names for the same parameter are used, I am trying to use serde's #[serde(alias = "something")]
directive. The issue I have is that when you try to merge two providers, one using the serde alias and one with the rust name, you get a duplicate field error. With each provider on its own they work.
use figment::{ Figment, providers::{Env, Format, Serialized, Toml} };
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub struct Config {
pub a: String,
#[serde(alias = "other")]
pub b: String
}
figment::Jail::expect_with(|jail| {
jail.set_env("TEST_A", "first");
jail.set_env("TEST_OTHER", "second");
jail.create_file("test.toml", r#"
a = "test1"
b = "test2"
"#)?;
let config: Result<Config, figment::Error> = Figment::new()
.merge(Toml::file("test.toml"))
.merge(Env::prefixed("TEST_"))
.extract();
println!("{:?}", config);
Ok(())
});
running this code produces
Err(Error { tag: Tag::Default, profile: Some(Profile(Uncased { string: "default" })), metadata: None, path: [], kind: DuplicateField("b"), prev: None })
if you only merge the Toml file it produces:
Ok(Config { a: "test1", b: "test2" })
if you only merge the environment variables it produces:
Ok(Config { a: "first", b: "second" })
Hi, I am relatively new to rust and am exploring cli and config parsing.
Just wanted to here your thoughts on what is the best way to interface with a command line parser such as https://github.com/TeXitoi/structopt ?
I want a typical linux cli use case where configuration options can be set by config files and cli options with any cli option overriding existing config values.
Figment does not appear to find environment variables for structs which are renamed to kebab-case. Here is code to reproduce the issue:
use figment::{
providers::{Env, Format, Toml},
Figment,
};
use serde::Deserialize;
#[derive(Deserialize)]
#[serde(rename_all = "kebab-case")]
struct Greet {
pub user_name: String,
}
#[derive(Deserialize)]
#[serde(rename_all = "kebab-case")]
struct Config {
pub greet: Greet,
}
fn main() {
let config: Config = Figment::new()
.merge(Toml::file("config.toml")) // behaviour still happens even if I remove this line
.merge(Env::prefixed("TEST_").split("_"))
.extract()
.expect("Failed to get config");
println!("Hello, {}!", config.greet.user_name);
}
To run it, I suggest:
TEST_GREET_USER_NAME=world cargo run
The program crashes with the following error:
thread 'main' panicked at 'Failed to get config: Error { tag: Tag(Default, 1), profile: Some(Profile(Uncased { string: "default" })), metadata: Some(Metadata { name: "`TEST_` environment variable(s)", source: None, provide_location: Some(Location { file: "src/main.rs", line: 21, col: 10 }), interpolater: }), path: ["greet"], kind: MissingField("user-name"), prev: None }', src/main.rs:23:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The program runs and prints
Hello, world!
I want to conditionally merge a provider based on the current profile, and I'm wondering if there's already a nice way to handle this.
Right now, I'm doing it as follows:
let mut app_config_figment = Figment::new()
.merge(Toml::file("./src/config/base.toml"))
.merge(Toml::file("./src/config/production.toml").profile("production"));
let maybe_profile = Profile::from_env_or("environment_type", "default");
app_config_figment = app_config_figment.select(maybe_profile);
if app_config_figment.profile() == "production" {
app_config_figment = app_config_figment.merge(Env::prefixed("COREOSION_PRODUCTION_").split("_"));
}
Add in a feature to integrate configuration with crates which provide standard directories to store config files, such as directories
or platform-dirs
?
Would enable even simpler config management at the cost of more dependencies, hence make into an optional feature.
I can implement it myself, but wanted to get the go-ahead from the project maintainer.
I would like to prioritise clap arguments over values from environment variables:
let args: DbArgs = Figment::new()
.merge(Serialized::defaults(args))
.join(Env::prefixed("DB_")) // clap arguments take precedence.
.extract()
.expect("error parsing environment for config");
Here is the struct used for clap. Option is used so that database_url
is not required to be passed though CLI:
#[derive(Parser, Debug, Serialize, Deserialize)]
struct DbArgs {
#[arg(long, default_value_t = 15)]
connect_timeout_ms: u64,
#[arg(long)]
database_url: Option<String>,
}
However, this does not work as Figment treats the None
value for Option<String>
to be present and does not overwrite the value from environment variable when using join
.
The example in the docs works, but this prioritises the environment variables over those specified by clap.
let args: DbArgs = Figment::new()
.merge(Env::prefixed("DB_")) // Environment variables take precedence.
.join(Serialized::defaults(args))
.extract()
.expect("error parsing environment for config");
Would it be possible to make it such that None
is treated as empty and can be overwritten by join
?
I'm building an application (atop Rocket) with three access modes:
So we've (really @132ikl) built up a configuration structure that looks something like this:
#[derive(Serialize, Deserialize, Clone)]
struct Config {
default_access: Access,
mode: Mode,
}
#[derive(Serialize, Deserialize, Clone)]
enum Mode {
Public,
Privileged((Vec<String>, Access)),
Private(Vec<String>),
}
/// In the Privileged case, this is probably more lax than the default_access
#[derive(Serialize, Deserialize, Clone)]
struct Access {
max_size: u64,
force_something: bool,
}
impl Default for Config {
fn default() -> Self {
Self {
default_access: Access {
max_size: 4096,
force_something: true,
},
mode: Mode::Private(vec!["memes lol".into()]),
}
}
}
So far so good. However, this appears to get... mangled when put into a figment. Here's how serde_json
serializes the default value, for comparison:
{"default_access":{"max_size":4096,"force_something":true},"mode":{"Private":["memes lol"]}}
Here, mode
's variant is indicated as you would expect for an enum
by default in Serde.
However, if we put the same structure into a Figment
with Figment::new().merge(Serialized.defaults(Config::default()))
, we don't seem to get the same tagging:
// from dbg!(figment)
value: Ok(
{
Profile(
Uncased {
string: "default",
},
): {
"default_access": Dict(
Tag(Default, 1),
{
"force_something": Bool(
Tag(Default, 1),
true,
),
"max_size": Num(
Tag(Default, 1),
U64(
4096,
),
),
},
),
"mode": Array( // Where's the enum tag?
Tag(Default, 1),
[
String(
Tag(Default, 1),
"memes lol",
),
],
),
},
},
),
This causes problems for a Rocket application:
#[get("/")]
fn dummy() -> &'static str {
"there's a strange dummy endpoint outside my home\n"
}
#[launch]
fn rocket() -> _ {
let fig =
Figment::from(rocket::Config::default()).merge(Serialized::defaults(Config::default()));
let rocket = rocket::custom(fig)
.mount("/", routes![dummy])
.attach(AdHoc::config::<Config>()); // this fairing fails because...
rocket
}
$ cargo run
Error: Rocket configuration extraction from provider failed.
>> invalid type: found sequence, expected enum Mode
>> for key default.mode
>> in src/main.rs:45:56 figment_broke::Config
Error: Rocket failed to launch due to failing fairings:
>> figment_broke::Config
Hi,
I am newish to Rust and Rocket. I am trying to use Figment with Rocket but it doesn't compile. I am getting this error:
|
= note: see issue #47809 rust-lang/rust#47809 for more information
= help: add#![feature(track_caller)]
to the crate attributes to enableerror[E0658]: the
#[track_caller]
attribute is an experimental feature
--> /home/yg/.cargo/registry/src/github.com-1ecc6299db9ec823/figment-0.9.4/src/figment.rs:176:5
|
176 | #[track_caller]
| ^^^^^^^^^^^^^^^
|
= note: see issue #47809 rust-lang/rust#47809 for more information
= help: add#![feature(track_caller)]
to the crate attributes to enable
error[E0658]: use of unstable library feature 'track_caller': uses #[track_caller] which is not yet stable
--> /home/yg/.cargo/registry/src/github.com-1ecc6299db9ec823/figment-0.9.4/src/providers/serialized.rs:80:18
|
80 | loc: Location::caller()
| ^^^^^^^^^^^^^^^^
This is from my Cargo.toml:
serde = { version = "1.0", features = ["derive"] }
figment = { version = "0.9", features = ["toml", "env"] }
This is my main function:
fn main() {
let port = env::var("PORT").expect("$PORT must be set");
let figment = Figment::new()
.merge(Toml::file("Rocket.toml"))
.merge(Env::prefixed("ROCKET_"));
rocket::custom(figment)
.attach(DbConn::fairing())
.mount("/", routes![
posts::list,
posts::new
]).launch();
}
I added #[macro_use] extern crate figment;
to my main.rs as well but didnt help
Many tools and their config files support the concept of extending additional config files through a field within the file, with the values being a relative file system path (extends: './other/config.yml'
) or URL (extends: 'https://domain.com/config.yml'
). With this approach, the extends
file is applied before the file doing the extending, so that the current document can override or merge when necessary. This is pretty great for composibility and reusability.
However, this seems to be extremely difficult in Figment, and I'm curious on the best way to approach it. The current problems are:
merge()
or join()
because we don't know what file to extend until after the config has been parsed and extracted.I have a working solution to this problem but it requires resolving and parsing the config twice, as demonstrated here: https://github.com/moonrepo/moon/pull/142/files#diff-752c2babea244e138a59a074967c9c3bb0faf51bf0a413ac4128505e2b58fb7fR146 The second figment extraction contains an additional merge()
for the extending file (which is requested via URL).
In an ideal world, something like this would be pretty great.
Figment::from(Serialized::defaults(WorkspaceConfig::default()))
.merge(Yaml::file(&path))
.merge_first(Yaml::from_field("extends"))
.extract()
It seems that when merging different providers, values aren't properly overwritten. Either that, or I'm not understanding properly how precedence works...
As a reproducer, I added the following doctest:
--- a/src/figment.rs
+++ b/src/figment.rs
@@ -42,6 +42,33 @@ use crate::coalesce::{Coalescible, Order};
/// assert_eq!(joined, "replaced");
/// ```
///
+/// This can also be used to set defaults that may be overridden with a config
+/// file or an env variable:
+///
+/// ```rust
+/// use figment::{Figment, providers::{Env, Format, Toml}};
+///
+/// figment::Jail::expect_with(|jail| {
+/// jail.create_file("Config.toml", r#"
+/// foobar = "file"
+/// "#)?;
+///
+/// let provider = Figment::from(("foobar", "tuple"))
+/// .merge(Toml::file("Config.toml"))
+/// .merge(Env::prefixed("CONFIG_"));
+/// assert_eq!(provider.extract_inner::<String>("foobar").unwrap(), "file".to_string());
+///
+/// jail.set_env("CONFIG_FOOBAR", "env");
+///
+/// let provider = Figment::from(("foobar", "tuple"))
+/// .merge(Toml::file("Config.toml"))
+/// .merge(Env::prefixed("CONFIG_"));
+/// assert_eq!(provider.extract_inner::<String>("foobar").unwrap(), "env".to_string());
+///
+/// Ok(())
+/// });
+/// ```
+///
/// ## Extraction
///
/// The configuration or a subset thereof can be extracted from a `Figment` in
This test fails:
$ cargo test --all-features --doc figment
...
---- src/figment.rs - figment::Figment (line 48) stdout ----
Test executable failed (exit code 101).
stderr:
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `"tuple"`,
right: `"file"`', src/figment.rs:14:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
From how I understand merging and joining, the latter providers should overwrite the earlier providers, but this doesn't seem to happen.
This has landed in serde_yaml
here: https://docs.rs/serde_yaml/0.9.4/serde_yaml/enum.Value.html#method.apply_merge
It would be nice if figment supported it natively. I briefly looked into it, but there's a lot of abstraction going on.
When I write this test function
#[test]
fn mytest() {
let v:Value = Value::from(5000i64);
println!("{:?} {:?}", v, v.to_i128());
assert_eq!(v.to_i128(), Some(5000i128));
}
I got this
Num(Tag::Default, I64(5000)) None
thread 'mytest' panicked at 'assertion failed: `(left == right)`
left: `None`,
right: `Some(5000)`', src/main.rs:22:5
I think it's wrong. please check it. And check Value::to_u128
for more.
I have a deeply nested configuration structure which uses the Env
provider. I added a new field, but figment does not seem to be loading the correct value, even though I have set the correct environment variable. I suspect it is because I am using a wrong environment variable for it.
Can we get a method that allows us to inspect the names of all expected env variables?
use figment::{providers::Env, Figment};
use serde::Deserialize;
#[derive(Deserialize, Debug, Clone)]
pub struct DatabaseConfig {
pub url: String,
}
#[derive(Deserialize, Debug, Clone)]
pub struct OpenlibraryConfig {
pub url: String,
}
#[derive(Deserialize, Debug, Clone)]
pub struct BookConfig {
pub openlibrary: OpenlibraryConfig,
}
#[derive(Deserialize, Debug, Clone)]
pub struct AppConfig {
pub book: BookConfig,
pub database: DatabaseConfig
}
pub fn get_figment_config() -> Figment {
Figment::new().merge(Env::prefixed("APP_").split("__"))
}
let env_names = get_figment_config.env_var_names();
// ["APP_BOOK__OPENLIBRARY__URL", "APP_DATABASE__URL"]
Hello, I found a problem in the sending system, where I enter a number, and I expect to receive a string and I end up getting an error, and I believe it was supposed to be a conversion
use figment::{providers::Env, Figment};
#[derive(Deserialize, Debug)]
pub struct Config {
pub secret: String,
}
impl Config {
pub fn figment() -> Figment {
Figment::new().merge(Env::prefixed("APP_").global())
}
}
Error
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { tag: Tag(Global, 1), profile: Some(Profile(Uncased { string: "global" })), metadata: Some(Metadata { name: "`APP_` environment variable(s)", source: None, provide_location: Some(Location { file: "src\\config.rs", line: 10, col: 24 }), interpolater: }), path: ["secret"], kind: InvalidType(Unsigned(123), "a string"), prev: None }', src\main.rs:9:54
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Reproduction: https://github.com/Gabriel-Paulucci/FigmentTest
I am trying to construct a Config
instance using rust
figment crate (figment = {version ="0.10.4", features = ["toml", "env"]}
with an objective of
config.toml
fileuse serde::{Serialize, Deserialize};
use figment::Figment;
use figment::providers::{Toml, Env, Format};
#[derive(Serialize, Deserialize, Clone)]
pub struct DatabaseConfig {
pub host: String,
pub port: i64,
}
#[derive(Serialize, Deserialize, Clone)]
pub struct Config {
database: DatabaseConfig,
}
fn main() {
let _res: Config = Figment::new()
.merge(Toml::file("config.toml"))
.merge(
Env::raw().split("_")
)
.extract()
.expect("expected to construct config");
}
Config file
[database]
host = "dbhost"
port = 5321
Cargo dependencies are
[dependencies]
figment = {version ="0.10.4", features = ["toml", "env"]}
serde = { version = "1.0", features = ["derive"] }
I tried running it using
DATABASE_HOST="dummyhost" cargo run
cargo run
In both cases the program panics with the error
Finished dev [unoptimized + debuginfo] target(s) in 0.03s
Running `target/debug/playground`
thread 'main' panicked at 'key is non-empty: must have dict', /Users/asnimansari/.cargo/registry/src/github.com-1ecc6299db9ec823/figment-0.10.4/src/providers/env.rs:486:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Looks like something is wrong with Env::raw().split("_")
.
What am I missing here?
i see that it's possible to extract values from configuration using a period-delimited 'path' str
. Is there anyway to do the inverse operation? That is, set a nested value using the path to the key.
The use case i'm envisaging is that
I had imagined an interface like
APP --config path.to.key=value
what's the best way to approach this using your library?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.