ardanlabs / conf Goto Github PK
View Code? Open in Web Editor NEWPackage conf provides support for using environmental variables and command line arguments for configuration.
License: Apache License 2.0
Package conf provides support for using environmental variables and command line arguments for configuration.
License: Apache License 2.0
what do you think would be good approach to do it
we need to add list of ids (ints) and later filter by them
for now i don't see any option but get string and split later and convert to ints
may be there is better option
we need something like --ids 222,333,444
Hi
I was experimenting with the latest version and found out that I can't use []string ( or []int )
I have a structure like this:
type ConfigOptions struct {
auxEndpoints []string conf:"default:127.0.0.1:200,127.0.0.1:829"
}
The problem is that comma is used to separate fields, so the "processField" can't recognize that last value and processes it as a field.
I suggest changing it to a semicolon.
Ex:
type ConfigOptions struct {
auxEndpoints []string conf:"default:127.0.0.1:200,127.0.0.1:829;short:d;help:test"
}
If you agree with this, I have a fork with the fix....
There is a circleci config in the project, README is green but looking at circleci there are not workflows running.
Is it intentional?
Hi
There's a problem with the required field.
`
var provided bool
for _, sourcer := range sources {
if sourcer == nil {
continue
}
var value string
if value, provided = sourcer.Source(field); !provided {
continue
}
// A value was found so update the struct value with it.
if err := processField(value, field.field); err != nil {
return &FieldError{
fieldName: field.name,
typeName: field.field.Type().String(),
value: value,
err: err,
}
}
}
// If this key is not provided by any source, check if it was
// required to be provided.
if !provided && field.options.required {
return fmt.Errorf("required field %s is missing value", field.name)
}`
Since there are multiple sources, the required field could have been in the 1st source the for loop processes.
No problem here.
But when you process the 2nd (..or any other), the "provided flag" is marked as false.
So although the field was processed with success 1 time, the final condition will still return an error.
I fix this in the code bellow (conf.go, line 82)
for _, sourcer := range sources { if sourcer == nil || provided { continue }
Hey Bill, I was just curious.. when overriding a variable name, as in the following example:
type ip struct {
Name string `conf:"default:localhost,env:IP_NAME_VAR"`
IP string `conf:"default:127.0.0.0"`
}
The package defines the variable using the namespace, as shown in the help options:
OPTIONS
--ip-name/$CRUD_IP_NAME_VAR (default: localhost)
Is that the desired behavior? Why doesn't it use the variable as defined when overridden: $IP_NAME_VAR ?
Thanks
type Config struct {
...
SlackBotToken string `conf:"default:'',noprint:true,env:SLACK_BOT_TOKEN,flag:slack-bot-token,help:Bot token"`
...
}
func main() {
...
cfg, err := config.Config()
if err != nil {
return errors.Wrapf(err, "error reading config\n")
}
out, err := conf.String(cfg)
if err != nil {
return errors.Wrap(err, "error generating config for output")
}
fmt.Println("starting service ===", "version:", cfg.Version.SVN)
fmt.Printf("configuration\n%v", out)
...
}
execution:
starting service === version: develop
configuration
--svn=develop
--slack-bot-token=xoxx-XDF...
Using 1.5.0
witg go
1.17
Sorry in advance for opening a question as an issue, could not find a relevant community to direct this.
How can we use the env vars without prefix?
When I pass the empty string as prefix:
if err := conf.Parse(os.Args[1:], "", &cfg);
I get a leading underscore in the env vars generated
--web-service-port/$_SERVICE_PORT <int> (default: 98765)
This was previously defined in a cfg
struct as follows
Web struct {
ServicePort int `conf:"default:98765,env:SERVICE_PORT"`
}
It looks like the library cannot parse multiple camel-cased embedded types (couldn't find the exact pattern, but camel-case looks to be the reason).
Sample code: https://go.dev/play/p/iSlHbeWHRHZ
Sample output (go run main.go -h
):
Usage: sample [options] [arguments]
OPTIONS
--database-new-sql-volt-db-pass/$DATABASE_NEW_SQL_VOLT_DB_PASS <string>
--database-new-sql-volt-db-pass/$DATABASE_NEW_SQL_VOLT_DB_PASS <int>
--database-new-sql-volt-db-pass/$DATABASE_NEW_SQL_VOLT_DB_PASS <string>
--database-new-sql-volt-db-pass/$DATABASE_NEW_SQL_VOLT_DB_PASS <string>
--database-new-sql-clustrix-db-pass/$DATABASE_NEW_SQL_CLUSTRIX_DB_PASS <string>
--database-new-sql-clustrix-db-pass/$DATABASE_NEW_SQL_CLUSTRIX_DB_PASS <int>
--database-new-sql-clustrix-db-pass/$DATABASE_NEW_SQL_CLUSTRIX_DB_PASS <string>
--database-new-sql-clustrix-db-pass/$DATABASE_NEW_SQL_CLUSTRIX_DB_PASS <string>
--database-no-sql-mongo-db-pass/$DATABASE_NO_SQL_MONGO_DB_PASS <string>
--database-no-sql-mongo-db-pass/$DATABASE_NO_SQL_MONGO_DB_PASS <int>
--database-no-sql-mongo-db-pass/$DATABASE_NO_SQL_MONGO_DB_PASS <string>
--database-no-sql-mongo-db-pass/$DATABASE_NO_SQL_MONGO_DB_PASS <string>
--database-no-sql-couch-db-pass/$DATABASE_NO_SQL_COUCH_DB_PASS <string>
--database-no-sql-couch-db-pass/$DATABASE_NO_SQL_COUCH_DB_PASS <int>
--database-no-sql-couch-db-pass/$DATABASE_NO_SQL_COUCH_DB_PASS <string>
--database-no-sql-couch-db-pass/$DATABASE_NO_SQL_COUCH_DB_PASS <string>
--help/-h
display this help message
All the variables are being suffixed with DB_PASS
.
Optional ones for JSON, YAML, TOML, etc so coders can choose which Sourcers they want beyond env
and flag
?
They would be included in a sources_extra.go
file.
It's not clear from the documentation that Parse should be called with os.Args[1:]
.
Does seem to matter for the short example
var cfg struct {
Port int
Args conf.Args
}
func main() {
if err := conf.Parse(os.Args, "CRUD", &cfg); err != nil {
log.Fatalf("main : Parsing Config : %v", err)
}
}
The bellow example doesn't work with os.Args.
Should conf.Parse(os.Args, ..
return an error in this example?
package main
import (
"fmt"
"log"
"os"
"time"
"github.com/ardanlabs/conf"
)
type ip struct {
Name string `conf:"default:localhost,env:IP_NAME_VAR"`
IP string `conf:"default:127.0.0.0"`
Endpoints []string `conf:"default:127.0.0.1:200;127.0.0.1:829"`
}
type Embed struct {
Name string `conf:"default:bill"`
Duration time.Duration `conf:"default:1s,flag:e-dur,short:d"`
}
type config struct {
AnInt int `conf:"default:9"`
AString string `conf:"default:B,short:s"`
Bool bool
Skip string `conf:"-"`
IP ip
Embed
}
func main() {
cfg := config{}
if err := conf.Parse(os.Args[1:], "TEST", &cfg); err != nil { // <- can't be os.Args,
log.Fatalf("main : Parsing Config : %v", err)
}
us, err := conf.Usage("Test", &cfg)
if err != nil {
fmt.Printf("err %v\n", err)
}
fmt.Printf("%+v\n", us)
fmt.Printf("%+v\n", cfg)
}
Is it intentional that masked field values are printed during the help output?
--db-password/$CLOUD-API_DB_PASSWORD (default: postgres)
It's possible to add an option -v/--version to show the application version information?
Instead of entirely omitting from printouts any variables set as noprint
in the cfg
struct, perhaps it would be a good idea just to omit / mask its value.
This would help the user/developer realise that there is indeed a configuration variable (set/expected) named (say) AWS_SECRET_ACCESS_KEY
but it is sensitive. (it helps precision in terms of having the logs reflect exactly (with the highest level of precision) the actual configuration.
The yaml package is registered as a Parser
which fills the struct before it is passed into the parse
function that evaluates all fields with the two sources:
During this they also check if the value has ever been provided, since the yaml parsing happens before the parse
function during the parser.Process
loop, the evaluation fails and you get the error that you did not provide a required field. Attaching a debugger and looking at the config struct throughout the flow of the parse
function shows that the fields are set correctly by the yaml parser.
Not sure what the correct implementation would be here, it feels like that the yaml parsing should be a Source
instead this would add some extra parameters to the API of Parse
though to figure out where the config file is located to read it.
Repository: https://github.com/Deichindianer/yaml-parse-error
package main
import (
"github.com/ardanlabs/conf/v2"
"github.com/ardanlabs/conf/v2/yaml"
"os"
)
type Config struct {
RequiredKey bool `conf:"required" yaml:"required"`
}
func main() {
var cfg Config
f, err := os.ReadFile("./config.yml")
if err != nil {
panic(err)
}
_, err = conf.Parse("", &cfg, yaml.WithData(f))
if err != nil {
panic(err)
}
}
This yields:
panic: parsing config: required field RequiredKey is missing value
goroutine 1 [running]:
main.main()
/home/bp/Code/tmp/yaml-parse-error/main.go:23 +0xc5
exit status 2
Parse unexpectedly returns an error when parsing a bool flag followed by an arg. This behavior differs from the standard library flag package.
I've put together a playground demonstrating the issue.
As a developer, I'd like to be able to override ,mask
status, so I can see values while debugging.
I'm trying to implement a Sourcer
(to be provided to Parse()
) for loading configuration from a YAML file, but apparently it's not possible due the fact that field
is unexported.
I think that field
type should be exported to make Sourcer
useful.
PS: thanks for your amazing work :-D
I think it will be great to implement mutator func. It will be useful if need to modify environment variable values before processing so developer can specify a custom mutator func.
One of use case can be to send a request to secret storage (like HashiCorp Vault) to retrieve secrets and update the env var before process it.
So we can create new type
type MutatorFunc func(key, value string) (string, error)
and new ProcessWithMutator func
func ProcessWithMutator(args []string, namespace string, cfgStruct interface{}, mutators ...MutatorFunc) error { return Parse(args, namespace, cfgStruct, mutators) }
and need to update Parse func to work with mutator
Following the Service 2.0
class and using the conf
package:
▶ ./sales-api --help
Usage: sales-api [options] [arguments]
OPTIONS
--web-api-host/$SALES_WEB_API_HOST <string> (default: 0.0.0.0:3000)
--web-debug-host/$SALES_WEB_DEBUG_HOST <string> (default: 0.0.0.0.:4000)
--web-read-timeout/$SALES_WEB_READ_TIMEOUT <duration> (default: 5s)
--web-write-timeout/$SALES_WEB_WRITE_TIMEOUT <duration> (default: 5s)
--web-shutdown-timeout/$SALES_WEB_SHUTDOWN_TIMEOUT <duration> (noprint,default: 5s)
--help/-h
display this help message
--version/-v
display version information
Perhaps it would be a good idea to have the cmd arguments named exactly (lowercased, hyphens instead of underscores) as the expected env vars, i.e.
--sales-web-api-host/$SALES_WEB_API_HOST
instead of
--web-api-host/$SALES_WEB_API_HOST
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.