I think that the net.URL is a very common type to be used in configuration.
I already found myself several times defining a URL configuration argument as a string type, and performing url.Parse to get a *url.URL object.
Since this type is not implementing the Setter or Decoder interface, it can't be used as is.
I think that adding a support to this type would be very useful.

I have a relatively simple use-case: I would like to primarily use envconfig, and occasionally provide flags to change some (otherwise required) configs.

Consider:

config := struct {
	Port int `required:"true"`
}{}

flag.IntVar(&config.Port, "port", 0, "Set port via flag or via PORT=8080")
flag.Parse()

err := envconfig.Process("", &config)

Currently, envconfig will produce an error, even if the program is started via ./program -port 8080. Considering the non-zero values in spec would allow this program to work without changes.

For contrast, this can trivially be rewritten to work with envconfig as-is:

port := flag.Int("port", 0, "Set port via flag or via PORT=8080")
flag.Parse()

if *port != 0 {
	os.Setenv("PORT", strconv.Itoa(*port))
}

I realize this proposal is not without it's own set of problems. For example, what if the user wants to provide a zero value as the default. Then either os.Setenv is needed or the type should be a pointer (*int).

I also considered if a new tag was merited (e.g. use_nonzero:"true"), but this is not ideal and expanding the API for this use-case just isn't worth it.

Most applications today provide a -h or --help command line argument that will display the usage of the command and describe, briefly, the available command line arguments and their default values.

If an application is configured by its environment there should be a mechanism by which a user can be provided with a usage description at runtime.

(see discussion for more information: #40)

Thinking out loud and asking for comment -- what about allowing non-struct types in Process()?

This use would be kind of like a type-decoding os.Getenv, just read a single env var and decode it into a pointer. The advantage is that it would allow use of all of our type-specific decoders, and even custom decoders, but wouldn't require a struct wrapper.

func init() {
    var timeout int
    envconfig.MustProcess("THING_TIMEOUT", &timeout)
}

We already have a few recursive Process() calls with the new slice and embedded struct support, this would potentially make the current common case a recursive one as well.

Hi,

The following playground link should illustrate this issue: http://play.golang.org/p/XHSJ8ihZh-

If MYAPP_USER is not defined but USER is, the value of USER will be used instead. This seems to be a side effect of #10 , in https://github.com/kelseyhightower/envconfig/blob/master/envconfig.go#L51 we use fieldName as key without actually checking if a struct tag was actually set.

I can submit a PR for this but want to make sure the current behaviour is not intentional and this is not just a documentation issue.

I came across the need to have two separate Process functions: one for default values, and another for env values. The reason is that sometimes there're other config sources (like config file), and in this case env value is used as a custom runtime option. If that's ok I can add a PR with these two functions

I noticed that some code paths use == "true". This means that, for example, True will not work. In the same time, bool's default description is True or False.

It's better to use strconv.ParseBool everywhere.

macOS 10.13.3 High Sierra
Golang go1.10rc1

When I run the example app, it just gives me the zero-values for the struct, not the values from the export commands. I seem to remember I had similar problems when I used os.Getenv but I had hoped this package solved those problems.

Hi,

I am embedding one config struct in another. The embedded struct is used across multiple projects. What I would need is to be able to set different default values based on the project, which is not possible since the tag value is static.

An example could be that I would like to use the value of SERVICE_NAME environment variable in the embedded config struct as the default value for something.

I am probably going to fork this package and implement it so that when there is default:"$SERVICE_NAME", the relevant environment variable is used as the default value. I am just wondering whether to send a PR afterwards. Does it seem handy for other people?

Or perhaps you want to implement it yourself quickly. It's just about checking whether default starts with $ or any other magical prefix and act accordingly...

Cheers!

It would be useful to prevent silent misconfigurations to have envconfig optionally be able to check if any of the environment variables currently set match the application prefix but is not in the set of allowed variable names.

type config struct {
  a int
  b int `ignored:"true"`
}

assuming the application prefix is MYAPP, the optional check should return the corresponding error if supplied with each env var:

MYAPP_A=1 # no error
MYAPP_B=1 # error: attempting to set ignored variable MYAPP_B
MYAPP_C=1 # error: attempting to set unknown variable MYAPP_C

Could this be made self-documenting similar to spf13/cobra? What would be the best way to implement this?

I personally like markdown tables for repositories of applications intended to be run through docker. Is this out of scope for the package?

I have the following Type:

type Mode int

const (
    Development Mode = iota
    Production
)

func (m *Mode) Decode(value string) error {
    lower := strings.ToLower(value)
    switch {
    case strings.HasPrefix(lower, "d"):
        *m = Development
    case strings.HasPrefix(lower, "p"):
        *m = Production
    default:
        return fmt.Errorf("'%s' is an unrecognized mode", value)
    }
    return nil
}

If I use the the enviromment varialbe APP_MODE=d and APP_MODE=p everything works as designed.

But if I use APP_MODE=t I get the error:

panic: envconfig.Process: assigning MULE_MODE to Mode: converting 't' to type main.Mode

But I would prefer the error message from the Decode function, like:

panic: envconfig.Process: assigning MULE_MODE to Mode: 't' s an unrecognized mode

Is that somehow possible?

I work in a team that uses docker-compose. Commonly, we have docker-compose.yml files for each of our environments. It's a fairly common issue that we define new environment variables for a service, but forget to include them in the production environment's compose file, since it doesn't tend to be used locally.

I don't propose to make envconfig aware of docker compose, but I would like a way to 'check' an arbitrary set of keys/values against a spec. I can put together a tool to get the environment variables from the docker-compose file, but I can't currently pass those in to envconfig. Would you be open to adding a Check(prefix string, spec interface{}, kvs map[string]string) method? I'd be happy to submit a PR for it.

Hi.

I see the latest tag is 12 behind the current master branch.

The feature I'm looking for is not having to specify the prefix on envconfig.Process() in snake case format. It seems odd to specify the env prefix like my_app where MY_APP is the standard format to specify environment variables.

I see this is fixed on master branch though, so since locking versions is always a good practice, when a new tag can be expected?

Cheers!

I noticed that the order of the struct tags is important and determines whether the split_words functionality words.

I didn't see this mentioned anywhere.

package main
  
import (
        "os"
        "fmt"
        "github.com/kelseyhightower/envconfig"
)

type backwordTag struct {
        MyBool bool `default:"false",split_words:"true"`
}

type forwardTag struct {
        MyBool bool `split_words:"true",default:"false"`
}

func main() {
        var bt backwordTag
        var ft forwardTag

        os.Setenv("MY_BOOL", "true")

        err := envconfig.Process("", &bt)
        if err != nil {
                fmt.Println("Backword Tag", err)
        }

        fmt.Printf("%t\n", bt.MyBool)

        err = envconfig.Process("", &ft)
        if err != nil {
                fmt.Println("Forward Tag", err)
        }

        fmt.Printf("%t\n", ft.MyBool)
}

[test]$ go run main.go 
false
true

Would be pretty useful to have a configuration option in Process that automatically inserts underscores for camel case field names, as in your MultiWordVar -> MYAPP_MULTI_WORD_VAR example. Just because it's tedious to specify custom envconfig annotations for every field.

This is useful for urls for either keys or values. As of now, envconfig fails with -- invalid map item at https://github.com/kelseyhightower/envconfig/blob/master/envconfig.go#L274.

Here's my idea to allow envconfig to set values in nested structs. Unfortunately, it's not functional because I still don't have enough knowledge about reflect package.

First, I replaced envconfig.go:50 by

if value == "" && f.Kind() != reflect.Struct {

then added the following case to switch:

case reflect.Struct:
    obj := f.Interface()
    err := Process(key, &obj)

    if err != nil {
        return err
    }

Please let me know your thoughts.

Would be nice to have this available so that I could build a command line Usage for the Environment Variables that are available to override.

For example, you could build a mapping of the env vars defined in your struct to some Usage doc that describes the fields individually.

#22 added support for embedded nested structs, this proposes supporting nested structs in named fields as well.

There is a potential name collision issue if we don't do something about the naming convention:

var Config = struct {
    HTTPClient http.Client
    Timeout int
}{}

func init() {
    envconfig.MustProcess("widget", &Config)
}

Does WIDGET_TIMEOUT correspond to Config.Timeout, or Config.HTTPClient.Timeout?

So I propose we underscore-delimit the field names: WIDGET_TIMEOUT and WIDGET_HTTPCLIENT_TIMEOUT.

envconfig tags would apply globally for nested structs as they do outer structs today (clobbering any prefix), so any field at any position can specify its full env var name. But we can't rely on struct tags alone and still support dropping in a stdlib or third-party-lib struct like the example above.

Let's say I define a bunch of variables but one of them has an override e.g.

LogLevel string desc:"Log level override" envconfig:"LOG_LEVEL"

then I would expect that a call to envconfig.Usage("CMD", &cfg) should print it as LOG_LEVEL ..., instead of that I get CMD_LOG_LEVEL.

Same thing seems to be happening with the error from envconfig.Process("CMD", &cfg). When missing a required variable with an override I get required key CMD_LOG_LEVEL missing value as a message.

Thanks,
Chris

The releases page list 1.3.0 as the latest. Is there merit to cut a new release (maybe minor, like 1.3.1) since there have been three changes?

So far if we export an environment variable with an empty value the required tag will not complaint.
For example in this sceneario, Process() method will not return an error.
Is this an expected behavior ?

export MYAPP_USER=
export MYAPP_USERS=""

type Specification struct {
    User    string
    Users   []string
}

Usage function is documented to write to stderr, but writes to stdout.

Package flag and third-party packages use stderr. Please fix this bug in implementation, not documentation. 😄

For a struct field like JSONFile, envconfig expects the environment variable to be named JSONF_ILE. This seems like a bug and would encourage writing JsonFile instead (which is against golint).

Expected result: JSON_FILE.

Example:

spec := struct {
	JSONFile string `split_words:"true"`
}{}
envconfig.Usage("", &spec)

Version: master (dd1402a)

I'm guessing this is due to me using the latest "Release" which glide uses by default, could you please publish a new release with the split_words enabled?

Thanks,

Just curious where "Specification" came as a name for the example that stores configuration coming from the environment.

Thanks!

I have a string field with an absolute path as the default value, running envconfig.Process() fills it with an empty string.

Example:

type Paths struct {
	Foo string `default:"/Applications/one/two/tree/"`
	Bar string `default:"./output/"`
}

Foo is empty after processing, while Bar is naturally filled with ./output/. It seems like the starting slash is the culprit.

Any hints?

It doesn't seem like it supports more than one tag for a field

I tried the followings and it doesnt seem to work,

with semicolon

type config struct {
myfield1 string `required:"true";split_words:"true"`
}

with comma

type config struct {
myfield1 string `required:"true",split_words:"true"`
}

I tried with colon

type config struct {
myfield1 string `required:"true":split_words:"true"`
}

Am I missing something or is it not supported?

I was expecting the below to succeed, since ENV is set.
However, the error I get says that MYAPP_ENV must exist.
Perhaps we need a required_any:"true" tag?
I may be able to make a PR, but I want to know what you think first?

export ENV="dev"
export MYAPP_STUFF="stuff"

type AllEnv struct {
    Env    string  `required:"true"`
    Stuff  string  `required:"true"`
}

func TestNewEnvVars(t *testing.T) {
    var env AllEnv
    envconfig.MustProcess("MYAPP", &env)
}

Can a reverse to Process() method be added? Essentially set env variables from a Struct?

Would it make sense to have split_words equal to true as the default? Assuming environment variable names whose parts are split (e.g., MONGO_URI, SERVER_PORT) may be more readable.

Unless there is a reason this does not make sense as the default, I would be happy to submit a corresponding PR.

What do you think?

I had the same idea to use a environment variable configuration for applications. I would like to load environment variable from a file ([]byte) instead of using the os package. Would you like also to have an extra function to pass in a io.Reader for reading configuration?

It would be great to set slice values of default types through environment variables.

A possible syntax for slices could be:
STRING_SLICE="item1, item2, item3, item4"
or
STRING_SLICE="'item1', 'item2', 'item3', 'item4'"

I've been thinking about what to try and get into the next release, and thought I'd put it here for comment.

What I'm largely focused on is getting to rough parity with stdlib examples of the "use reflect to deserialize into interface{}" -- thinking mainly about encoding/json and database/sql.

Nested struct support

This will be especially useful for eliminating boilerplate when configuring option or client structs for libraries. I'd like to get this in soon.

Pointer field support

Particularly for nested structs above (fields of pointer-to-struct are so common), but in the general case if we support decoding into a type then it should be pretty straightforward to decode into a *type. In our case it's not as simple as it should be, there's a little refactoring necessary.

json and sql both do this, and it can be useful for differentiating the zero value of a type from "wasn't set at all" (in which case it remains nil).

Anything else I'm missing?

It could use a special case for "":

package main

import (
    "fmt"
    "os"
    "github.com/kelseyhightower/envconfig"
)

var conf = struct {
    Field string `required:"true"`
}{}

func main() {
    os.Setenv("FIELD", "foobar")
    envconfig.MustProcess("", &conf)
    fmt.Println(conf.Field)
}

panic: required key _FIELD missing value

This way Usage() can be used after processing to print applied configuration as a better-looking alternative to fmt.Printf("%+v\n", config).

Today there's already a support for the Decoder interface that is great for new types. But if we added support to encoding.TextUnmarshaler we would automatically add support for std library types such as time.Time, net.IP, etc.

What you guys think about it? Can I start working on a PR for that?

Hi, this use case was working before commit 31176c5

package main

import (
    "fmt"
    "net/url"
    "os"

    "github.com/kelseyhightower/envconfig"
)

type Config struct {
    One string `envconfig:"one"`

    WebUrl *url.URL

    Two string `envconfig:"two"`
}

func main() {

    os.Setenv("TEST_ONE", "one")
    os.Setenv("TEST_TWO", "two")

    c := Config{}

    envconfig.Process("test", &c)
    fmt.Println(c.One)
    fmt.Println(c.Two) // two should be parsed and it's not

}

@elgris @teepark @kelseyhightower 🔔

I was wondering if there was interest in supporting an emerging pattern in the container world:

As an alternative to passing sensitive information via environment variables, _FILE may be appended to the previously listed environment variables, causing the initialization script to load the values for those variables from files present in the container.

For example, the POSTGRES_PASSWORD environment variable can also be sourced from a file, where POSTGRES_PASSWORD_FILE provides the path to the file. This pattern is very useful when you are targeting kubernetes and swarm environments, that provide the ability to source secrets from a mounted file.

This could be supported with an optional tag:

type Specification struct {
    Password string `envconfig:"POSTGRES_PASSWORD", file:"true"`
}

I would be open to sending a pull request, if there was interest in this capability.

Morning - so I have the following:

type Specification struct {
    Hostname        string
    // RDBMSDBUser     string
    // RDBMSDBPassword string
}

func GetConf() Specification {
    var s Specification
    err := envconfig.Process("myapp", &s)
    if err != nil {
        panic(err.Error())
    }
    log.Log("#######################")
    log.Log(s)
    log.Log(&s)

    // s.Hostname, s.RDBMSDBUser, s.RDBMSDBPassword
    return s
}

and I am running my app in docker - so it is running it as root - so I assume I need to export those variables as root, also I did it as my primary user too.

root@mdere-VirtualBox:/home/mdere# echo $MYAPP_HOSTNAME
localhost:8080

mdere:~$ echo $MYAPP_HOSTNAME
localhost:8080

the logs are spitting out {} or &{} for s variable

Background

when running go test ./... and the tested code uses environment variables, go-1.10 now caches the access to those env vars: golang/go@29be20a#diff-b30ffd1cf7612ccebac97fced83d5106R84

Symptoms

Tests can appear to pass when they should fail! They are not run due to prior cached successes.

(When they've previously been run successfully and that result is cached, and only the env vars have since changed, and the env vars are used only by envconfig.)

Cause

syscall.Getenv is not compatible with the cacheing used by go1.10 - it does not log accesses to env vars. So separate runs of go test ./... (with differing env vars) are treated as cached and appear to pass (if passed previously and cached) when they should fail.

Quick fixes

one of:

• replace the lookupEnv call in envconfig.go with os.LookupEnv and it solves the problem
• use go clean -cache a lot more. 😄
• use go test -count=1 ./... to force cache-invalidation

Fix

use go1.5 tag in env_*.go - see patch below

I want to use both the tags envconfig:"multi_word_var" and default:"value" together on a struct field but it doesn't seem to work and I am getting some strange behaviour when I combine these. The behaviour even changes when you change the order of them (either default: first or envconfig: first).

It is working fine when I am only using one of these tags on a field, but it seems to not work properly when I put both tags on a field.

I was expecting it would try to use the environment variable first (as defined by the envconfig: tag) and if that didn't exist, use the default value defined by the default: tag. This is not the behaviour I am getting however.

It is looking like you can't use these tags together.on a single field.

Currently, environment variables can only be populated from basic primitives (int, float, string, bool, etc.). It'd be really useful if a user could take a comma-separated value given via an environment variable and populate it into a slice field on a struct.

For example, given the following struct:

type Specification struct {
    AdminUsers                   []string
    MagicNumbers                 []int

a user should be able to set ADMINUSERS = "john,joe,matt" in the environment and have the values populated into the Specification.AdminUsers field on the above struct as []string{"john", "joe", "matt"}

I was curious if a patch to add map support would be accepted, I found MendelGusmao's work on this (https://github.com/MendelGusmao/envconfig) is very dated by seems like it could be a reasonable approach.

Hi, can you plz create git tag for stable version?
This will enable your package to be installed using http://labix.org/gopkg.in .

There are 27 commits since 1.1.0.
Also, may you please add tag v1.1.0 to 1.1.0? Right now you have v1.0.0 and 1.1.0, styles are different.

If I have the following code and haven't actually defined any of the environment variables:

package main

import (
	"github.com/davecgh/go-spew/spew"
	"github.com/kelseyhightower/envconfig"
)

type SomeItem struct {
	Blah string
}

type Config struct {
	Data        string
	ItemVal     SomeItem
	ItemPointer *SomeItem
}

func main() {
	conf := Config{}
	envconfig.Process("wtf", &conf)
	spew.Dump(conf)
}

I'd expect to get this:

(main.Config) {
 Data: (string) "",
 ItemVal: (main.SomeItem) {
  Blah: (string) ""
 },
 ItemPointer: (*main.SomeItem)(<nil>)
}

but I actually get:

(main.Config) {
 Data: (string) "",
 ItemVal: (main.SomeItem) {
  Blah: (string) ""
 },
 ItemPointer: (*main.SomeItem)(0xc42000e400)({
  Blah: (string) ""
 })
}

For some reason, envconfig has created an empty SomeItem and assigned its address to conf.ItemPointer.

My application uses envconfig with a configuration struct like this:

// Config provides the configuration for this application
type Config struct {
	AppName string `split_words:"true"`
	Env string

	MongoPoolSize int `split_words:"true"`
	MongoPort string `split_words:"true"`
	MongoUri string `required:"true" split_words:"true"`

	Port int
	Version string
}

It would be useful to have an option that I could pass to the Process/MustProcess functions to configure the split_words behaviour for all variables, so that I can cut down on boilerplate in my struct tags.

I'm happy to work on a PR for this once this issue is acknowledged.

I wanna set my env in file .env using package https://github.com/joho/godotenv
Can I use this package to get env from .env