Giter Club home page Giter Club logo

gookit / config Goto Github PK

View Code? Open in Web Editor NEW
522.0 12.0 57.0 610 KB

📝 Go configuration manage(load,get,set,export). support JSON, YAML, TOML, Properties, INI, HCL, ENV and Flags. Multi file load, data override merge, parse ENV var. Go应用配置加载管理,支持多种格式,多文件加载,远程文件加载,支持数据合并,解析环境变量名

Home Page: https://pkg.go.dev/github.com/gookit/config/v2

License: MIT License

Go 100.00%
config config-management json yaml toml ini hcl goconfig flags gookit

config's Introduction

Config

GitHub go.mod Go version Codacy Badge Build Status Actions Status Coverage Status Go Report Card Go Reference

config - Simple, full-featured Go application configuration management tool library.

中文说明

Features

  • Support multi format: JSON(default), JSON5, INI, Properties, YAML, TOML, HCL, ENV, Flags
    • JSON content support comments. will auto clear comments
    • Other drivers are used on demand, not used will not be loaded into the application.
      • Possibility to add custom driver for your specific format
  • Support multi-file and multi-data loading
  • Support for loading configuration from system ENV
  • Support for loading configuration data from remote URLs
  • Support for setting configuration data from command line(flags)
  • Support listen and fire events on config data changed.
    • allow events: set.value, set.data, load.data, clean.data, reload.data
  • Support data overlay and merge, automatically load by key when loading multiple copies of data
  • Support for binding all or part of the configuration data to the structure
    • Support init default value by struct tag default:"def_value"
    • Support init default value from ENV default:"${APP_ENV | dev}"
  • Support get sub value by key-path, like map.key arr.2
  • Support parse ENV name and allow with default value. like envKey: ${SHELL|/bin/bash} -> envKey: /bin/zsh
  • Generic API: Get Int Uint Int64 Float String Bool Ints IntMap Strings StringMap ...
  • Complete unit test(code coverage > 95%)

Only use INI

If you just want to use INI for simple config management, recommended use gookit/ini

Load dotenv file

On gookit/ini: Provide a sub-package dotenv that supports importing data from files (eg .env) to ENV

go get github.com/gookit/ini/v2/dotenv

GoDoc

Install

go get github.com/gookit/config/v2

Usage

Here using the yaml format as an example(testdata/yml_other.yml):

name: app2
debug: false
baseKey: value2
shell: ${SHELL}
envKey1: ${NotExist|defValue}

map1:
    key: val2
    key2: val20

arr1:
    - val1
    - val21

Load data

examples code please see _examples/yaml.go:

package main

import (
    "github.com/gookit/config/v2"
    "github.com/gookit/config/v2/yaml"
)

// go run ./examples/yaml.go
func main() {
	// config.ParseEnv: will parse env var in string value. eg: shell: ${SHELL}
	config.WithOptions(config.ParseEnv)

	// add driver for support yaml content
	config.AddDriver(yaml.Driver)

	err := config.LoadFiles("testdata/yml_base.yml")
	if err != nil {
		panic(err)
	}

	// load more files
	err = config.LoadFiles("testdata/yml_other.yml")
	// can also load multi at once
	// err := config.LoadFiles("testdata/yml_base.yml", "testdata/yml_other.yml")
	if err != nil {
		panic(err)
	}

	// fmt.Printf("config data: \n %#v\n", config.Data())
}

Usage tips:

  • More extra options can be added using WithOptions(). For example: ParseEnv, ParseDefault
  • You can use AddDriver() to add the required format driver (json is loaded by default, no need to add)
  • The configuration data can then be loaded using LoadFiles() LoadStrings() etc.
    • You can pass in multiple files or call multiple times
    • Data loaded multiple times will be automatically merged by key

Bind Structure

Note: The default binding mapping tag of a structure is mapstructure, which can be changed by setting the decoder's option options.DecoderConfig.TagName

type User struct {
    Age  int  `mapstructure:"age"`
    Key  string `mapstructure:"key"`
    UserName  string `mapstructure:"user_name"`
    Tags []int  `mapstructure:"tags"`
}

user := User{}
err = config.BindStruct("user", &user)

fmt.Println(user.UserName) // inhere

Change struct tag name

config.WithOptions(func(opt *Options) {
    options.DecoderConfig.TagName = "config"
})

// use custom tag name.
type User struct {
  Age  int  `config:"age"`
  Key  string `config:"key"`
  UserName  string `config:"user_name"`
  Tags []int  `config:"tags"`
}

user := User{}
err = config.Decode(&user)

Can bind all config data to a struct:

config.Decode(&myConf)
// can also
config.BindStruct("", &myConf)

config.MapOnExists like BindStruct,but map binding only if key exists

Direct read data

  • Get integer
age := config.Int("age")
fmt.Print(age) // 100
  • Get bool
val := config.Bool("debug")
fmt.Print(val) // true
  • Get string
name := config.String("name")
fmt.Print(name) // inhere
  • Get strings(slice)
arr1 := config.Strings("arr1")
fmt.Printf("%#v", arr1) // []string{"val1", "val21"}
  • Get string map
val := config.StringMap("map1")
fmt.Printf("%#v",val) // map[string]string{"key":"val2", "key2":"val20"}
  • Value contains ENV var
value := config.String("shell")
fmt.Print(value) // "/bin/zsh"
  • Get value by key path
// from array
value := config.String("arr1.0")
fmt.Print(value) // "val1"

// from map
value := config.String("map1.key")
fmt.Print(value) // "val2"
  • Setting new value
// set value
config.Set("name", "new name")
name = config.String("name")
fmt.Print(name) // "new name"

Load from flags

Support simple flags parameter parsing, loading

// flags like: --name inhere --env dev --age 99 --debug

// load flag info
keys := []string{"name", "env", "age:int" "debug:bool"}
err := config.LoadFlags(keys)

// read
config.String("name") // "inhere"
config.String("env") // "dev"
config.Int("age") // 99
config.Bool("debug") // true

Load from ENV

// os env: APP_NAME=config APP_DEBUG=true
// load ENV info
config.LoadOSEnvs(map[string]string{"APP_NAME": "app_name", "APP_DEBUG": "app_debug"})

// read
config.Bool("app_debug") // true
config.String("app_name") // "config"

New config instance

You can create custom config instance

// create new instance, will auto register JSON driver
myConf := config.New("my-conf")

// create empty instance
myConf := config.NewEmpty("my-conf")

// create and with some options
myConf := config.NewWithOptions("my-conf", config.ParseEnv, config.ReadOnly)

Listen config change

Now, you can add a hook func for listen config data change. then, you can do something like: write data to file

Add hook func on create config:

hookFn := func(event string, c *Config) {
    fmt.Println("fire the:", event)
}

c := NewWithOptions("test", config.WithHookFunc(hookFn))
// for global config
config.WithOptions(config.WithHookFunc(hookFn))

After that, when calling LoadXXX, Set, SetData, ClearData methods, it will output:

fire the: load.data
fire the: set.value
fire the: set.data
fire the: clean.data

Watch loaded config files

To listen for changes to loaded config files, and reload the config when it changes, you need to use the https://github.com/fsnotify/fsnotify library. For usage, please refer to the example ./_example/watch_file.go

Also, you need to listen to the reload.data event:

config.WithOptions(config.WithHookFunc(func(event string, c *config.Config) {
    if event == config.OnReloadData {
        fmt.Println("config reloaded, you can do something ....")
    }
}))

When the configuration changes, you can do related things, for example: rebind the configuration to your struct.

Dump config data

Can use config.DumpTo() export the configuration data to the specified writer, such as: buffer,file

Dump to JSON file

buf := new(bytes.Buffer)

_, err := config.DumpTo(buf, config.JSON)
ioutil.WriteFile("my-config.json", buf.Bytes(), 0755)

Dump pretty JSON

You can set the default var JSONMarshalIndent or custom a new JSON driver.

config.JSONMarshalIndent = "    "

Dump to YAML file

_, err := config.DumpTo(buf, config.YAML)
ioutil.WriteFile("my-config.yaml", buf.Bytes(), 0755)

Available options

// Options config options
type Options struct {
	// parse env in string value. like: "${EnvName}" "${EnvName|default}"
	ParseEnv bool
    // ParseTime parses a duration string to time.Duration
    // eg: 10s, 2m
    ParseTime bool
	// config is readonly. default is False
	Readonly bool
	// enable config data cache. default is False
	EnableCache bool
	// parse key, allow find value by key path. default is True eg: 'key.sub' will find `map[key]sub`
	ParseKey bool
	// the delimiter char for split key path, if `FindByPath=true`. default is '.'
	Delimiter byte
	// default write format
	DumpFormat string
	// default input format
	ReadFormat string
	// DecoderConfig setting for binding data to struct
	DecoderConfig *mapstructure.DecoderConfig
	// HookFunc on data changed.
	HookFunc HookFunc
	// ParseDefault tag on binding data to struct. tag: default
	ParseDefault bool
}

Examples for set options:

config.WithOptions(config.WithTagName("mytag"))
config.WithOptions(func(opt *Options) {
    opt.SetTagNames("config")
})

Options: Parse default

Support parse default value by struct tag default

// add option: config.ParseDefault
c := config.New("test").WithOptions(config.ParseDefault)

// only set name
c.SetData(map[string]any{
    "name": "inhere",
})

// age load from default tag
type User struct {
    Age  int `default:"30"`
    Name string
    Tags []int
}

user := &User{}
goutil.MustOk(c.Decode(user))
dump.Println(user)

Output:

&config_test.User {
  Age: int(30),
  Name: string("inhere"), #len=6
  Tags: []int [ #len=0
  ],
},

API Methods Refer

Load Config

  • LoadOSEnvs(nameToKeyMap map[string]string) Load data from os ENV
  • LoadData(dataSource ...any) (err error) Load from struts or maps
  • LoadFlags(keys []string) (err error) Load from CLI flags
  • LoadExists(sourceFiles ...string) (err error)
  • LoadFiles(sourceFiles ...string) (err error)
  • LoadFromDir(dirPath, format string) (err error) Load custom format files from the given directory, the file name will be used as the key
  • LoadRemote(format, url string) (err error)
  • LoadSources(format string, src []byte, more ...[]byte) (err error)
  • LoadStrings(format string, str string, more ...string) (err error)
  • LoadFilesByFormat(format string, sourceFiles ...string) (err error)
  • LoadExistsByFormat(format string, sourceFiles ...string) error

Getting Values

  • Bool(key string, defVal ...bool) bool
  • Int(key string, defVal ...int) int
  • Uint(key string, defVal ...uint) uint
  • Int64(key string, defVal ...int64) int64
  • Ints(key string) (arr []int)
  • IntMap(key string) (mp map[string]int)
  • Float(key string, defVal ...float64) float64
  • String(key string, defVal ...string) string
  • Strings(key string) (arr []string)
  • SubDataMap(key string) maputi.Data
  • StringMap(key string) (mp map[string]string)
  • Get(key string, findByPath ...bool) (value any)

Mapping data to struct:

  • Decode(dst any) error
  • BindStruct(key string, dst any) error
  • MapOnExists(key string, dst any) error

Setting Values

  • Set(key string, val any, setByPath ...bool) (err error)

Useful Methods

  • Getenv(name string, defVal ...string) (val string)
  • AddDriver(driver Driver)
  • Data() map[string]any
  • SetData(data map[string]any) set data to override the Config.Data
  • Exists(key string, findByPath ...bool) bool
  • DumpTo(out io.Writer, format string) (n int64, err error)

Run Tests

go test -cover
// contains all sub-folder
go test -cover ./...

Projects using config

Check out these projects, which use https://github.com/gookit/config :

Gookit packages

  • gookit/ini Go config management, use INI files
  • gookit/rux Simple and fast request router for golang HTTP
  • gookit/gcli build CLI application, tool library, running CLI commands
  • gookit/event Lightweight event manager and dispatcher implements by Go
  • gookit/cache Generic cache use and cache manager for golang. support File, Memory, Redis, Memcached.
  • gookit/config Go config management. support JSON, YAML, TOML, INI, HCL, ENV and Flags
  • gookit/color A command-line color library with true color support, universal API methods and Windows support
  • gookit/filter Provide filtering, sanitizing, and conversion of golang data
  • gookit/validate Use for data validation and filtering. support Map, Struct, Form data
  • gookit/goutil Some utils for the Go: string, array/slice, map, format, cli, env, filesystem, test and more
  • More, please see https://github.com/gookit

See also

License

MIT

config's People

Contributors

aadog avatar aermolaev avatar beckend avatar codacy-badger avatar dependabot-preview[bot] avatar dependabot[bot] avatar fiksn avatar inhere avatar jandedobbeleer avatar morontt avatar nacho692 avatar refs avatar vipcxj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

config's Issues

ReloadFiles方法在重载json格式的文件时会发生错误但toml正常

// 监听配置文件热修改
func watchConfigFiles(cfg *config.Config) {
	// 开一个新线程防止主线程被卡住
	readyTask := new(sync.WaitGroup)
	readyTask.Add(1)
	go func() {
		watcher, err := fsnotify.NewWatcher()
		if err != nil {
			cliutil.Errorln(err.Error())
			return
		}
		defer watcher.Close()
		// 获取加载的配置文件
		files := cfg.LoadedFiles()
		if len(files) == 0 {
			cliutil.Errorln("未读取到配置文件")
			return
		}

		// 处理出错或通道关闭时的退出问题
		eventsTask := new(sync.WaitGroup)
		eventsTask.Add(1)
		go func() {
			for {
				select {
				case event, ok := <-watcher.Events:
					if !ok {
						eventsTask.Done()
						return
					}
					// 只有写入时才重新创建数据
					switch event.Op.String() {
					case "WRITE":
						// 重载数据
						if err := cfg.ReloadFiles(); err != nil {
							eventsTask.Done()
							cliutil.Errorf("重载%s数据出错,err:%s\n", event.Name, err.Error())
							return
						}
						cliutil.Infof("监听到%s变动\n", event.Name)
						fmt.Println(cfg.Data())
					case "REMOVE":
						eventsTask.Done()
						cliutil.Errorf("重载%s数据出错,err:文件被删除,请不要删除配置文件\n", event.Name)
						return
					default:
						cliutil.Infof("监听到%s变动 Op->%s\n", event.Name, event.Op.String())
					}
				case err, ok := <-watcher.Errors:
					if ok {
						cliutil.Errorln(err.Error())
					}
					if err != nil {
						cliutil.Errorln(err.Error())
					}
					eventsTask.Done()
					return
				}
			}
		}()
		// 加载文件的监听
		for _, path := range files {
			if err := watcher.Add(path); err != nil {
				cliutil.Errorln(err.Error())
			}
		}
		// 加载文件监听成功后释放创建监听的线程
		readyTask.Done()
		// 等待事件释放
		eventsTask.Wait()
	}()
	// 等待监听成功
	readyTask.Wait()
}

以上是监听代码

json文件使用的是github.com/gookit/config/v2/json这个驱动
toml文件使用的是github.com/gookit/config/v2/toml这个驱动

出错时报的错误:ReadMapCB: expect { or n, but found , error found in #0 byte of ...||..., bigger context ...||...

json格式的数据

{
  "app": {
    "name": "ceshi",
    "mode": "debug2"
  }
}

toml格式的数据

[app]
name = "ceshi"
mode = "debug"

并发调用 Structure,存在获取不到数据的情况

System (please complete the following information):

  • OS: linux
  • GO Version: 1.18
  • Pkg Version: 2.1.2

Describe the bug

func (c *Config) Structure(key string, dst interface{}) error
该方法在并发调用时,dst 会存在获取失败的情况

To Reproduce

func (c *Config) Structure(key string, dst interface{}) error

Expected behavior

func (c *Config) Structure(key string, dst interface{}) error
该方法在并发调用时,dst 会存在获取失败的情况

bindConf.Result = dst // set result struct ptr

在并发调用时,bindConf.Result 的值存在被覆盖的情况

LoadData doesn't merge configs like LoadFiles

System (please complete the following information):

  • OS: windows
  • GO Version: 1.20
  • Pkg Version: 2.2.3

Describe the bug

When loading config from Files

paths := []string{"base.json", "mod.json"}
err = tmpCfg.LoadFiles(paths...)

configuration is appended but when using LoadData() is overriding by last source

To Reproduce

// MODELS

type PostgresConfig struct {
// cut for simplicity
	Postgres *Postgres
}

type Postgres struct {
	Host         string
	Port         int
	User         string
	Password     string
	PasswordFile string
	Database     string
	SearchPath   string
	MinPoolSize  int32
	MaxPoolSize  int32 
}

// GOT
initConfig := map[string]any{
 "Postgres": &Postgres{
 Host:         "init-host",
 Port:         1000,
 User:         "init-user",
 Password:     "init-password",
 PasswordFile: "init-password-file",
 Database:     "init-database",
 SearchPath:   "init-search-path",
 MinPoolSize:  10,
 MaxPoolSize:  100,
 }}
appendConfig := map[string]any{
 "Postgres": &Postgres{Host: "append-host"}}

cfg := config.New("")
result := &PostgresConfig{}
cfg.LoadData(initConfig, appendConfig)
cfg.Decode(result)

Current behavior

image

Expected behavior

LoadData Merges all data loaded by parameter list
image

Additional context

If needed some more explanation, code or screens, fell free to write.
If there is any other method to load data like from files but from source maps or maybe there is possibility to create something like Merge(any...)

如何将config配置文件的默认值保存为一个默认配置文件?

type Option struct {
	Test            string `config:"test"`
}

func TestConfig(t *testing.T) {
	var opt Option
	opt.Test = "test"
	con := config.New("test", func(opt *config.Options) {
		opt.DecoderConfig.TagName = "config"
		opt.ParseDefault = true
	})

	con.AddDriver(yaml.Driver)
	err := con.BindStruct("", &opt)
	var buf bytes.Buffer
	_, err = con.DumpTo(&buf, config.Yaml)
	if err != nil {
		return
	}
	fmt.Println(buf.String())
}

这段代码不生效, buf中没有任何数据.

如果可以, 是否可以提供一组更方便的api, 例如:

  • 允许设置defualt config 文件, 如果不存在则创建, 如果存在则自动加载
  • 允许将defualt的值自动填入default config 文件中

Got panic when load json then yaml to overwrite a key in json

config/test0.json:

{
    "lang": {
        "allowed": {
            "en": "ddd"
        }
    }
}

config/test1.yaml:

lang:
  allowed:
    en: "666"

run the following code:

package main

import (
	"fmt"
	"os"
	"path/filepath"

	"github.com/gookit/config/v2"
	"github.com/gookit/config/v2/yaml"
)

func main() {
	config.AddDriver(yaml.Driver)
	config.Default().Options().TagName = "json"

	filepath.Walk("config", func(path string, info os.FileInfo, err error) error {
		if err != nil {
			return err
		}

		if info.IsDir() {
			return nil
		}

		fmt.Printf("loading %s...\n", path)
		return config.LoadFiles(path)
	})
}

got this panic:

loading config/test0.json...
loading config/test1.yaml...
panic: reflect.Value.MapIndex: value of type interface {} is not assignable to type string

if reverse the loading order, i.e. load test1.yaml first then test0.json, it works

String Durations are not parsed with ParseTime and ParseEnv

System (please complete the following information):

  • OS: macOS
  • GO Version: 1.18
  • Pkg Version: 2.2.1

Describe the bug

When enabling both ParseEnv and ParseTime, time is not parsed when loading from environment, with or without default values

To Reproduce

Added a test to my own fork

func TestIssues_XXX(t *testing.T) {
	c := config.NewWithOptions("test",
		config.ParseDefault,
		config.ParseEnv,
		config.ParseTime,
	)

	type conf struct {
		Env        time.Duration
		DefaultEnv time.Duration
		NoEnv      time.Duration
	}

	err := os.Setenv("ENV", "5s")
	assert.NoError(t, err)

	err = c.LoadStrings(config.JSON, `{
		"env": "${ENV}",
		"defaultEnv": "${DEFAULT_ENV| 10s}",
		"noEnv": "15s"
	}`)
	assert.NoErr(t, err)

	var cc conf
	err = c.Decode(&cc)
	assert.NoErr(t, err)

	assert.Eq(t, 5*time.Second, cc.Env)
	assert.Eq(t, 10*time.Second, cc.DefaultEnv)
	assert.Eq(t, 15*time.Second, cc.NoEnv)
}

Expected behavior

Durations should be parsed

Screenshots

Screenshot 2023-06-02 at 15 53 38

Make parameter expansion scheme great again

There is nothing wrong using | for separating substitutions/replacements. Nevertheless, I though I would give it a whirl because using the form :- is something that probably everyone in the "development world" is familiar with, and while I don't fully understand if it's more difficult to implement or not, it gives some other flexibility.

Would you consider adding or replacing | for (the well known) :- form?

Consider the notion of profiles/environments for configuration files

Description

Loading and binding configuration data to Go's structs is probably all within the scope of this library, but I think that almost always, any modern application is delivered throughout different environments. Moreover, following this is a premise for any application that follows the twelve-factor methodology.

I think it would be great if gookit's config supports this in a certain way. For instance:

---
application:
  key: default_value___useless_because_it_s_overwritten_in_all_environments
datasource:
  username: postgres # Common to all, unless overwritten
---
"%dev":
  application:
    key: value_for_dev
  datasource:
    password: postgres_secret_for_dev
    url: postgresql:host-for-dev:5432/database
---
"%prod":
  application:
    key: value_for_production
  datasource:
    username: postgres_in_prod
    password: postgres_secret_for_prod
    url: postgresql:host-for-prod:5432/database
---
"%test":
  application:
    key: value_for_test
  datasource:
    password: postgres_secret_for_test
    url: postgresql:host-for-test:5432/database

Currently the only way to achieve this is to make use of environment variables, and that's a viable route, but it gets really messy when an application uses a lot of configuration settings — like the one I'm dealing with 😬

Regression in yml file support introduced

System:

  • OS: linux
  • GO Version: 1.15.6
  • Pkg Version: 2.0.19

Describe the bug

This change (see #30) introduced a regression in the supported file types. Before that, the function LoadFiles supported yaml and .yml extensions after loading the yaml.Driver driver.

To Reproduce

AddDriver(yaml.Driver)
...
LoadFiles("file.yml")
..

Expected behavior

I expect that once the yaml.Driver is loaded, both .yaml and .yml files are supported.

Actual v2.0.6 release file has incorrect ini version

Hey there,
I have been fighting with this for longer than I care to admit when I finally realized that when my project tries to download v2.0.6 it has the incorrect ini version. Your go.mod in the repo is correct but if you download v2.0.6 from the release tab, ini has v1 instead of v2 as you can see highlighted below.

Thanks,
-MH

``Config.Set()`` is Not Override New Value if Value is Not The Same Type

The config.Set() should override the value even though the old value is not the same type of the new one.
EX:

{
  "parent": {
    "child": "Test Var"  
  }
}

After config.Set("parent.child.grandChild", "New Val") the object should be:

{
  "parent": {
    "child": {
      "grandChild": "New Val"  
    } 
  }
}

As for now, it does not change anything or not even throw any error. I think it is because parent.child value was a string so it doesn't allow me to add a new struct ({grandChild: "New Val"}) to it.

Any idea how to fix it?

yaml Binding ist not working correctly

System (please complete the following information):

  • OS: macOS
  • GO Version: go version go1.20.1 darwin/amd64
  • Pkg Version: v2.2.1

Describe the bug

yaml Binding ist not working correctly

To Reproduce

config/config.yml

server:
  portNumber: 8080

oauth2:
  redirectUrl:  "http://localhost:8080/login/oauth2/code/dbwebsso",
  clientId: ${CLIENT_ID}
  clientSecret: ${CLIENT_SECRET}
  scopes:
    - "1234/.default"
    - "openid"
    - "profile"
    - "email"
  endPoint: "1234"
package myConfig

import (
	"fmt"
	"github.com/gookit/config/v2"
	"github.com/gookit/config/v2/yamlv3"
)

type Binder struct {
	Server server `mapstructure:"server"`
	Oauth2 oauth2 `mapstructure:"oauth2"`
}

type server struct {
	PortNumber string `mapstructure:"portNumber"`
}

type oauth2 struct {
	RedirectUrl  string   `mapstructure:"redirectUrl"`
	ClientId     string   `mapstructure:"clientId"`
	ClientSecret string   `mapstructure:"clientSecret"`
	Scopes       []string `mapstructure:"scopes"`
	EndPoint     string   `mapstructure:"endPoint"`
}

var MyConfig Binder

func LoadConfig() {

	config.WithOptions(config.ParseEnv)

	// only add decoder
	// config.SetDecoder(config.Yaml, yamlv3.Decoder)
	// Or
	config.AddDriver(yamlv3.Driver)

	err := config.LoadFiles("config/config.yml")
	if err != nil {
		panic(err)
	}

	config.BindStruct("", &MyConfig)
	//fmt.Printf("config data: \n %#v\n", config.Data())
	fmt.Printf("config data: \n %v\n", MyConfig)
}

Output

config data: 
 {{8080} {http://localhost:8080/login/oauth2/code/dbwebsso   [] }}

Expected behavior

Object Binding is working properly. If I am doing wrong, i am please to get example.

mergo 版本

mergo 是否有意更新到1.x版本,我看新版本改到 module dario.cat/mergo 地址了

Add http:// Support

	"address": [
		"192.168.1.XXX:2379",
		"192.168.1.XXX:2379",
		"192.168.1.XXX:2379"
	]

is OK, but
"address": [
"http://192.168.1.XXX:2379",
"http://192.168.1.XXX:2379",
"http://192.168.1.XXX:2379"
]
is panic.
panic: ReadString: invalid control character found: 10, error found in #10 byte of ...|79"
Strings containing “http://” are very common, Don't be confused with comments。
Tanks.

support parsedefault from defaultfile

support setting the default file when parsedefault complex structures.

type ConfigA struct {
	config ConfigB `config:"config" default:"configb.yaml"`
}

type ConfigB struct {
	A string `config:"a"`
	B string `config:"b"`
}

configb.yaml

a: aaa
b: bbb

Questions on implementation

Hey there,
I was hoping that you might not mind me asking a few questions and perhaps get your opinion? I wrote an application originally in Python and I thought it would be a good exercise to help me better learn Go to convert it. There were some Python packages I really liked that I used and I have been looking around to try and find similar ones in Go and so far this one looks pretty close to have I was using. (Which was this one here). It was my first time trying to make a full on application, so most of it was slapped together and "worked". This time around I want to try and make it a bit more structured and just, better, lol.

That being said, I was going to have a config.go and in there was going to have some structs to hold the config data so that this time around I am not constantly referencing the data via the actual loading mechanism to the config file. Before I was getting data like this all throughout the application:

config = jsoncfg.load_config('instance/config/config.json')

    @staticmethod
    def start():
        token = config.token()
        if token:
            # The rest of the code
        else:
            print("Missing token in config.json")

I wanted to use Yaml this time around for ease of use on the end user side, as my previous config file, while informative, ended up being littered with commends and what not since I was capable of doing so. I have my mockup Yaml file for the main settings config and am my plan was to have a MainSettings struct {} and figured it might be a good idea to try and structure it similar to how my yaml file was laid out.

Here is a portion of the config file layout:

settings:
  system:
    token: '12f23f23fFDS32f23123r13r3f'
    commandprefix: '!cmd'
    requireemail: 'Yes'
    consoleloglevel: 'DEBUG'
    fileloglevel: "DEBUG"
  integrations:
    wordpress: 'Yes'
    connection: 'Oauth2'
    webaddress: 'http://webaddress.com'
  server:
    id: 321321321321
    userids:
      - 123123123123
      - 123123123123
    roles:
      Verified: 123123123123
      Unverified: 123123123123

I made a struct to follow suit:

type MainSettings struct {
	System struct {
		Token           string
		CommandPrefix   string
		RequireEmail    string
		ConsoleLogLevel string
		FileLogLevel    string
	}
	Integrations struct {
		WordPress  string
		Connection string
		WebAddress string
	}
	Server struct {
		Id    int
		Userids []int
		Roles    map[string]int
	}
}

So my questions are as follows. I am currently attempting to learn Go so I am not 100% certain I am doing this "the right way", but what would be the best way to try and populate the data properly using this config package? For each one field should I assign them, similar to something like this?

func LoadConfig(filename string) MainSettings {
	config.WithOptions(config.ParseEnv)
	config.AddDriver(yaml.Driver)

	err := config.LoadFiles(filename)
	if err == nil {
		panic(err)
	}

	// The structure of this was autogenerated by my IDE GoLand, I am hoping this is how you go about it. It feels like it might be a bit much?
	mainsettings := MainSettings{System: struct {
		Token           string
		CommandPrefix   string
		RequireEmail    string
		ConsoleLogLevel string
		FileLogLevel    string
	}{Token: config.String("settings.system.token"),
		CommandPrefix:   config.String("settings.system.commandprefix"),
		RequireEmail:    config.String("settings.system.requireemail"),
		ConsoleLogLevel: config.String("settings.system.consoleloglevel"),
		FileLogLevel:    config.String("settings.system.fileloglevel")}}

	return mainsettings
  }
}

Do you have any recommendations perhaps if I am going about this incorrectly? I did attempt to run it as is, but unfortunately it came up empty.

func main() {
	settings := config.GetConfig()
	fmt.Printf("Token: %s", settings.System.Token)
}

I realize this became much longer than I intended, so I apologize about that. I am hoping that perhaps you might be able to at least point me in the right direction?

I appreciate your time,
-MH

关于 yaml解析遇到的问题

在解析yaml时候遇到了解析出错:
yam文件格式如下:

---
id: VEHICLE-RFID-0807112517
name: vehicle rfid pipeline
properties:
  kafka.bootstrap.servers: hdh98:9092,hdh197:9092,hdh199:9092
  kafka.zookeeper.connector: hdh98:2181,hdh197:2181,hdh199:2181
  hbase.zookeeper.quorum: hdh98:2181,hdh197:2181,hdh199:2181
  hbase.zookeeper.property.clientPort: 2181

查看了错误原因是因为解析时候对key的分割处理导致的。
read.go 文件中,144行处。

	if !strings.Contains(key, ".") {
		// c.addError(errNotFound)
		return
	}

	keys := strings.Split(key, ".")
	topK := keys[0]

修改后:

	if !strings.Contains(key, ":") {
		// c.addError(errNotFound)
		return
	}

	keys := strings.Split(key, ":")
	topK := keys[0]

在解析文件中希望可以修改,修改后查询使用的方式如下:
value2 := cfg.String("topics:0:properties:cleanup.policy")

[FEAT] Support Set with Slice (Array) Index Using Square Brackets?

It would be nice to support slice index in .Set()
Example:

config.Set("parent.child[0]", "Test1")
config.Set("parent.child[1]", "Test2")

Currently, the result will be like:

{
    "parent": {
        "child[0]": "Test1",
        "child[1]": "Test1"
    }
}

==> Expecting result would look like:

{
    "parent": {
        "child": [
          "Test1",
          "Test2"
        ]
    }
}

Thanks,

BindStruct doesn't seem to work with env var substitution

conf.gookit.json

{
  "http": {
    "port": "${HTTP_PORT|8080}"
  }
}

main.go

import (
	gookit "github.com/gookit/config/v2"
	gookitJson "github.com/gookit/config/v2/json"
)

type Http struct {
	Port int
}

func main() {
	c := gookit.NewWithOptions("conf", gookit.ParseEnv, gookit.Readonly, gookit.EnableCache)
	c.AddDriver(gookitJson.Driver)

	err := c.LoadFiles("conf.gookit.json")
	if err != nil {
		panic(err)
	}

	var t Http
	err = c.BindStruct("http", &t)
	if err != nil {
		panic(err)
	}

Error is "cannot parse 'Port' as int: strconv.ParseInt: parsing "${HTTP_PORT|8080}": invalid syntax"

mapstructure.DecoderConfig.Squash support config in Options()

cfgLoader := config.NewWithOptions("", func(options *config.Options) {
    options.TagName = "json"
    options.Squash  = true
})

type Test1 struct {
    B int `json:"b"`
}
type Test2 struct {
    Test1
    C int `json:"c"`
}
cfg := &Test2{}

cfgLoader.BindStruct("", cfg)
mapConf := &mapstructure.DecoderConfig{
		Metadata: nil,
		Result:   dst,
		TagName:  c.opts.TagName,
                Squash :  c.opts.Squash ,                   // support embedded structs
		// will auto convert string to int/uint
		WeaklyTypedInput: true,
	}

Support slice with ParseEnv

System (please complete the following information):

  • OS: linux
  • GO Version: 1.19.4
  • Pkg Version: 2.1.8

Describe the bug

Slice not parser with environment variables.

To Reproduce

package main

import (
	"github.com/gookit/config/v2"
)

var version = "dev"

type conf struct {
	Name  string   `mapstructure:"name" default:"${NAME | Bob}"`
	Value []string `mapstructure:"value" default:"${VAL | val1}"`
}

func main() {

	config.WithOptions(
		config.ParseDefault,
		config.ParseEnv,
		config.Readonly,
	)

	err := config.LoadExists("")
	if err != nil {
		panic(err)
	}

	var cc conf
	if err := config.Decode(&cc); err != nil {
		panic(err)
	}
}

Actual behavior

panic: convert value type error

goroutine 1 [running]:
main.main()
        /run/media/unikum/UNIKUM-STORAGE/work/centrofinans/dev/kafka-tb-translator/tools/conf/main.go:29 +0xd1
exit status 2

Panic on missing key/value

Hi,

is there any support for panic on missing config keys and values?

Right now I have a wrapper code that calls the config tool with direct reads to accomplish that:

// Code snippet for panic on missing key/value
func (*ConfigFileReader) MustHaveString(key string) string {
	value := config.String(key)

	if len(value) == 0 {
		log.Panic("Config file key not found or has no value: ", key)
	}

	return value
}

Best
Gabriele

Add Duration type support

conf.Duration(key) = time.Duration
---
Save Stepstimeout := time.Duration(Conf.Int64("timeout"))
  timeoutDuration := time.Second * timeout 

LoadOSEnvs not work

System (please complete the following information):

  • OS: linux [e.g. linux, macOS]
  • GO Version: 1.13 [e.g. 1.13]
  • Pkg Version: 1.1.1 [e.g. 1.1.1]

Describe the bug

type ConferenceConfigure struct {
  AuthServerEnable   bool   `mapstructure:"authServerEnable" default:"true"`
}

var ENVS = map[string]string{
  "CONF_AUTH_SERVER_ENABLE":    "authServerEnable",
}

config.WithOptions(config.ParseEnv, config.ParseTime, config.ParseDefault)
config.LoadOSEnvs(ENVS)
print(config.Bool("authServerEnable")) // false
err = config.LoadExists(XXX) // XXX not exists
print(config.Bool("authServerEnable")) // false
config.Decode(CONFIGURE)
print(config.Bool("authServerEnable")) // false
// but CONFIGURE.AuthServerEnable  == true

A clear and concise description of what you expected to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

More of a question setDecoder manually

we have our legacy system where I would want to replace our existing config manager with the gookit the problem I'm facing straight up with integration is past developers have used .conf for ini file.

In my attempt to let gookit/config to consider .conf to consider as an INI decoder I stumble

config.SetDecoder("conf", ini.Driver.GetDecoder())

All works well but the API seem to be deprecated and I'm not sure how to ensure that how this can be done with the latest API

        config.SetDecoder("conf", ini.Driver.GetDecoder())
	config.AddDriver(ini.Driver)
	config.WithOptions(
		config.ParseEnv,
		config.WithHookFunc(func(event string, c *config.Config) {
			if event == config.OnReloadData {
				zap.L().Info("config is reloaded")
			}
		}),
	)

	err := config.LoadFiles(
		constants.CONFIG_INI,
		constants.SUGARBOX_INI,
	)

	if err != nil {
		panic(err)
	}

	go watchConfigFiles(config.Default())

Let me know anything that can be done here that not deprecated.

config does not use ParseEnv option on nested keys.

System (please complete the following information):

  • OS: linux
  • GO Version: 1.16.4
  • Pkg Version: 2.0.23

Describe the bug

config does not use ParseEnv option on nested keys.

To Reproduce

import (
    config "github.com/gookit/config/v2"
)

func BugReport() {
	jsonStr := `{
    "envKey": "${SHELL}",
    "map1": {
        "key1": "${SHELL}"
    }
}`

	config.AddDriver(config.JSONDriver)
	config.WithOptions(config.ParseEnv)
	_ = config.LoadStrings(config.JSON, jsonStr)
	// works ok /bin/my-shell
	fmt.Println(config.String("envKey", ""))
	map1 := config.StringMap("map1")
	// "${SHELL}"
	fmt.Println("get map", map1["key1"])
}

Expected behavior
should give me /bin/bash

fmt.Println("get map", map1["key1"])

文件名后缀作为判断依据的问题

对于一些工具的配置文件,格式虽然可能是ini,yaml或者json
但是为了区分这些文件的作用,可能不再是ini,yaml这样的后缀,
而是会以rules,system这样的后缀结尾,
而我看你的parseSourceCode()中会检查这样的后缀,作为文件格式的判断依据
但是对于上述描述到的文件,就会被认定为不合法的文件格式,

对于这样的情况能否给个方法,可以让我直接指定文件格式,从而解决这样的问题呢?

Config.Error()

System (please complete the following information):

  • OS: macOS
  • GO Version: 1.17.5
  • Pkg Version: v2.1.1

Describe the bug
当我的操作引发err并获取err后,再进行其他合法操作时,最后一次err没有被清除

To Reproduce

var readonly atomic.Value

func Load(bytes []byte) error {
	c := config.New("xxx")

	c.AddDriver(toml.Driver)

	err := c.LoadSources("toml", bytes)
	if err != nil {
		return err
	}

	readonly.Store(c)
	return nil
}

func GetConf() *config.Config {
	return readonly.Load().(*config.Config)
}
func TestConf(t *testing.T) {
	bytes := []byte(`[redis]
sentinel.urls = [
    "xxxx:32501",
    "xxxx:32501",
    "xxxx:32501",
]
database = 7`)

	err := Load(bytes)
	if err != nil {
		t.Fatal(err)
	}

	conf := GetConf()

	conf.Strings("redis.database")
	fmt.Println(conf.Error())

	urls := conf.Strings("redis.sentinel.urls")
	fmt.Println(conf.Error())
}
value cannot be convert to []string, key is 'redis.database'
value cannot be convert to []string, key is 'redis.database'

Expected behavior

是不是应该改成这样,不知道拷贝err这种方式合不合适

func (c *Config) Error() error {
         err := c.err
         c.err = nil
	return err
}

默认值解析空数组时,会自动生成一个元素(附带上默认值)

关联: #141
示例(示例代码来自 issue 141的测试代码,删掉 json content)

// https://github.com/gookit/config/issues/141
func TestIssues_141(t *testing.T) {
	type Logger struct {
		Name     string `json:"name"`
		LogFile  string `json:"logFile"`
		MaxSize  int    `json:"maxSize" default:"1024"` // MB
		MaxDays  int    `json:"maxDays" default:"7"`
		Compress bool   `json:"compress" default:"true"`
	}

	type LogConfig struct {
		Loggers []*Logger `default:""` // mark for parse default
	}

	c := config.New("issues_141", config.ParseDefault)
	err := c.LoadStrings(config.JSON, `
{
}
`)

	assert.NoErr(t, err)

	opt := &LogConfig{}
	err = c.Decode(opt)
	dump.Println(opt)

output:

PRINT AT github.com/gookit/config/v2_test.TestIssues_141(issues_test.go:351)
&config_test.LogConfig {
  Loggers: []*config_test.Logger [ #len=1,cap=1
    &config_test.Logger {
      Name: string(""), #len=0
      LogFile: string(""), #len=0
      MaxSize: int(1024),
      MaxDays: int(7),
      Compress: bool(true),
    },
  ],
},

feat: support parse ENV var on struct default tag

Examples:

type MyConf struct {
	Env string `default:"${APP_ENV | dev}"`
	Debug bool `default:"${APP_DEBUG | false}"`
}

cfg := NewWithOptions("test", ParseEnv, ParseDefault)

mc := &MyConf{}
_ = cfg.Decode(mc)

Output:

&config.MyConf {
  Env: string("dev"), #len=3
  Debug: bool(false),
},

Support Pretty Print?

Would be nice to have support for pretty print out-of-the-box like Viper.
Currently: config.Set()

{"parent":{"child":{"grandChild":"Test Val"}}}

Expecting:

{
  "parent": {
    "child": {
      "grandChild": "Test Val"
    }
  }
}

Is this posible?
Or any work-around for this?

Dependabot can't resolve your Go dependency files

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

go: github.com/BurntSushi/[email protected] requires
	github.com/BurntSushi/[email protected] requires
	github.com/BurntSushi/[email protected] requires
	github.com/BurntSushi/[email protected] requires
	github.com/BurntSushi/[email protected] requires
	github.com/BurntSushi/[email protected] requires
	github.com/BurntSushi/[email protected]: invalid version: unknown revision

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

How to parse ini to struct if data within [brackets] is variable?

Hello,
I was wondering how I might go about parsing to a struct a file like below:

[variablename1]
type = 
client_id = 
client_secret = 
scope = 
token = 

[variablename1] will always be unique as it is a name assigned at the time of creation of the ini, the rest of the fields will always have the same key name.

I was going to have:

type Settings struct {
	Remote struct {
		Type         string `json:"type"`
		ClientId     string `json:"client_id"`
		ClientSecret string `json:"client_secret"`
		Scope        int    `json:"scope"`
		Token        int    `json:"token"`
	} `json:"???"` // --- This will always be different
}

Is there a way to deal with this?
Thanks,
-MH

[FEAT] Support Encrypted Config File

It would be nice to have a built-in encrypted config file so sometime we are only allow user to change setting directly from the app.

I have 2 function to encrypt and decrypt the string to base64, just not sure how to implement it...

// Encrypt a string with a secret key.
// Secretkey must be 16, 24 or 32 characters long.
func EncryptStr(text, secretKey string) (string, error) {
	var randBytes = []byte{35, 46, 57, 24, 85, 35, 24, 74, 87, 35, 88, 98, 66, 32, 14, 05}
	block, err := aes.NewCipher([]byte(secretKey))
	if err != nil {
		return "", err
	}
	plainText := []byte(text)
	cfb := cipher.NewCFBEncrypter(block, randBytes)
	cipherText := make([]byte, len(plainText))
	cfb.XORKeyStream(cipherText, plainText)
	endCodeCipherText := base64.StdEncoding.EncodeToString(cipherText)
	return endCodeCipherText, nil
}

// Decrypt am encrypt string with the same secret key used in encrypt.
// Secretkey must be 16, 24 or 32 characters long.
func DecryptStr(eText, secretKey string) (string, error) {
	var randBytes = []byte{35, 46, 57, 24, 85, 35, 24, 74, 87, 35, 88, 98, 66, 32, 14, 05}
	block, err := aes.NewCipher([]byte(secretKey))
	if err != nil {
		return "", err
	}
	cipherText, _ := base64.StdEncoding.DecodeString(eText)
	cfb := cipher.NewCFBDecrypter(block, randBytes)
	plainText := make([]byte, len(cipherText))
	cfb.XORKeyStream(plainText, cipherText)
	return string(plainText), nil
}

Thought?

Fail fast for required values

I'm integrating gookit/config in an existing project in order to manage the configuration. I've ran into some issues, but I've been able to workaround them in one way or another — I'm pretty new to Go as well 😬

What I've been able to do is to be able to interrupt the application bootstrapping when there is a missing value. For instance, consider the following configuration:

---
datasource:
  password: ${DATABASE_PASSWORD|unset}
  type: postgres
  username: ${DATABASE_USERNAME|postgres}
  url: ${DATABASE_URL|unset}

It would be great to have the behavior that Docker Compose provides to fail whenever a value for an environment variable is not present. Basically, if implemented exactly the same (bonus points!) the YAML file would be like this:

---
datasource:
  password: ${DATABASE_PASSWORD|?error}
  type: postgres
  username: ${DATABASE_USERNAME|postgres}
  url: ${DATABASE_URL|?error}

ini: invalid data to encode as ini

解析ini文件时候,使用了ini.Driver
但在把data写会文件时候,发生了"ini: invalid data to encode as ini"
使用(* config)DumpTo() method
查看代码发现ini的Encode代码是这么写的:

func Encode(v interface{}, defSection ...string) (out []byte, err error) {
    switch vd := v.(type) {
    case map[string]interface{}: // from full mode
        return EncodeFull(vd, defSection...)
    case map[string]map[string]string: // from simple mode
        return EncodeSimple(vd, defSection...)
    default:
        err = errors.New("ini: invalid data to encode as ini")
    }
    return
}

但是在func (c *Config) DumpTo(out io.Writer, format string) (n int64, err error)却是这么调用的

    // encode data to string
    encoded, err := encoder(&c.data)
    if err != nil {
        return
    }

取的是c.data的地址(&c.data),所以type永远匹配不上
所以这里或许是个bug,还是说,这里我的用法有问题,我的代码如下:

func main() {
    r := config.New("yyyy")
    r.WithOptions(config.ParseEnv)

    r.AddDriver(ini.Driver)

    err := r.LoadFiles("yyyy.ini")
    if err != nil {
        panic(err)
    }

    fmt.Printf("config data: \n %#v\n", r.Data())

    f, e := os.OpenFile("xxxx.ini", os.O_RDWR|os.O_CREATE, 0666)
    if e != nil {
        fmt.Println(e)
        return
    }
    defer f.Close()

    n, e := r.DumpTo(f, config.Ini)
    fmt.Println(n, e)
}

请问路径分隔符,能否支持自定义?

假设 key 存在 . 符号,是否会存在误区,分隔符能否自定义?假设 key 中存在域名怎么办?

若能自定义分隔符,是否可以 map 多维到一维互转?

准备用 go 做一个开源程序,主要涉及 json yaml ini 这一块。
需要实现一个支持将多维扁平化处理,将一维多维处理。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.