Giter Club home page Giter Club logo

demystifyfp / fsconfig Goto Github PK

View Code? Open in Web Editor NEW
156.0 9.0 17.0 479 KB

FsConfig is a F# library for reading configuration data from environment variables and AppSettings with type safety.

Home Page: https://www.demystifyfp.com/FsConfig/

License: The Unlicense

Batchfile 0.06% F# 92.28% Shell 0.47% Dockerfile 0.58% JavaScript 2.42% CSS 4.18%
fsharp functional-programming generic-programming environment-variables 12-factor appsettings configuration configuration-management

fsconfig's Introduction

FsConfig

FsConfig is a F# library for reading configuration data from environment variables and AppSettings with type safety

Nuget Build master

Why FsConfig?

To understand FsConfig, let's have a look at an use case from the FsTweet application.

The FsTweet application follows The Twelve-Factor App guideline for managing the configuration data. During the application bootstrap, it retrieves its ten configuration parameters from their respective environment variables.

open System

let main argv =

  let fsTweetConnString = 
   Environment.GetEnvironmentVariable  "FSTWEET_DB_CONN_STRING"

  let serverToken =
    Environment.GetEnvironmentVariable "FSTWEET_POSTMARK_SERVER_TOKEN"

  let senderEmailAddress =
    Environment.GetEnvironmentVariable "FSTWEET_SENDER_EMAIL_ADDRESS"

  let env = 
    Environment.GetEnvironmentVariable "FSTWEET_ENVIRONMENT"

  let streamConfig : GetStream.Config = {
      ApiKey = 
        Environment.GetEnvironmentVariable "FSTWEET_STREAM_KEY"
      ApiSecret = 
        Environment.GetEnvironmentVariable "FSTWEET_STREAM_SECRET"
      AppId = 
        Environment.GetEnvironmentVariable "FSTWEET_STREAM_APP_ID"
  }

  let serverKey = 
    Environment.GetEnvironmentVariable "FSTWEET_SERVER_KEY"

  let port = 
    Environment.GetEnvironmentVariable "PORT" |> uint16

  // ...

Though the code snippet does the job, there are some shortcomings.

  1. The code is verbose.
  2. There is no error handling to deal with the absence of values or wrong values.
  3. Explicit type casting

With the help of FsConfig, we can overcome these limitations by specifying the configuration data as a F# Record type.

type StreamConfig = {
  Key : string
  Secret : string
  AppId : string
}

[<Convention("FSTWEET")>]
type Config = {

  DbConnString : string
  PostmarkServerToken : string
  SenderEmailAddress : string
  ServerKey : string
  Environment : string

  [<CustomName("PORT")>]
  Port : uint16
  Stream : StreamConfig
}

And then read all the associated environment variables in a single function call with type safety and error handling!

let main argv =

  let config = 
    match EnvConfig.Get<Config>() with
    | Ok config -> config
    | Error error -> 
      match error with
      | NotFound envVarName -> 
        failwithf "Environment variable %s not found" envVarName
      | BadValue (envVarName, value) ->
        failwithf "Environment variable %s has invalid value %s" envVarName value
      | NotSupported msg -> 
        failwith msg

Supported Data Types

FsConfig supports the following data types and leverages their respective TryParse function to do the type conversion.

  • Int16, Int32, Int64, UInt16, UInt32, UInt64
  • Byte, SByte
  • Single, Double, Decimal
  • Char, String
  • Bool
  • TimeSpan, DateTimeOffset, DateTime
  • Guid
  • Enum
  • list of all the above types
  • option of all the above types

Option Type

FsConfig allows us to specify optional configuration parameters using the option type. In the previous example, if the configuration parameter Port is optional, we can define it like

type Config = {
   ...
-  Port : uint16
+  Port : uint16 option
}

Discriminated Union Type

FsConfig supports Discriminated Union Types that has cases alone.

type Color =
| Red
| Green
| Blue 

type Config = {
  ConsoleColor : Color
}

With this configuration declaration, FsConfig read the environment variable CONSOLE_COLOR and populates the ConsoleColor field of type Color.

List of Discriminated Union Types also supported!

List Type

FsConfig also supports list type, and it expects comma separated individual values.

For example, to get mulitple ports, we can define the config as

type Config = {
  Port : uint16 list
}

and then pass the value 8084,8085,8080 using the environment variable PORT.

The default separator for the list can be changed if needed using the ListSeparator attribute.

  [<Convention("MYENV")>]
  type CustomListSeparatorSampleConfig = {
    ProcessNames : string list
    [<ListSeparator(';')>]
    ProcessIds : uint16 list
    [<ListSeparator('|')>]
    PipedFlow : int list    
  }

With this configuration declaration, FSConfig would be able to read the following entries from App.settings.

  <add key="MYENVProcessNames" value="conhost.exe,gitter.exe"/>
  <add key="MYENVProcessIds" value="4700;15680"/>
  <add key="MYENVPipedFlow" value="4700|15680|-1" />

A definition similar to the one shown below will allow parsing of standalone lists.

  type IntListUsingSemiColonsConfig = {
    [<ListSeparator(';')>]
    IntListUp : int list
  }

E.g. an environment containing

INT_LIST_UP=42;43;44 

Record Type

As shown in the initial example, FsConfig allows us to group similar configuration into a record type.

type AwsConfig = {
  AccessKeyId : string
  DefaultRegion : string
  SecretAccessKey : string
}

type Config = {
  Aws : AwsConfig
}

With this configuration declaration, FsConfig read the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION and populates the Aws field of type AwsConfig.

Default Value

If you'd like to use a default value in the absence of a field value, you can make use of the DefaultValue attribute.

type Config = {
  [<DefaultValue("8080")>]
  HttpServerPort : int16
  [<DefaultValue("Server=localhost;Port=5432;Database=FsTweet;User Id=postgres;Password=test;")>]
  DbConnectionString: string
}

Environment Variable Name Convention & Customization

By default, FsConfig follows Underscores with uppercase convention, as in UPPER_CASE, for deriving the environment variable name.

type Config = {
  ServerKey : string
}

Using this configuration declaration, FsConfig read the environment variable SERVER_KEY and populates the ServerKey field

To specify a custom prefix in the environment variables, we can make use of the Convention attribute.

[<Convention("FSTWEET")>]
type Config = {
  ServerKey : string
}

For this configuration declaration, FsConfig read the environment variable FSTWEET_SERVER_KEY and populates the ServerKey field.

We can also override the separator character _ using the Convention attribute's optional field Separator

[<Convention("FSTWEET", Separator="-")>]
type Config = {
  ServerKey : string
}

In this case, FsConfig derives the environment variable name as FSTWEET-SERVER-KEY.

If an environment variable name is not following a convention, we can override the environment variable name at the field level using the CustomName attribute.

type Config = {
  [<CustomName("MY_SERVER_KEY")>]
  ServerKey : string
}

Here, FsConfig uses the environment variable name MY_SERVER_KEY to get the ServerKey.

We can also merely customise (or control) the environment variable name generation by passing an higher-order function while calling the Get function

open FsConfig

// Prefix -> string -> string
let lowerCaseConfigNameCanonicalizer (Prefix prefix) (name : string) = 
  let lowerCaseName = name.ToLowerInvariant()
  if String.IsNullOrEmpty prefix then 
    name.ToLowerInvariant()
  else
    sprintf "%s-%s" (prefix.ToLowerInvariant()) lowerCaseName


[<Convention("FSTWEET")>]
type Config = {
  ServerKey : string
}

let main argv =
  let config = 
    match EnvConfig.Get<Config> lowerCaseConfigNameCanonicalizer with
    | Ok config -> config
    | Error error -> failwithf "Error : %A" error

FsConfig computes the environment variable name as fstweet-server-key in this scenario.

Getting Individual Environment Variables

FsConfig also supports reading value directly by explicitly specifying the environment variable name

EnvConfig.Get<decimal> "MY_APP_INITIAL_BALANCE" // Result<decimal, ConfigParseError>

App Config

FsConfig supports App Config for both DotNet Core and Non DotNet Core Applications.

DotNet Core Configuration (Supported from V2.0.0 or above)

FsConfig abstracts the configuration provider by depending on IConfigurationRoot.

let configurationRoot : IConfigurationRoot = // ...
let appConfig = new AppConfig(configurationRoot)

After creating an instance appConfig (of type AppConfig from FsConfig), you can use it to read the configuration values as below

// Reading Primitive
let result = 
  appConfig.Get<int> "processId" // Result<int, ConfigParseError>

// A Sample Record
type SampleConfig = {
  ProcessId : int
  ProcessName : string
}

// Reading a Record type
let result = 
  appConfig.Get<SampleConfig> () // Result<SampleConfig, ConfigParseError>

// A Sample Nested Record
type AwsConfig = {
  AccessKeyId : string
  DefaultRegion : string
  SecretAccessKey : string
}

type Config = {
  MagicNumber : int
  Aws : AwsConfig
}

// Reading a Nested Record type
let result = 
  appConfig.Get<Config> () // Result<Config, ConfigParseError>

Refer below for creating configurationRoot based on the file type and using FsConfig to read the values.

JSON

{
  "processId" : "123",
  "processName" : "FsConfig",
  "magicNumber" : 42,
  "aws" : {
    "accessKeyId" : "Id-123",
    "defaultRegion" : "us-east-1",
    "secretAccessKey" : "secret123"
  },
  "colors" : "Red,Green"
}

This JSON file can be read using

// Requires NuGet package
// Microsoft.Extensions.Configuration.Json
let configurationRoot =  
  ConfigurationBuilder().SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("settings.json").Build()

let appConfig = new AppConfig(configurationRoot)
let result = 
  appConfig.Get<Config> () // Result<Config, ConfigParseError>

XML

<Settings>
  <ProcessId>123</ProcessId>
  <ProcessName>FsConfig</ProcessName>
  <MagicNumber>42</MagicNumber>
  <Aws>
    <AccessKeyId>Id-123</AccessKeyId>
    <DefaultRegion>us-east-1</DefaultRegion>
    <SecretAccessKey>secret123</SecretAccessKey>
  </Aws>
  <Colors>Red,Green</Colors>
</Settings>

This XML file can be read using

// Requires NuGet package
// Microsoft.Extensions.Configuration.Xml
let configurationRoot =  
  ConfigurationBuilder().SetBasePath(Directory.GetCurrentDirectory())
    .AddXmlFile("settings.xml").Build()

let appConfig = new AppConfig(configurationRoot)
let result = 
  appConfig.Get<Config> () // Result<Config, ConfigParseError>

INI

ProcessId=123
ProcessName=FsConfig
MagicNumber=42
Colors=Red,Green

[Aws]
AccessKeyId=Id-123
DefaultRegion=us-east-1
SecretAccessKey=secret123

This INI file can be read using

// Requires NuGet package
// Microsoft.Extensions.Configuration.Ini
let configurationRoot =  
  ConfigurationBuilder().SetBasePath(Directory.GetCurrentDirectory())
    .AddIniFile("settings.ini").Build()

let appConfig = new AppConfig(configurationRoot)
let result = 
  appConfig.Get<Config> () // Result<Config, ConfigParseError>

appSettings (Only Supported in V0.0.6 or below)

We can read the appSettings values using the AppConfig type instead of EnvConfig type.

FsConfig uses the exact name of the field to derive the appSettings key name and doesn't use any separator by default.

type AwsConfig = {
  AccessKeyId : string
  DefaultRegion : string
  SecretAccessKey : string
}

type Config = {
  Port : uint16
  Aws : AwsConfig
}

let main argv =
  let config = 
    match AppConfig.Get<Config>() with
    | Ok config -> config
    | Error error -> failwithf "Error : %A" error

The above code snippet looks for appSettings values with the name Port, AwsAccessKeyId, AwsDefaultRegion, AwsSecretAccessKey and populates the respective fields.

All the customisation that we have seen for EnvConfig is applicable for AppConfig as well.

How FsConfig Works

If you are curious to know how FsConfig works and its internals then you might be interested in my blog post, Generic Programming Made Easy that deep dives into the initial implementation of FsConfig.

Feedback

We all need people who will give us feedback. That's how we improve - Bill Gates.

Your suggestions/feedback are welcome!

Acknowledgements

The idea of FsConfig is inspired by Kelsey Hightower's golang library envconfig.

FsConfig uses Eirik Tsarpalis's TypeShape library for generic programming.

Maintainer(s)

fsconfig's People

Contributors

2mol avatar aurecchia avatar demystifyfp avatar eugene-g avatar mtnrbq avatar piaste avatar queil avatar rajivhost avatar tamizhvendan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fsconfig's Issues

Optional subsections does not work.

Optional subsections does not work for .json files

type SubSection = {
    valueA : string
}

type Settings = {
   subSection : SubSection option
}

Reading a json with the subsection in place gives a null value.

Provide override to change list separator

Description

I have legacy apps using ';' as a list separator

Expected behavior

Ideally, I'd like to be able to specify a list separator, e.g. in a similar way to how the environment variable name separator is specified:

[<Convention("FSTWEET", Separator="-",ListSeparator=";")>]

Actual behavior

Current implementation only supports ',' as list separator

Known workarounds

No known workaround

I'm happy to submit a PR for the change if you believe it is worthwhile

FsConfig doesn't support JSON Array

Description

FsConfig doesn't support JSON Array

Repro steps

  1. Define config.json
    here is a JSON:
    {
    data: ["1", "2"]
    }

  2. Define F# type
    type Config = {
    Data: string list
    }

  3. Parse
    let config = ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("config.json")
    .Build()
    |> AppConfig

let result = config.Get()

Expected behavior

result should contain not empty Config.Data

Actual behavior

result contains empty Config.Data

  • Operating system Windows 10 x64

Consecutive defaults

When working with configurations, sometimes they get overwritten depending on the origin. The order of the defaulting is discussable, but I think it is

  • read environment vars
  • read configuration file 1, maybe from server
  • read local configuration file
  • read arguments

and of course starting with settings that the programmer considers perfect as defaults.

type Settings { showAsWindow: bool; windowXsize: int; windowYsize: int }
let DefaultSettings = { true; 1024; 768 }

wouldn't it be nice to have a configuration chain like

let setup = DefaultSettings
|> EnvConfig.Get<Settings>
|> IniConfig.Get<Settings>
|> ArgConfig.Get<Settings>

Is this possible somehow?

TypeShape conflict

Hi,
I have a TypeShape conflict when I try to install FsConfig and Equinox.Codec as shown below

image

Can we hope to have an update of the dependency here?
Best regards,
Rajiv.

Docs: fix rendering

Describe the bug
At the moment docs generation is broken.

To Reproduce

  1. Run ./build.sh ReleaseDocs

Error:

Fatal error: System.Exception: error while formatting file /home/queil/gh/FsConfig/docsSrc/content/index.fsx. Errors are:
seq
  [SourceError
     ((105, 2), (105, 5), ErrorKind.Error,
      "The block following this 'let' is unfinished. Every code block is an expression and must have a result. 'let' cannot be the final code element in a block. Consider giving this block an explicit result.");
   SourceError
     ((48, 21), (48, 30), ErrorKind.Error,
      "The namespace or module 'GetStream' is not defined.");
   SourceError
     ((49, 6), (49, 12), ErrorKind.Error,
      "The record label 'ApiKey' is not defined.");
   SourceError
     ((83, 2), (83, 12), ErrorKind.Error,
      "The type 'Convention' is not defined."); ...]
   at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1639.Invoke(String message) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1639

It seems (*** do-not-eval-file ***) is not respected.

Expected behavior
A new version of docs gets published to the docs dir.

Split Microsoft.Extensions.Configuration-dependent features into a separate NuGet

Description

FsConfig depends on Microsoft.Extensions.Configuration but this dependency is only needed when using IConfigurationRoot. Wouldn't it make sense to split the package in two so the users not using Microsoft's extension do not need to needlessly deal with this dependency?

Repro steps

Add FsConfig package to a project and do dotnet restore.

Expected behavior

Can reference a "core" FsConfig without dependency on Microsoft.Extensions.Configuration and the NU1603 warning does not appear.

Actual behavior

Getting this warning:

warning NU1603: FsConfig 2.1.5 depends on Microsoft.Extensions.Configuration (>= 0.0.0) but Microsoft.Extensions.Configuration 0.0.0 was not found. An approximate best match of Microsoft.Extensions.Configuration 1.0.0 was resolved.

Context

  • Os: Ubuntu 18.04
  • .NET Core 3.1.101
  • FsConfig 2.1.5

Possible to supply dynamic keys?

Description

Just curious if there's any way to supply dynamic keys, like a Map<string, string>? eg. if I had a top-level Connections key, I could add in an entry for each DB connection.

I tried using Dictionary<string, string> but got a not implemented error.

Related information

  • macOS Mojave
  • FSconfig 2.1.6
  • 3.1.300 .NET core

Could not load file or assembly 'TypeShape, Version=8.0.1.0'

Description

Retrieving the config fails with Could not load file or assembly 'TypeShape, Version=8.0.1.0,...

Repro steps

  1. Create a new project referencing FsConfig
  2. Create a config record type and use the default match expression from the docs to retrieve the config

Expected behavior

Config gets successfully retrieved.

Actual behavior

The following exception gets thrown:

Unhandled exception. System.IO.FileLoadException: Could not load file or assembly 'TypeShape, Version=8.0.1.0, Culture=neutral, PublicKeyToken=null'. The located assembly's manifest definition does not match the assembly reference. (0x80131040)
File name: 'TypeShape, Version=8.0.1.0, Culture=neutral, PublicKeyToken=null'
   at FsConfig.Core.parseInternal[T](IConfigReader configReader, FSharpFunc`2 fieldNameCanonicalizer, FieldValueParseArgs args)
   at Program.main(String[] argv) in

Known workarounds

Add TypeShape directly to the project:

    <PackageReference Include="TypeShape" Version="8.0.1" />

Related information

  • Operating system: Ubuntu 18.04
  • FsConfig: 2.1.5
  • .NET Core 3.1.101

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.