Currently we have a read that returns ConfigReport and A.
Same time we have doc(configDesc) that writes a man page.

What we ideally need is
doc(configDesc, A) and doc(configDesc), of which the first function will provide the values held by each path as well.

In that way read looks just simpler for the user, and avoid them working with tuple.

A Map[String, String] as the basis for properties means that no nested configuration can be supported, which is inherently anti-modular; in addition, many third-party libraries that could work well with ZIO Config support nesting (HOCON, YAML, JSON, etc.).

To support nesting, it would be necessary to transition away from Map[String, String] to an alternate data structure. This data structure would support nesting but not require it: in the sense that it should be possible to create a value of this type from any Map[String, String], so that flat formats are supported. Similarly, it should be possible (probably?) to "flatten" the type into something completely flat, so we can read even nested structures from flat sources like environment variables.

If we take into consideration the design discussion around Pure, then we will end up with a design similar to the following:

sealed trait PropertyTree
object PropertyTree {
  final case class Leaf(value: String) extends PropertyTree
  final case class Record(value: Map[String, PropertyTree]) extends PropertyTree

  def fromMap(map: Map[String, String]): PropertyTree = ???
}

Probably, we should make that polymorphic and defer specialization to the point where we need it:

sealed trait PropertyTree[+K, +V] {
  def flatten(zero: K, append: (K, K) => K): Map[K, V] = ???
}
object PropertyTree {
  final case class Leaf[V](value: V) extends PropertyTree[Nothing, V]
  final case class Record[K, V](value: Map[K, PropertyTree[K, V]]) extends PropertyTree[K, V]

  def fromMap(map: Map[String, String]): PropertyTree[String, String] = ???

  def unflatten[K, V](map: Map[List[K], V]): PropertyTree[K, V] = ???
}

Using a structure like this, it should be possible to support nested configuration formats.

Note that currently we will have in all cases PropertyTree[String, String]. That's because our PropertyType is a string-based codec. We could think about the connection there and whether to break the dependency on string or not, but moving away from strings would definitely require a change to PropertyType.

Probably report the source during the read as well as a tuple.

read(config).map{case(source, value) => generateDocsWithValue(...))

@jdegoes This can be a quite nice way of putting it through within a couple of hours. I liked about it is, we still keep orthogonality. What the opinion?

1. Better layout.
2. More accessibility to the contents with in each page.
3. Better home page (probably with images)

currently in the config package those methods are using different data type

  def read[A](config: => Config[A]): Read[A]     = Read.read[A](config)
  def write[A](config: => Config[A]): Write[A]   = Write.write[A](config)
  def report[A](config: => Config[A]): Report[A] = Report.report[A](config)

Move the implementation of Read, Write and Report to config and make them methods instead of data types.
those methods should return ZIO

  def read[A](config: => Config[A]): ZIO[ConfigSource, List[ReadError], (ConfigReport, A)]  
  def write[A](config: => Config[A]): Map[String, String]
  def report[A](config: => Config[A]): ConfigReport

Example:

object ConfigSpec
    extends DefaultRunnableSpec(
      suite("config suite")(
        testM("optional Int example") {

          case class ConfigInternal private (
            domain: String,
            maybePort: Option[Int]
          )
          val envVars: Map[String, String] = Map(
            "DOMAIN" -> "dom1",
            "PORT"   -> "NOT_AN_INT"
          )
          val configDescriptor: ConfigDescriptor[String, String, ConfigInternal] =
            (string("DOMAIN") |@| int(
              "PORT"
            ).optional)(ConfigInternal.apply, ConfigInternal.unapply)

          val configM: IO[ReadErrors[Vector[String], String], config.Config[ConfigInternal]] =
            zio.config.Config.fromMap(envVars, configDescriptor)

          for {
            c        <- configM
            internal <- c.config.config
          } yield assert(internal, equalTo(ConfigInternal("dom1", None)))
          // succeeds but we would expect Optional[Int] field to fail when given data "NOT_AN_INT"
        }
      )
    )

As of now, generateDocs and generateDocsWithValue returns ConfigDocs, a datastructure that sort of act as a manpage and report respectively. We need to pipe this to much more readable format.

1. Markdown
2. A simple json (EDIT: deprioritised)
3. Help Page
4. Or a simple table.

In-fact, we need support for all of it..

• Being able to read and write hoccon
• Being able to support for all type safe config formats.

zio-config, from its design, already support nested formats. For the same reason, there shouldn't be any major roadblock towards this integration. This is probably high priority issue.

Also called collectAll

This behaviour is visible here:
https://github.com/leigh-perry/zio-config/blob/master/examples/src/main/scala/zio/config/examples/SequenceExample.scala#L44

Fix available.... change foldLeft to foldRight

Is desirable to have this feature?

We will have to support reading config values as a list, and this involves a bit of design discussions, specifically about generalising the Map[String, String] data type at the core

Depends on
#18

 trait Config[A] {
    def <?>(description: String): Config[A] =
      Config.Describe(self, description) 
}

And the new functions that made use of PropertyType descriptions will be

  def int(path: String): Config[Int] =
    Config.Source(path, PropertyType.IntType) <?> "value of type int"

This allows more generic descriptions to be tagged against the overall config (and its nodes)

Short answer is no. However views are welcome.

case class Id(value: String)
case class X(id: Id, priority: Int )
case class PgmConfig(id: Either[Id, X] ) 


val id = string("PATH").xmap(Id)(_.value)

// just for the sake of reuse, if we do
val dev = id.withPrefix("DEV")
val prod = id.withPrefix("PROD")
val prodRelated = (prod <*> int("PRIORITY")(X.apply, X.unapply)

val config: Config[PgmConfig] = 
  (dev or prodRelated)(Config.apply, Config.unapply)

A few points:

I would imagine users hardly need this by the way.
If both Id was formed from the same path, then as far as Id exists in the environment, prodRelated (the X) will never occur. withPrefix can help them differentiate, however it still sounds fragile, and users hardly need this situation to occur.

Another use of withPrefix is to filter out the sys.env map to have only things that we need.
However, with our library, report is based on config description and already filters out other parameters in sys.env or properties.

Rename Errors.scala file to ReadErrors.scala file;
Remove WriteError. Replace with String in places where WriteError is used.

We will change the encoding of ReadError to the following

sealed trait ReadError {
  def key: String
}

object ReadError {
 final case class MissingValue(key: String) extends ReadError(key)
 final case class ParseError(key: String, provided: String, `type`: String) extends ReadError(key)
}

This will make it consistent with zio

def write[A](config: Config[A], value: A): Map[String, String]

Remove KeyValue datatype and use Map[String, String] as part of simplifying write method

None of the mentioned combinators appear in the following code exactly as shown, leading to potential confusion.

The doc operator is ?, I suggest changing to ?? to make it less common and compatible with ZIO Test, which uses ?? for adding annotations to assertions.

Add property based tests for generateDocs and generateDocsWithValue functionality.

Possibly include a xmap2Either in config data type, that is a form of invariant applicative functor that handle errors.

In Config type,

  def xmap2Either[B, C](that: Config[B])(f: (A, B) => Either[ReadError, C])(g: C => Either[WriteError, (A, B)]): Config[C] = {
    (self |@| that).apply[(A, B)]((a, b) => (a, b), t => Some((t._1, t._2))).mapEither(b => f(b._1, b._2))(g)
  }

This allows us to implement the following in the companion object

 def sequence[A](list: List[Config[A]]): Config[List[A]] = {
    list.foldLeft(Pure(Nil: List[A]): Config[List[A]])((a, b) => b. xmap2Either(a)((aa, bb) => Right( aa :: bb))(t => Right((t.head, t.tail))))
  }

// where Pure is

  final case class Pure[A](a: A) extends Config[A]

This enables user parse complicated groupings in the env. Although, not a convincing example below, it is essential to make config traversable and allow users to do such things.

final case class Variables(variable1: Int, variable2: Option[Int])

  val listOfConfig: List[Config[Variables]] =
    (0 to 3).toList.map(t => (int(s"${t}_VARIABLE1") |@| opt(int(s"${t}_VARIABLE2")))(Variables.apply, Variables.unapply))

  val configOfList: Config[List[Variables]] =
    Config.sequence(listOfConfig)

  val map = mapSource(
    Map(
      "0_VARIABLE1" -> "1",
      "0_VARIABLE2" -> "2",
      "1_VARIABLE1" -> "3",
      "1_VARIABLE2" -> "4",
      "2_VARIABLE1" -> "5",
      "2_VARIABLE2" -> "6"
      "3_VARIABLE1" -> "7",
    )
  )


val result = read(configOfList).run.provide(map)
// Result: List(Variables(7,None), Variables(5,Some(6)), Variables(3,Some(4)), Variables(1,Some(2)))

val written = write(configOfList).run.provide(result._2)

// Result:
 Map(
  0_VARIABLE1 -> 1,
  0_VARIABLE2 -> 2, 
  1_VARIABLE1 -> 3, 
  1_VARIABLE2 -> 4,
  2_VARIABLE1 -> 5, 
  2_VARIABLE2 -> 6, 
  3_VARIABLE1 -> 7
)

Because this project is ZIO Config, the top-level project names could be shortened / simplified to match ZIO conventions, e.g.:

• zio-config => core
• zio-config-refined => refined
• etc.

System property is delegated to system env itself, and this is a bug

Remove unnecessary functions in package object.
Clean it up based on the discussions and reviews.

Take a look at NestedConfig example, and bring more property laws for it.

One piece that is missing for us to look into adopting this library is support for the typesafe configuration format. It was alluded to in the doc pages (that more formats were coming) but that format would be high priority for us.

Move: int, double, string,long, short,uri, read, write, and report to the companion object Config and make sure the examples still work.

This is related to #39

As of now, describing a nested config is

  val database =
    (string("connection") |@| int("port"))(Database.apply, Database.unapply)

  val appConfig =
    (nested("south") { database } ? "South details" |@|
      nested("east") { database } ? "East details" |@|
      string("appName"))(AwsConfig, AwsConfig.unapply)

Should this be somthing like

  val database =
    (string("connection") |@| int("port"))(Database.apply, Database.unapply)

  val appConfig =
    ("south" /  database  ? "South details" |@|
     "east" /   database  ? "East details" |@|
      string("appName"))(AwsConfig, AwsConfig.unapply)

@jdegoes ?

zio-confing currently lacks functionality to compose config sources. I believe it would be advantageous for users to be able to create new config sources by composition. This would allow, for example, a user to override a config setting from a file using a system property.

A work in progress PR has been opened-
https://github.com/zio/zio-config/pull/80/files

type ReadErrors = ::[ReadError] in the package object

User ReadErrors instead of List[ReadError]
Ensures non empty list representing errors.

Some ideas can be found here in PureConfig which does a pretty good job organizing content.

Some ideas:

• Central link on the home page should be "quick start", and it should be a short 1 page file with the bare minimum necessary to get someone started. There will be very little explanation, mostly very tiny code snippets and 1-3 sentences for each code snippet. Quick start will have:
  • How to add project to SBT / Mill / etc. build file
  • How to add optional dependency with Magnolia to SBT / Mill / etc. build file
  • How to create config descriptor (links to "creating config descriptor" section)
    • How to derive descriptor config from data type using Magnolia
    • How to create descriptor config "by hand" with added docs
  • How to specify where config information should be read from (links to "specifying config source" section)
  • How to use config descriptor (links to "using config descriptor" section)
    • How to get readers from config descriptor
    • How to get documentation from config descriptor
    • How to get report generation from config descriptor and value
    • How to get writers from config descriptor
  • How to consume config from from ZIO environment
• Section on creating configuration descriptor
  • Subsection on creating configuration descriptor by hand
  • Subsection on creating configuration descriptor by Magnolia
• Section on specifying config source
  • Specifying source as map
  • Specifying source as properties
  • Specifying source as environment variables
  • Specifying source as hocon file
  • Specifying source as json file
  • Specifying source as yaml file
• Section on using configuration descriptor
  • Subsection on deriving readers for various file formats (properties, env vars, json, yaml, hocon)
  • Subsection on deriving documentation for various formats (html, markdown, plaintext)
  • Subsection on deriving report generation for various formats (html, markdown, plaintext)
  • Subsection on deriving writers for various file formats
• Section on consuming config from ZIO environment
  • The Config module
  • Creating the config module
  • Adding the config module to the whole application environment
  • Accessing application config from anywhere in app using module helpers
• Section on interop
  • Interop with HOCON
  • Interop with etc...

While it may sound natural to have the support of a recursive config build,
a recursive structure to hold the environment configuration of an application does seem to be an overkill.

However, this is posted as a question and views are invited.

trait Config[A] { self =>
  def optional: Config[Option[A]] = 
     Config.Optional(self)
   
   def describe(description: String): Config[A] = 
      Config.Describe(self, description)
}

object Config {
    case class Describe[A](config: Config[A], description: String) extends Config[A] 
    case class Optional[A](config: Config[A]) extends Config[Option[A]]
    case class Default[A](value: A, propertyValue: Sting) extends Config[A]

    def default(value: A, propertyValue: String): Config[A] =  
       Config.Default(value, propertyValue) 
}

We need Optional for man page.

And default could be used as below:

 import zio.config._, Config._

 case class Port(value: Int) extends AnyVal

(int("PORT") orElse default(Port(42), "42")).xmap(Port)(_.value)

Currently, the config report is converted to string as follows-

  override def toString: String =
    list.mkString("\n")

The use of a hardcoded newline will cause problems for Windows users. There are a couple of approaches I think we can take -

1. Still override toString but use ", " instead and provide a separate method for using newlines.
2. Do not override toString and provide a separate method for using newlines.
3. Still override toString but use System.lineSeparator() instead of "\n".

I believe that the third option is against the philosophy of ZIO so I suggest either option one or two. Either way, I believe we need a separate method for using newlines.

In ProductBuilder.scala, currently we support only 9 parameters in the config.
We need to extend the boiler plate to support maximum number of variables.
It is fairly straight forward to add them, but it will be manual addition.

Remove Pure Config and add Empty Config which describes an empty configuration and preserve the type of the config

final case class Empty[A]() extends Config

Move the example to a sub project called examples in the top level of zio-config
and put the code src in a sub project called core

@jdegoes As of now write function returns a PropertyTree.
By write, I think, what we need is write(config, value).asHoccon or write(config, value).asJson and write(config, value).asMap, of which the last one already exists. Does this issue sound reasonable?

When writing back the config, we end up in a situation where the first config description's write is executed regardless of the type.

More details on this issue soon.

In ConfigSource.scala, change UIO[Option[String]] to IO[Unit, String].
Also, rename getString to getConfigValue.

Right now we use ZIO Environment for ConfigSource, but I think we should refactor that a bit, and use ZIO Environment directly for accessing configuration.

The most natural name to use for the module would be Config, probably, which would mean changing the existing Config to ConfigDescriptor or ConfigFormat or something similar.

Then we could construct a module like this:

trait Config[A] {
  def config: Config.Service[Any, A]
}
object Config {
  trait Service[A] {
    def config: UIO[A]
  }
}
object config {
  def config[A]: ZIO[Config, Nothing, A] = ZIO.accessM(_.config)
}

Now users could access config in their application like this:

import zio.config._ 

...
case class MyConfig(port: Int, server: String)
...
for {
  myConfig <- config[MyConfig]
} yield (myConfig.port, myConfig.server)

This leaves what to do with ConfigSource. I think it could become not a module, but a trait or even just a function, that can load configuration. Then we can have:

object Config {
  ...
  def make[A](source: ConfigSource, configDescriptor: ConfigDescriptor[A]): IO[ReadError, Config[A]] = ???
  ...
}

which effectfully builds a Config module given a way to read it. Or perhaps that's too fancy and we should just put fromMap, fromEnv, etc., methods directly on Config`?