japgolly / clear-config Goto Github PK
View Code? Open in Web Editor NEWScala FP configuration library with a focus on runtime clarity
License: Apache License 2.0
Scala FP configuration library with a focus on runtime clarity
License: Apache License 2.0
my application.conf file contains variable to be read from .env and when i run this i get an error.Though other values have been read only variables are the problem.
// app.conf
myapp.dbconfig.threads-pool-count = 2
myapp.dbconfig.threads-pool-count = ${?DB_THREAD_POOL_COUNT}
// queueSize The size of the job queue, 0 for direct hand-off or -1 for unlimited size.
myapp.dbconfig.queue-size = 50
myapp.dbconfig.queue-size = ${?DB_QUEUE_SIZE}
myapp.dbconfig.search-limit = 10
myapp.dbconfig.search-limit = ${?SEARCH_LIMIT}
error:-
Exception in thread "main" java.lang.RuntimeException: 3 errors:
[error] - Error reading key [myapp.dbconfig.queue-size] from source [cp:/application.conf] with value [${DB_QUEUE_SIZE}]: Int expected.
[error] - Error reading key [myapp.dbconfig.search-limit] from source [cp:/application.conf] with value [${SEARCH_LIMIT}]: Int expected.
[error] - Error reading key [myapp.dbconfig.threads-pool-count] from source [cp:/application.conf] with value [${DB_THREAD_POOL_COUNT}]: Int expected.
[error] 5 sources (highest to lowest priority):
[error] - Env
For example:
val port = ConfigDef.getOrUse[Port]("server.port", port"8085")
def configSources[F[_]: Applicative]: ConfigSources[F] = {
// Allow property names like `foo.bar` to map to environment variables like `FOO_BAR`
// Taken from https://github.com/japgolly/clear-config/issues/132
def propertyToEnv(k: ConfigKey): List[ConfigKey] =
k :: k.map(_.toUpperCase.replace('.', '_')) :: Nil
ConfigSource.environment[F].mapKeyQueries(propertyToEnv) >
ConfigSource.propFile[F]("deploy.properties", optional = true)
ConfigSource.system[F]
}
def readConfig[F[_]: MonadThrow](log: Logger[F]): F[MainConfig] =
config.withReport
.run(configSources[F])
.flatMap(_.toEither.leftMap(e => new Exception(e) with NoStackTrace {}).liftTo[F])
.flatMap { case (cfg, report) =>
// Don't bother reporting unused environment variables, or "system" settings
val reportMessage = report
.mapUnused(_.withoutSources(ConfigSourceName.environment, ConfigSourceName.system))
.full
log
.info(reportMessage)
.as(cfg)
}
Results in a config report like this:
Used keys (5):
+-----------------+------+-----------------------------------------------------------------+---------+
| Key | Env | /Users/gavin/blahblah/blahblahblahblahblah/local-dev.properties | Default |
+-----------------+------+-----------------------------------------------------------------+---------+
| SERVER_PORT | 8089 | | |
| server.port | | | 8085 |
+-----------------+------+-----------------------------------------------------------------+---------+
Look at this travesty:
val cfg = RestServiceConfig.configParserHostLookup.parse(
japgolly.clearconfig.internals.Lookup.Found(
Report.Settings.default.copy(valueDisplay0 = valueDisplay, maxValueLen = Some(100))
Poor Tostein...
And provide a sensible error message showing all possible options.
Also have oneOf[A](preProcess: String => String, f: (String, A)*)
No more manually doing the following:
ConfigDef.getOrUse[String]("LOG_APPENDER", "DEV") *>
ConfigDef.getOrUse[String]("LOG_LEVEL_ROOT", "INFO") *>
ConfigDef.getOrUse[String]("LOG_LEVEL_SQL", "INFO") *>
ConfigDef.unit
I saw the note in the README - opening this as a ticket for people to track if they care.
If you're interested in getting a hand for it, I might be able to find some time.
eg:
A_1=a
A_2=b
// becomes
List("a", "b")
A.1.A = 10
A.1.B = 11
A.2.A = 20
A.2.B = 21
// becomes
List( X(10,11), x(20,21))
(Key,Int) => Key
?
(Key,Int) => Suffix
?
def poolConfig: ConfigDef[JedisPoolConfig => Unit] =
ConfigDef.consumerFn[JedisPoolConfig](
_.get("BLOCK_WHEN_EXHAUSTED", _.setBlockWhenExhausted),
_.get("EVICTION_POLICY_CLASS_NAME", _.setEvictionPolicyClassName),
_.getOrUse("FAIRNESS", _.setFairness)(true), // <------------------ look
_.get("JMX_ENABLED", _.setJmxEnabled),
_.get("JMX_NAME_BASE", _.setJmxNameBase),
_.get("JMX_NAME_PREFIX", _.setJmxNamePrefix),
_.get("LIFO", _.setLifo),
_.get("MAX_IDLE", _.setMaxIdle),
_.get("MAX_TOTAL", _.setMaxTotal),
_.get("MAX_WAIT_MILLIS", _.setMaxWaitMillis),
_.get("MIN_EVICTABLE_IDLE_TIME_MILLIS", _.setMinEvictableIdleTimeMillis),
_.getOrUse("MIN_IDLE", _.setMinIdle)(2), // <------------------ look
_.get("NUM_TESTS_PER_EVICTION_RUN", _.setNumTestsPerEvictionRun),
_.get("SOFT_MIN_EVICTABLE_IDLE_TIME_MILLIS", _.setSoftMinEvictableIdleTimeMillis),
_.get("TEST_ON_BORROW", _.setTestOnBorrow),
_.get("TEST_ON_CREATE", _.setTestOnCreate),
_.get("TEST_ON_RETURN", _.setTestOnReturn),
_.get("TEST_WHILE_IDLE", _.setTestWhileIdle),
_.get("TIME_BETWEEN_EVICTION_RUNS_MILLIS", _.setTimeBetweenEvictionRunsMillis)
)
.withPrefix("POOL_")
and the report
Used keys (59):
+------------------------------------------------------+----------+------------------+
| Key | Fake env | Default |
+------------------------------------------------------+----------+------------------+
| CACHE_REDIS_POOL_BLOCK_WHEN_EXHAUSTED | | |
| CACHE_REDIS_POOL_EVICTION_POLICY_CLASS_NAME | | |
| CACHE_REDIS_POOL_FAIRNESS | | |
| CACHE_REDIS_POOL_JMX_ENABLED | | |
| CACHE_REDIS_POOL_JMX_NAME_BASE | | |
| CACHE_REDIS_POOL_JMX_NAME_PREFIX | | |
| CACHE_REDIS_POOL_LIFO | | |
| CACHE_REDIS_POOL_MAX_IDLE | | |
| CACHE_REDIS_POOL_MAX_TOTAL | | |
| CACHE_REDIS_POOL_MAX_WAIT_MILLIS | | |
| CACHE_REDIS_POOL_MIN_EVICTABLE_IDLE_TIME_MILLIS | | |
| CACHE_REDIS_POOL_MIN_IDLE | | |
| CACHE_REDIS_POOL_NUM_TESTS_PER_EVICTION_RUN | | |
| CACHE_REDIS_POOL_SOFT_MIN_EVICTABLE_IDLE_TIME_MILLIS | | |
| CACHE_REDIS_POOL_TEST_ON_BORROW | | |
| CACHE_REDIS_POOL_TEST_ON_CREATE | | |
| CACHE_REDIS_POOL_TEST_ON_RETURN | | |
| CACHE_REDIS_POOL_TEST_WHILE_IDLE | | |
| CACHE_REDIS_POOL_TIME_BETWEEN_EVICTION_RUNS_MILLIS | | |
val exclude = List(SourceName.environment, SourceName.system)
val configReport = configReport0
.withUnusedSettings(
_.withColFilter(_ && ConfigReport.ColFilter.excludeSources(exclude: _*))
.withRowFilter(_ && ConfigReport.RowFilter((_, m) => m.keys.exists(!exclude.contains(_)))))
// .withRowFilter(_ && ConfigReport.RowFilter.excludeEmpty)) // TODO Fix microlibs
println(configReport.reportUsed)
// TODO This is a temporary hack - need a way to know if empty after filter-application
if (configReport.reportUnused.length > 23)
println("Unused config:\n" + configReport.reportUnused)
It will fail if one value is provided but another needed value is not. I want a version that just returns None
without failing
I'm always doing the same thing. Make this easier out-of-the-box:
def sources: ConfigSources[IO] =
// Highest pri
ConfigSource.environment[IO].expandInlineProperties("APP_PROPS").mapKeyQueries(acceptExternalKeyFormat) >
ConfigSource.propFileOnClasspath[IO]("app.properties", optional = true) >
ConfigSource.system[IO]
// Lowest pri
private def acceptExternalKeyFormat(k: ConfigKey): List[ConfigKey] =
k :: k.map(_.toUpperCase.replace('.', '_')) :: Nil
def load: IO[Option[AppConfig]] =
config
.withReport
.run(sources)
.map { result =>
result.toEither match {
case Right((config, report)) =>
if (config.showUnusedConfig)
logger.info("Config report\n" + report.full)
else
logger.info("Config report\n" + report.sources + "\n\n" + report.used)
Some(config)
case Left(err) =>
logger.error("Invalid config:\n" + err)
None
}
}
Make this easier
if (config.showUnusedConfig)
logger.info("Config report\n" + report.full)
else
logger.info("Config report\n" + report.sources + "\n\n" + report.used)
I didn't see an example in the README - is this something that could be added, perhaps as a separate module?
This seems like a good utility, but it seems far from being a complete solution.
Is there plans to add migrations / upgrades / in-app mutation/setting of config values?
Probably just wrap /^"\s*"$/
in quotes
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.