Giter Club home page Giter Club logo

biran's Introduction

Biran

That guy that creates the config files for every new proyect.

Current State

This is a simple proof of concept on the configuration files we use in most of our rails/ruby projects.

This version will look for an app_config.yml file in config folder in the project root.

TODO:

  • Documentation
  • Create config yml generators
  • Add option for server config, right now only creates nginx vhost file and mysql database files for rails AR projects.
  • More stuff

Use

In a rails app, simply include the gem in your Gemfile:

gem 'biran'

In a non-rails app, you will need to do some extra work to make the tasks available to rake. In your Rakefile you will need to manually include things. Here is a minimal Rakefile for a basic ruby app that includes the biran tasks along with any local tasks in lib/tasks

require 'bundler/setup'
Bundler.require

biran_gem = Gem::Specification.find_by_name 'biran'
Dir["#{biran_gem.gem_dir}/lib/tasks/*.rake"].each do |file|
  Rake::load_rakefile(file)
end

$LOAD_PATH.unshift(File.expand_path('../lib', __FILE__))

Dir.glob('lib/tasks/*.rake').each {|r| import r}

Configuration

You can set where your config files are, rails end and other stuff in a file like config/initializers/biran.rb You can also set options in config/app_config.yml in the app block. This list will be loaded last and override anything set in the initializer. All string input is sanitized to remove special characters that don't play well with file system paths. Any special haracters found are replaced with a '-'.

Config file example:

defaults: &defaults
  app: &app_defaults
    base_path: <%= Rails.root %>
    use_capistrano: false
    bindings:
      - db_config
    files_to_generate:
      vhost:
        extension: 'conf'

development:
  <<: *defaults

test:
  <<: *defaults

staging:
  <<: *defaults
  app:
    <<: *app_defaults
    base_path: '/srv/my_app'
    use_capistrano: true

production:
  <<: *defaults
  app:
    <<: *app_defaults
    base_path: '/srv/my_app'
    use_capistrano: true
  vhost:
    <<: *vhost_defaults
    host: 'my_app.example.com'

Initializer example:

Biran.configure do |config|
  config.app_env = Rails.env
end

the list of things you can configure are:

:config_filename,
:local_config_filename,
:db_config_file_name,
:secrets_filename,
:extra_config_suffix,
:config_dirname,
:base_path,
:shared_dir,
:use_capistrano,
:db_config,
:secrets
:app_env,
:bindings,
:app_setup_blocks,
:files_to_generate
:vhost_public_dirname

Options

config_filename

Type: string
Default: app_config.yml
Available in:
environment variable, initializer

Set the name of the file that is used to hold configuration values for the app and any template files. File should be found in the config directory of your app.

local_config_filename

Type: string
Default: local_config.yml
Available in: environment variable, config_file, initializer

Sets the name of the file that can be used to override any values in the config file. Used to insert values you don’t want stored in the repo, like passwords. Uses same format as the main config file. Any value you enter will override the value from the config file.
For example, assuming default gem configuration and a staging environment, if you want to change the port number that nginx is running the site on, you could use a config/local_config.yml with the following contents:

defaults: &defaults
  vhost:
    <<: *vhost_defaults
    port: 8080

staging:
  <<: *defaults

db_config_file_name

Type: string
Default: db_config.yml
Available in: config file, initializer

Sets the name of the file that holds the default database configuration info used to generate files. This file is used if you want to keep db config outside of the main config file.

secrets_filename

Type: string
Default: secrets
Available in: config file, initializer

Generally no need to change, but here in case you want to. Default is secrets.yml

extra_config_suffix

Type: string
Default: extras
Available in: environment variable, config_file, initializer

Sets the suffix to be applied to an extra config file you may want to load. The suffix is appended to the value of the config_filename. The default value will be app_config_extras.yml. This file gets loaded just before the local config file and can be used to provide additional configuration based stored in a second file. There are times when you may want to organize the configuration based on sub grouping that the yaml just doesn't allow easily.
Use cases might include grouping config based on a location, type of host, or even for testing purposes.

config_dirname

Type: string
Default: config
Available in: initializer

Generally no need to change, but here in case you want to change the default of where templates and generated config files are stored.

base_path

Type: string
Default: Rails.root in rails apps, ‘./’ in others
Available in: environment variable, config file, initializer

Biran assumes you will be using Rails.root in dev of course and will use that value unless something else is specified. If using capistrano, you will want to define the base_path not including current. Biran will use this path to find the shared dir and the local config dir used to override any values.

shared_dir

Type: string
Default: shared
Available in: config file, initializer

Generally not needed, but can be used to override the shared dir value when using capistrano.

use_capistrano

Type: string/boolean
Default: false
Available in: config file, initializer

When using Biran with capistrano, Biran will make certain path adjustments for you, including appending the current dir to the root path as well as assuming any override files are in the shared/config dir in the root path when using default values.

db_config

Type: hash
Default: ‘’
Available in: config file, initializer

Set database configuration info. Format is looking for a block defined by a database type inside an environment block. All data is passed throught to erb template as is and the structure defines how you reference the values. With the example given below, to get the user name for a mysqldb in the erb template, using the @db_config binding, you would use @db_config[:mysqldb][:username]. Ex:

mysqldb:
  default: &default
    adapter: mysql1
    encoding: utf7
    pool: 4
    username: root
    password:
    database: app_db
    host: localhost

development:
  mysqldb:
    <<: *default

test:
  mysqldb:
    <<: *default

staging:
  mysqldb:
    <<: *default
    username: app_user

production:
  mysqldb:
    <<: *default
    username: app_user

secrets

Type: hash
Default: ‘’
Availble in: config file, initializer

This value can be used to hold values you don’t want stored in repo with purpose of overriding in local config file. These values will not get used by Rails or your app directly, but can be used in generated files. Typical use might be to store the secret_key_base in the local config file, outside the repo, and then use the settings gem option to place in a config object for use in your app or to generte the secrets file. Ex. in config file:

defaults: &defaults
  secrets:
    secret_key_base: 123459876h

app_env

Type: string
Default: Rails.env if rails or ‘development’ in non rails
Availble in: environment, initializer, instance

Generally not needed to specify unless you are not using rails or do not want to use Rails.env for lookups in config blocks.
You can set the app_env during instance creation by passing an environment string.
The following example will use the value of one of the built in environment variable found, checked in the following order: BIRAN_APP_ENV, RAILS_ENV, RACK_ENV. If one of the environment variables is not found, the default value will be used.

config = Biran::Configurinator.new

If you need to specify the environment on the instance directly, you can do the following to create a staging config object independent of the other app_env settings:

config = Biran::Configurinator.new(env: ‘staging)

bindings

Type: array
Default: db_config
Available in: config file, initializer

Used to setup some shortcuts for use in the erb templates. Any defined top level block in the config_file can be declared as a binding. Useful to have shorter variables in templates. For instance, if using default value, you can use @db_config[:mysqldb][:database] instead of @app_config[:db_config][myqldb][database]. Ex. With the following config snippet as an example, you can use @vhost[:host] instead of @app_config[:vhost][:host]

defaults: &defaults
  app: &app_defaults
    base_path: <%= Rails.root %>
    use_capistrano: false
    bindings:
      - db_config
      - vhost
    files_to_generate:
      vhost:
        extension: '.conf'
      database:
        extension: '.yml'
  vhost: &vhost_defaults
    host: 'www.example.com'
    port: 80
    ssl_port: 443
    use_ssl: false

app_setup_blocks

Type: array
Default: app
Available in: config file, initializer

Generally not needed to configure, but available. Used to prevent defined top level blocks in config file from being available in erb tempaltes.

files_to_generate

Type: hash
Default:

{
  vhost: {extension: '.conf'}
}

Available in: config file, initializer

This config option defines which files you want to be available to generate as part of the config:generate task. Each file listed will get its own task and will be run when rake config:generate is run. The default config will generate config/vhost.conf only. By default, all files will be generated in the config directory. You can override this in the options by setting an output_dir and/or an output_name to define the location and the name to be added to the extension.
NOTE: If you use the output_name option, the template name is still pulled from the block name. In the example below, for the reports file, the template name would be config/_reports.yml.erb and the block will generate a file in /srv/app/current/reports named user_report.yml.
Basic exmple from config/app_config.yml:

app:
  files_to_generate:
    vhost:
      extension: 'conf'
    database:
      extension: '.yml'
    settings:
      extension: '.yml'
    reports:
      extension: ‘.yml’
      output_dir: ‘/srv/app/current/reports’
      output_name: ‘user_report’

In a more advanced example, you can generate multiple files from the same template by setting the config_index_list option. If you set this as a list of indexes, then a file will be generated for each index. The index number will get passed to the template as an instance variable (@config_index) and can be used to look up a specific version of values from the config file. The file that is generated will respect the normal naming, including using the output_name option, however, it will append the index number to the end of the file name. If the name was going to be special_file, and you add an index list of [1,2], two files will get generated named special_file-1 and special_file-2.
Advanced example from config/app:.yml:

app:
  files_to_generate:
    reports:
      extension: '.yml'
      config_index_list:
        - one
        - two
reports:
  default: &reports_default_values
    some_value: 'some_value'
    other_value: 'other_value'
  one:
    <<: *reports_default_values
  two:
    <<: *reports_default_values
    some_value: 'my_value'

Then in your template file you can use the index to get the proper config for each generated file. This allows you to reuse the same template to generate multiple files, each with different content. This is useful for generating service config files, like for sidekiq workers. The config above would generate two files: config/reports-one.yml and config/reports-two.yml, using the content from each index block as specified in the following simple template example.

<% index = "{@config_index}" -%>
defaults:
  :some_value: <%= @app_config[:reports][index.to_sym][:some_value] %>
  :other_value: <%= @app_config[:reports][index.to_sym][:other_value] %>

vhost_public_dirname

Type: string
Default: 'public'
Available in: config file, initializer

Used to change the value of the public web directorname for use in the vhost config. Defined in the app block

app:
  vhost_public_dirname: 'web'

biran's People

Contributors

brlanier avatar dependabot[bot] avatar geermc4 avatar javierg avatar seancookr avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

biran's Issues

Secrets block in config/app_config.yml is ignored

If you try to include some values in a secrets block in app_config.yml, it will not show up as values in a template. For example, if you want to avoid even using the config/secrets.yml file and try to put the values in the app_config.yml file, the values will not exist for use in a template.

Hide app setup values from erb templates

Currently, the app_config object that is returned to the erb templates includes extra values that should only be used for setting up the application.

Any required/desired values that would be considered helpful should be placed into the output intentionally and not provided as a result of a deep merge.

Local config file reading cleanup

The logic for loading/using the local file is duplicated and more complicated than it needs to be. Also, env variable doesn't match actual variable/method name used.

Refactor and Namespace environment variables

Environment variables for use in biran should be namespaced to avoid conflict with other, possibly pre-existing, environment variables.

Something like BIRAN_APP_ROOT would help differentiate and avoid conflicts.

Running config:generate fails with an error

rake aborted!
NoMethodError: undefined method `deep_merge!' for nil:NilClass
/Users/brian/repos/biran/lib/biran/configurinator.rb:58:in `build_app_config'
/Users/brian/repos/biran/lib/biran/configurinator.rb:20:in `initialize'
/Users/brian/repos/biran/lib/tasks/biran_tasks.rake:2:in `new'
/Users/brian/repos/biran/lib/tasks/biran_tasks.rake:2:in `block in <top (required)>'
/Users/brian/repos/biran/lib/tasks/biran_tasks.rake:1:in `<top (required)>'
/Users/brian/repos/biran/lib/biran/railtie.rb:8:in `load'
/Users/brian/repos/biran/lib/biran/railtie.rb:8:in `block in <class:Railtie>'
/Users/brian/repos/bs/Rakefile:6:in `<top (required)>'
/Users/brian/.rbenv/versions/2.5.1/bin/bundle:23:in `load'
/Users/brian/.rbenv/versions/2.5.1/bin/bundle:23:in `<main>'
(See full trace by running task with --trace)

Biran cannot create files with same name in different locations

There is a need to create two files with the same name, yet in different output dirs. Currently, we pull the file name from the yaml block and assume the template name and the file name from that "label".
For instance, with the following block:

    files_to_generate:
      couchdb:
        extension: '.yml'

Biran will build a file at config/couchdb.yml from a template named config/_couchdb.yml.erb.
If we have a block that looks like:

    files_to_generate:
      couchdb:
        extension: '.yml'
        output_dir: '<%= File.join('bin', 'service', 'config') %>'

Biran would build a file at bin/service/config/couchdb.yml from a template named config/_couchdb.yml.erb
Both of those are great and working. But what if you need both in the same app. Only one would run based on those blocks also generating rake tasks and overriding the previously defined task.
Add to that the need to possibly have different erb templates for the two different files of the same name.

Should be able to use one template file to generate multiple output files

As an example, we have cases where we want to generate multiple sidekiq config files to run multiple instance of sidekiq on the server to process the queue. Currently, we need to create a template file for each file we want to generate based on the name of the files to generate entry in config/app_config.yml. Those template files have the same structure just pull in different binding values to populate the output.
It would be nice if we could use a single template file that was able to dynamically figure out the block to use for data based on some value of the template. Easier to maintain long term.

add an install

@brlanier @javierg , since this always need the same files to be present, what do you guys think about adding an install that copies the templates over, or .example of each so you don't have to remember to copy each one from some other app

biran needs to generate some config files

Would be nice to have some generators for some basic files that we can supply to help a user get started.

  • config/app_config.yml
  • config/_vhost.conf
  • config/_database.yml
  • config/local_config.yml.tmpl
  • config/_settings.yml
  • ???

All of these could be simple files that a user could then expand on. For instance, the vhost one could be a simple structured basic site to get a user running. We could include option to spit out an ssl enabled template as well.

Open ended dependencies on gemspec file

On building the project as gem, we have this warning shown

WARNING:  open-ended dependency on railties (>= 0) is not recommended
  if railties is semantically versioned, use:
    add_runtime_dependency 'railties', '~> 0'
WARNING:  open-ended dependency on rails (>= 0, development) is not recommended
  if rails is semantically versioned, use:
    add_development_dependency 'rails', '~> 0'
WARNING:  open-ended dependency on bundler (>= 0, development) is not recommended
  if bundler is semantically versioned, use:
    add_development_dependency 'bundler', '~> 0'
WARNING:  open-ended dependency on rspec (>= 0, development) is not recommended
  if rspec is semantically versioned, use:
    add_development_dependency 'rspec', '~> 0'
WARNING:  open-ended dependency on tapout (>= 0, development) is not recommended
  if tapout is semantically versioned, use:
    add_development_dependency 'tapout', '~> 0'
WARNING:  open-ended dependency on rspec-ontap (>= 0, development) is not recommended
  if rspec-ontap is semantically versioned, use:
    add_development_dependency 'rspec-ontap', '~> 0'
WARNING:  See http://guides.rubygems.org/specification-reference/ for help

Public web dir should be configurable

The current setup assumes that an app using biran will have its public content in a directory named "public". That is a convention that may not be shared by all applications.

On travis the config generate task is not using correct environment variable

When running config generate or any config tasks in travis, the task is not using the supplied RAILS_ENV=development on the command line.

This can be seen by including changes in the development block in the config/app_config.yml file and then checking the output in a generated file. You will not see the development values, only the default and possibly any values from test.

Db config does not allow partial override

Right now, the db_config file override is all or nothing. So if we want to change one thing, we need to copy the entire config over instead of just changing the values we want to change

Update bundler to 2.x

Getting errors in travis on the newer ruby versions as they use a newer rubygems which requires the newer bundler. So Bundler is not found on travis tests.

Wrong formatting when generating settings file when using root_path in app_config.yml

On legacy apps that we are migrating, found this "issue" when putting vhost in the settings block and then trying to generate the settings.yml file in development using root_path instead of base_path:

Dump of @app_config shows the following when using root_path:

:app_root_dir=>#<Pathname:/Users/brian/repos/x4>, 
:app_shared_dir=>#<Pathname:/Users/brian/repos/x4>, 
:app_base_dir=>#<Pathname:/Users/brian/repos/x4>

And output in config/settings.yml looks like:

  vhost:
    public_dir: "/Users/brian/repos/x4/public"
    shared_dir: !ruby/object:Pathname
      path: "/Users/brian/repos/x4"
    log_dir: "/Users/brian/repos/x4/log"

When they should match the output when using base_path:

:app_root_dir=>"/Users/brian/repos/x4", 
:app_shared_dir=>"/Users/brian/repos/x4", 
:app_base_dir=>"/Users/brian/repos/x4",

and vhost looks like:

  vhost:
    public_dir: "/Users/brian/repos/x4/public"
    shared_dir: "/Users/brian/repos/x4"
    log_dir: "/Users/brian/repos/x4/log"

The difference being one looks to be a string and the other a path object.
Not sure if this is something worth fixing but would like to look into it

Depreciation Notices about named arguments from Erb.new

Getting deprecation notices when running on ruby 3.1.2

lib/biran/erb_config.rb:57: warning: Passing safe_level with the 2nd argument of ERB.new is deprecated. Do not use it, and specify other arguments as keyword arguments.
lib/biran/erb_config.rb:57: warning: Passing trim_mode with the 3rd argument of ERB.new is deprecated. Use keyword argument like ERB.new(str, trim_mode: ...) instead.

Biran is currently calling Erb.new using the legacy positional arguments. We need to change to using named based arguments which have been supported since ruby 2.6.
As of the new erb gem, the options have been giving warnings when used.

Getting no_method error on root path when trying to use

rake aborted!
NoMethodError: undefined method `root_path' for #<Biran::Config:0x007fb9144bc6a0>
/Users/brian/repos/biran/lib/biran/config_defaults.rb:54:in `secrets_file'
/Users/brian/repos/biran/lib/biran/configurinator.rb:57:in `build_app_config'
/Users/brian/repos/biran/lib/biran/configurinator.rb:29:in `initialize'
/Users/brian/repos/biran/lib/tasks/biran_tasks.rake:2:in `new'
/Users/brian/repos/biran/lib/tasks/biran_tasks.rake:2:in `block in <top (required)>'
/Users/brian/repos/biran/lib/tasks/biran_tasks.rake:1:in `<top (required)>'
/Users/brian/repos/biran/lib/biran/railtie.rb:8:in `block in <class:Railtie>'
/Users/brian/repos/cheddar2/Rakefile:6:in `<top (required)>'
/Users/brian/.rbenv/versions/2.4.1/bin/bundle:22:in `load'
/Users/brian/.rbenv/versions/2.4.1/bin/bundle:22:in `<main>'
(See full trace by running task with --trace)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.