Giter Club home page Giter Club logo

salt-formula's Introduction

salt-formula

Travis CI Build Status Semantic Release

Yes, Salt can Salt itself!

See the full SaltStack Formulas installation and usage instructions.

If you are interested in writing or contributing to formulas, please pay attention to the Writing Formula Section.

If you want to use this formula, please pay attention to the FORMULA file and/or git tag, which contains the currently released version. This formula is versioned according to Semantic Versioning.

See Formula Versioning Section for more details.

Commit message formatting is significant!!

Please see How to contribute for more details.

Meta-state (This is a state that includes other states).

This calls all runable states based on configured pillar data.

Install a minion

Install a master.

Install a syndic.

Install salt cloud.

Install salt-ssh with roster file. Configure pillar data under salt:ssh_roster to feed the template.

Install salt api Requisite: Configure salt-master with rest_cherrypy or rest_tornado.

Install a minion and configure it in standalone mode.

Install gitfs backend dulwich dependencies. Set salt:master:gitfs_provider: dulwich in your pillar.

Install gitfs backend GitPython dependenciess. Set salt:master:gitfs_provider: gitpython in your pillar.

Install ssh keys to be used by gitfs

Install gitfs backend libgit2/pygit2 dependenciess. Set salt:master:gitfs_provider: pygit2 in your pillar. For EL distributions, pygit is installed from packages from EPEL.

It is recommended to use SaltStack repository for Debian, RedHat, and SuSE, to benefit from the latest stable salt release. Refer to official documentation at <http://docs.saltstack.com/en/latest/topics/installation/index.html#platform-specific-installation-instructions>`_.

Undo the effects of salt.pkgrepo on Debian, RedHat, and SuSE.

Clone selected Salt formulas Git repositories under /srv/formulas and makes them available in the relevant file_roots settings. Please note that in order for file_roots to be updated, salt.master must be called after salt.formulas. For example:

base:
  'saltmain':
    - salt.formulas
    - salt.master

Pillar data can be used to customize all paths, URLs, etc. Here's a minimal pillar sample installing two formulas in the base environment:

salt_formulas:
  list:
    base:
      - salt-formula
      - openssh-formula

See pillar.example for an exhaustive list of settings available via pillar. Note that by default this state:

  • downloads the latest formulas from the saltstack-formulas project on GitHub.
  • does not update the local repositories after the initial clone. This is a safety measure since you do not control how the official repositories evolve.

If you configure the state to download the formulas from repositories that you control, then you can safely enable the salt_formulas:git_opts:default:update pillar setting to True.

Every option available in the templates can be set in pillar. Settings under 'salt' will be overridden by more specific settings under salt['master'], salt['minion'] or salt['cloud']. Options specified in salt['minion'] which are not present in the default configuration file will be added to the end of the configuration file.

salt:
  ret_port: 4506
  master:
    user: saltuser
    ...
  minion:
    user: saltuser
    ...
  cloud:
    providers: ec2
    ...

Additional templates can be added by the user under salt/files/minion.d and master.d. This might be useful if, for example, a recently-added configuration option is not yet provided by the default template.

Executing the provided Vagrantfile will create a Ubuntu 14.04 VM, add the default Saltstack Repository and install the current stable version.

The folders inside the VM will be set up in a way that enables you to simply execute 'sudo salt "*" state.highstate' to apply the salt formula to the VM, using the pillar.example config. You can check /etc/salt/ for results.

Remember, you will have to run state.highstate or state.sls salt.(master|minion|cloud) manually.

As MacOS has no native package management that pkg.installed can leverage appropriately, and brew does not count, the salt.minion state manages salt minion package upgrades by way of .pkg file download which is then installed using the macpackage.installed state.

salt-minion packages on MacOS will not be upgraded by default. To enable package management you must set the following at a minimum,

install_packages: True
version: 2017.7.4
salt_minion_pkg_source: https://repo.saltproject.io/osx/salt-2017.7.4-py3-x86_64.pkg

install_packages must indicate that the installation of a package is desired. If so, version will be used to compare the version of the installed .pkg against the downloaded one. If version is not set and a salt.pkg is already installed the .pkg will not be installed again.

A future update to the formula may include extraction of version from the downloaded .pkg itself; but for the time being you MUST set version to indicate what you believe it to be.

Refer to pillar.example for more information.

Linux testing is done with kitchen-salt.

  • Ruby
  • Docker
$ gem install bundler
$ bundle install
$ bin/kitchen test [platform]

Where [platform] is the platform name defined in kitchen.yml, e.g. debian-9-2019-2-py3.

Creates the docker instance and runs the salt main states, ready for testing.

Runs the inspec tests on the actual instance.

Removes the docker instance.

Runs all of the stages above in one go: i.e. destroy + converge + verify + destroy.

Gives you SSH access to the instance for manual testing.

Windows/FreeBSD/OpenBSD testing is done with kitchen-salt.

  • Ruby
  • Virtualbox
  • Vagrant
$ gem install bundler
$ bundle install --with=vagrant
$ bin/kitchen test [platform]

Where [platform] is the platform name defined in kitchen.vagrant.yml, e.g. windows-81-latest-py3.

When testing using Vagrant you must set the environment variable KITCHEN_LOCAL_YAML to kitchen.vagrant.yml. For example:

$ KITCHEN_LOCAL_YAML=kitchen.vagrant.yml bin/kitchen test      # Alternatively,
$ export KITCHEN_LOCAL_YAML=kitchen.vagrant.yml
$ bin/kitchen test

Then run the following commands as needed.

Creates the Vagrant instance and runs the salt main states, ready for testing.

Runs the inspec tests on the actual instance.

Removes the Vagrant instance.

Runs all of the stages above in one go: i.e. destroy + converge + verify + destroy.

Gives you RDP/SSH access to the instance for manual testing.

salt-formula's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

salt-formula's Issues

no salt-cloud vmware provider

as of 2015.8.x, vmware as a provider is available, as such i've manually altered the formula to add that functionality locally:

  • added a cloud.provider.d vmware.conf template
  • added a cloud.profile.d vmware.conf template
  • altered cloud.sls to install pyvmomi if required

I'm raising this issue, because this functionality should probably exist in the cloud formula going forward.

if people are interested, i can prep a PR for review.

Ability to run from Git clone

It would be awesome if the salt-formula could be configured to clone Salt from the Git repo (e.g. giving it a version tag), and then it set up a virtualenv and made sure the minion/master were running from source, simultaneously disabling any distro-wide installations.

formula fails on CentOS when run from gitfs

I've recently converted a bunch of formulas I cloned and then referenced via file_roots to local git repos referenced from gitfs_remotes. A couple of formulas now break whenever a dictionary item can't be resolved.

I am using CentOS, and this formula fails because it can't resolve salt.pkrepo.centos. Since this was working fine when pulled from file_roots I am not sure what to say, although wrapping the contents of salt/pkgrepo/init.sls in an if checking for os_family == 'Debian' eliminates the problem for me.

salt.master fails in salt-ssh: TemplateNotFound: salt/formulas.jinja

I'm attempting to use salt-ssh 2015.5.0 to set up a salt master. The salt.master state is failing.

----------
          ID: salt-master
    Function: file.recurse
        Name: /etc/salt/master.d
      Result: False
     Comment: #### /etc/salt/master.d/f_defaults.conf ####
              Unable to manage file: Jinja error: salt/formulas.jinja
              Traceback (most recent call last):
                File "/tmp/.root_679d2e__salt/salt/utils/templates.py", line 306, in render_jinja_tmpl
                  output = template.render(**decoded_context)
                File "/tmp/.root_679d2e__salt/jinja2/environment.py", line 969, in render
                  return self.environment.handle_exception(exc_info, True)
                File "/tmp/.root_679d2e__salt/jinja2/environment.py", line 742, in handle_exception
                  reraise(exc_type, exc_value, tb)
                File "<template>", line 15, in top-level template code
                File "/tmp/.root_679d2e__salt/salt/utils/jinja.py", line 110, in get_source
                  raise TemplateNotFound(template)
              TemplateNotFound: salt/formulas.jinja

              ; line 15

              ---
              [...]
              {{ configname }}: {{ cfg_salt[configname] }}
              {%- else -%}
              #{{ configname }}: {{ default_value }}
              {%- endif -%}
              {%- endmacro -%}
              {%- from 'salt/formulas.jinja' import file_roots, formulas with context -%}    <======================
              ##### Primary configuration settings #####
              ##########################################
              # This configuration file is used to manage the behavior of the Salt Master
              # Values that are commented out but have no space after the comment are
              # defaults that need not be set in the config. If there is a space after the
              [...]
              ---
     Started: 04:18:21.410948
    Duration: 271.237 ms
     Changes:   

I cloned the formula manually and have it referenced in my file_roots. Not sure if that would have an impact over bringing it in via gitfs_remotes, but I'm trying to avoid adding the gitfs dependencies.

Jinja2 mapping test failure on RHEL 6

Salting minion fails on all RHEL 6 boxes with python-jinja2-2.2.1-2.el6_5.x86_64. Same run succeeds on RHEL 7 boxes with python-jinja2-2.7.2-2.el7.noarch.

----------
          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: #### /etc/salt/minion.d/f_defaults.conf ####
              Unable to manage file: Jinja syntax error: no test named 'mapping'
              /var/cache/salt/minion/files/prod/salt/formulas.jinja(14):
              ---
              [...]

              {%- macro formulas_git_opt(env, opt) -%}
              {%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
                   salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
                     defaults[opt])) -%}
              {%- if value is mapping -%}    <======================
              {{ value|yaml }}
              {%- else -%}
              {{ value }}
              {%- endif -%}
              {%- endmacro -%}
              [...]
              ---
     Started: 22:55:39.879651
    Duration: 133.819 ms
     Changes:

[CentOS 6] `mapping` test not supported, master config cannot be generated

Unfortunately, since CentOS 6 only ships with Jinja2 2.2.1, the mapping test is not supported. Looking through the documentation, I don't see anything else equivalent to it (both iterable and sequence test true for dicts and lists), so I'm not sure how to maintian functionality without that test.

Tapping @Sjd-Risca, just in case they have some idea

[salt-master-2015.8.0-2.el6] salt.minion ends up with values from salt.master's grains in config file, cannot start

Configuration:

This is surprisingly easy to reproduce:

/srv/salt/top.sls:

base:
  '*':
    - salt.minion

  'G@roles:master':
    - salt.master

/srv/pillar/master.sls:

salt:
  master:
    fileserver_backend:
      - roots
      - git

    file_roots:
      base:
        - /srv/salt

    pillar_roots:
      base:
        - /srv/pillar

    gitfs_provider: pygit2

    gitfs_remotes:
      - git://github.com/saltstack-formulas/salt-formula

/srv/pillar/minion.sls:

salt:
  minion:
    - master: 'salt'

/srv/pillar/top.sls:

base:
  '*':
    - minion

  'G@roles:master':
    - master

Results:

Given those values, I end up with the following in /etc/salt/minion.d/f_defaults.conf:

[root@pluto ~]# grep -v '^#' /etc/salt/minion.d/f_defaults.conf | uniq

master:
  - fileserver_backend
  - gitfs_remotes
  - pillar_roots
  - file_roots
  - gitfs_provider


[root@pluto ~]#

Salt-cloud crypto dependency fails.

I rebuilt my salt master today and the salt.cloud state refuses to run, saying that the attempt to pip.install package 'crypto' failed. I tried it manually and it also failed. I looked at the package index located here: https://pypi.python.org/pypi?%3Aaction=index and see no package named 'crypto.'

I am confused, because this worked a month ago. Did the package get renamed in pip recently, or something? Shouldn't the distribution-provided crypto package be getting used anyway, if possible?

salt-cloud provider configuration file permissions are too open

The salt-cloud provider configuration files (.../cloud.providers.d/*.conf) may contain keying material (e.g., passwords or secret keys). The permissions of these files should be restricted to some group of authorized users (e.g., root or wheel) in order to prevent unauthorized access.

Default value for minion "master" parameter does not work

When the master is also being configured. The problem is this line in salt/files/minion:

{{ get_config('master', 'salt') }}

If pillars['salt']['minion']['master'] does not exist, it will try pillars['salt']['master'], which of course is itself a dict.

salt.minion fails when applied with salt-ssh

I just posted this issue with SaltStack as I believe the bug exists in salt-ssh rather than in this formula directly. This is really just for visibility.

Brief summary:
When calling salt.minion via salt-ssh, the formulas.jinja file is not included and so the minion.d/f_defaults.conf file fails to render.

Workaround is to simply modify the following lines in salt/files/minion.d/f_defaults.conf:

@@ -12,7 +12,6 @@
 #{{ configname }}: {{ default_value }}
 {%- endif -%}
 {%- endmacro -%}
-{%- from 'salt/formulas.jinja' import file_roots, formulas with context -%}
 ##### Primary configuration settings #####
 ##########################################

@@ -235,8 +234,6 @@ file_client: local
 {{ file_roots(cfg_minion['file_roots']) }}
 {%- elif 'file_roots' in cfg_salt -%}
 {{ file_roots(cfg_salt['file_roots']) }}
-{%- elif formulas|length -%}
-{{ file_roots({'base': ['/srv/salt']}) }}
 {%- endif %}

 # limit fileserver traversal

Reduce amount of configuration in rendered configuration files

For example in this one we can see many default comments.
Although this is important for new users, the comments make much harder to navigate among all this lines.
Moreover if meaning of some parameter changes and we don't syncronize the change with original, our words will be confusing.

What do you think about removing these comments?

Doesn't entirely play nice with salt-cloud

Ran into an issue today: salt.minion does all of its management in minion.d and doesn't touch the /etc/salt/minion file. salt-cloud does the opposite. Usually the settings in the former override the latter so everything is okay. However, for features that can be configured either from the minion config or the pillar root (such as mine_functions), the cloud-deployed file takes precedence over pillar, and salt.minion has no way of cleaning it up after the fact.

I don't think this is really a bug, but removing those settings after the fact is surprisingly irritating given that salt.minion won't touch that file.

I don't know the correct way to deal with this, but it seems like some option for managing /etc/salt/minion or otherwise neutering it (perhaps off by default) would be a helpful thing to have.

salt.minion is not working centos5

2015-07-29 15:09:16,376 [salt.state                                 ][ERROR   ][8517] #### /etc/salt/minion.d/f_defaults.conf ####
Unable to manage file: Jinja syntax error: no test named 'mapping'
/var/cache/salt/minion/files/itg/salt/formulas.jinja(14):

---
[...]

{%- macro formulas_git_opt(env, opt) -%}
{%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
     salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
       defaults[opt])) -%}
{%- if value is mapping -%}    <======================
{{ value|yaml }}
{%- else -%}
{{ value }}
{%- endif -%}
{%- endmacro -%}
[...]

Mapping test is available from jinja version >= 2.6. But for centos5 available is only 2.5 version.

salt.formulas fails on highstate if not data

I removed the salt_formulas: thing from my pillar, but forgot the salt/top.sls with the salt.formulas there and it started to fail.

[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/minion.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/formulas.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/formulas.jinja'
[CRITICAL] Rendering SLS 'base:salt.formulas' failed: Jinja variable 'str object' has no attribute 'iteritems'
local:
    Data failed to compile:
----------
    Rendering SLS 'base:salt.formulas' failed: Jinja variable 'str object' has no attribute 'iteritems'

Manage salt-key as well

Accepting minions after installation with this formula will make provisioning much easier.

10 failed states when provisioning Vagrant with all defaults

(Warning: I'm a Salt newb)

Running the following from a clean checkout of salt-formula:

vagrant up
vagrant ssh
vagrant@salt:~$ sudo su -
root@salt:~# salt-call state.highstate

I get the following failed states:

----------
          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: Recurse failed: none of the specified sources were found
     Started: 23:03:24.860639
    Duration: 8292.687 ms
     Changes:   
----------
          ID: salt-minion
    Function: service.running
      Result: False
     Comment: One or more requisite failed: {'salt.minion.salt-minion': 'Recurse failed: none of the specified sources were found'}
     Started: 
    Duration: 
     Changes:   
----------
          ID: cloud-cert-aws-pem
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/key/aws.pem
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.656005
    Duration: 8.735 ms
     Changes:   
----------
          ID: cloud-cert-gce-pem
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/key/gce.pem
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.666232
    Duration: 11.163 ms
     Changes:   
----------
          ID: salt-cloud-profiles-ec2
    Function: file.managed
        Name: /etc/salt/cloud.profiles.d/ec2.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.677963
    Duration: 7.45 ms
     Changes:   
----------
          ID: salt-cloud-profiles-gce
    Function: file.managed
        Name: /etc/salt/cloud.profiles.d/gce.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.686129
    Duration: 12.509 ms
     Changes:   
----------
          ID: salt-cloud-providers-ec2
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/ec2.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.699382
    Duration: 3.817 ms
     Changes:   
----------
          ID: salt-cloud-providers-gce
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/gce.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.703812
    Duration: 5.745 ms
     Changes:   
----------
          ID: salt-cloud-maps-ec2
    Function: file.managed
        Name: /etc/salt/cloud.maps.d/ec2.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.710319
    Duration: 6.986 ms
     Changes:   
----------
          ID: salt-cloud-maps-gce
    Function: file.managed
        Name: /etc/salt/cloud.maps.d/gce.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.722172
    Duration: 10.791 ms
     Changes:   

After this failure if I run:

salt-run fileserver.update
salt-call state.highstate

all states succeed.

What is the ideal set of commands to go from 'vagrant up' to all states passing with no failures? Not sure if I am missing something... for a new person seeing a successful run of this formula from a clean repo would be a good confidence builder :).

Error when running salt.minion

The following error occurs:

          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: #### /etc/salt/minion.d/_defaults.conf ####
              Unable to manage file: Jinja syntax error: no test named 'mapping'
              /var/cache/salt/minion/files/base/salt/formulas.jinja(14):
              ---
              [...]

              {%- macro formulas_git_opt(env, opt) -%}
              {%- set value = salt['pillar.get']('salt_formulas:git_opts:{}:{}'.format(env, opt),
                   salt['pillar.get']('salt_formulas:git_opts:default:{}'.format(opt),
                     defaults[opt])) -%}
              {%- if value is mapping -%}    <======================
              {{ value|yaml }}
              {%- else -%}
              {{ value }}
              {%- endif -%}
              {%- endmacro -%}
              [...]
              ---

Which appears to come from line 14 in ../salt/formulas.jinja

Not sure what the right value should be.

Use `repo.saltstack.com` where appropriate/possible

Currently, the following distributions/releases are covered by https://repo.saltstack.com:

  • Debian
    • Jessie (8)
  • Ubuntu
    • Precise Pangolin (12.04)
    • Trusty Tahr (14.04)
  • RHEL/CentOS
    • RHEL/CentOS 5
    • RHEL/CentOS 6
    • RHEL/CentOS 7

salt.pkgrepo should make use of repo.saltstack.com where appropriate/possible.

I might take care of providing a PR for Ubuntu. Can't promise yet to add support for other distributions as well.

Not obvious how to use it

I'm new to Salt. I've read the documentation fairly extensively but without a "here's how to run this" in the README.md I'm unlikely to have time to figure out how to use this, even though I would really like to.

Can I use salt-ssh from my local machine to use this to deploy salt onto a remote machine? If so, how? Can it be explained in the README?

Restart salt-minion *after* end of run

Hi all,

Is there a reason why something like : http://docs.saltstack.com/en/latest/faq.html#what-is-the-best-way-to-restart-a-salt-daemon-using-salt is not used in this formula ?

I changed salt-formula (locally), it's a huge win : I can now spawn a new salt minion (with salt-cloud) in one highstate run, without interruption.

salt/minion.sls :

  cmd.wait:
    - name: echo salt-call --local service.restart salt-minion | at now + 1 minute
    - python_shell: True
    - order: last
    - watch:
{% if salt_settings.install_packages %}
      - pkg: salt-minion
{% endif %}
      - file: salt-minion
      - file: remove-old-minion-conf-file

I didn't manage "atd" since it's on by default on Debian, but it's definitely doable.

Errors rendering salt/formulas.sls or salt/files/master.d/f_defaults.conf

Running the salt.formulas state results in the following error:

root@uxeprdbsdsalt01:~ # salt-call state.sls salt.formulas saltenv=production test=True
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'production', ** skipped ** latest already in cache 'salt://salt/formulas.sls'
[INFO    ] Fetching file from saltenv 'production', ** done ** 'salt/formulas.jinja'
[CRITICAL] Rendering SLS 'production:salt.formulas' failed: mapping values are not allowed here; line 18

---
[...]

# Setup the directory hosting the Git repository


/usr/local/etc/salt/formulas
...:    <======================
  file.directory:
    - makedirs: True
    - group: wheel
    - user: root
    - mode: 755
[...]

---
local:
    Data failed to compile:
----------
    Rendering SLS 'production:salt.formulas' failed: mapping values are not allowed here; line 18

---
[...]

# Setup the directory hosting the Git repository


/usr/local/etc/salt/formulas
...:    <======================
  file.directory:
    - makedirs: True
    - group: wheel
    - user: root
    - mode: 755
[...]

---

Running the salt.master state results in the following incorrect rendering (truncated for brevity's sake):

root@uxeprdbsdsalt01:~ # salt-call state.sls salt.master saltenv=development test=True
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO    ] Fetching file from saltenv 'development', ** done ** 'salt/map.jinja'
[INFO    ] Fetching file from saltenv 'development', ** done ** 'salt/defaults.yaml'
[INFO    ] Executing command '/sbin/zfs -?' in directory '/root'
[INFO    ] Running state [py27-salt] at time 14:50:01.749847
[INFO    ] Executing state pkg.installed for py27-salt
[INFO    ] Executing command 'pkg info -ao' in directory '/root'
[INFO    ] Package py27-salt is already installed.
[INFO    ] Completed state [py27-salt] at time 14:50:01.963569
[INFO    ] Running state [/usr/local/etc/salt/master.d] at time 14:50:01.968229
[INFO    ] Executing state file.recurse for /usr/local/etc/salt/master.d
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache u'salt://salt/files/master.d/f_defaults.conf'
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache 'salt://salt/formulas.jinja'
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache u'salt://salt/files/master.d/reactor.conf'
[INFO    ] {'/usr/local/etc/salt/master.d/f_defaults.conf': {'diff': '--- \n+++ \n@@ -431,77 +431,146 @@\n     - /code/twgs-formula\n     - /code/users-formula\n   production:\n-    - /usr/local/etc/salt/formulas/active-directory-formula\n-    - /usr/local/etc/salt/formulas/epel-formula\n-    - /usr/local/etc/salt/formulas/fail2ban-formula\n-    - /usr/local/etc/salt/formulas/firewalld-formula\n-    - /usr/local/etc/salt/formulas/git-formula\n-    - /usr/local/etc/salt/formulas/jenkins-formula\n-    - /usr/local/etc/salt/formulas/mysql-formula\n-    - /usr/local/etc/salt/formulas/ntp-formula\n-    - /usr/local/etc/salt/formulas/nux-formula\n-    - /usr/local/etc/salt/formulas/openstack-formula\n-    - /usr/local/etc/salt/formulas/openssh-formula\n-    - /usr
...
----------
          ID: salt-master
    Function: file.recurse
        Name: /usr/local/etc/salt/master.d
      Result: None
     Comment: #### /usr/local/etc/salt/master.d/f_defaults.conf ####
              The file /usr/local/etc/salt/master.d/f_defaults.conf is set to be changed
     Started: 14:50:01.968229
    Duration: 890.319 ms
     Changes:
              ----------
              /usr/local/etc/salt/master.d/f_defaults.conf:
                  ----------
                  diff:
                      ---
                      +++
                      @@ -431,77 +431,146 @@
                           - /code/twgs-formula
                           - /code/users-formula
                         production:
                      -    - /usr/local/etc/salt/formulas/active-directory-formula
                      -    - /usr/local/etc/salt/formulas/epel-formula
                      -    - /usr/local/etc/salt/formulas/fail2ban-formula
                      -    - /usr/local/etc/salt/formulas/firewalld-formula
                      -    - /usr/local/etc/salt/formulas/git-formula
                      -    - /usr/local/etc/salt/formulas/jenkins-formula
                      -    - /usr/local/etc/salt/formulas/mysql-formula
                      -    - /usr/local/etc/salt/formulas/ntp-formula
                      -    - /usr/local/etc/salt/formulas/nux-formula
                      -    - /usr/local/etc/salt/formulas/openstack-formula
                      -    - /usr/local/etc/salt/formulas/openssh-formula
                      -    - /usr/local/etc/salt/formulas/os-hardening-formula
                      -    - /usr/local/etc/salt/formulas/owncloud-formula
                      -    - /usr/local/etc/salt/formulas/postgres-formula
                      -    - /usr/local/etc/salt/formulas/rabbitmq-formula
                      -    - /usr/local/etc/salt/formulas/salt-formula
                      -    - /usr/local/etc/salt/formulas/shibboleth-formula
                      -    - /usr/local/etc/salt/formulas/snmp-formula
                      -    - /usr/local/etc/salt/formulas/spigotmc-formula
                      -    - /usr/local/etc/salt/formulas/sudoers-formula
                      -    - /usr/local/etc/salt/formulas/tomcat-formula
                      -    - /usr/local/etc/salt/formulas/twgs-formula
                      -    - /usr/local/etc/salt/formulas/users-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../active-directory-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../epel-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../fail2ban-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../firewalld-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../git-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../jenkins-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../mysql-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../ntp-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../nux-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../openstack-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../openssh-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../os-hardening-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../owncloud-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../postgres-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../rabbitmq-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../salt-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../shibboleth-formula

move changed config files to master.d and minion.d

I would like to propose the following change:

if a config file is changed by pillar data put the file in:

/etc/salt/master.d/master.conf
/etc/salt/minion.d/minion.conf

So not to overwrite the defaults salt package provides ?
If i'm not mistaken salt will prefer the files in master.d and minion.d
because all defaults are commented.

minion/master file.recurse needs to stop using clean: True

The salt code (and myself) put files in /etc/salt/{master,minion}.d

The "clean: True" is killing all that. It probably shouldn't do that.

salt-minion:
  file.recurse:
    - name: {{ salt.get('config-path', '/etc/salt') }}/minion.d
    - template: jinja
    - source: salt://salt/files/minion.d
    - clean: True
    - context:
        standalone: False

Master template gitfs_remotes must have children

{% if 'gitfs_remotes' in master -%}
gitfs_remotes:
{%- for remote in master['gitfs_remotes'] %}
{% if remote is iterable %}
{%- for repo, children in remote.iteritems() -%}
- {{ repo }}:
{%- for child in children %}
{% for key, value in child.iteritems() -%}
- {{ key }}: {{ value }}
{%- endfor %}
{%- endfor %}
{%- endfor %}
{%- else -%}
- {{ remote }}
{%- endif %}
{%- endfor %}
{%- endif %}

is iterable doesn't seem to be a deep enough check.

My salt.sls pillar looks something like this:

salt:
  master:
    fileserver_backend:
      - git
      - roots
    gitfs_remotes:
      - [email protected]:saltstack-formulas/salt-formula.git

And if the remote item doesn't have a child as the above doesn't (note that there's no data about the remote, just the address) it fails with the following jinja error:

I get the following jinja error when running state.highstate:

          ID: salt-master
    Function: file.recurse
        Name: /etc/salt/master.d
      Result: False
     Comment: #### /etc/salt/master.d/_defaults.conf ####
              Unable to manage file: Jinja variable 'str object' has no attribute 'iteritems'
     Started: 03:18:47.515161
    Duration: 153.94 ms
     Changes:

If I add a remote child, it starts working:

    gitfs_remotes:
      - [email protected]:saltstack-formulas/salt-formula.git:
        - base: master

salt versions-report is:

           Salt: 2014.7.0
         Python: 2.7.5 (default, Jun 17 2014, 18:11:42)
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 4.0.4
           Mako: Not Installed

salt version 2015.8.0rc1 can't reference {{ slspath }}

in the latest salt version 2015.8.0rc1 the source salt://{{ slspath }} doesn't work anymore

file_|-salt-master_|-/etc/salt/master.d_|-recurse:
        ----------
        __run_num__:
            1
        changes:
            ----------
        comment:
            #### /etc/salt/master.d/reactor.conf ####
            Unable to manage file: [Errno 2] No such file or directory: u'|/var/cache/salt/minion/files/base/salt/files/master.d/reactor.conf'

            #### /etc/salt/master.d/f_defaults.conf ####
            Unable to manage file: [Errno 2] No such file or directory: u'|/var/cache/salt/minion/files/base/salt/files/master.d/f_defaults.conf'
        duration:
            52.152 ms
        name:
            /etc/salt/master.d
        result:
            False
        start_time:
            21:36:23.706337

if I change the {{ slspath }} back to the hardcoded salt://salt/files/ it works

Rename to avoid fork conflict

One can not fork "saltstack/salt" and " saltstack-formulas/salt".
And having a "salt" outside saltstack-formulas is misleading.
This issue applies to saltstack-formulas/*
Maybe prefix with something like salt-formula-* ?

Error while trying to use salt.gitfs.gitpython

I'm getting an error when trying use this state.

johncc@aws-dev-c-salt-1:~⟫ sudo salt-call -l debug state.sls salt.gitfs.gitpython                                                                                                                                                              
[sudo] password for johncc: 
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG   ] Configuration file path: /etc/salt/minion
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG   ] Initializing new SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] Decrypting the current master AES key
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded state.sls
[DEBUG   ] LazyLoaded saltutil.is_running
[DEBUG   ] LazyLoaded grains.get
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[INFO    ] Loading fresh modules for state activity
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/gitfs/gitpython.sls'
[DEBUG   ] LazyLoaded cmd.run
[DEBUG   ] Jinja search path: ['/var/cache/salt/minion/files/base']
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[DEBUG   ] LazyLoaded grains.filter_by
[DEBUG   ] LazyLoaded pillar.get
[CRITICAL] Rendering SLS 'base:salt.gitfs.gitpython' failed: Jinja variable 'dict object' has no attribute 'gitpython'
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] LazyLoaded highstate.output
local:
    Data failed to compile:
----------
    Rendering SLS 'base:salt.gitfs.gitpython' failed: Jinja variable 'dict object' has no attribute 'gitpython'

It was suggested in the IRC channel by AndreasLutro that the problem is caused by incorrect merging of the default and OS specific settings dictionaries.

In particular, it was suggested that this line is at fault https://github.com/saltstack-formulas/salt-formula/blob/master/salt/map.jinja#L79 and should be changed to do a deep merge instead of a shallow merge.

salt_cloud_certs broken by pull #106 ("Don't overwrite salt variable")

In context, see cb3aa80#diff-cc4a14e8ec642fa03a6038c172bd52b0L46.

The original Pillar lookup had nothing to do with the salt variable that was renamed to salt_settings. Instead, it was querying the value of the Pillar key "salt_cloud_certs". This commit changes the Pillar key to "salt:salt_cloud_certs" without also updating https://github.com/saltstack-formulas/salt-formula/blob/master/salt/files/key and https://github.com/saltstack-formulas/salt-formula/blob/master/pillar.example, and it violates POLA in that it breaks working salt-cloud configurations managed by salt-formula.

`gitfs` support doesn't ensure dependency on `pip` is fulfilled

gitfs support makes use of a pip.installed state in several places.
When pip isn't installed, applying the salt.gitfs state will fail like this:

# salt-call state.sls salt.gitfs.pygit2 test=true
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'base', ** done ** u'salt/gitfs/pygit2.sls'
[INFO    ] Fetching file from saltenv 'base', ** done ** u'salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** done ** u'salt/defaults.yaml'
[ERROR   ] You should upgrade pyOpenSSL to at least 0.14.1 to enable the use of X509 extensions
[INFO    ] Running state [git] at time 11:41:13.008687
[INFO    ] Executing state pkg.installed for git
[INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}\n', '-W'] in directory '/root'
[INFO    ] Package git is already installed
[INFO    ] Completed state [git] at time 11:41:13.499474
[INFO    ] Running state [libgit2-22] at time 11:41:13.499758
[INFO    ] Executing state pkg.installed for libgit2-22
[INFO    ] The following packages are set to be installed/updated: libgit2-22
[INFO    ] Completed state [libgit2-22] at time 11:41:13.504759
[INFO    ] Running state [pygit2 == 0.22.1] at time 11:41:13.505978
[INFO    ] Executing state pip.installed for pygit2 == 0.22.1
[ERROR   ] An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
[INFO    ] Completed state [pygit2 == 0.22.1] at time 11:41:13.506796
local:
----------
          ID: pygit2-git
    Function: pkg.installed
        Name: git
      Result: True
     Comment: Package git is already installed
     Started: 11:41:13.008687
    Duration: 490.787 ms
     Changes:   
----------
          ID: libgit2-22
    Function: pkg.installed
      Result: None
     Comment: The following packages are set to be installed/updated: libgit2-22
     Started: 11:41:13.499758
    Duration: 5.001 ms
     Changes:   
----------
          ID: install-pygit2
    Function: pip.installed
        Name: pygit2 == 0.22.1
      Result: False
     Comment: An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
     Started: 11:41:13.505978
    Duration: 0.818 ms
     Changes:   

Summary for local
------------
Succeeded: 2 (unchanged=1)
Failed:    1
------------
Total states run:     3
Total run time: 496.606 ms

salt.minion fails with syntax error: no test named 'mapping' RHEL6/Jinja 2.2.1

It looks like mapping test requires Jinja2 v2.6 or higher and I'm on 2.2.1 - which is the best RHEL 6 provides.

          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: #### /etc/salt/minion.d/f_defaults.conf ####
              Unable to manage file: Jinja syntax error: no test named 'mapping'
              /var/cache/salt/minion/files/base/salt/formulas.jinja(14):
              ---
              [...]

              {%- macro formulas_git_opt(env, opt) -%}
              {%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
                   salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
                     defaults[opt])) -%}
              {%- if value is mapping -%}    <======================
              {{ value|yaml }}
              {%- else -%}
              {{ value }}
              {%- endif -%}
              {%- endmacro -%}
# salt-master --versions
           Salt: 2015.5.3
         Python: 2.6.6 (r266:84292, Nov 21 2013, 10:50:32)
         Jinja2: 2.2.1
       M2Crypto: 0.20.2
 msgpack-python: 0.4.6
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.5.0
           RAET: Not Installed
            ZMQ: 4.0.5
           Mako: 0.3.4
        Tornado: Not Installed

Windows is missing

How about salting/upgrading salt minions on windows in a stable way? I cant see support for that in this formula.

I have some work implemented for this but its very "hacky" due to that nssm.exe often hangs and locks files after an uninstall. To come around this I'm rebooting the server and continue the installation afterwards but this is far from optimal.

I can however, share the work if anyone is interested.

Use as submodule

I can not see how to use this has a git submodule.
I'm forced to create a state to override all salt:// declarations with include and extend to get things work.
Not being able to use this as submodule reduce the interest of the project from "plug and play" to "cherry pick". And cherry pick won't bring contributions..
I've found no solution for now. But I'm new to saltstack. I may have missed something.

salt.syndic fails to compile

# salt-call state.sls salt.syndic test=True
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'base', ** done ** 'salt/syndic.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
local:
    Data failed to compile:
----------
    A recursive requisite was found, SLS "salt.syndic" ID "salt-syndic" ID "salt-syndic"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.