amendlik / gitstack-pillar Goto Github PK

Will you do pull request for saltstack repository for gitstack.py? It's quite handy.

according to salt version 2017.7.2 and develop branch:
here and here

the directory _pillars should be renamed to _pillar or fix docs for fileserver_backend to use git section, saying:

gitfs_remotes:
  - https://github.com/amendlik/gitstack-pillar.git:
    - root: _pillars
    - mountpoint: _pillar

The API that GitStack uses changed in 2017.7.5 and 2018.3.1. As a result GitStack does not work with those versions or newer. The 2016.11 branch was not affected by this change.

This was originally reported in saltstack/salt#47841

Questions

1. Is Gitstack a fork from https://github.com/bbinet/pillarstack or a rewrite? If so, is it intended to exist separately or be merged back in to Pillarstack?
2. Are you interested in maintaining feature parity with pillarstack? In particular the Jinja environment features of Salt, (and consequently pillarstack), are always expanding, are you interested in supporting up to date Jinja in GitStack?
3. Any particular advice for people interested in switching from pillarstack to GitStack?
4. Are you interested in feeding gitstack into Salt so that it can be available by default in a future release?
5. Is GitStack intended as a cut-over replacement for Pillarstack or more of a redesign replacement?
6. Any other interesting plans for the future for this module?

I ask as someone who is interested in switching from Pillarstack to GitStack.

• bbinet/pillarstack#16
• bbinet/pillarstack#24

When using the older config style with a list in the stack keyword, an error is produced:

[CRITICAL] Pillar render error: Failed to load ext_pillar gitstack: local variable 'stack_config_kwargs' referenced before assignment

Here's an example of the gitstack config I'm using:

ext_pillar:
  - git:
    - master https://github.com/genesis-matrix/gema-plr-core
  - gitstack:
      repo: https://github.com/genesis-matrix/gema-plr-core
      stack:
        - pillarstack.cfg
        - pillarstack.cfg
        - pillarstack.cfg
        - pillarstack.cfg

Apparently this line https://github.com/amendlik/gitstack-pillar/blob/develop/_pillar/gitstack.py#L86
cannot works on PY3

2019-04-11 23:12:07,106 [salt.pillar      :984 ][ERROR   ][6684] Exception caught loading ext_pillar 'gitstack':
  File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 974, in ext_pillar
    key)
  File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 902, in _external_pillar_data
    *val)
  File "/var/cache/salt/master/extmods/pillar/gitstack.py", line 86, in ext_pillar
    if len(salt.utils.gitfs.GitPillar.__init__.im_func.func_code.co_varnames) > 2:

2019-04-11 23:12:07,339 [salt.pillar      :1036][CRITICAL][6684] Pillar render error: Failed to load ext_pillar gitstack: 'function' object has no attribute 'im_func'

I'm not sure if I'm missing something, but I can't seem to get this to work. What I'm trying to do is something similar to the PillarStack example:

ext_pillar:  
    - stack:  
      - /path/to/stack1.cfg  
      - /path/to/stack2.cfg

So I'm trying this:

ext_pillar:
    - git:
      - master [email protected]:salt/pillar.git
    - gitstack:
      - master [email protected]:salt/pillar.git:
        - stack:
          - _stack/stack_1.cfg
          - _stack/stack_2_roles.cfg
          - _stack/stack_3_minions.cfg

But I get an error on the Salt master:

2018-09-27 16:07:43,836 [salt.pillar      :972 ][ERROR   ][16772] Execption caught loading ext_pillar 'gitstack':
  File "/usr/lib/python2.7/site-packages/salt/pillar/__init__.py", line 962, in ext_pillar
    key)
  File "/usr/lib/python2.7/site-packages/salt/pillar/__init__.py", line 890, in _external_pillar_data
    *val)
  File "/var/cache/salt/master/extmods/pillar/gitstack.py", line 144, in ext_pillar
    return stack_pillar(minion_id, pillar, *stack_config, **stack_config_kwargs)
  File "/usr/lib/python2.7/site-packages/salt/pillar/stack.py", line 415, in ext_pillar
    if not os.path.isfile(cfg):
  File "/usr/lib64/python2.7/genericpath.py", line 29, in isfile
    st = os.stat(path)

2018-09-27 16:07:44,001 [salt.pillar      :1019][CRITICAL][16772] Pillar render error: Failed to load ext_pillar gitstack: coercing to Unicode: need string or buffer, list found

I've tried following the flow but I'm still learning and having trouble figuring out exactly where things are going wrong. My guess is that it's something in the section that resolves the paths or in the _resolve_stack function.

I'll try to play around with it later to see if I can figure out what's going wrong but I'd appreciate some help from someone with more expertise. Thank you!

Hi - I'm currently trying to use gitstack following the pillarstack docs with the example of iterating over the 'roles' as shown here: https://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.stack.html#pillarstack-configuration-files

My config looks like this:

top.sls

{% set hostname, domain = opts['id'].split('.', 1) %}

base:
 '*':
    - minions/{{ hostname }}

minions/server1.sls

roles:
  - a
  - b

stack.cfg

{%- for role in pillar.get('roles', []) %}
roles/{{ role }}.sls
{%- endfor %}

When I call salt 'server1' pillar.items I see:

server1:
    ----------
    roles:
        - a
        - b

However none of the data in roles/a.sls or roles/b.sls is ever included.

I wasn't sure the best way to debug the stack state, so for testing I put the following in stack.cfg:

TESTING/{{ pillar }}

This results in the following in the logs:

Ignoring pillar stack template "TESTING/{}" ...

Any ideas why pillar is an empty dict?

With Pillarstack, files are only searched relative to the directory of the stack.cfg file.

GitStack now supports defining multiple git pillar repos as pillarstack targets, but they are processed sequentially, as distinct invocation/passes.

This feature requests the option to include other pillar directories into search path used in a single stack.cfg pass.

I'd like to be able to break pillar configuration into multiple git repos that overlay on each other into a single tree that could be passed to pillarstack.

This would allow files in different directory trees to be processed by precedence within the same pass.

This would allow for greater modularity, which would support greater re-use.