saltstack-formulas / packages-formula Goto Github PK
View Code? Open in Web Editor NEWA simple 'packages manager' formula, to install/remove packages without further ado.
License: Apache License 2.0
packages-formula's People
Stargazers
Watchers
Forkers
netmanagers noelmcloughlin camlog nuancesalt ikogan stooj abele ops-sandy jcwoltz steffenritschel century-tech ze42 amitchotaliya trexglobal myii contegix cvillers wanless-systems daks corpitsysadmins 14west-devops kartzone1 arthurzenika kiniou opendatasoft joniw aboe76 apfm-salt marrold-salt ypid-geberit wilful fgionghi sifive mdschmitt dyap7em onestic cetanu onesdata tiebreak-solutions fleetwood-systems mbonastre parisbite niclor jeff350 safanaj didiermfbpackages-formula's Issues
[FEATURE] (archive.sls) Keep idempotency if the wanted package is a file inside a tarball?
Is your feature request related to a problem?
Every time I run state.apply, it module.runs file.check_hash and extracts the binary from the tarball.
Describe the solution you'd like
I'm not exactly sure how, but it'd be neat if things were idempotent when installing packages that are really just binaries inside tarballs.
Describe alternatives you've considered
N/A
Additional context
I could absolutely be misusing it, so maybe there's a better way?
Pillar:
packages:
tmpdir: /var/cache/salt/minion/extrn_files
archives:
wanted:
rg:
dest: /usr/local/bin
dl:
format: tar
source: https://github.com/BurntSushi/ripgrep/releases/download/13.0.0/ripgrep-13.0.0-x86_64-unknown-linux-musl.tar.gz
hashsum: ee4e0751ab108b6da4f47c52da187d5177dc371f0f512a7caaec5434e711c091
options: '--strip-components=1 --no-same-owner ripgrep-13.0.0-x86_64-unknown-linux-musl/rg'Output on state.apply regardless of whether or not the file's already there and correct:
ID: packages-archive-wanted-rg-check-hashsum
Function: module.run
Name: file.check_hash
Result: True
Comment: Module function file.check_hash executed
Started: 00:45:00.061936
Duration: 6.53 ms
Changes:
----------
ret:
True
ID: packages-archive-wanted-install-rg
Function: archive.extracted
Name: /usr/local/bin/
Result: True
Comment: /var/cache/salt/minion/extrn_files/ripgrep-13.0.0-x86_64-unknown-linux-musl.tar.gz extracted to /usr/local/bin/, due to absence of one or more files/dirs. Output was trimmed to False number of lines
Started: 00:45:00.068953
Duration: 112.007 ms
Changes:
----------
extracted_files:
- ripgrep-13.0.0-x86_64-unknown-linux-musl/rg
Update the formula to follow template-formula guidelines
Update the formula to follow the guidelines of template-formula as much as possible.
[BUG] Fix failing CI instances
Your setup
Formula commit hash / release tag
Just pushed 592e3d1.
Versions reports (master & minion)
N/A.
Pillar / config used
As used by the CI.
Bug details
Describe the bug
Updated the CI to use the latest pre-salted images. Have three platforms failing which really shouldn't be. Disabled at the moment:
packages-formula/.gitlab-ci.yml
Lines 136 to 151 in 592e3d1
Errors:
- https://gitlab.com/myii/packages-formula/-/jobs/1027075752
- https://gitlab.com/myii/packages-formula/-/jobs/1027075761
- https://gitlab.com/myii/packages-formula/-/jobs/1027075777
Perhaps the easiest answer is just adding more suites. In any case, these entries should be enabled once working.
Steps to reproduce the bug
Expected behaviour
Attempts to fix the bug
Additional context
Add pillar merging support
The concept is simple but it's difficult for me to put into words so hopefully this all makes sense.
I have some server types that needs some extra packages that I don't want installed elsewhere but that don't necessarily need a whole role/formula defined for them. Everyone I talk to suggests that putting too much logic in a single init.sls pillar is not best practice, and that makes sense to me. It seems the best way to do this is leave the logic in top.sls like this:
#/srv/pillar/top.sls
base:
'*':
- packages
'srvtype*':
- packages.srvtype
#/srv/pillar/packages/init.sls
packages:
pkgs:
wanted:
- bash
#/srv/pillar/packages/srvtype.sls
packages:
pkgs:
wanted:
- otherpackage
With this setup the 'srvtype*' minions only see 'otherpackage' as wanted as it overrides the packages in init.sls instead of merging.
I know it's possible to set salt master to always merge lists but I don't think that's the best way to go.
It would be fantastic if the formula could merge them, either by logic in the formula I guess, or by setting up the formula to get pillar data as a dictionary instead of lists? I know vaguely what is needed but don't have the chops to actually do it myself.
[BUG] with empy wanted_pkgs and 2019.2 fails on pkg.installed
Your setup
Formula commit hash / release tag
Versions reports (master & minion)
salt-call --versions-report
Salt Version:
Salt: 2019.2.2
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.5.3
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.9.4
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.8
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
python-gnupg: Not Installed
PyYAML: 3.12
PyZMQ: 16.0.2
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.4.3
ZMQ: 4.2.1
System Versions:
dist: debian 9.11
locale: UTF-8
machine: x86_64
release: 4.9.0-11-amd64
system: Linux
version: debian 9.11
Pillar / config used
packages:
pkgs:
wanted:
Bug details
Describe the bug
----------
ID: wanted_pkgs
Function: pkg.installed
Result: False
Comment: Attempt 1: Returned a result of "False", with the following comment: "Problem encountered installing package(s). Additional info follows:
errors:
- Running scope as unit: run-r277f85bc77b94f668eefedd7cedd7a54.scope
E: Unable to locate package wanted_pkgs"
Attempt 2: Returned a result of "False", with the following comment: "Problem encountered installing package(s). Additional info follows:
errors:
- Running scope as unit: run-red3c26eba2cf440487d115a396506e48.scope
E: Unable to locate package wanted_pkgs"
Problem encountered installing package(s). Additional info follows:
errors:
- Running scope as unit: run-rec8c9a5c750d4ce9882b5ec09b725ccc.scope
E: Unable to locate package wanted_pkgs
Started: 13:17:32.463431
Duration: 131452.439 ms
Changes:
Steps to reproduce the bug
Expected behaviour
Attempts to fix the bug
Additional context
Improve README to be less repetitive
As the formula tries to be consistent among its different states, most of the options have the same meaning/behaviour, ie pkgs:required:states and pips:required:states can just be explained describing what *:required:states does.
Moving these parameters' description to a 'Common parameters valid for all states' section (or something like that) would improve the README removing unneeded repetition.
Add channel support to Snap
Would like to be able to install packages in Snap other than stable. Is this possible?
RFE: Generic Upgrade OS support wanted
This formula could include 'upgrade_os: True/False' support. This and #8 would be useful features.
problem with UTF?
I'm using this formula and since I installed salt 2019.2.0 I have this problem (tested in ubuntu 16.04 and redhat 6):
# salt-call state.apply packages
[INFO ] Loading fresh modules for state activity
[INFO ] Running state [pkg_req_pkgs] at time 15:29:33.789483
[INFO ] Executing state pkg.installed for [pkg_req_pkgs]
[INFO ] Executing command [u'dpkg-query', u'--showformat', u'${Status} ${Package} ${Version} ${Architecture}', u'-W'] in directory '/root'
[INFO ] Executing command [u'apt-cache', u'-q', u'policy', u"u'curl'"] in directory '/root'
[INFO ] Executing command [u'apt-cache', u'-q', u'policy', u"u'gzip'"] in directory '/root'
[INFO ] Executing command [u'apt-cache', u'-q', u'policy', u"u'bzip2'"] in directory '/root'
[INFO ] Executing command [u'apt-get', u'-q', u'update'] in directory '/root'
[INFO ] Executing command [u'dpkg', u'--get-selections', u'*'] in directory '/root'
[INFO ] Executing command [u'systemd-run', u'--scope', u'apt-get', u'-q', u'-y', u'-o', u'DPkg::Options::=--force-confold', u'-o', u'DPkg::Options::=--force-confdef', u'install', u"u'curl'", u"u'gzip'", u"u'bzip2'"] in directory '/root'
[ERROR ] Command '[u'systemd-run', u'--scope', u'apt-get', u'-q', u'-y', u'-o', u'DPkg::Options::=--force-confold', u'-o', u'DPkg::Options::=--force-confdef', u'install', u"u'curl'", u"u'gzip'", u"u'bzip2'"]' failed with return code: 100
[ERROR ] stdout: Reading package lists...
Building dependency tree...
Reading state information...
[ERROR ] stderr: Running scope as unit run-r08177169660649899531c302403fa65e.scope.
E: Unable to locate package u'curl'
E: Unable to locate package u'gzip'
E: Unable to locate package u'bzip2'
[ERROR ] retcode: 100
[INFO ] Executing command [u'dpkg-query', u'--showformat', u'${Status} ${Package} ${Version} ${Architecture}', u'-W'] in directory '/root'
[ERROR ] Problem encountered installing package(s). Additional info follows:
errors:
- Running scope as unit run-r08177169660649899531c302403fa65e.scope.
E: Unable to locate package u'curl'
E: Unable to locate package u'gzip'
E: Unable to locate package u'bzip2'
I think there is a problem with UTF and python because the formula try to install the package u'curl' instead of curl.
Thank you
EDIT: same problem with ubuntu 18.04 with salt python2, but there is no problem with ubuntu 18.04 and salt with python3
[FEATURE] Use chocolatey.bootstrap instead of winrepo-ng to install chocolatey
Describe the solution you'd like
I don't have any solution yet but I was thinking that maybe instead of using winrepo-ng
to just install chocolatey, we could use the available bootstrap module.
In this way we can install chocolatey packages out of the box, without additional configuration and formulas.
RFE: Consider repo handling feature
Would it be in scope, for this formula, to remove list of unwanted repos? For example, apache formula adds a known bad repo so formula fails. This is OS annoyance but just recording this.
ID: wanted_packages
Function: pkg.installed
Result: False
Comment: An error was encountered while installing package(s): W: The repository 'http://httpredir.debian.org/debian xenial Release' does not have a Release file.
E: Failed to fetch http://cdn-fastly.deb.debian.org/debian/dists/xenial/non-free/binary-amd64/Packages 404 Not Found [IP: 151.101.40.204 80]
E: Failed to fetch http://cdn-fastly.deb.debian.org/debian/dists/xenial/non-free/binary-i386/Packages 404 Not Found [IP: 151.101.40.204 80]
E: Failed to fetch http://cdn-fastly.deb.debian.org/debian/dists/xenial/non-free/binary-all/Packages 404 Not Found [IP: 151.101.40.204 80]
E: Failed to fetch http://cdn-fastly.deb.debian.org/debian/dists/xenial/non-free/i18n/Translation-en 404 Not Found [IP: 151.101.40.204 80]
E: Failed to fetch http://cdn-fastly.deb.debian.org/debian/dists/xenial/non-free/dep11/Components-amd64.yml 404 Not Found [IP: 151.101.40.204 80]
E: Failed to fetch http://cdn-fastly.deb.debian.org/debian/dists/xenial/non-free/dep11/icons-64x64.tar 404 Not Found [IP: 151.101.40.204 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
RFE: Pinning packages by name
The 2/21 Ubuntu kernel upgrade fiasco bashed a bunch of my machines, breaking a kernel module.
https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/1750937
I wonder could this formula have a "pin" and "unpin" package-by-name feature? Supporting any package managers (yum/zypp/apt) which have hold and unhold support?
[BUG] some issues on CENT8/RHEL8
Your setup
Formula commit hash / release tag
Versions reports (master & minion)
Salt 3000
Pillar / config used
Bug details
Describe the bug
The packages state throws errors on CentOS8:
[ERROR ] Made the following changes:
'python3-dnf-plugin-versionlock' changed from 'absent' to '4.0.8-3.el8'
[ERROR ] The following packages failed to install/update: yum-plugin-versionlock
The following packages were already installed: curl, bzip2, gzip
[ERROR ] The following packages failed to install/update: yum-plugin-versionlock
The following packages were already installed: curl, bzip2, gzip
[ERROR ] Command '['systemd-run', '--scope', 'yum', '-y', 'install', 'python-devel', 'python2-pip']' failed with return code: 1
[ERROR ] stdout: Running scope as unit: run-r8905c4fd894b4d7993d8b3e9b79c8aa3.scope
SaltStack repo for RHEL/CentOS 8 510 B/s | 557 B 00:01
Failed to download metadata for repo 'saltstack'
Ignoring repositories: saltstack
Last metadata expiration check: 0:02:27 ago on Tue Apr 14 12:46:27 2020.
No match for argument: python-devel
Error: Unable to find a match: python-devel
[ERROR ] retcode: 1
[ERROR ] Error occurred installing package(s). Additional info follows:
errors:
- Running scope as unit: run-r8905c4fd894b4d7993d8b3e9b79c8aa3.scope
SaltStack repo for RHEL/CentOS 8 510 B/s | 557 B 00:01
Failed to download metadata for repo 'saltstack'
Ignoring repositories: saltstack
Last metadata expiration check: 0:02:27 ago on Tue Apr 14 12:46:27 2020.
No match for argument: python-devel
Error: Unable to find a match: python-devel
[ERROR ] Command '['systemd-run', '--scope', 'yum', '-y', 'install', 'policycoreutils-python']' failed with return code: 1
This is related saltstack/salt#54798
Steps to reproduce the bug
Expected behaviour
Attempts to fix the bug
I was thinking about introducing an osfingermap like this but not sure that is best way. Maybe osfamilymap should check major version. Not sure of ideal solution here.
CentOS-7:
pkgs:
required:
pkgs:
- yum-plugin-versionlock
pips:
required:
pkgs:
- python-devel
- python2-pip
CentOS-8:
pkgs:
required:
pkgs:
- python3-dnf-plugin-versionlock
pips:
required:
pkgs:
- python3-devel
- python3-pip
Red Hat Enterprise Linux Server-6: {}
Additional context
Small feature request: Allow naming of archive binary via pillar
Is your feature request related to a problem?
Sort of. Basically it's common that an upstream project will name a binary release to include a version number or platform suffix. Example: https://github.com/mikefarah/yq/releases . I want that binary named yq in /usr/local/bin, not yq_linux_amd64.
I think the target binary filename should based on the name of the pillar value, not the URL suffix.
Describe the solution you'd like
I'd propose the following tweaks:
--- a/packages/archives.sls
+++ b/packages/archives.sls
@@ -26,7 +26,11 @@ packages-archive-unwanted-{{ file_or_directory }}:
# wanted 'archive' software
{%- for package, archive in wanted_archives.items() %}
- {%- set archivename = archive.dl.source.split('/')[-1] %}
+ {%- if archive.dl.format in ('bin',) %}
+ {%- set archivename = package %}
+ {%- else %}
+ {%- set archivename = archive.dl.source.split('/')[-1] %}
+ {%- endif %}
packages-archive-wanted-target-{{ package }}-directory:
file.directory:--- a/pillar.example
+++ b/pillar.example
@@ -146,12 +146,14 @@ packages:
# yamllint disable-line rule:line-length
source: https://raw.githubusercontent.com/openstack/cinder/master/contrib/block-box/docker-compose.yml
hashsum: 1751f8e4f6b4cddd8c4843a0f4473274
- kubectl:
+ # If the format is "bin", the resource name will be used as the filename.
+ # Thus, the following block creates /usr/local/bin/yq
+ yq:
dest: /usr/local/bin
dl:
format: bin
# yamllint disable-line rule:line-length
- source: https://storage.googleapis.com/kubernetes-release/release/v1.12.0/bin/darwin/amd64/kubectl
+ source: https://github.com/mikefarah/yq/releases/download/v4.13.4/yq_linux_amd64
unwanted:
- /usr/local/boring_archive_softwareDescribe alternatives you've considered
n/a
Additional context
n/a
[FEATURE] Use `saltstack/salt` proposed `snap` modules to improve support in this formula (replace `cmd.run` states)
Use the snap modules from the following Salt PR:
- saltstack/salt#56953 (port to
masterfor saltstack/salt#51625).
This will both allow for more features as well as remove the current cmd.run states, to achieve a more robust solution.
As I've discussed in that PR's thread, I've already tested this out and it is working well. I'll provide a PR when have some time available.
This will also close #38.
pip.installed was not found on Suse
Running the following high state on OpenSUSE
- packages.pips
- packages.pkgs
resulted in
[ERROR ] State 'pip.installed' was not found in SLS 'packages.pips'
Reason: 'pip.installed' is not available.
[ERROR ] State 'pip.installed' was not found in SLS 'packages.pips'
Reason: 'pip.installed' is not available.
Add Helm support
Investigate Helm package manager - see also helm-github
RFE: Support holding packages by name only
The formula currently supports holding packages by name: version .
We should also support packages by name only, the following pillar data for example.
held:
- xserver-xorg
- xserver-xorg-core
- kernel-4.13.0-36-generic
[CRITICAL] Rendering SLS 'base:packages.pkgs' failed: Jinja variable 'list object' has no attribute 'items'.
Add kubectl package handling
Just an idea. The SLS would be simple enough.
Add:
{%- for yamlurl in packages.kube.pkgs.add %}
cmd.run: kubectl apply -f {{ yamlurl }}
Remove:
{%- for yamlurl in packages.kube.pkgs.remove %}
cmd.run: kubectl delete -f {{ yamlurl }}
Kitchen test failure on Debian: "udevadm" not found
Kitchen test failed on Debian testing PR #13
----------
ID: packages-snapd.service-service
Function: service.running
Name: snapd.service
Result: True
Comment: Service snapd.service is already enabled, and is running
Started: 17:17:53.652801
Duration: 238.097 ms
Changes:
----------
snapd.service:
True
Name: snapd.socket - Function: service.running - Result: Clean Started: - 17:17:53.892751 Duration: 69.846 ms
----------
ID: packages-snapd-hello-world-wanted
Function: cmd.run
Name: snap install hello-world
Result: False
Comment: Command "snap install hello-world" run
Started: 17:17:53.965852
Duration: 7962.479 ms
Changes:
----------
pid:
20577
retcode:
1
stderr:
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
error: cannot perform the following tasks:
- Setup snap "core" (4206) security profiles (cannot setup udev for snap "core": cannot reload udev rules: exec: "udevadm": executable file not found in $PATH
udev output:
)
- Setup snap "core" (4206) security profiles (cannot reload udev rules: exec: "udevadm": executable file not found in $PATH
udev output:
)
[INFO ] Running state [snapd.service] at time 17:17:53.652802
[INFO ] Executing state service.running for [snapd.service]
[INFO ] Executing command ['systemctl', 'status', 'snapd.service', '-n', '0'] in directory '/home/kitchen'
[INFO ] Executing command ['systemctl', 'is-active', 'snapd.service'] in directory '/home/kitchen'
[INFO ] Executing command ['systemctl', 'is-enabled', 'snapd.service'] in directory '/home/kitchen'
[INFO ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'snapd.service'] in directory '/home/kitchen'
[INFO ] Executing command ['systemctl', 'is-active', 'snapd.service'] in directory '/home/kitchen'
[INFO ] Executing command ['systemctl', 'is-enabled', 'snapd.service'] in directory '/home/kitchen'
[INFO ] Executing command ['systemctl', 'is-enabled', 'snapd.service'] in directory '/home/kitchen'
[INFO ] {'snapd.service': True}
[INFO ] Completed state [snapd.service] at time 17:17:53.890898 duration_in_ms=238.097
[INFO ] Running state [snapd.socket] at time 17:17:53.892750
[INFO ] Executing state service.running for [snapd.socket]
[INFO ] Executing command ['systemctl', 'status', 'snapd.socket', '-n', '0'] in directory '/home/kitchen'
[INFO ] Executing command ['systemctl', 'is-active', 'snapd.socket'] in directory '/home/kitchen'
[INFO ] Executing command ['systemctl', 'is-enabled', 'snapd.socket'] in directory '/home/kitchen'
[INFO ] The service snapd.socket is already running
[INFO ] Completed state [snapd.socket] at time 17:17:53.962597 duration_in_ms=69.846
[INFO ] Running state [snap install hello-world] at time 17:17:53.965852
[INFO ] Executing state cmd.run for [snap install hello-world]
[ERROR ] {'pid': 20577, 'retcode': 1, 'stderr': '/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)\nerror: cannot perform the following tasks:\n- Setup snap "core" (4206) security profiles (cannot setup udev for snap "core": cannot reload udev rules: exec: "udevadm": executable file not found in $PATH\nudev output:\n)\n- Setup snap "core" (4206) security profiles (cannot reload udev rules: exec: "udevadm": executable file not found in $PATH\nudev output:\n)', 'stdout': '\r\x1b[7mEnsure prerequisites for "he\x1b[0mllo-world" are available \rE\x1b[7mnsure prerequisites for "hel\x1b[0mlo-world" are available \rDo\x1b[7mwnload snap "core" (4206) f\x1b[0mrom channel "stable" \rDow\x1b[7mnload snap "core" (4206) fr\x1b[0mom channel "stable" \rDown\x1b[7mload snap "core" (4206) fro\x1b[0mm channel "stable" \rDown\x1b[7mload snap "core" (4206) from\x1b[0m channel "stable" \rDownl\x1b[7moad snap "core" (4206) from \x1b[0mchannel "stable" \rDownlo\x1b[7mad snap "core" (4206) from \x1b[0mchannel "stable" \rDownloa\x1b[7md snap "core" (4206) from c\x1b[0mhannel "stable" \rDownload\x1b[7m snap "core" (4206) from ch\x1b[0mannel "stable" \x1b[?25l\r\x1b[7mDo\x1b[0mwnload snap "core" (4206) from channel "stable" 3% 26.0MB/s 3.19s\r\x1b[7mDownl\x1b[0moad snap "core" (4206) from channel "stable" 7% 29.8MB/s 2.67s\r\x1b[7mDownload \x1b[0msnap "core" (4206) from channel "stable" 11% 32.0MB/s 2.37s\r\x1b[7mDownload sna\x1b[0mp "core" (4206) from channel "stable" 15% 32.7MB/s 2.22s\r\x1b[7mDownload snap "\x1b[0mcore" (4206) from channel "stable" 20% 33.5MB/s 2.05s\r\x1b[7mDownload snap "cor\x1b[0me" (4206) from channel "stable" 24% 33.3MB/s 1.96s\r\x1b[7mDownload snap "core" (\x1b[0m4206) from channel "stable" 28% 33.6MB/s 1.84s\r\x1b[7mDownload snap "core" (420\x1b[0m6) from channel "stable" 32% 33.8MB/s 1.72s\r\x1b[7mDownload snap "core" (4206) f\x1b[0mrom channel "stable" 36% 34.0MB/s 1.60s\r\x1b[7mDownload snap "core" (4206) from\x1b[0m channel "stable" 40% 34.0MB/s 1.51s\r\x1b[7mDownload snap "core" (4206) from ch\x1b[0mannel "stable" 45% 34.5MB/s 1.37s\r\x1b[7mDownload snap "core" (4206) from channe\x1b[0ml "stable" 49% 34.6MB/s 1.26s\r\x1b[7mDownload snap "core" (4206) from channel "sta\x1b[0mble" 57% 37.2MB/s 982ms\r\x1b[7mDownload snap "core" (4206) from channel "stable" \x1b[0m 65% 39.4MB/s 752ms\r\x1b[7mDownload snap "core" (4206) from channel "stable" \x1b[0m 73% 41.4MB/s 550ms\r\x1b[7mDownload snap "core" (4206) from channel "stable" 81%\x1b[0m 43.0MB/s 369ms\r\x1b[7mDownload snap "core" (4206) from channel "stable" 90% 44.5M\x1b[0mB/s 202ms\r\x1b[7mDownload snap "core" (4206) from channel "stable" 98% 45.9MB/s 38\x1b[0mms\r\x1b[7mDownload snap "core" (4206) from channel "stable" 100% 44.4MB/s 0.0ns\x1b[0m\r\x1b[7mDownload snap "core" (4206) from channel "stable" 100% 42.1MB/s 0.0ns\x1b[0m\r\x1b[7mDownload snap "core" (4206) from channel "stable" 100% 40.1MB/s 0.0ns\x1b[0m\r\x1b[7mDownload snap "core" (4206) from channel "stable" 100% 38.3MB/s 0.0ns\x1b[0m\r\x1b[7mDownload snap "core" (4206) from channel "stable" 100% 36.7MB/s 0.0ns\x1b[0m\r\x1b[7mDownload snap "core" (4206) from channel "stable" 100% 35.1MB/s 0.0ns\x1b[0m\rFetch an\x1b[7md check assertions for snap \x1b[0m"core" (4206) \rFetch and\x1b[7m check assertions for snap "\x1b[0mcore" (4206) \rFetch and \x1b[7mcheck assertions for snap "\x1b[0mcore" (4206) \rFetch and c\x1b[7mheck assertions for snap "c\x1b[0more" (4206) \rFetch and ch\x1b[7meck assertions for snap "co\x1b[0mre" (4206) \rFetch and ch\x1b[7meck assertions for snap "cor\x1b[0me" (4206) \rFetch and che\x1b[7mck assertions for snap "core\x1b[0m" (4206) \rFetch and chec\x1b[7mk assertions for snap "core\x1b[0m" (4206) \rFetch and check\x1b[7m assertions for snap "core"\x1b[0m (4206) \rFetch and check \x1b[7massertions for snap "core" \x1b[0m(4206) \rFetch and check \x1b[7massertions for snap "core" (\x1b[0m4206) \rMount snap "core"\x1b[7m (4206) \x1b[0m \rMount snap "core" \x1b[7m(4206) \x1b[0m \rMount snap "core" (\x1b[7m4206) \x1b[0m \rSetup snap "core" (4\x1b[7m206) security profiles \x1b[0m \r\x1b[0m\x1b[?12;25h\x1b[K'}
[INFO ] Completed state [snap install hello-world] at time 17:18:01.928331 duration_in_ms=7962.479
[INFO ] Running state [snap remove goodbye-world] at time 17:18:01.929115
[INFO ] Executing state cmd.run for [snap remove goodbye-world]
[INFO ] onlyif execution failed
[INFO ] Completed state [snap remove goodbye-world] at time 17:18:01.956409 duration_in_ms=27.293
How to handle extends better
Today, we use following pattern in pip / gems SLS.
{% set req_states = packages.pips.required.states %}
{% set req_pkgs = packages.pips.required.pkgs %}
{% set wanted_pips = packages.pips.wanted %}
{% set unwanted_pips = packages.pips.unwanted %}
### REQ PKGS (without these, some of the WANTED PIPS will fail to install)
pip_req_pkgs:
pkg.installed:
- pkgs: {{ req_pkgs }}
The snap SLS follows the pip/gems pattern but introduces extends/include.
{% set req_states = packages.snaps.required.states %}
{% set wanted_snaps = packages.snaps.wanted %}
{% set unwanted_snaps = packages.snaps.unwanted %}
{% if packages.snaps.wanted or packages.snaps.unwanted %}
### REQ PKGS (without this, SNAPS can fail to install/uninstall)
include:
- packages.pkgs
extend:
... etc ...
I think this is fine for now. but overall not optimal. Other formulae seem to place extends/includes inside init.SLS. Just flagging this as maybe an improvement is needed
zypper doesn't understand param ```hold``` and ```update_holds```
excerpt from : https://docs.saltstack.com/en/latest/ref/states/all/salt.states.pkg.html
hold: Currently works with YUM/DNF & APT based systems. New in version 2014.7.0.
update_holds: Currently works with YUM/DNF & APT based systems. New in version 2016.11.0.
[BUG] Salt minion is not caching all the formula's files on windows machines
Your setup
Formula commit hash / release tag
Release tag: 0.14.0
Versions reports (master & minion)
Salt master:
Salt Version:
Salt: 3003.3
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: 2.0.6
gitpython: 3.0.7
Jinja2: 2.10.1
libgit2: 0.28.3
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 0.6.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.6.1
pygit2: 1.0.3
Python: 3.8.10 (default, Jun 2 2021, 10:49:15)
python-gnupg: 0.4.5
PyYAML: 5.3.1
PyZMQ: 18.1.1
smmap: 2.0.5
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.2
System Versions:
dist: ubuntu 20.04 focal
locale: utf-8
machine: x86_64
release: 5.8.0-59-generic
system: Linux
version: Ubuntu 20.04 focal
Salt minion:
Salt Version:
Salt: 3003.3
Dependency Versions:
cffi: 1.14.5
cherrypy: 18.6.0
dateutil: 2.8.1
docker-py: Not Installed
gitdb: 4.0.5
gitpython: 3.1.13
Jinja2: 2.11.3
libgit2: Not Installed
M2Crypto: Not Installed
Mako: 1.1.4
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.20
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: Not Installed
Python: 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
python-gnupg: 0.4.6
PyYAML: 5.4.1
PyZMQ: 18.0.1
smmap: 3.0.4
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.1
System Versions:
dist:
locale: cp1252
machine: AMD64
release: 10
system: Windows
version: 10 10.0.19041 SP0
Pillar / config used
packages:
chocolatey:
wanted:
prometheus-windows-exporter.install:
version: '0.16.0'
Bug details
Describe the bug
When I run the command salt skyros.pc.itc.it state.test packages.chocolatey, I get the error:
skyros.pc.itc.it:
Data failed to compile:
----------
Rendering SLS 'base:packages.chocolatey' failed: Jinja error: ./map.jinja
Traceback (most recent call last):
File "c:\salt\bin\lib\site-packages\salt-3003.3-py3.7.egg\salt\utils\templates.py", line 500, in render_
jinja_tmpl
output = template.render(**decoded_context)
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1090, in render
self.environment.handle_exception()
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "<template>", line 3, in top-level template code
File "c:\salt\bin\lib\site-packages\salt-3003.3-py3.7.egg\salt\utils\jinja.py", line 198, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: ./map.jinja
; line 3
---
# -*- coding: utf-8 -*-
# vim: ft=sls
{%- from "./map.jinja" import packages with context %} <======================
{%- if grains['os'] == 'Windows' %}
{%- if packages.chocolatey %}
{%- set req_states = packages.chocolatey.required.states %}
[...]
---
ERROR: Minions returned with non-zero exit code
and in the cache folder on the target (C:\salt\var\cache\salt\minion\files\base\packages) the only file is chocolately.sls without any map.jinja
We tried with different version of salt-minion, with different windows version (win10, win server 2016/2019) and with two different salt-master (same version of salt-master).
When I run salt skyros.pc.itc.it state.test packages in the cache folder on the target machine I have more files (for example the map.jinja) but not all files (the default.yaml is missing). In fact I get multiple errors regarding the default.yaml file, like this:
----------
Rendering SLS 'base:packages.golang.goget' failed: Jinja error: ./defaults.yaml
c:\salt\var\cache\salt\minion\files\base\packages/map.jinja(4):
---
# -*- coding: utf-8 -*-
# vim: ft=jinja
{%- import_yaml './defaults.yaml' as defaults %} <======================
{%- import_yaml './osfamilymap.yaml' as osfamilymap %}
{%- import_yaml './osmap.yaml' as osmap %}
{%- import_yaml './osfingermap.yaml' as osfingermap %}
{%- set packages = salt['grains.filter_by'](
[...]
---
Traceback (most recent call last):
File "c:\salt\bin\lib\site-packages\salt-3003.3-py3.7.egg\salt\utils\templates.py", line 500, in render_
jinja_tmpl
output = template.render(**decoded_context)
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1090, in render
self.environment.handle_exception()
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "<template>", line 6, in top-level template code
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1155, in make_module
return TemplateModule(self, self.new_context(vars, shared, locals))
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1238, in __init__
body_stream = list(template.root_render_func(context))
File "c:\salt\var\cache\salt\minion\files\base\packages/map.jinja", line 4, in top-level template code
{%- import_yaml './defaults.yaml' as defaults %}
File "c:\salt\bin\lib\site-packages\salt-3003.3-py3.7.egg\salt\utils\jinja.py", line 198, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: ./defaults.yaml
Expected behaviour
It should install the package prometheus-windows-exporter using chocolatey.
Attempts to fix the bug
I submitted a PR that seems to fix the problem: #77
Add example of version specific pkg.install in the pillar
As far as I understand it, the pkg:wanted pillar key is fed directly to pkg.installed: pkgs so one can use that to have version specific specs :
wanted:
- git
- less
- bc
- curl
- fail2ban
replaced by
wanted:
- git
- less
- bc
- curl
- fail2ban: 0.10.2-2.1
Travis CI - snap state fails
Travis CI has following failed state for Ubuntu:
**error: too early for operation, device not yet seeded or device model not acknowledged**
Logs.
----------
ID: packages-snapd-hello-world-wanted
Function: cmd.run
Name: snap install hello-world
Result: False
Comment: Command "snap install hello-world" run
Started: 14:24:07.177879
Duration: 43.539 ms
Changes:
----------
pid:
7842
retcode:
10
stderr:
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
error: too early for operation, device not yet seeded or device model not acknowledged
stdout:
[FEATURE] Use `saltstack/salt` proposed `flatpak` modules to provide support in this formula
Similar in nature to #87, use the flatpak modules from the following Salt PR:
- saltstack/salt#56907 (port to
masterfor saltstack/salt#51993).
[FIX NEEDED] Deprecated functionality `module.run` used in archives.sls
Your setup
Formula commit hash / release tag
Versions reports (master & minion)
Pillar / config used
Bug details
Describe the bug
On more recent versions of Salt, this warning is spit out when running the archives.sls state
[WARNING ] The function "module.run" is using its deprecated version and will expire in version "Phosphorus".
Steps to reproduce the bug
Expected behaviour
No warnings issued.
Attempts to fix the bug
Additional context
pip.installed reload_modules true
Might be useful to include reload_modules: True in pip.installed function so modules get refreshed.
Add npm package support
Adding npm package install/remove would be great addition - it's same idea as pip.
Setup:
sudo apt install nodejs npm (or yum or zypper)
There are a few different command forms per documentation but I think the following forms (only) should be supported because they are most common.
Install:
npm install <package-name>
uninstall:
npm uninstall <package-name>
Bug with snap.sls breaks pkgs.purged
PR #13 introduced a bug. The bug is related to #15 (how best to use extends).
Basically**unwanted packages** are not removed for some reason.
My workaround was to remove extend.unwanted_pkgs stanza from snaps.sls
extend:
unwanted_pkgs:
pkg.removed:
- pkgs: {{ packages.snaps.collides }}
{% if req_states %}
- require:
{% for dep in req_states %}
- sls: {{ dep }}
{% endfor %}
{% endif %}
Pillars tested
packages:
pips:
wanted:
- docker
unwanted:
- docker-py
pkgs:
unwanted:
- unattended-upgrades
- avahi-daemon
- lxc-docker*
- docker.io*
- unattended-upgrades
held:
- xserver-xorg-core: '2:1.18*'
wanted:
- iptables
- ca-certificates
- apt-transport-https
- python-apt
snaps:
wanted:
- hello-world
unwanted:
- goodbye-world
Once the snaps.extends.unwanted_pkgs is removed formula works.
[INFO ] Executing state pkg.purged for [unwanted_pkgs]
...
[INFO ] Executing command ['systemd-run', '--scope', 'apt-get', '-q', '-y', 'purge', 'unattended-upgrades'] in directory '/home/messi'
[TRACE ] stdout: Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be REMOVED:
unattended-upgrades*
RFE: Add support for Archive file format
Computer files, used for purpose of software bundle distribution, are basically packages. Could we add generic support for Archive file format, and specifically tarball format installation?
Handling archive file format is very consistent across formats. Could I suggest following-
New States
packages.archive.wanted
- prepare tmp-dir (
file.directory) - download archive to tmp-dir (
cmd.run) - checksum archive in tmp-dir (
module-run.checksum) - extract archive to DEST (
archive.extractedorcmd.run) - cleanup tmp-dir/archive (
file.absent)
packages.archive.unwanted
- cleanup DEST (
file.absent)
New Pillars
packages:
archive:
wanted:
bob:
source: http://repo.org/release/bob_latest.tar.gz
dest: /opt/bob/
format: tarball
unwanted:
- /opt/peter/
New format support
- Tarball (tar.gz)
pkg.installed failure on MacOS
Noting an error is being reported on MacOS with simple pillar data.
Pillars
packages:
pkgs:
wanted:
- wget
- curl
Failed state
ID: pkg_req_pkgs
Function: pkg.installed
Result: False
Comment: Brew command failed. Additional info follows:
result:
----------
pid:
30227
retcode:
1
stderr:
Error: No available formula with the name "tar"
Warning: homebrew/core is shallow clone. To get complete history run:
git -C "$(brew --repo homebrew/core)" fetch --unshallow
Error: No previously deleted formula found.
==> Searching taps on GitHub...
Error: No formulae found in taps.
stdout:
==> Searching for a previously deleted formula (in the last month)...
==> Searching for similarly named formulae...
These similarly named formulae were found:
atari800
fstar
git-archive-all
gnu-tar
hatari
internetarchive
juju-quickstart
kstart
libtar
qtfaststart
rakudo-star
sagittarius-scheme
star
startup-notification
tarantool
tarsnap
tarsnap-gui
tarsnapper
xmlstarlet
To install one of them, run (for example):
brew install atari800
==> Searching taps...
Started: 14:57:33.937366
Duration: 4854.738 ms
Changes:
ID: wanted_pkgs
Function: pkg.installed
Result: False
Comment: One or more requisite failed: packages.pkgs.pkg_req_pkgs
DEBUG
Error: No previously deleted formula found.
==> Searching taps on GitHub...
Error: No formulae found in taps.
[ERROR ] retcode: 1
[ERROR ] Brew command failed. Additional info follows:
result:
----------
pid:
30227
retcode:
1
stderr:
Error: No available formula with the name "tar"
Warning: homebrew/core is shallow clone. To get complete history run:
git -C "$(brew --repo homebrew/core)" fetch --unshallow
Error: No previously deleted formula found.
==> Searching taps on GitHub...
Error: No formulae found in taps.
stdout:
==> Searching for a previously deleted formula (in the last month)...
==> Searching for similarly named formulae...
These similarly named formulae were found:
atari800
fstar
git-archive-all
gnu-tar
hatari
internetarchive
juju-quickstart
kstart
libtar
qtfaststart
rakudo-star
sagittarius-scheme
star
startup-notification
tarantool
tarsnap
tarsnap-gui
tarsnapper
xmlstarlet
To install one of them, run (for example):
brew install atari800
==> Searching taps...
[INFO ] Completed state [pkg_req_pkgs] at time 14:57:38.792104 (duration_in_ms=4854.738)
This error is not observed during a standard execution of brew install.
$ brew install wget
Updating Homebrew...
Warning: wget 1.19.5 is already installed and up-to-date
To reinstall 1.19.5, run `brew reinstall wget`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.