A python package containing functions that help interacting with various versions of Ansible
License: MIT License
A python package contains functions that facilitate working with various versions of Ansible 2.14 and newer.
Documentation is available at ansible-compat.readthedocs.io.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![pre-commit-ci[bot] avatar](https://avatars.githubusercontent.com/in/68672?v=4 "pre-commit-ci[bot]")
See #266 (comment).
To test my role locally, "molecule test" with the vagrant driver fails by showing me:
Computed fully qualified role name of role.test does not follow current galaxy requirements.
Please edit meta/main.yml and assure we can correctly determine full role name:
galaxy_info:
role_name: my_name # if absent directory name hosting role is used instead
namespace: my_galaxy_namespace # if absent, author is used instead
it keeps failing even if I add "role_name" and "namespace" on meta/main.yml.
I had to delete the three lines: 394, 395 and 396 on the file: "...../lib/python3.6/site-packages/ansible_compat/runtime.py" to prevent the error from occurring and so that I could run the test of my role.
Newer versions of pytest-plus are complaining because the naming of the tests is not safe:
[ 32s] ============================= test session starts ==============================
[ 32s] platform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.3.0 -- /usr/bin/python3.11
[ 32s] cachedir: .pytest_cache
[ 32s] rootdir: /home/abuild/rpmbuild/BUILD/ansible-compat-4.1.10
[ 32s] configfile: pyproject.toml
[ 32s] testpaths: test
[ 32s] plugins: mock-3.12.0, plus-0.0.0
[ 32s] collecting ... ERROR: Failed run due to following issues being identified:
[ 32s] Test <Function test__update_env_no_old_value_no_default[value1-a:b]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s] Test <Function test__update_env_no_old_value_no_default[value2-a:b:c]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s] Test <Function test__update_env_no_old_value[a:b-value0-c:a:b]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s] Test <Function test__update_env_no_old_value[a:b-value1-c:d:a:b]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s] Test <Function test__update_env_no_default[a:b-value0-c:a:b]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s] Test <Function test__update_env_no_default[a:b-value1-c:d:a:b]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s] Test <Function test__update_env[a--value1-e:a]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s] Test <Function test__update_env[a-c-value3-e:f:a]> has an id that does not match our safe pattern '^[\w_\-\.]+$' for use with a terminal.
[ 32s]
[ 32s] collected 93 items
I have not found a fix in the list of commits, did I just miss it or is there none yet?
Kind Regards,
Johannes
Move most/all code from ansiblelint.prerun into ansiblecompat so we can replace molecule dependency on ansiblelint with a dependency on this module, same would apply to ansiblelint.
It is not guaranteed that ~/.ansible/collections is the correct place to store collections and load_collections will fail if that directory cannot be created.
load_collections should probably use $ANSIBLE_HOME/collections as the directory to store collections in.
Hi,
As @john-bailey report yesterday here : #257
This update broke molecule.
We have to downgrade back to 3.0.2.
I have today the same issue of @john-bailey and the same downgrade work.
I take this issue here because perhaps commentary is not the best place to report an issue.
Thanks for you're work
FYI: This update broke Molecule, and I had to downgrade back to 3.0.2 Here's the traceback
Traceback (most recent call last): File "/apps/ansible_test/.venv/bin/molecule", line 8, in <module> sys.exit(main()) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/molecule/command/init/role.py", line 174, in role r.execute() File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/molecule/command/init/role.py", line 84, in execute result = util.run_command(cmd) File "/apps/ansible_test/.venv/lib64/python3.9/site-packages/molecule/util.py", line 149, in run_command result = app.runtime.exec( AttributeError: 'Runtime' object has no attribute 'exec'
One of our ansilbe-lint jobs has started failing with 4.0.1. Last good run with 3.0.2 is [1], failure is [2]. Full traceback is
Traceback (most recent call last):
File "/home/iwienand/programs/openstack-infra/openstack-zuul-jobs/.tox/linters/bin/ansible-lint", line 8, in <module>
sys.exit(_run_cli_entrypoint())
^^^^^^^^^^^^^^^^^^^^^
File "/home/iwienand/programs/openstack-infra/openstack-zuul-jobs/.tox/linters/lib/python3.11/site-packages/ansiblelint/__main__.py", line 266, in _run_cli_entrypoint
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "/home/iwienand/programs/openstack-infra/openstack-zuul-jobs/.tox/linters/lib/python3.11/site-packages/ansiblelint/__main__.py", line 157, in main
initialize_options(argv[1:])
File "/home/iwienand/programs/openstack-infra/openstack-zuul-jobs/.tox/linters/lib/python3.11/site-packages/ansiblelint/__main__.py", line 105, in initialize_options
options.cache_dir = get_cache_dir(options.project_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/iwienand/programs/openstack-infra/openstack-zuul-jobs/.tox/linters/lib/python3.11/site-packages/ansible_compat/prerun.py", line 13, in get_cache_dir
basename = project_dir.resolve().name.encode(encoding="utf-8")
^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'resolve'
Note versions of ansible-lint (6.4.0) and ansible (2.12) didn't change between the two runs, just ansible_compat.
[1] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6e7/881890/1/check/openstack-zuul-jobs-linters/6e77077/job-output.txt
[2] https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a9f/881890/1/gate/openstack-zuul-jobs-linters/a9f991b/job-output.txt
Hi! We're currently doing the Python 3.11 rebuilds on Arch Linux.
Unfortunately it seems that ansible-compat now has a few more failing tests that I can't figure out:
test_prepare_environment_with_collections
test_require_collection_wrong_version
test_require_collection
test_install_collection
test_install_collection_dest
test_upgrade_collection
test_require_collection_no_cache_dir
test_install_collection_from_disk[normal]
test_install_collection_from_disk[deep]
test_runtime_example
=================================== FAILURES ===================================
__________________ test_prepare_environment_with_collections ___________________
tmp_path = PosixPath('/tmp/pytest-of-builduser/pytest-0/test_prepare_environment_with_0')
def test_prepare_environment_with_collections(tmp_path: pathlib.Path) -> None:
"""Check that collections are correctly installed."""
runtime = Runtime(isolated=True, project_dir=str(tmp_path))
> runtime.prepare_environment(required_collections={"community.molecule": "0.1.0"})
test/test_runtime.py:164:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_dir/usr/lib/python3.11/site-packages/ansible_compat/runtime.py:385: in prepare_environment
self.install_collection(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <ansible_compat.runtime.Runtime object at 0x7f29d2001590>
collection = 'community.molecule:>=0.1.0'
destination = '/build/.cache/ansible-compat/5e37ff/collections', force = False
def install_collection(
self,
collection: str,
destination: Optional[Union[str, pathlib.Path]] = None,
force: bool = False,
) -> None:
"""Install an Ansible collection.
Can accept version constraints like 'foo.bar:>=1.2.3'
"""
cmd = [
"ansible-galaxy",
"collection",
"install",
"-vvv", # this is needed to make ansible display important info in case of failures
]
if force:
cmd.append("--force")
# As ansible-galaxy install is not able to automatically determine
# if the range requires a pre-release, we need to manuall add the --pre
# flag when needed.
matches = version_re.search(collection)
if matches and Version(matches[1]).is_prerelease:
cmd.append("--pre")
if destination:
cmd.extend(["-p", str(destination)])
cmd.append(f"{collection}")
_logger.info("Running from %s : %s", os.getcwd(), " ".join(cmd))
run = self.exec(
cmd,
retry=True,
)
if run.returncode != 0:
msg = f"Command returned {run.returncode} code:\n{run.stdout}\n{run.stderr}"
_logger.error(msg)
> raise InvalidPrerequisiteError(msg)
E ansible_compat.errors.InvalidPrerequisiteError: Command returned 250 code:
E ansible-galaxy [core 2.14.4]
E config file = /etc/ansible/ansible.cfg
E configured module search path = ['/build/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
E ansible python module location = /usr/lib/python3.11/site-packages/ansible
E ansible collection location = /build/.ansible/collections:/usr/share/ansible/collections
E executable location = /usr/bin/ansible-galaxy
E python version = 3.11.3 (main, Apr 5 2023, 15:52:25) [GCC 12.2.1 20230201] (/usr/bin/python)
E jinja version = 3.1.2
E libyaml = True
E Using /etc/ansible/ansible.cfg as config file
E Starting galaxy collection install process
E Process install dependency map
E the full traceback was:
E
E Traceback (most recent call last):
E File "/usr/lib/python3.11/site-packages/ansible/cli/__init__.py", line 647, in cli_executor
E exit_code = cli.run()
E ^^^^^^^^^
E File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 714, in run
E return context.CLIARGS['func']()
E ^^^^^^^^^^^^^^^^^^^^^^^^^
E File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 115, in method_wrapper
E return wrapped_method(*args, **kwargs)
E ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 1363, in execute_install
E self._execute_install_collection(
E File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 1400, in _execute_install_collection
E install_collections(
E File "/usr/lib/python3.11/site-packages/ansible/galaxy/collection/__init__.py", line 730, in install_collections
E dependency_map = _resolve_depenency_map(
E ^^^^^^^^^^^^^^^^^^^^^^^
E File "/usr/lib/python3.11/site-packages/ansible/galaxy/collection/__init__.py", line 1766, in _resolve_depenency_map
E collection_dep_resolver = build_collection_dependency_resolver(
E ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E File "/usr/lib/python3.11/site-packages/ansible/galaxy/dependency_resolution/__init__.py", line 43, in build_collection_dependency_resolver
E return CollectionDependencyResolver(
E ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E TypeError: CollectionDependencyResolver() takes no arguments
E
E [WARNING]: The specified collections path '/build/.cache/ansible-
E compat/5e37ff/collections' is not part of the configured Ansible collections
E paths '/build/.ansible/collections:/usr/share/ansible/collections'. The
E installed collection will not be picked up in an Ansible run, unless within a
E playbook-adjacent collections directory.
E ERROR! Unexpected Exception, this is probably a bug: CollectionDependencyResolver() takes no arguments
test_dir/usr/lib/python3.11/site-packages/ansible_compat/runtime.py:265: InvalidPrerequisiteError
------------------------------ Captured log call -------------------------------
WARNING ansible_compat.runtime:runtime.py:188 Retrying execution failure 250 of: ansible-galaxy collection install -vvv -p /build/.cache/ansible-compat/5e37ff/collections community.molecule:>=0.1.0
ERROR ansible_compat.runtime:runtime.py:264 Command returned 250 code:
ansible-galaxy [core 2.14.4]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/build/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.11/site-packages/ansible
ansible collection location = /build/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible-galaxy
python version = 3.11.3 (main, Apr 5 2023, 15:52:25) [GCC 12.2.1 20230201] (/usr/bin/python)
jinja version = 3.1.2
libyaml = True
Using /etc/ansible/ansible.cfg as config file
Starting galaxy collection install process
Process install dependency map
the full traceback was:
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/ansible/cli/__init__.py", line 647, in cli_executor
exit_code = cli.run()
^^^^^^^^^
File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 714, in run
return context.CLIARGS['func']()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 115, in method_wrapper
return wrapped_method(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 1363, in execute_install
self._execute_install_collection(
File "/usr/lib/python3.11/site-packages/ansible/cli/galaxy.py", line 1400, in _execute_install_collection
install_collections(
File "/usr/lib/python3.11/site-packages/ansible/galaxy/collection/__init__.py", line 730, in install_collections
dependency_map = _resolve_depenency_map(
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/ansible/galaxy/collection/__init__.py", line 1766, in _resolve_depenency_map
collection_dep_resolver = build_collection_dependency_resolver(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/ansible/galaxy/dependency_resolution/__init__.py", line 43, in build_collection_dependency_resolver
return CollectionDependencyResolver(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: CollectionDependencyResolver() takes no arguments
[WARNING]: The specified collections path '/build/.cache/ansible-
compat/5e37ff/collections' is not part of the configured Ansible collections
paths '/build/.ansible/collections:/usr/share/ansible/collections'. The
installed collection will not be picked up in an Ansible run, unless within a
playbook-adjacent collections directory.
ERROR! Unexpected Exception, this is probably a bug: CollectionDependencyResolver() takes no arguments
[... other errors skipped for brevity, more in logs below! ]
=============================== warnings summary ===============================
../../../../usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:432
/usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:432: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/nodeids
config.cache.set("cache/nodeids", sorted(self.cached_nodeids))
../../../../usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:386
/usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:386: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/lastfailed
config.cache.set("cache/lastfailed", self.lastfailed)
../../../../usr/lib/python3.11/site-packages/_pytest/stepwise.py:56
/usr/lib/python3.11/site-packages/_pytest/stepwise.py:56: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/stepwise
session.config.cache.set(STEPWISE_CACHE_DIR, [])
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED ../../../../dev/test/test_runtime.py::test_prepare_environment_with_collections
FAILED ../../../../dev/test/test_runtime.py::test_require_collection_wrong_version
FAILED ../../../../dev/test/test_runtime.py::test_require_collection - ansibl...
FAILED ../../../../dev/test/test_runtime.py::test_install_collection - ansibl...
FAILED ../../../../dev/test/test_runtime.py::test_install_collection_dest - a...
FAILED ../../../../dev/test/test_runtime.py::test_upgrade_collection - ansibl...
FAILED ../../../../dev/test/test_runtime.py::test_require_collection_no_cache_dir
FAILED ../../../../dev/test/test_runtime.py::test_install_collection_from_disk[normal]
FAILED ../../../../dev/test/test_runtime.py::test_install_collection_from_disk[deep]
FAILED ../../../../dev/test/test_runtime_example.py::test_runtime_example - a...
=========== 10 failed, 68 passed, 2 deselected, 3 warnings in 29.03s ===========
python-ansible-compat-3.0.1-2-x86_64-build.log
python-ansible-compat-3.0.1-2-x86_64-check.log
Maybe you have any pointers as to what is happening here?
Hi! When trying to package 2.2.7 for Arch Linux I ran into the two failing tests test_prerun_reqs_v1 and test_prerun_reqs_v2.
=================================== FAILURES ===================================
_____________________________ test_prerun_reqs_v1 ______________________________
caplog = <_pytest.logging.LogCaptureFixture object at 0x7f9feb9b8b50>
runtime = <ansible_compat.runtime.Runtime object at 0x7f9feb9ba950>
def test_prerun_reqs_v1(caplog: pytest.LogCaptureFixture, runtime: Runtime) -> None:
"""Checks that the linter can auto-install requirements v1 when found."""
cwd = os.path.realpath(
os.path.join(
os.path.dirname(os.path.realpath(__file__)), "..", "examples", "reqs_v1"
)
)
with remember_cwd(cwd):
with caplog.at_level(logging.INFO):
> runtime.prepare_environment()
test/test_runtime.py:224:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_dir/usr/lib/python3.10/site-packages/ansible_compat/runtime.py:356: in prepare_environment
self.install_requirements(req_file, retry=retry, offline=offline)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <ansible_compat.runtime.Runtime object at 0x7f9feb9ba950>
requirement = 'requirements.yml', retry = False, offline = False
def install_requirements(
self, requirement: str, retry: bool = False, offline: bool = False
) -> None:
"""Install dependencies from a requirements.yml.
param: offline: bypass installation, may fail if requirements are not met
"""
if not os.path.exists(requirement):
return
reqs_yaml = yaml_from_file(requirement)
if not isinstance(reqs_yaml, (dict, list)):
raise InvalidPrerequisiteError(
f"{requirement} file is not a valid Ansible requirements file."
)
if isinstance(reqs_yaml, list) or "roles" in reqs_yaml:
cmd = [
"ansible-galaxy",
"role",
"install",
"-vr",
f"{requirement}",
]
if self.cache_dir:
cmd.extend(["--roles-path", f"{self.cache_dir}/roles"])
if offline:
_logger.warning(
"Skipped installing old role dependencies due to running in offline mode."
)
else:
_logger.info("Running %s", " ".join(cmd))
result = self.exec(cmd, retry=retry)
if result.returncode != 0:
_logger.error(result.stdout)
> raise AnsibleCommandError(result)
E ansible_compat.errors.AnsibleCommandError: Got 1 exit code while running: ansible-galaxy role install -vr requirements.yml --roles-path /build/.cache/ansible-compat/04df9e/roles
test_dir/usr/lib/python3.10/site-packages/ansible_compat/runtime.py:309: AnsibleCommandError
------------------------------ Captured log call -------------------------------
INFO ansible_compat.runtime:runtime.py:304 Running ansible-galaxy role install -vr requirements.yml --roles-path /build/.cache/ansible-compat/04df9e/roles
WARNING ansible_compat.runtime:runtime.py:162 Retrying execution failure 1 of: ansible-galaxy role install -vr requirements.yml --roles-path /build/.cache/ansible-compat/04df9e/roles
ERROR ansible_compat.runtime:runtime.py:308 Using /etc/ansible/ansible.cfg as config file
Starting galaxy role install process
_____________________________ test_prerun_reqs_v2 ______________________________
caplog = <_pytest.logging.LogCaptureFixture object at 0x7f9feb9bab90>
runtime = <ansible_compat.runtime.Runtime object at 0x7f9feb9bbd90>
def test_prerun_reqs_v2(caplog: pytest.LogCaptureFixture, runtime: Runtime) -> None:
"""Checks that the linter can auto-install requirements v2 when found."""
cwd = os.path.realpath(
os.path.join(
os.path.dirname(os.path.realpath(__file__)), "..", "examples", "reqs_v2"
)
)
with remember_cwd(cwd):
with caplog.at_level(logging.INFO):
> runtime.prepare_environment()
test/test_runtime.py:243:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_dir/usr/lib/python3.10/site-packages/ansible_compat/runtime.py:356: in prepare_environment
self.install_requirements(req_file, retry=retry, offline=offline)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <ansible_compat.runtime.Runtime object at 0x7f9feb9bbd90>
requirement = 'requirements.yml', retry = False, offline = False
def install_requirements(
self, requirement: str, retry: bool = False, offline: bool = False
) -> None:
"""Install dependencies from a requirements.yml.
param: offline: bypass installation, may fail if requirements are not met
"""
if not os.path.exists(requirement):
return
reqs_yaml = yaml_from_file(requirement)
if not isinstance(reqs_yaml, (dict, list)):
raise InvalidPrerequisiteError(
f"{requirement} file is not a valid Ansible requirements file."
)
if isinstance(reqs_yaml, list) or "roles" in reqs_yaml:
cmd = [
"ansible-galaxy",
"role",
"install",
"-vr",
f"{requirement}",
]
if self.cache_dir:
cmd.extend(["--roles-path", f"{self.cache_dir}/roles"])
if offline:
_logger.warning(
"Skipped installing old role dependencies due to running in offline mode."
)
else:
_logger.info("Running %s", " ".join(cmd))
result = self.exec(cmd, retry=retry)
if result.returncode != 0:
_logger.error(result.stdout)
> raise AnsibleCommandError(result)
E ansible_compat.errors.AnsibleCommandError: Got 1 exit code while running: ansible-galaxy role install -vr requirements.yml --roles-path /build/.cache/ansible-compat/04df9e/roles
test_dir/usr/lib/python3.10/site-packages/ansible_compat/runtime.py:309: AnsibleCommandError
------------------------------ Captured log call -------------------------------
INFO ansible_compat.runtime:runtime.py:304 Running ansible-galaxy role install -vr requirements.yml --roles-path /build/.cache/ansible-compat/04df9e/roles
WARNING ansible_compat.runtime:runtime.py:162 Retrying execution failure 1 of: ansible-galaxy role install -vr requirements.yml --roles-path /build/.cache/ansible-compat/04df9e/roles
ERROR ansible_compat.runtime:runtime.py:308 Using /etc/ansible/ansible.cfg as config file
Starting galaxy role install process
=============================== warnings summary ===============================
../../../../usr/lib/python3.10/site-packages/_pytest/cacheprovider.py:433
/usr/lib/python3.10/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/nodeids
config.cache.set("cache/nodeids", sorted(self.cached_nodeids))
../../../../usr/lib/python3.10/site-packages/_pytest/cacheprovider.py:387
/usr/lib/python3.10/site-packages/_pytest/cacheprovider.py:387: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/lastfailed
config.cache.set("cache/lastfailed", self.lastfailed)
../../../../usr/lib/python3.10/site-packages/_pytest/stepwise.py:52
/usr/lib/python3.10/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/stepwise
session.config.cache.set(STEPWISE_CACHE_DIR, [])
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED ../../../../dev/test/test_runtime.py::test_prerun_reqs_v1 - ansible_co...
FAILED ../../../../dev/test/test_runtime.py::test_prerun_reqs_v2 - ansible_co...
================== 2 failed, 76 passed, 3 warnings in 41.96s ===================
python-ansible-compat-2.2.7-1-x86_64-build.log
python-ansible-compat-2.2.7-1-x86_64-check.log
Build and test steps can be found here: https://github.com/archlinux/svntogit-community/blob/packages/python-ansible-compat/trunk/PKGBUILD
The link in the readme gives a 404 not found error, I dont know where it should point so I am opening this issue instead of a PR.
Line 12 in f51159b
It seems that ansible-compat==1.0.0 doesn't allow fqrn names of the role to test.
Steps to reproduce:
virtualenv venv
cd venv
. bin/activate
pip install molecule
# Successfully installed Jinja2-3.0.3 MarkupSafe-2.0.1 PyYAML-5.4.1 ansible-compat-1.0.0 arrow-1.2.1 bcrypt-3.2.0 binaryornot-0.4.4 cerberus-1.3.2 certifi-2021.10.8 cffi-1.15.0 chardet-4.0.0 charset-normalizer-2.0.10 click-8.0.3 click-help-colors-0.9.1 colorama-0.4.4 commonmark-0.9.1 cookiecutter-1.7.3 cryptography-36.0.1 distro-1.6.0 enrich-1.2.7 idna-3.3 jinja2-time-0.2.0 molecule-3.5.2 packaging-21.3 paramiko-2.9.2 pluggy-1.0.0 poyo-0.5.0 pycparser-2.21 pygments-2.11.2 pynacl-1.5.0 pyparsing-3.0.6 python-dateutil-2.8.2 python-slugify-5.0.2 requests-2.27.1 rich-11.0.0 selinux-0.2.1 six-1.16.0 subprocess-tee-0.3.5 text-unidecode-1.3 urllib3-1.26.8
pip install molecule-vagrant
# Successfully installed molecule-vagrant-1.0.0 python-vagrant-0.5.15
molecule init role -d vagrant foo
cat << EOT > meta/main.yml
---
galaxy_info:
author: author
namespace: my_namespace
role_name: foo
EOT
sed -i 's/foo/my_namespace.foo/' molecule/default/converge.yml
molecule test
# TASK [Include my_namespace.foo] ************************************************
# ERROR! the role 'my_namespace.foo' was not found in /tmp/venv/foo/molecule/default/roles:/home/user/.cache/molecule/foo/default/roles:/tmp/venv:/home/user/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/tmp/venv/foo:/tmp/venv/foo/molecule/default
#
# The error appears to be in '/tmp/venv/foo/molecule/default/converge.yml': line 7, column 15, but may be elsewhere in the file depending on the exact syntax problem.
#
# The offending line appears to be:
#
# include_role:
# name: "my_namespace.foo"
# ^ here
# ...
# CRITICAL Ansible return code was 2, command was: ['ansible-playbook', '--inventory', '/home/user/.cache/molecule/foo/default/inventory', '--skip-tags', 'molecule-notest,notest', '/tmp/venv/foo/molecule/default/converge.yml']
Now the same test with ansible-compat==0.5.0:
pip install ansible-compat==0.5.0
# Successfully uninstalled ansible-compat-1.0.0
# Successfully installed ansible-compat-0.5.0
molecule test
# ...
# PLAY RECAP *********************************************************************
# instance : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
The following symlink will be created with ansible-compat==0.5.0, but not with version 1.0.0:
ls -l ~/.ansible/roles/my_namespace.foo
lrwxrwxrwx. 1 user user 13 14. Jan 13:36 ~/.ansible/roles/my_namespace.foo -> /tmp/venv/foo
Hello,
As one of the maintainer of the Ansible stack on Arch Linux's side, I noticed that the v4.1.12 tag has been deleted.
Apart from Debian, that still has the said tag in their source's clone, multiple distributions now have an ansible-compat package based on an non-existing upstream tag, preventing rebuilds and breaking reproducibility (until another stable tag is made at least).
Is there any specific reason for this deletion? If so, does some more tags removal should be expected in the future?
When handling a role that's inside of a collection, ansible-compat is unable to determine that the collection exists. Therefore, it errors because a role name and namespace are not specified inside of meta/main.yml. Here is an example CI run where this occurs.
Hi! 👋
There is a tag for 4.1.12 in this repository, but no release 4.1.12 is available on PyPI.
Please see comments appended here: #114
As stated in ansible/ansible#79408 (comment) -- At least ansible-galaxy is not supposed to be run in parallel, ever.
As per this discussion we will be changing releases to CalVer moving forward.
This issue is to track the status of that transition.
As development of 2.14 started recently, we found that it broke ansible-compat. We need a fix for it.
Apparently a change introduced in 2.14 devel branch broke the output format of ansible-config dump, which is no longer parseable:
ansible-config dump|grep COLLECTIONS_PATHS
COLLECTIONS_PATHS(default) = {{ ANSIBLE_HOME ~ "/collections:/usr/share/ansible/collections" }}
Before this the values were loadable using python eval.
Hi! When packaging 2.2.1 for Arch Linux I noticed, that packaging seems to be a requirement, but it is not listed as a direct project requirement:
if this is run from a role directory where the role directory name is also name for the fqrn, you get this failure:
self._install_galaxy_role(self.project_dir, ignore_errors=True)
File "/home/user/.ansible/roles/limepepper.nginx/.venv/lib64/python3.9/site-packages/ansible_compat/runtime.py", line 412, in _install_galaxy_role
if not exists or os.readlink(link_path) != str(target):
OSError: [Errno 22] Invalid argument: '/home/user/.ansible/roles/limepepper.nginx'
os.readlink(link_path) assumes that link_path is a symlink
which if you name your role directories with the namespace.role_name convention, it isn't
$ file /home/user/.ansible/roles/limepepper.nginx
/home/user/.ansible/roles/limepepper.nginx: directory
As noted by @zhan9san the inclusion of sys.path entries is limited isolated=False.
The intent of this issue is to question the validity of this.
Related: #318 (comment)
Failed to load packit config file:
Cannot parse package config: ValidationError({'_schema': ["Keys: {'branch'} are defined outside job metadata dictionary. Mixing obsolete metadata dictionary and new job keys is not possible. Remove obsolete nested job metadata dictionary."]}).
For more info, please check out the documentation or contact the Packit team.
As we want to minimize dependencies introduced by this library we should avoid use of tenacity.
As part of the same change we should allow user of the library to use the library without being forced to use our hardcoded retry logic (make the retries optional).
@webknjaz suggested use of backoff or writing our own.
Hi,
I think I came across a weird corner case in which the recent "load_collections" addition broke ansible-lint v6.16.1 when the ansible verbosity is increased.
Exact change:
def load_collections(self) -> None:
...
proc = self.run(["ansible-galaxy", "collection", "list", "--format=json"])
...
...
data = json.loads(proc.stdout)
This new method invokes the ansible-galaxy collection list --format=json command and if the user previously set the 'ANSIBLE_VERBOSITY', the output will be more than the json response thus causing this cryptic issue
$ export ANSIBLE_VERBOSITY=2
$ ansible-lint pb-test.yml
Traceback (most recent call last):
File "/usr/local/bin/ansible-lint", line 8, in <module>
sys.exit(_run_cli_entrypoint())
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/ansiblelint/__main__.py", line 310, in _run_cli_entrypoint
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/ansiblelint/__main__.py", line , in main
app = get_app()
^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/ansiblelint/app.py", line 387, in get_app
app.runtime.prepare_environment(
File "/usr/local/lib/python3.11/dist-packages/ansiblelint/_vendor/ansible_compat/runtime.py", line 5, in prepare_environment
self.load_collections()
File "/usr/local/lib/python3.11/dist-packages/ansiblelint/_vendor/ansible_compat/runtime.py", line 176, in load_collections
data = json.loads(proc.stdout)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
This is what would be the normal output that the json.loads is trying to parse
$ ansible-galaxy collection list --format=json
ansible-galaxy [core 2.14.5]
config file = /builds/ansible.cfg
configured module search path = ['/builds/library']
ansible python module location = /usr/local/lib/python3.11/dist-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible-galaxy
python version = 3.11.3 (main, Apr 5 2023, 14:15:06) [GCC 10.2.1 20210110] (/usr/bin/python3.11)
jinja version = 3.1.2
libyaml = True
Using /builds/ansible.cfg as config file
{"/usr/local/lib/python3.11/dist-packages/ansible_collections": {"hetzner.hcloud": {"version": "1.11.
Ansible added --pre in 2.10 to ansible-galaxy collection install, but ansible-compat uses this also for Ansible 2.9, thus resulting in errors.
ansible-compat/src/ansible_compat/runtime.py
Lines 219 to 224 in f6b3871
usage: ansible-galaxy [-h] [--version] [-v] TYPE ...
ansible-galaxy: error: unrecognized arguments: --pre
This was removed (because it apparently already didn't work correctly before that) in #37
For more context see ansible/ansible-lint#2042 - functionality in ansible-lint is currently broken/bugged due to this library automatically exec'ing out to ansible-galaxy whenever a certain file exists in the root folder of a repository.
Ansible galaxy collection install --force can fail if the target collection was symlinked. While this seems like a galaxy bug, we cannot rely on ansible-galaxy to fix it in all ansible versions, so we need a workaround.
Installing 'cisco.nxos:0.0.0' to '/Users/ssbarnea/.cache/ansible-compat/15cb31/collections/ansible_collections/cisco/nxos'
E to see the full traceback, use -vvv
E
E ERROR! Unexpected Exception, this is probably a bug: Cannot call rmtree on a symbolic link
E
E Command returned 250 code:
Hi! I package this project for Arch Linux.
Since setup.py has been removed with 1.0.0, I need to use a new mix of tools to build and install this package.
Unfortunately this leads to /usr/lib/python3.10/site-packages/ansible_compat-0.0.0.dist-info/METADATA containing VERSION: 0.0.0 and not 1.0.0, which in turn breaks molecule.
I am using
python -m build --wheel --skip-dependency-check --no-isolation
python -m install --optimize=1 --destdir="$pkgdir" dist/*.whl
to build and install the package, which is sort of what you do in tox (minus the installation).
As this is a fundamental change in how this package is supposed to be built and installed, it would be great if this could be resolved. It currently breaks molecule on Arch Linux.
Hello"
I'm maintaining ansible-compat on Debian.
Let me ask:
The last update I made to Debian was v4.1.12
In the last release it went from v4.1.12 to v24.5.1. That's right? Or did ansible compat change the versioning system?
Would you be able to clarify?
Dear,
Nilson Silva
After commit was merged, py311-devel is failing.
Once requirement for using exec() inside molecule is to be able to get real-time output from the executed command. That is because most molecule commands are running playbooks and the user wants to see which tasks are executed in real time.
For linter we have the other use-case where we do not want to show the output but we want to analyze the output (that is already covered with current implementation).
Molecule makes use of https://github.com/pycontribs/subprocess-tee/blob/main/src/subprocess_tee/__init__.py in order to do that but we could consider doing that ourselves and benefit from reducing the number of dependencies we have.
We can defer this feature for another release as for the moment molecule could continue to use subprocess-tee instead of using Runtime.exec() but longer term, we will need a solution, as it would prevent molecule from using execution environments if we do not to it.
We should ensure we have full code-coverage for the library before effectively starting to use it in molecule and linter, especially as the code-base is quite small.
molecule's login command calls exec from ansible_compat/runtime.py. This function calls subprocess.run with a variable args, which is one of the arguments to exec. However, if args is a string with spaces, which is the case for molecule's login call, subprocess.run raises a FileNotFoundError. e.g.:
Traceback (most recent call last):
File "/usr/bin/molecule", line 8, in <module>
sys.exit(main())
File "/usr/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.10/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/lib/python3.10/site-packages/molecule/command/login.py", line 166, in login
base.execute_subcommand(scenario.config, subcommand)
File "/usr/lib/python3.10/site-packages/molecule/command/base.py", line 149, in execute_subcommand
return command(config).execute()
File "/usr/lib/python3.10/site-packages/molecule/logger.py", line 188, in wrapper
rt = func(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/molecule/command/login.py", line 101, in execute
self._get_login(hostname)
File "/usr/lib/python3.10/site-packages/molecule/command/login.py", line 146, in _get_login
app.runtime.exec(cmd)
File "/usr/lib/python3.10/site-packages/ansible_compat/runtime.py", line 138, in exec
result = run_func(
File "/usr/lib/python3.10/subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.10/subprocess.py", line 966, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.10/subprocess.py", line 1842, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/env docker exec -e COLUMNS=88 -e LINES=29 -e TERM=bash -e TERM=xterm -ti molecule-minio-ubuntu1804 bash'
I've obviously tried copy pasting the underlying command it's trying to run and it works. I think maybe subprocess.run needs to be called with shell=True for this to work? Or the args variable has to be a list rather than a string?
I'm on archlinux, these are the versions of the relevant packages:
$ molecule --version
molecule 3.6.1 using python 3.10
ansible:2.12.2
delegated:3.6.1 from molecule
docker:1.1.0 from molecule_docker requiring collections: community.docker>=1.9.1
podman:1.1.0 from molecule_podman requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
$ ansible --version && molecule --version
ansible [core 2.11.5]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/Users/jgeerling/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /opt/homebrew/lib/python3.9/site-packages/ansible
ansible collection location = /Users/jgeerling/.ansible/collections:/usr/share/ansible/collections
executable location = /opt/homebrew/bin/ansible
python version = 3.9.5 (default, May 3 2021, 19:12:05) [Clang 12.0.5 (clang-1205.0.22.9)]
jinja version = 3.0.1
libyaml = True
molecule 3.5.1 using python 3.9
ansible:2.11.5
delegated:3.5.1 from molecule
Molecule installation method (one of):
Ansible installation method (one of):
Detail any linters or test runners used:
N/A
I have the following .ansible-lint file set up:
skip_list:
- 'yaml'
- 'role-name'When I run molecule test on any of my roles named with dashes (which are grandfathered in on Galaxy) like geerlingguy.aws-inspector (role here: https://github.com/geerlingguy/ansible-role-aws-inspector), I get the error:
ansible_compat.errors.InvalidPrerequisiteError: Computed fully qualified role name of geerlingguy.aws-inspector does not follow current galaxy requirements.
Please edit meta/main.yml and assure we can correctly determine full role name:
galaxy_info:
role_name: my_name # if absent directory name hosting role is used instead
namespace: my_galaxy_namespace # if absent, author is used instead
Namespace: https://galaxy.ansible.com/docs/contributing/namespaces.html#galaxy-namespace-limitations
Role: https://galaxy.ansible.com/docs/contributing/creating_role.html#role-names
As an alternative, you can add 'role-name' to either skip_list or warn_list.
skip_list should work as documented, and it has until today. It seems like something in the ability to skip the role-name check has changed and now all my roles like the one linked above are failing to run in Molecule :(
See above—if I run molecule test inside the repo for geerlingguy.aws-inspector (or any other role I have with a dash in the name), I get a fail immediately with the same error message.
Some users may not want to make use of our embedded retry mechanism used by network-dependent Runtime() methods and we should allow them to decide if they want a retry mechanism or not.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.