noelmcloughlin / airflow-component Goto Github PK
View Code? Open in Web Editor NEWLightweight IaC Installer of Federated Apache-Airflow
License: Apache License 2.0
Lightweight IaC Installer of Federated Apache-Airflow
License: Apache License 2.0
something is wrong on ldap
After cent7 install we should expect all airflow dependencies are installed by pywinrm[credssp,kerberos]
https://github.com/saltstack-formulas/airflow-formula/blob/9869f01db7ebad033da9657febe4f29eed67ab90/airflow/defaults.yaml#L174
- pywinrm[kerberos,credssp]
But some failing tasks and conflicting error messages suggests otherwise. Checking...
$ sudo systemctl status airflow-celery-worker -l
... [active] ...
Sep 06 19:24:38 myfqdn airflow[29409]: File "/home/_airflowservice@myhostl/.local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 99, in _execute_in_fork
Sep 06 19:24:38 myfqdn airflow[29409]: raise AirflowException('Celery command failed on host: ' + get_hostname())
Sep 06 19:24:38 myfqdn airflow[29409]: airflow.exceptions.AirflowException: Celery command failed on host: myhost
hostname_callable
in ~/airflow/airflow.cfg
looks sensible: # If using IP address as hostname is preferred, use value ``airflow.utils.net.get_host_ip_address``
hostname_callable = socket.getfqdn
#hostname_callable = socket:gethostname
/airflow_home/dags/
path is identical on scheduler & worker node. Enforce using symlink if necessary:$ ls -l /home | grep airflow
$ tail -2 /home/_airflowservice@mydomain/airflow/logs/WinRM_Testing/winrmtest1-2021-09-06T18:29:46.431741+00:00/1.log
2021-09-06T18:29:46.431741+00:00/1.log:[2021-09-06 18:29:51,887] {base_task_runner.py:118} INFO - Job 495: Subtask winrmtest1 ***.exceptions.AirflowException: Error connecting to host: myotherhost, error: requests auth method is credssp, but requests-credssp is not installed
requests-credssp
is installed$ pip freeze | egrep 'credssp|nego'
requests-credssp
pyspnego
$ python -c "import requests_credssp"
$ echo $?
0
$ pip install pywinrm[credssp,kerberos]
Looking in indexes: https://pypi.gbetechnologies.com, https://pypi.org/simple
Requirement already satisfied: pywinrm[credssp,kerberos] in ./.local/lib/python3.6/site-packages (0.4.2)
Requirement already satisfied: requests-ntlm>=0.3.0 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (2.26.0)
Requirement already satisfied: xmltodict in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (0.12.0)
Requirement already satisfied: six in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.16.0)
Requirement already satisfied: requests-credssp>=1.0.0 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.2.0)
Requirement already satisfied: pykerberos<2.0.0,>=1.2.1 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.2.1)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (2021.5.30)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (1.26.6)
Requirement already satisfied: idna<4,>=2.5 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (3.2)
Requirement already satisfied: pyasn1>=0.3.1 in ./.local/lib/python3.6/site-packages (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (0.4.8)
Requirement already satisfied: cryptography in ./.local/lib/python3.6/site-packages (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (3.4.8)
Requirement already satisfied: pyspnego in ./.local/lib/python3.6/site-packages (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (0.1.6)
Collecting pyOpenSSL>=16.0.0
Using cached pyOpenSSL-20.0.1-py2.py3-none-any.whl (54 kB)
Requirement already satisfied: cffi>=1.12 in ./.local/lib/python3.6/site-packages (from cryptography->requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (1.14.6)
Requirement already satisfied: pycparser in ./.local/lib/python3.6/site-packages (from cffi>=1.12->cryptography->requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (2.20)
Requirement already satisfied: ntlm-auth>=1.0.2 in ./.local/lib/python3.6/site-packages (from requests-ntlm>=0.3.0->pywinrm[credssp,kerberos]) (1.5.0)
Installing collected packages: pyOpenSSL
Successfully installed pyOpenSSL-20.0.1
So why is PyOpenSSL
being collected and installed?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.