Giter Club home page Giter Club logo

airflow-component's People

Contributors

noelmcloughlin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

airflow-component's Issues

[pywinrm] auth method is credssp, but requests-credssp (PyOpenSSL?) is not installed

After cent7 install we should expect all airflow dependencies are installed by pywinrm[credssp,kerberos]

https://github.com/saltstack-formulas/airflow-formula/blob/9869f01db7ebad033da9657febe4f29eed67ab90/airflow/defaults.yaml#L174
- pywinrm[kerberos,credssp]

But some failing tasks and conflicting error messages suggests otherwise. Checking...

Is worker running
$ sudo systemctl status airflow-celery-worker -l
... [active] ...
Sep 06 19:24:38 myfqdn airflow[29409]: File "/home/_airflowservice@myhostl/.local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 99, in _execute_in_fork
Sep 06 19:24:38 myfqdn airflow[29409]: raise AirflowException('Celery command failed on host: ' + get_hostname())
Sep 06 19:24:38 myfqdn airflow[29409]: airflow.exceptions.AirflowException: Celery command failed on host: myhost
Check value of hostname_callable in ~/airflow/airflow.cfg looks sensible:
    # If using IP address as hostname is preferred, use value ``airflow.utils.net.get_host_ip_address``
    hostname_callable = socket.getfqdn
    #hostname_callable = socket:gethostname
Ensure /airflow_home/dags/ path is identical on scheduler & worker node. Enforce using symlink if necessary:
$ ls -l /home | grep airflow
Check Task logfile:
$ tail -2 /home/_airflowservice@mydomain/airflow/logs/WinRM_Testing/winrmtest1-2021-09-06T18:29:46.431741+00:00/1.log
2021-09-06T18:29:46.431741+00:00/1.log:[2021-09-06 18:29:51,887] {base_task_runner.py:118} INFO - Job 495: Subtask winrmtest1 ***.exceptions.AirflowException: Error connecting to host: myotherhost, error: requests auth method is credssp, but requests-credssp is not installed
Check/ensure requests-credssp is installed
$ pip freeze | egrep 'credssp|nego'
requests-credssp
pyspnego
$ python -c "import requests_credssp"
$ echo $?
0
Check if dependencies installed
$ pip install pywinrm[credssp,kerberos]
Looking in indexes: https://pypi.gbetechnologies.com, https://pypi.org/simple
Requirement already satisfied: pywinrm[credssp,kerberos] in ./.local/lib/python3.6/site-packages (0.4.2)
Requirement already satisfied: requests-ntlm>=0.3.0 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (2.26.0)
Requirement already satisfied: xmltodict in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (0.12.0)
Requirement already satisfied: six in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.16.0)
Requirement already satisfied: requests-credssp>=1.0.0 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.2.0)
Requirement already satisfied: pykerberos<2.0.0,>=1.2.1 in ./.local/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]) (1.2.1)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (2021.5.30)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (1.26.6)
Requirement already satisfied: idna<4,>=2.5 in ./.local/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm[credssp,kerberos]) (3.2)
Requirement already satisfied: pyasn1>=0.3.1 in ./.local/lib/python3.6/site-packages (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (0.4.8)
Requirement already satisfied: cryptography in ./.local/lib/python3.6/site-packages (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (3.4.8)
Requirement already satisfied: pyspnego in ./.local/lib/python3.6/site-packages (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (0.1.6)
Collecting pyOpenSSL>=16.0.0
  Using cached pyOpenSSL-20.0.1-py2.py3-none-any.whl (54 kB)
Requirement already satisfied: cffi>=1.12 in ./.local/lib/python3.6/site-packages (from cryptography->requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (1.14.6)
Requirement already satisfied: pycparser in ./.local/lib/python3.6/site-packages (from cffi>=1.12->cryptography->requests-credssp>=1.0.0->pywinrm[credssp,kerberos]) (2.20)
Requirement already satisfied: ntlm-auth>=1.0.2 in ./.local/lib/python3.6/site-packages (from requests-ntlm>=0.3.0->pywinrm[credssp,kerberos]) (1.5.0)
Installing collected packages: pyOpenSSL
Successfully installed pyOpenSSL-20.0.1

So why is PyOpenSSL being collected and installed?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.