tulibraries / ansible-role-airflow Goto Github PK

View Code? Open in Web Editor NEW

6.0 6.0 6.0 319 KB

TU Libraries Airflow & DAGs Deployment Ansible Role

License: MIT License

Python 72.89% Jinja 27.11%

airflow ansible-roles

ansible-role-airflow's People

Contributors

Stargazers

Watchers

Forkers

izhidkikh gnishanth4 jrd281 bziobnic mrsipan hajaalin

ansible-role-airflow's Issues

Failure when running bundle install as airflow user

From /var/lib/airflow/tul_cob

bundle install
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5/ext/unf_ext
/var/lib/airflow/.rbenv/versions/2.4.1/bin/ruby -r ./siteconf20190403-30967-waln2k.rb extconf.rb
checking for -lstdc++... no
creating Makefile

current directory: /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5/ext/unf_ext
make "DESTDIR=" clean

current directory: /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5/ext/unf_ext
make "DESTDIR="
compiling unf.cc
make: g++: Command not found
make: *** [unf.o] Error 127

make failed, exit code 2

Gem files will remain installed in /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5 for inspection.
Results logged to /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/extensions/x86_64-linux/2.4.0-static/unf_ext-0.0.7.5/gem_make.out

An error occurred while installing unf_ext (0.0.7.5), and Bundler cannot continue.
Make sure that `gem install unf_ext -v '0.0.7.5' --source 'https://rubygems.org/'` succeeds before bundling.

If I try to install that specific gem

gem install unf_ext -v '0.0.7.5'
Building native extensions.  This could take a while...
ERROR:  Error installing unf_ext:
	ERROR: Failed to build gem native extension.

    current directory: /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5/ext/unf_ext
/var/lib/airflow/.rbenv/versions/2.4.1/bin/ruby -r ./siteconf20190403-20269-zbj4kp.rb extconf.rb
checking for -lstdc++... no
creating Makefile

current directory: /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5/ext/unf_ext
make "DESTDIR=" clean

current directory: /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5/ext/unf_ext
make "DESTDIR="
compiling unf.cc
make: g++: Command not found
make: *** [unf.o] Error 127

make failed, exit code 2

Gem files will remain installed in /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unf_ext-0.0.7.5 for inspection.
Results logged to /var/lib/airflow/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/extensions/x86_64-linux/2.4.0-static/unf_ext-0.0.7.5/gem_make.out```

Issue with package while deploying airflow

Hi,

I tried to use your role but it failed with this error

failed: [dmpServer] (item={'name': 'python-virtualenv'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "python-virtualenv"}, "msg": "No package matching 'python-virtualenv' is available"}
failed: [dmpServer] (item={'name': 'python-pip'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "python-pip"}, "msg": "No package matching 'python-pip' is available"}

I could not use this role due to this error.

Regards
Rajaniesh

way to load variables faster when removing & adding known variables during deploy

Use dotenv files or similar for managing envvars

Becomes more important given airflow library's turn towards preferring environment variables to config values

Find Airflow community way to handle Public RBAC role creating errors by default

Basically, it cannot run a task by default that is required of the Public role. We can either update the Public role during deploy or update hte task. The airflow cli isn't great for either of those options yet, and the database is a black pit.

Upgrade to 1.10.6

Feature request: Add airflow user to ssl-cert group on Debian/Ubuntu

A small feature request:

Add airflow user to ssl-cert group on Debian/Ubuntu so that it can be easily given the option to read certificates for https

Getting a deprecation warning in airflow 1.10 on generated config file

configuration.py:569: DeprecationWarning: Specifying both AIRFLOW_HOME environment variable and airflow_home in the config file is deprecated. Please use Please use only the AIRFLOW_HOME environment variable and remove the config file entry.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.