Comments (1)
shishircc
commented on August 25, 2024
emr_dag_automation_blueprint.py.txt
Attached DAG can be used to create the EMR with the above configuration
Here is the requirements.txt
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.6.3/constraints-3.10.txt"
apache-airflow==2.6.3
apache-airflow-providers-salesforce
apache-airflow-providers-apache-spark
apache-airflow-providers-amazon
apache-airflow-providers-postgres
apache-airflow-providers-mongo
apache-airflow-providers-ssh
apache-airflow-providers-common-sql
astronomer-cosmos
boto3
simplejson
pymongo
pymssql
smart-open
psycopg2==2.9.5
simple-salesforce
from dbt-spark.
Related Issues (20)
- [Bug] New test failure for `tests/functional/adapter/test_constraints.py::TestSparkIncrementalConstraintsRollback::test__constraints_enforcement_rollback` HOT 2
- [CI/CD Improvements] Migrate to `hatch` and standardize workflows
- Import relevant pytest(s) for cross-database `cast` macro
- Cross-database `date` macro
- [Bug] New test failure for `tests/functional/adapter/test_python_model.py::TestPythonIncrementalModelSpark::test_incremental`
- [Bug] dbt run will fail if default namespace doesn't exist. HOT 1
- [Bug] Cannot run unit tests against Spark/Hudi, receiving "NoneType is not iterable" error HOT 3
- [ADAP-1093] [Feature] Run integration tests against all supported python versions
- [Feature] Support HTTP transport protocol for Thrift method
- [Feature] Support OCI Dataflow as a backend for dbt-spark
- `dbt-core` Dockerfile does not work for `dbt-spark` due to `PyHive` HOT 2
- [Bug] CI is broken on `main` due to dependency resolution and timeout issues HOT 1
- [Feature] Spike on supporting Py3.12 in dbt-spark
- [Bug] The tblproperties are not applied when using Python Model to create a table HOT 1
- [Issue] sasl as a dependency HOT 3
- [Feature] Livy connection support for Spark SQL models
- [Unit Testing] Add functional tests for unit testing: HOT 1
- [Bug] dbt docs generate does not include Data Type (data stored in iceberg format) HOT 2
- [Bug] flake8 code smells HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbt-spark.