meltanolabs / tap-dbt Goto Github PK
View Code? Open in Web Editor NEWSinger Tap for dbt API v2 built with the Meltano SDK
Home Page: https://pypi.org/p/tap-dbt
License: Apache License 2.0
Singer Tap for dbt API v2 built with the Meltano SDK
Home Page: https://pypi.org/p/tap-dbt
License: Apache License 2.0
Looks like the last successful test run was 6 months ago. Tests are throwing 404's on:
singer_sdk.exceptions.FatalAPIError: 404 Client Error: Not Found for path: /api/v2/accounts/***/
From a Slack conversation:
- Incremental replication where the replication key is sometimes null:
Again in tap-dbt, the runs stream is set to replicate incrementally using finished_at as the replication key. However this field is sometimes NULL for our runs.Are there any workarounds for this, aside from tweaking our local code to replicate the runs table with full table replication?
I need to dig into the API docs to see what's going and maybe come up with a workaround (other than overriding the replication method in the Singer catalog).
Help from other users of this tap is more than welcome!
404 warning - Some endpoints are only available to privileged users, and the API appears to handle this at our site by return 404 if the endpoint is not reachable. I don't know how to handle this error properly in the Meltano SDK - the fix for me would be to "not select" the stream, e.g. I have had varying success querying
{accountId}/repositories/
with different API keys.
From #162.
Perhaps selected_by_default
can be used for this.
cc @mjsqu
Python 3.7 reached EOL back in 2023/06.
Upvote (:+1:) or downvote (๐) this issue.
Regardless, I plan do this before the end of 2023 or Jan 2024 at the latest.
Cannot start plugin tap-dbt: Catalog discovery failed
...
KeyError: 'Account'
...
jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'components/schemas/Account'
May be caused by changes to the open api spec recently: dbt-labs/dbt-cloud-openapi-spec@b88cc21
From a Slack conversation:
Trying to run tap-dbt and I'm running into some issues that I think would apply more generally to other taps too, but I couldn't find an answer in the forums or slack:
- Allowing NULL types globally: The expected schema for most of the fields in tap-dbt does not allow NULL values. For example in the runs stream, the field dbt_version is configured to expect a type of string like:
"dbt_version": { "type": "string" }
The solution would involve one of two alternatives:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.