meltanolabs / tap-dbt Goto Github PK

Looks like the last successful test run was 6 months ago. Tests are throwing 404's on:

singer_sdk.exceptions.FatalAPIError: 404 Client Error: Not Found for path: /api/v2/accounts/***/

404 warning - Some endpoints are only available to privileged users, and the API appears to handle this at our site by return 404 if the endpoint is not reachable. I don't know how to handle this error properly in the Meltano SDK - the fix for me would be to "not select" the stream, e.g. I have had varying success querying {accountId}/repositories/ with different API keys.

From #162.

Perhaps selected_by_default can be used for this.

cc @mjsqu

From a Slack conversation:

Trying to run tap-dbt and I'm running into some issues that I think would apply more generally to other taps too, but I couldn't find an answer in the forums or slack:

1. Allowing NULL types globally: The expected schema for most of the fields in tap-dbt does not allow NULL values. For example in the runs stream, the field dbt_version is configured to expect a type of string like:

          "dbt_version": {
            "type": "string"
          }

The solution would involve one of two alternatives:

• Manually add the "null" type in all the schemas
• Refactor the schemas to use the SDK typing helpers, which make fields nullable by default.

• Slack
• discuss.meltano.com

From a Slack conversation:

2. Incremental replication where the replication key is sometimes null:
   Again in tap-dbt, the runs stream is set to replicate incrementally using finished_at as the replication key. However this field is sometimes NULL for our runs.

Are there any workarounds for this, aside from tweaking our local code to replicate the runs table with full table replication?

I need to dig into the API docs to see what's going and maybe come up with a workaround (other than overriding the replication method in the Singer catalog).

Help from other users of this tap is more than welcome!

• Slack
• discuss.meltano.com

Cannot start plugin tap-dbt: Catalog discovery failed
...
KeyError: 'Account'
...
jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'components/schemas/Account'

May be caused by changes to the open api spec recently: dbt-labs/dbt-cloud-openapi-spec@b88cc21

https://docs.getdbt.com/docs/dbt-cloud-apis/sl-graphql

cc @mjsqu

Python 3.7 reached EOL back in 2023/06.

Upvote (:+1:) or downvote (👎) this issue.

Regardless, I plan do this before the end of 2023 or Jan 2024 at the latest.