gclunies / reflekt Goto Github PK

Hello !

We noticed whilst using reflekt report -s Cafeyn/ --to-file returns all properties as required and hence does not match the required status of the JSON files for some reason (we have not yet investigated)

Pulling a segment tracking plan that has track events defined without descriptions with reflect pull will cause the command to fail with KeyError: 'description'. It's good practice to always have descriptions defined for tracking events, however the segment UI allows this to be optional so reflekt should probably handle this use case. Looks like a .get("description", "") would just need to be added here https://github.com/GClunies/Reflekt/blob/bc3e4ece44ffe5f23905b43429ea5e4e54fb3614/reflekt/registry/segment.py#L380

Use the jaffle shop seeds as a basis to create an example dataset of events capturing orders, customers, and payments.

This will make demos more relatable as the jaffle shop is becoming a de-facto eample dataset. This data can also be used to power better tests for reflekt.

Work likely includes:

• Script to to generate Segment/Rudder/etc events.
• OPTIONAL: use reflekt-registry as the event validation
• Data stored in Snowflake/Redshift/BiqQuery. For BigQuery, this work should happen after #82.
• Build private dbt package using example data.

It seems there are very few blockers for BigQuery support. I believe its just the SQL generation which uses some non ANSI casting syntax ::.

Congrats on the 3.0 upgrade, looks nice!

The Rudderstack SDKs are similar to Segment, so Reflekt should be able to work with event data from Rudderstack as well.

Similar to Segment, Rudderstack has tracking plans and an Event Audit API that we could leverage:

• https://www.rudderstack.com/docs/features/data-governance/tracking-plans/
• https://www.rudderstack.com/docs/api/event-audit-api/

Work likely includes:

• Add a RudderRegistry class at reflekt/registry/rudder.py to sync schemas to Rudderstack's API.
• Add a common schema for Rudderstack events similar to what is done for segment here.
• Refector DbtBuilder in reflekt/builder/dbt.py to support building dbt artifacts for Rudder event data.

Great tool
It would be great to enahnce this to support databricks
Would you be open for a PR to support this feature?

Hi there !

I tried configuring a new reflekt repository by running the command reflekt init, and putting the following content in the prompts:

But then it returned the following issue:

Meaning the init command does not seem to be able to create the profile folder and file.

Add support to push/pull schemas to/from a reflekt-registry. This will remove reflekt's dependence on SaaS/CDP schema registries.

Related to issue #86, after having created on my own /Users/amaury/.reflekt/reflekt_profiles.yml then there was a mistake as the init command could not update the profiles files, as displayed here :