smbgs / syml Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
syml's People
Contributors
Stargazers
Watchers
syml's Issues
Consider the metadata abstraction and prototype client for non-relational doc-oriented databases (mongo, couchdb etc.)
Schema: define the "destination" section and schema
Define the analytics layer and components in general and for snowflake
Consider the metadata abstraction and protorype client for Neo4j
Schema: define & maintain the overall semantics and schema
Design the HTML/JS based report visualization tool and POC
Schema: define the "include" section and schema
Define the CI integrations for SchemaMigrations creation using schema/db comparison and git commit
Choose the schema definition format
- yamale
- jsonschema (for yaml)
- k8s based?
Define the metadata abstraction layer for the relational databases (postgres, mysql, snowflake)
SchemaReportResults: define & maintain overall semantics and schema
Implement table creation POC based on Schema definition
Define the CI integrations for creating SchemaAnalyticsReports
Create the test container that creates the Postgres database with MusicBrainz schema in docker/vagrant
Define the general operator features
SchemaMigration: define & maintain overall semantics and schema
Implement the base layer using sqlalchemy core able to connect to the Postgres instance and retrieve the database metadata (tables, views)
Define the CLI interface for extra modules
All modules should be able to register actions and events that they support. Actions should be automatically extend the CLI set of commands.
Arguments should be properly validated and results should be retrieved either as synchronous "results" or, preferably typed "events" rendered as CLI output.
Note, other orchestration tools will use the same interface as CLI so the definition should be shared with "k8s operator" etc.
Design the Markdown prototype for report data visualization and implement the POC
Create the high-level syml system diagram and describe the core concepts (svg, md)
SchemaReport: define & maintain overall semantics and schema
Implement the POC for postgres analytics, including integrated pg metrics about space and indexes
Define the general CLI approach
Choose from:
- simple unix-style CLI with modular sub-commands (possibly using python Click, Argparse)
- interactive shell with PyInquirer, python-nubia or something else
Schema: define the "sources" section semantics and schema
Implement table migration POC based on SchemaMigration
Implement operator POC and Schemas for CRDs.
Implement Snowflake connector and retrieve tables and views metadata
Implement the openapi -> SYML Schema (similar to what reverse module is doing, can be combined)
Define the CI integrations for migrating the target database using SchemaMigrations
Schema: define the "fields" section and schema
Implement the POC for snowflake analytics
Create the SYML SDK and guidelines for contributors
Define the benefits of this bridge
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.