Comments (2)
Hey @elgalu,
(for context, I'm a product manager at google working on a few things, including MLMD. Views here are mine, not through any official google spokesperson)
Great question! There are some fascinating HCI and product strategy opinions behind the answer.
There are multiple efforts in the ML industry to solve the problem of keeping track of all your ML artifacts: Models, datasets, features, etc. The same way classical software engineering iterated through multiple ways of keeping track of code (central repos like subversion, decentralized repos like git, etc.), ML engineering is evolving the best way to keep track of ML Engineering.
IMO, ML-Flow is a model-first view. Kind of like a star-schema in RDBMs, if you're familiar. Contrast to TFX's pipeline-first view. MLMD can be part of how TFX's pipeline records the DAG history of each run. MLFlow and MLMD are both under active development, so these opinions are moving targets :-)
Each view (model first or pipeline first) can be, in theory, recovered from the other. It is easier to recover the model-first view from the pipeline-first view because it only requires hiding information (rest of the pipeline). But a model-first view may not at first store extra information outside the model like its training data, analysis, comparisons with other models trained on the same pipeline, the model's full ineage, etc.
So MLFlow is kinda coming at the problem from a different angle than ML Metadata. Pipeline-first instead of Model-first. We at google have found making the pipeline a first-class citizen more useful.
from ml-metadata.
Some further reading and watching on this from the relevant google teams
https://www.youtube.com/watch?v=drYM04t57tU&feature=youtu.be
from ml-metadata.
Related Issues (20)
- conda distribution HOT 4
- Cannot install ml-metadata v1.0.0 HOT 15
- Garbage collection of underlying artifacts HOT 11
- How do I get the stack trace? HOT 3
- Data too long for column 'string_value' at row 1 HOT 3
- Example for connecting to Google Cloud Vertex Metadata HOT 1
- Suitable for computer vision project? HOT 2
- Google.com HOT 1
- Python2_brutegram HOT 1
- fully support mysql 8.0 (`caching_sha2_password` authentication) HOT 3
- Fb.me Bruterforce attack
- Reaching the dev team to ask questions about mlmd implementation and roadmap HOT 2
- CVE-2023-44487 and CVE-2023-39325 affecting mlmd gRPC remote server? HOT 4
- ML Metadata as a Model Registry solution HOT 3
- Add support for command line server ssl options HOT 1
- Using ML metadata postgres SSL connection options don't work for `sslMode=require` HOT 1
- Is there a plan to support Linux on the ARM64 architecture?
- pip install ml-metadata==1.14.0 doesn't work with Python >= 3.11 HOT 2
- Add cmdline ssl options for mysql
- String property max chars when using MLMD with PostgreSQL HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ml-metadata.