hazyresearch / data-centric-ai Goto Github PK

Use this template to propose a new area page. A good rule-of-thumb to decide whether a topic merits an area page is whether it would pass muster as a workshop at a machine learning venue (e.g. ICML).

Please address the following questions when raising this issue:

• what story you might tell about the topic's importance to data-centric AI
• whether this topic is related to other areas in data-centric AI, and why existing discussions may not be sufficient
• what subtopics, resources and related work you may discuss in the area page

If your request to add the new area is approved, you can submit a PR with the following changes:

• add an additional area in README.md
• add an area page with relevant discussion

Dataframe preprocessing platform:

Automunge is a resource for encoding dataframes for supervised learning. Originally built as a resource for missing data infill, the library evolved to include a unique and refined API for engineering sets of feature transformations, including some novel approaches for integrating stochasticity into supervised learning training or inference. The library has been shared in workshops at venues like NeurIPS, ICML, and ICLR. We suggest the tutorials folder in the github account as a starting point, or full (and comprehensive) documentation is provided in the readme file. Further write-ups are available in the arxiv literature authored by the developer (Nicholas J. Teague) and linked from automunge.com

Data selection methods, such as active learning and core-set selection, are useful and important tools for machine learning on large datasets. Major AI/ML conferences such as NeurIPS and ICML have consistently featured workshops and tutorials on these topics:

• SubSetML: Subset Selection in Machine Learning: From Theory to Practice, ICML 2021
• Workshop on Dataset Curation and Security, NeurIPS 2020
• Active Learning From Theory to Practice, ICML 2019

what story you might tell about the topic's importance to data-centric AI
Large-scale unlabeled datasets can contain millions or billions of examples covering a wide variety of underlying concepts. Yet, these massive datasets often skew towards a relatively small number of common concepts, for example ‘cats’, ‘dogs’, and ‘people’. Rare concepts, such as ‘harbor seals’, tend to only appear in a small fraction of the data (usually less than 1%). However, performance on these rare concepts is critical in many settings. For example, harmful or malicious content may comprise only a small percentage of user-generated content, but it can have a disproportionate impact on the overall user experience. Similarly, when debugging model behavior for safety-critical applications like autonomous vehicles, or when dealing with representational biases in models, obtaining data that captures rare concepts allows machine learning practitioners to combat blind spots in model performance. Even a simple task, such as stop sign detection by an autonomous vehicle, can be difficult due to the diversity of real-world data. Stop signs may appear in a variety of conditions (e.g., on a wall or held by a person), can be heavily occluded, or have modifiers (e.g., “Except Right Turn”). Large-scale datasets are essential but not sufficient; finding the relevant examples for these long-tail tasks is challenging. Data selection methods, active learning, active search, and core-set selection methods, have the potential to automate the process of identifying these rare, high-value data points. (See "Similarity Search for Efficient Active Learning and Search of Rare Concepts" for more detail)

whether this topic is related to other areas in data-centric AI, and why existing discussions may not be sufficient
All of the other areas focus on how we process data, not which data should we process.

what subtopics, resources and related work you may discuss in the area page

• Active learning
• Active search
• Core-set selection
• Cooperative learning

Catalogue data-centric tools that are related to existing area pages, e.g. tools for data programming, weak supervision, data cleaning, data privacy, robustness, evaluation, monitoring, etc.

[Describe an awesome list that you would like to add or contribute to: what does the contribution require, and what would you add]

《The Re-Label Method For Data-Centric Machine Learning》
《Learning From How Humans Correct》
《Automatic Label Error Correction》
《Re-Label By Data Pattern For Controllable Deep Learning》

Dear Concern,
I was trying to access the discourse on data-centric-ai, but when I "click to join if you are a new user", I get this.

Catalogue startups that are working to develop data-centric AI solutions, including MLOps tools, ML platforms, data management solutions for ML, etc. Link here

This is definitely an awesome list, however, I notice there exist two similar lists that might need our attention:
https://github.com/Renumics/awesome-open-data-centric-ai
https://github.com/Data-Centric-AI-Community/awesome-data-centric-ai

How do we differentiate the list from each other in terms of the goal?