Giter Club home page Giter Club logo

microsoft / cdm Goto Github PK

View Code? Open in Web Editor NEW
1.6K 153.0 509.0 115.49 MB

The Common Data Model (CDM) is a standard and extensible collection of schemas (entities, attributes, relationships) that represents business concepts and activities with well-defined semantics, to facilitate data interoperability. Examples of entities include: Account, Contact, Lead, Opportunity, Product, etc.

Home Page: https://powerplatform.microsoft.com/en-us/common-data-model/

License: Creative Commons Attribution 4.0 International

C# 28.29% Java 27.05% Python 21.58% TypeScript 23.04% PowerShell 0.04%
csharp typescript java python microsoft sdk

cdm's Introduction

❗Important Notice❗

Starting with CDM SDK release 1.7.1 the CdmStandardsAdapter is no longer requiring CDM Schema Store to provision foundational definitions required for CDM metadata parsing, as they are now bundled in the SDK binaries directly. The bundled definitions do not include CDM application schemas (e.g. schemaDocuments/core, schemaDocuments/FinancialServices...), only the foundational definitions in the schemaDocuments root are included to reduce binary size. If you have a dependency to these additional schemas, please package them and reference them from your project.

The CDM Schema Store will be shut down by end of March '24, and any services still using the older CDM SDK releases may start failing due to unavailability of the store. If you are still using the SDK versions prior to 1.7.1, please upgrade now!

Common Data Model Official Packages

NuGet Maven Central PyPI npm

We welcome your feedback

Do you use the Common Data Model SDK? If you do, please can you tell us which version(s) you use (C#/.NET, Java, Python, or TypeScript) by taking this short survey.

Common Data Model (CDM) Schema

The Common Data Model is a declarative specification, and definition of standard entities that represent commonly used concepts and activities across business and productivity applications, and is being extended to observational and analytical data as well. CDM provides well-defined, modular, and extensible business entities such as Account, Business Unit, Case, Contact, Lead, Opportunity, and Product, as well as interactions with vendors, workers, and customers, such as activities and service level agreements. Anyone can build on and extend CDM definitions to capture additional business-specific ideas.

    
     
            

Introduction

The Common Data Model standard defines a common language for business entities covering, over time, the full range of business processes across sales, services, marketing, operations, finance, talent, and commerce and for the Customer, People, and Product entities at the core of a company's business processes. The goal of CDM is to enable data and application interoperability spanning multiple channels, service implementations, and vendors. CDM provides self-describing data (structurally and semantically), enabling applications to easily read and understand the data.

The CDM is undergoing a specification effort driven by Microsoft and the documents published are continuously being iterated upon.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Repository Layout

There are two ways to consume the information in this repository:

  1. Entity Reference Index
  2. Visual Entity Navigator for interactively exploring entities, entity extensions/inheritance, attributes, and relationships

Versioning

Maintaining forward and backward compatibility is a key goal of the CDM. Therefore, the CDM uses only additive versioning, which means any revision of the CDM following a "1.0" release will not:

  • Introduce new mandatory attributes on previously published entities, or change an optional attribute to be mandatory
  • Rename previously published attributes or entities
  • Remove previously published attributes

Legal Notices

Microsoft and any contributors grant you a license to the Microsoft documentation and other content in this repository under the Creative Commons Attribution 4.0 International Public License, see the LICENSE file, and grant you a license to any code in the repository under the MIT License, see the LICENSE-CODE file.

Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.

Privacy information can be found at https://privacy.microsoft.com/en-us/

Microsoft and any contributors reserve all others rights, whether under their respective copyrights, patents, or trademarks, whether by implication, estoppel or otherwise.

cdm's People

Contributors

burhandodhy avatar cdm-publisher avatar chandrsoni avatar dingbx avatar imkef avatar jeffbern avatar jimdaly avatar jinichu avatar llawwaii avatar mafrisci avatar matgos avatar mfrish avatar microsoft-github-policy-service[bot] avatar microsoftopensource avatar miroslavplese avatar msftgits avatar nenad1002 avatar oovanesy avatar robertbruckner avatar sukanyamsft avatar theresapalmer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cdm's Issues

Visual Entity Navigator won't load (offline) GitHub source files

I wanted to download the source files, modify them and then load them back for testing purposes but when I do so it starts loading and then simply hangs with the message "Loading and resolving entity documents, please wait...".

It is possible to get the source files used when you click on "Load from Github!" or fix the source files in Github so they can be loaded from the desktop and get the same results using either method?

Common Data Model Adoption

Hi,

Is it possible to talk with someone from Microsoft regarding this initiative? We're planning a more robust usage, but we need some clarifications regarding the future of the project.

Thanks,

Daniel

Primitive Data Types

Is it expected that primitive data types (from primitives.cdm.json) to always exist for CDM? Meaning, once defined... they basically will be the "foundation" of how other things are defined? I'm assuming new data types could be added in future, but there would always be the usual defaults (e.g. Boolean, Integer, etc.).

Versioning and labeling

Hi, I'm wondering if it is possible for you to label with a version that is the current release in the PowerApps based CDS so we can detect in our buildflow changes directly from git?

Cheers,
Mario

CDM Roadmap with Milestones

There appears to be a CDM roadmap here. It would be nice to add a more detailed plan for work for CDM and how this repo fits in. This will allow users to understand what's being worked on and more confidence there is a bright future. I like how other Microsoft teams add milestones (with issues attached to them) along with Projects to see basic status of Not Started, In Progress, and Completed. If the community had a little more insight, they could possibly help with PRs on bugs, docs, features and defining more data models.

Installing CDM ?

We are looking to build a solution on top of CDM using a PowerApps Plan 2 licence. Our solution needs to use the opportunity entity. When looking at the entity in PowerApps, we cannot see the Opportunity entity. How can we add this entity to our data model ? Do we need to install something ?

Thanks !

Versioning of mandatory properties

The versioning section states:

Maintaining forward and backwards compatibility is a key goal of the CDM. Therefore, the CDM uses purely additive versioning, which means any revision of the CDM following an initial release will not:

  • Introduce new mandatory properties on existing entities

Should the converse restriction also apply, that a mandatory field cannot be changed to optional?

question

Every business has unique data and operations. although it would be possible to standardise data from some businesses in similar industries, how does the possibility of a company with various business operations impact the ability to create a CDM?

Enums in model.json

When we create a model.json file, is there a best practice for dealing with attributes which have a data type of enum? For example attribute "freightTermsCode" on Sales > Order

Common Data Service On-Prem Examples

There are cases when we would like to use the CDM with on-prem data sources (e.g. SQL server, etc.) without relying on Azure. Are there examples how to create .NET APIs to connect with an on-prem data source using Common Data Service connection?

Support for more primitive unsigned data types

This request is similar to #79 and one I created for IoT PnP repo. It seems like there would be primitive data types for unsigned values (i.e. SmallUnsignedInteger, UnsignedInteger, BigUnsignedInteger, etc.).

There are a few more other data types I believe should be primitives, but I'm still pondering and probably warrant their own issue created for discussion.

Thx

No explanation on how to use Visual Entity Navigator, can't see relationships

So the Visual Entity Navigator here https://microsoft.github.io/CDM/ seems a potentially useful tool to understand which bits of an entity come from which bits of Dynamics 365 and Microsoft's various add-ins and add-ons for it.. (Something that has become increasingly hard to understand and keep up with recently). But without some more information I'm really not sure how to use it effectively.

For example at the moment I don't seem to be able to see relationships properly. I can see them to some limited extent in terms of lookup fields and the "Used by Selected" and "Uses the Selected" colours when I click on an entity and a list of related entities if I double-click on an entity.

But that's it - nothing about relationship cardinality, cascade rules, naming etc. etc. Am I missing something?

Regards

--
Simon

Avro Schema Generators

We should be able to generate Open API and Avro Schema from CDM , so that BigData Storage and Processing using Avro and exposing or sharing the data via API using OpenAPI can be achieved.

version question

Hi,
I am new to CDM, and I have a question about the cdm schedma version.

In this page https://microsoft.github.io/CDM/ when I click "Currency" under applicationCommon in the left, I can see the version of the "Currency" entity is "0.8.1" as below:

Entity Currency
displayName Currency
description Currency in which a financial transaction is carried out.
version 0.8.1
cdmSchemas [ "/core/applicationCommon/Currency.cdm.json/Currency/hasAttributes/attributesAddedAtThisScope" ]
sourceName TransactionCurrency

But when I download the related cdm json file(https://raw.githubusercontent.com/Microsoft/CDM/master/schemaDocuments/core/applicationCommon/Currency.cdm.json), which I get it from the broswer log, I see the version info : jsonSchemaSemanticVersion: "0.7.0" and I can NOT find the "0.8.1" in this json file.

Can you tell me what is the difference between this two version number?
And how can I get the displayed version(0.8.1) from the json file?

Thanks very much.

Docs on how to create C# classes

Is there any documentation how to create C# interfaces/classes based on CDM? I'm hoping to create/use a .NET Standard API based on a set of defined entities.

It would be nice to have a core API that includes only the objects defined in CDM. We could then have separate APIs to reference that API to perform the interaction between data stores/apps.

Any help is appreciated.

Clarification on File Pointers

From the documentation on this page : https://docs.microsoft.com/en-us/common-data-model/data-lake

The model.json metadata file provides pointers to the entity data files throughout the CDM folder.

This seems to imply that the model.json file would need to contain a pointer to every file in the tree. Is it every file or every folder? The examples in the documentation don't show a long list of files.

If it is every file, what is the vision for delta files? If I send a file with the changes for the last 24 hours is the expectation that I update the model.json to have an explicit link to the new file? What if someone deletes any file in the past?

Thanks!
Caleb

CDM Format and project information

This work looks really promising. We're also working on coming up with a standard set of entities to model work from a performance perspective. Are you working from a standard way of describing entities? Are the .cdm.json and .dplx based on a standard somewhere?

Is this project open source or seeking to become a community/shared standard?

[question] Resource type

We think of the resource type as being representative of everything inbetween human resources to licenses, repositories, or servers.

How come the resource type is under service, and isn't a common to be extended? Is our way of thinking not the CDM-way?

TypeScript NPM Package?

I see that the Tools/cdm-types/cdm-types.ts file exists, but including it in a project that requires stricter linting rules leads to compilation issues due to the loose use of "any".

Is there an NPM package published with the type definitions? If so, can you provide the package name here and somewhere prevalent in the documents?

If not, can you publish it up ASAP to help drive adoption?

Process of Proposal

What are the procedures and appropriate methods of creating/requesting a proposal for CDM? For example, if there is a new primitive or attribute that could enhance CDM, what would be required for internal team to review/approve/deny?

[question] is.nullable SDK correlation

We are building a tool to provision a CDS with entities based on .cdm.json files. I'm wondering what corresponds to the "is.nullable" trait in an AttributeMetadata from the SDK.

As far as I can tell is.nullable is always set when

                                    "traitReference": "is.requiredAtLevel",
                                        "arguments": [
                                            {
                                                "name": "level",
                                                "value": "none"
                                            }
                                        ]

exists, and value is set to none.

Is this always the case? Is there any documentation about these correlations that I have simply overlooked?

Thank you for any help!

Support for Parquet files?

Will Parquet files be supported as an alternative to CSVs? We'd prefer to store the data as parquet since the compression is so efficient compared to CSV.

Versioning of property changes

Are property level changes allowed with respect to ensuring backwards and forwards compatibility? There are multiple types of changes that could break the backwards compatibility requirement, for example:

  • changing the maxLength on a string attribute, by increasing from 250 to 300 could break an application on an older version of the schema expecting a maxLength of 250 and receiving 300 from an application running on a newer schema that accepts a maxLength of 300
    {
        "name": "address1_line1",
        "dataType": "String",
        "displayName": "Address 1: Street 1",
        "description": "Type the first line of the primary address.",
        "semanticDomain": {
            "semanticType": "Location.Address.Street",
            "maxLength": 250,
            "inherited": true
        }
    }
  • adding a new value to a constrainedList could break an old application/schema not expecting an new schema with an additional value of 5/Secondary
    {
        "name": "address1_addresstypecode",
        "dataType": "String",
        "displayName": "Address 1: Address Type",
        "description": "Select the primary address type.",
        "semanticDomain": {
            "semanticType": "Measurement.OptionCode",
            "inherited": true,
            "constrainedList": {
                "name": "account_address1_addresstypecode",
                "displayName": "Address 1: Address Type",
                "description": "Type of address for address 1, such as billing, shipping, or primary address.",
                "items": [
                    {
                        "value": 1,
                        "displayName": "Bill To"
                    },
                    {
                        "value": 2,
                        "displayName": "Ship To"
                    },
                    {
                        "value": 3,
                        "displayName": "Primary"
                    },
                    {
                        "value": 4,
                        "displayName": "Other"
                    }
                ]
            }
        }
    }

Specification Document

Understanding the data model is extensible, but is there an official document that describes the modeling language? It would be nice to understand some of the following...

  • Is the model purely based on JSON or can it be other formats (XML, CSV, etc.)
  • Primitive data types
  • Defined data types. For example, what is a LookupType or MemoType and what elements are they built upon?
  • Model properties link to other models
  • How to structure your own model (new and one extending a defined model)
  • How to create your own CDM Folder and related contents to include.
  • What data elements are required, how do you show that are required?
  • What data elements are nullable, how do you show they can be missing or their value can be null?
  • Explanation of trait information. is.dataFormat, is.dataFormat.big, means.content.text.HTML, means.identity.entityId, etc.

Link between schemaDocuments and model.json

Is there a way to generate the model.json from the set of schemaDocuments? The model.json schema seems like it has less definition but is used for Data Lake.

Ultimately, I'd like to be able to create a set of classes in C# as well as the Dynamics and Power BI implementations so looking in to whether it is based to define extensions in the model.json format or the full schema (my preference at the moment).

Thanks!

[question] About guidelines for reusing the existing models

We are in process of creating a complete domain models and relationships for Strategic Sourcing. In simple terms, Sourcing (Direct and Indirect) is a process of procurement of goods and services from a Supplier(s).

  1. Is there any guidelines on what/how models can/should be reused? For example, in Sourcing domain there is concept of Contract (between supplier and the org). The Services module in the CRM also has a model for Contract. Can/should the Services Contract be used/extended by the Sourcing module, given that Sourcing module does not sit inside Services (infact there is no relationship between Sourcing and Services).

  2. What is the difference between Solutions and Accelerators? Eventually the sourcing module will be provided as CRM solution, so what is the correct place for it (Solution or Accelerator)?
    @JeffBern

GoalMetric doesn't have a value

I was considering using the CDM for some metric analysis and when I look through it I think that it makes sense to have the Goal entity which represents something a team or user is working toward, and the GoalMetric which represents the metrics that are used to track towards the goal. However shouldn't the GoalMetric have a field for value or current value? For example if you take the example of a sales team that has a goal of a certain number of conversions of some type you could set up the goal, then the GoalMetric would be "conversions" which would hold some value, but I don't see anywhere in that entity where the current number of conversions would be. If that make sense I'm happy to submit a PR, let me know if I'm missing something. Thanks!

Which license applies to cdm.json files?

The legal notice states the following:

Microsoft and any contributors grant you a license to the Microsoft documentation and other content in this repository under the Creative Commons Attribution 4.0 International Public License, see the LICENSE file, and grant you a license to any code in the repository under the MIT License, see the LICENSE-CODE file.

What license do the cdm.json files, e.g. Contact.cdm.json, fall under, the CCA license or MIT?

Attributes with no meta data

In the Sales > Orders schema I found some attributes that do not have any metadata. Is an oversight or be design? If the latter how common will this be?

Microservice Patterns

Are there any docs related to microservice patterns using CDM? It would be nice to see how to possibly use with something like new ASP.NET Core gRPC.

Possibly related... There is an open issue related to contracts and possibly generating client/server code based on interfaces/classes.

[question] How are property instances related to products? Or, how do I know that the color of my grill is blue?

If my Product is a bbq grill, and the property is color, and the property instance is blue, how is everything related?

  • I have product as Grillco BBQ grill model 1234.
  • I have property as color.
  • I have property association linking the product and property.
  • I have property instance as blue.

How do I link the property instance with the product?

Am I understanding the CDM correctly? Should I be storing this data differently?

Manufacturing Related Entities

Understanding you could build a custom model for Manufacturing using other entities, is there any work going on to build a model with entities specific to Manufacturing? This would include things like work orders, equipment, testing (UUT - unit under test), inventory quantities, etc.

This would be a big effort when using with IoT and Digital Twin concepts.

I noticed there were Industry Accelerators. There seems like there would be one for Manufacturing.

Best Practice for reference fields

When looking over the schema I have found a number of fields which have as their data type, a reference to a field on another entity. Is there an established BP for how to cope with this, if the goal is to retrieve the data type for that field?

Load schema locally do not display attributes

Steps to reproduce:

  1. Clone CDM git repo.
  2. Click on CDM\docs\experimental\index.html
  3. Click on Load from files... button and select the CDM\schemaDocuments folder

The left pane is populated with the classes but is not populated the same way as the "load from GitHub!" button. The left pane do not show any attributes or other schema information.

[Question] Could someone please point me to the corresponding maven repository

Really, appreciate the effort that has been put into this initiative. It's wonderful and got lots of potential to become a silver bullet for metadata management in distributed systems.
Currently, we are trying to use CDM as part of our metadata management initiative. Could someone please point me to the maven repository?
Thank you.

Missing Entities

Inventory related Entities like Inventory On Hand, Purchase Orders, etc?

Seems like these are some pretty core entities that are missing.

Project missing from manifest

\core\applicationCommon\foundationCommon\crmCommon\projectCommon and its sub entites is missing from the "crmCommon.manifest.cdm" or one of its subordinates. This prevents the entity group from displaying on the CDM Entity Navigatior app.

Edit: I was refering the wrong manifest

C# implementation?

Is there a C# implementation of the spec? We are creating an ecommerce system and I think it would be helpful to build off of CDM. With PowerApps and Dynamics and other MS properties using or going to use CDM, I find it strange that I can't find an implementation of the schema. Are their plans to build one?

[question] Is it possible to provision CDS with json schemas

Hi, we're adopting some of our internal business system integration using the CDS and thus strives to match CDM. However we will need to extend some entities and create completely new ones.

We can manage the modeling and visualization very well. What I don't understand is how to provision CDS using eg a CI/CD flow from the json schema models. We will use a dev, QA, prod scenario for our CDS.

Is there any way of doing this?

Cheers,
Mario

Custom entities in CDM

I couldn't find relevant documentation regarding this so posting my questions here:

  • How do custom entities work and where are they stored themselves if the corresponding data can be stored in ADLS in CDM folders?
  • Can custom entities inherit from standard entities?
  • Does batching apply to entity operations? If so, what should be the size of the batch, and what should be the frequency of batching?
  • Can the datatype be specified in the schema?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.