Comments (16)
hello @dgcaron,
You're schema now is in what we call "Logical" form. In logical form you can have references to another entities that are added as relationships once you resolve
them. Resolution is the process in which the CDM SDK transforms you logical definitions into resolved/physical definitions.
In this line we save the resolved document into disk, you can take a look there :)
from cdm.
ok, i get the difference between logical and resolved now. so i need to save the logical form first and then resolve it using the methods in the example. two things:
- the example doesn't save the manifest, i need to resolve this too with same options?
- the is.linkedEntity.identifier isn't added by the example
{
"name": "address",
"type": "attributeDefinition",
"parent": "referenceOnly_Person/attributeContext/referenceOnly_Person",
"definition": "resolvedFrom/Person/hasAttributes/address",
"contents": [
{
"name": "projection",
"type": "projection",
"parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address",
"definition": "resolvedFrom/Person/hasAttributes/address/entity/projection",
"contents": [
{
"name": "source",
"type": "source",
"parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/projection",
"contents": [
{
"name": "Address",
"type": "entity",
"parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/projection/source",
"definition": "resolvedFrom/Address"
}
]
}
]
},
{
"name": "_generatedAttributeSet",
"type": "generatedSet",
"parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address",
"contents": [
{
"name": "operation/index1/operationReplaceAsForeignKey",
"type": "operationReplaceAsForeignKey",
"parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/_generatedAttributeSet",
"contents": [
{
"name": "_foreignKey",
"type": "addedAttributeIdentity",
"parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/_generatedAttributeSet/operation/index1/operationReplaceAsForeignKey",
"contents": [
"referenceOnly_Person/hasAttributes/addressFK"
]
}
]
}
]
}
]
},
is that the expected behaviour?
from cdm.
ok, so instead of resolving individual entities resolving the manifest actually yielded the expected result
const resolvedFolder = await this.corpus.fetchObjectAsync<cdm.types.CdmFolderDefinition>("local:/resolved/");
var manifestName = "default.manifest.cdm.json";
var options: cdm.types.copyOptions = {};
var resolvedManifest = await this.manifest.createResolvedManifestAsync(
manifestName,
"resolved/{n}.cdm.json",
undefined
);
resolvedFolder.documents?.push(resolvedManifest);
var saved = await resolvedManifest.saveAsAsync(manifestName, true, options);
the attribute now looks like this
{
"name": "FK_attr1",
"appliedTraits": [
"is.dataFormat.character",
"is.dataFormat.big",
"is.dataFormat.array",
"is.dataFormat.guid",
"means.identity.entityId",
{
"traitReference": "is.linkedEntity.identifier",
"arguments": [
{
"entityReference": {
"entityShape": "entitySet",
"constantValues": [
["/entity2.cdm.json/entity2", "attr1", "attr3_entity2"]
]
}
}
]
}
],
"attributeContext": "entity1/attributeContext/entity1/attr3/_generatedAttributeSet/operation/index1/operationReplaceAsForeignKey/_foreignKey",
"dataFormat": "Guid"
}
from cdm.
what i don't fully understand is that it defaults to a GUID. the field being referenced is a int64.. do i need to set this explicitly somewhere?
---- edit ----
for anyone stumbling upon this thread you need to explicitly set the type when you create the attribute and "entityId" resolves to a GUID.
const foreignKeyAttr = new cdm.types.CdmTypeAttributeDefinition(ctx, fkAttrName);
foreignKeyAttr.dataType = new cdm.types.CdmDataTypeReference(ctx, "string", true); // <--- replace "string" for the correct type being referenced
--- endedit ---
also when i do the following with the resolved manifest:
if (manifest.relationships?.length > 0) {
this.logger.write(`Manifest contains relationships directly`, LogLevel.Info);
return;
}
try {
this.logger.write(`Calculating relationships`, LogLevel.Info);
await corpus.calculateEntityGraphAsync(manifest);
} catch (e) {
this.logger.write(e.message, LogLevel.Error);
this.logger.write(JSON.stringify(e), LogLevel.Error);
return;
}
// this fails as there are no relationships
expect(manifest?.relationships.allItems).toHaveLength(1);
from cdm.
the issue that remains is that when i read the resolved manifest the attributes and relationships remain empty, is there an additional call i need to do to construct theses values?
from cdm.
Hi, is the problem you saved the resolved manifest and the file looks good but when reading it, it has empty relationship and attributes? Can you post the code how you save (if it's not the same as above code) and read the manifest?
from cdm.
yes, that is correct. i store the files using the createResolvedManifestAsync method as described above.
this is how i read the resolved manifest (simplified):
var objectPath = "default.manifest.cdm.json";
var manifest = await corpus.fetchObjectAsync<cdm.types.CdmManifestDefinition>(`local:/${objectPath}`);
for await (const entity of manifest?.entities.allItems.map((e) => corpus.fetchObjectAsync<cdm.types.CdmEntityDefinition>(e.atCorpusPath, manifest))) {
// use the entity
// the entity has a name and such but when i want to check the attributes or relationships they remain empty
}
from cdm.
It seems that the entity fetched is not properly defined. There are several things worth checking.
- the resolved manifest file
- please check if the resolved manifest's entities list has the entityPath defined properly, it should point to the resolved entity.
- the resolved entity file
- please check the resolved entity file has attributes.
You can also try modify the file manually and use fetchObjectAsync to check whether it works properly or not. From the error you get, it is most likely to be the file itself doesn't have attributes.
from cdm.
It seems that the entity fetched is not properly defined. There are several things worth checking.
- the resolved manifest file
- please check if the resolved manifest's entities list has the entityPath defined properly, it should point to the resolved entity.
- the resolved entity file
- please check the resolved entity file has attributes.
You can also try modify the file manually and use fetchObjectAsync to check whether it works properly or not. From the error you get, it is most likely to be the file itself doesn't have attributes.
from cdm.
i have tried what you suggested and i might be overlooking something here.. i have attached the manifest and the (very) simple example json files
maybe someone can spot the problem here?
from cdm.
ok, the issue with the attributes was caused by calling
corpus.fetchObjectAsync<cdm.types.CdmEntityDefinition>(e.atCorpusPath, manifest)))
instead of
// use the entityPath and not the corpuspath to fetch the entity
corpus.fetchObjectAsync<cdm.types.CdmEntityDefinition>(e.entityPath, manifest)))
so i can read the resolved entities now, however the relationships remain empty it that expected?
from cdm.
Hi @dgcaron,
When you create the resolved manifest that should make the correct calls to put the relationships in that newly created resolved manifest.
You have some other code where you call CalculateEntityGraphAsync, if that is done on the resolved manifest, it should not be necessary as the resolved manifest should have already contained this information. If you are doing this on the unresolved manifest, you will also need to call the api to add the relationships to the manifest like this:
// generate the relationship information
await corpus.calculateEntityGraphAsync(manifest);
// add the relationship information to the manfiest
await manifest.PopulateManifestRelationshipsAsync(CdmRelationshipDiscoveryStyle.Exclusive);
Again, these apis are called inside of CreateResolvedManifestAsync so the resolved manifest that is created should already contain the relationships. If the relationships still aren't created, then there may be an issue with the logical schema files. If that is the case, can you share the unresolved schema files? The zip you shared before only contained the resolved entity files
from cdm.
Yes, Sure! thank you for looking into this
from cdm.
It looks like there may be an issue with generating relationships on resolved manifests and entities when using projections (such as calling CreateResolvedManifestAsync, I am still investigating that. Looking at the files you sent, this code should work for generating relationships on the unresolved entity/manifest:
// generates the relationships from entities in the manifest
await corpus.CalculateEntityGraphAsync(manifest);
// add any relationships that are relevant to the entities found in this manifest
await manifest.PopulateManifestRelationshipsAsync();
from cdm.
Hi! Thank you for taking the time to look into this. I'll give these pointers a go and let you know what the result is.
from cdm.
Closing this issue, please open a new one if there is any new issue.
from cdm.
Related Issues (20)
- Python ADLSAdapter hostname end backslash HOT 2
- Gender fields are out of sync across CDM and should include nonbinary/non-binary HOT 2
- CDM Usage outside Microsoft Apps HOT 1
- Which CDM entity can be used for Product Attribute/Dimension/Batch Attribute from D365? HOT 1
- How to use the python CDM objectmodel HOT 1
- name of standard trait is automatically prefixed with 'cdm:/primitives.cdm.json/' HOT 5
- CDMPoster_a3 has a misspelling HOT 2
- Traits on ... traits?!?
- How to create our own data model in CDM HOT 2
- setting the dataFormat to date doesn't result in the expected defintion HOT 2
- Is there a way to get the CDM model in Excel sheet ? HOT 1
- Search is limited to Entity only, it should also match on attribtues HOT 3
- Converting JSON schema to ER Diagrams HOT 7
- Is the file ../standards.manifest.cdm.json missing?
- Error while loading the .sln file of the CDM Samples HOT 2
- New Life
- Encountering Errors with ADLS Adapter in Azure Functions HOT 3
- (De)serializing model.json of a PowerBI dataflow, some info gets lost HOT 1
- Standard Manifest Names Missing HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cdm.