how do i get from the operation replaceAsForeignKey to is.linkedEntity.identifier in a document about cdm HOT 16 CLOSED

hello @dgcaron,

You're schema now is in what we call "Logical" form. In logical form you can have references to another entities that are added as relationships once you resolve them. Resolution is the process in which the CDM SDK transforms you logical definitions into resolved/physical definitions.

In this line we save the resolved document into disk, you can take a look there :)

from cdm.

ok, i get the difference between logical and resolved now. so i need to save the logical form first and then resolve it using the methods in the example. two things:

1. the example doesn't save the manifest, i need to resolve this too with same options?
2. the is.linkedEntity.identifier isn't added by the example

{
                        "name": "address",
                        "type": "attributeDefinition",
                        "parent": "referenceOnly_Person/attributeContext/referenceOnly_Person",
                        "definition": "resolvedFrom/Person/hasAttributes/address",
                        "contents": [
                            {
                                "name": "projection",
                                "type": "projection",
                                "parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address",
                                "definition": "resolvedFrom/Person/hasAttributes/address/entity/projection",
                                "contents": [
                                    {
                                        "name": "source",
                                        "type": "source",
                                        "parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/projection",
                                        "contents": [
                                            {
                                                "name": "Address",
                                                "type": "entity",
                                                "parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/projection/source",
                                                "definition": "resolvedFrom/Address"
                                            }
                                        ]
                                    }
                                ]
                            },
                            {
                                "name": "_generatedAttributeSet",
                                "type": "generatedSet",
                                "parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address",
                                "contents": [
                                    {
                                        "name": "operation/index1/operationReplaceAsForeignKey",
                                        "type": "operationReplaceAsForeignKey",
                                        "parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/_generatedAttributeSet",
                                        "contents": [
                                            {
                                                "name": "_foreignKey",
                                                "type": "addedAttributeIdentity",
                                                "parent": "referenceOnly_Person/attributeContext/referenceOnly_Person/address/_generatedAttributeSet/operation/index1/operationReplaceAsForeignKey",
                                                "contents": [
                                                    "referenceOnly_Person/hasAttributes/addressFK"
                                                ]
                                            }
                                        ]
                                    }
                                ]
                            }
                        ]
                    },

is that the expected behaviour?

from cdm.

ok, so instead of resolving individual entities resolving the manifest actually yielded the expected result

const resolvedFolder = await this.corpus.fetchObjectAsync<cdm.types.CdmFolderDefinition>("local:/resolved/");

var manifestName = "default.manifest.cdm.json";
var options: cdm.types.copyOptions = {};
var resolvedManifest = await this.manifest.createResolvedManifestAsync(
      manifestName,
      "resolved/{n}.cdm.json",
      undefined
);

resolvedFolder.documents?.push(resolvedManifest);
var saved = await resolvedManifest.saveAsAsync(manifestName, true, options);

the attribute now looks like this

 {
  "name": "FK_attr1",
  "appliedTraits": [
    "is.dataFormat.character",
    "is.dataFormat.big",
    "is.dataFormat.array",
    "is.dataFormat.guid",
    "means.identity.entityId",
    {
      "traitReference": "is.linkedEntity.identifier",
      "arguments": [
        {
          "entityReference": {
            "entityShape": "entitySet",
            "constantValues": [
              ["/entity2.cdm.json/entity2", "attr1", "attr3_entity2"]
            ]
          }
        }
      ]
    }
  ],
  "attributeContext": "entity1/attributeContext/entity1/attr3/_generatedAttributeSet/operation/index1/operationReplaceAsForeignKey/_foreignKey",
  "dataFormat": "Guid"
}

from cdm.

what i don't fully understand is that it defaults to a GUID. the field being referenced is a int64.. do i need to set this explicitly somewhere?

---- edit ----
for anyone stumbling upon this thread you need to explicitly set the type when you create the attribute and "entityId" resolves to a GUID.

const foreignKeyAttr = new cdm.types.CdmTypeAttributeDefinition(ctx, fkAttrName);
foreignKeyAttr.dataType = new cdm.types.CdmDataTypeReference(ctx, "string", true); // <--- replace "string" for the correct type being referenced

--- endedit ---

also when i do the following with the resolved manifest:

if (manifest.relationships?.length > 0) {
  this.logger.write(`Manifest contains relationships directly`, LogLevel.Info);
  return;
}

try {
   this.logger.write(`Calculating relationships`, LogLevel.Info);
   await corpus.calculateEntityGraphAsync(manifest);
} catch (e) {
    this.logger.write(e.message, LogLevel.Error);
    this.logger.write(JSON.stringify(e), LogLevel.Error);
    return;
}

// this fails as there are no relationships
 expect(manifest?.relationships.allItems).toHaveLength(1);

from cdm.

the issue that remains is that when i read the resolved manifest the attributes and relationships remain empty, is there an additional call i need to do to construct theses values?

from cdm.

Hi, is the problem you saved the resolved manifest and the file looks good but when reading it, it has empty relationship and attributes? Can you post the code how you save (if it's not the same as above code) and read the manifest?

from cdm.

yes, that is correct. i store the files using the createResolvedManifestAsync method as described above.

this is how i read the resolved manifest (simplified):

var objectPath = "default.manifest.cdm.json";
var manifest = await corpus.fetchObjectAsync<cdm.types.CdmManifestDefinition>(`local:/${objectPath}`);

for await (const entity of manifest?.entities.allItems.map((e) => corpus.fetchObjectAsync<cdm.types.CdmEntityDefinition>(e.atCorpusPath, manifest))) {
// use the entity
// the entity has a name and such but when i want to check the attributes or relationships they remain empty
}

from cdm.

It seems that the entity fetched is not properly defined. There are several things worth checking.

1. the resolved manifest file
   • please check if the resolved manifest's entities list has the entityPath defined properly, it should point to the resolved entity.
2. the resolved entity file
   • please check the resolved entity file has attributes.

You can also try modify the file manually and use fetchObjectAsync to check whether it works properly or not. From the error you get, it is most likely to be the file itself doesn't have attributes.

from cdm.

It seems that the entity fetched is not properly defined. There are several things worth checking.

1. the resolved manifest file
   • please check if the resolved manifest's entities list has the entityPath defined properly, it should point to the resolved entity.
2. the resolved entity file
   • please check the resolved entity file has attributes.

You can also try modify the file manually and use fetchObjectAsync to check whether it works properly or not. From the error you get, it is most likely to be the file itself doesn't have attributes.

from cdm.

i have tried what you suggested and i might be overlooking something here.. i have attached the manifest and the (very) simple example json files

resolved.cdm.zip

maybe someone can spot the problem here?

from cdm.

ok, the issue with the attributes was caused by calling

corpus.fetchObjectAsync<cdm.types.CdmEntityDefinition>(e.atCorpusPath, manifest)))

instead of

// use the entityPath and not the corpuspath to fetch the entity
corpus.fetchObjectAsync<cdm.types.CdmEntityDefinition>(e.entityPath, manifest)))

so i can read the resolved entities now, however the relationships remain empty it that expected?

from cdm.

Hi @dgcaron,

When you create the resolved manifest that should make the correct calls to put the relationships in that newly created resolved manifest.

You have some other code where you call CalculateEntityGraphAsync, if that is done on the resolved manifest, it should not be necessary as the resolved manifest should have already contained this information. If you are doing this on the unresolved manifest, you will also need to call the api to add the relationships to the manifest like this:

// generate the relationship information
await corpus.calculateEntityGraphAsync(manifest);
// add the relationship information to the manfiest
await manifest.PopulateManifestRelationshipsAsync(CdmRelationshipDiscoveryStyle.Exclusive);

Again, these apis are called inside of CreateResolvedManifestAsync so the resolved manifest that is created should already contain the relationships. If the relationships still aren't created, then there may be an issue with the logical schema files. If that is the case, can you share the unresolved schema files? The zip you shared before only contained the resolved entity files

from cdm.

logical-replaceasfk.cdm.zip

Yes, Sure! thank you for looking into this

from cdm.

It looks like there may be an issue with generating relationships on resolved manifests and entities when using projections (such as calling CreateResolvedManifestAsync, I am still investigating that. Looking at the files you sent, this code should work for generating relationships on the unresolved entity/manifest:

// generates the relationships from entities in the manifest
await corpus.CalculateEntityGraphAsync(manifest);
// add any relationships that are relevant to the entities found in this manifest
await manifest.PopulateManifestRelationshipsAsync();

from cdm.

Hi! Thank you for taking the time to look into this. I'll give these pointers a go and let you know what the result is.

from cdm.

Closing this issue, please open a new one if there is any new issue.

from cdm.