Description

From #72, a record should be able to be created using custom controller. I have found that the reference is sometimes not set after a record is created for a custom controller.

Technical Information

This happens when a provided ID has an existing record. I believe the root cause is the check for an existing record is always done on the provided ID, but not the custom controller when provider. See here.

Waves is an aspiring blockchain solution, presenting a real alternative to Ethereum DeFi infrastructure with its Neutrino based services. IDX should support Waves due to extremely low transaction fees, popularity, and potential to become a real alternative to Ethereum infrastructure.

Describe the bug
Executing idx.setDefaults times out after about 1 minute in browser, and throw multiple error in the ceramic daemon console.

To Reproduce
Steps to reproduce the behavior:

1. Setup a project with the latest IDX connected to a Ceramic node running version 1.0.0.
2. Create a new IDX account.
3. Run idx.setDefaults() with test data of your choice. In my case I was setting the defaults of a custom record on the IDX account.
4. Open up browser console to see errors similar to below

Expected behavior
Defaults should be set like how it was previously in Ceramic 0.22.1

Screenshots

Additional context
Ceramic daemon logs the following after attempting to set defaults

TimeoutError: request timed out

Error: Invalid signature for signed commit. Error: Failed to resolve did:3:kjzl6cwe1jw14a0ajsnmusud2rmptpi2u7p8vqxsgvqm433o

When a command that has been depreciated due to it's migration to the glaze CLI we should output a message notifying the user & prompting them to install (or update) the Glaze CLI

(node:32812) UnhandledPromiseRejectionWarning: null
(Use `node --trace-warnings ...` to show where the warning was created)
(node:32812) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:32812) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Code looks like:

await idx.createDefinition({
        name: "profile",
        schema: schemas.BasicProfile,
      }),

Where schemas.BasicProfile is "ceramic://bagcqcera4jj6ienfkpazjc4dczxnayxqy34fgt6zagx7ncxht4wnva"

Fwiw - the definition does come back seemingly ok -
"profile":"ceramic://bagcqceraynhpmkz6vtermrsyvbqx2sow2sv3lfkj4smhg3e32iozak4nyeea"

I'm running:

identity-wallet: 2.0.0-alpha.5
@ceramicnetwork/ceramic-http-client: 0.5.12
@ceramicnetwork/ceramic-cli: 0.6.0, local commit: 7c75fff1917239086526763eee5303e33efb5482 (pulled develop today)
js-idx on commit 1a2b66159f3375974177f70662e90324878ae9a8
js-idx-schemas on commit 99f671f8a3f160e4f7b59b716b26ebd697d30185

This includes anything that is NOT covered by functions already in the Glaze CLI.

All stories under this are to Evaluate whether the feature is needed in the Glaze CLI or if it is already covered, and then to merge the functionality (and any underlying tests/utils) into the hybrid CLI.

I'm replicating some code in your tests on a nodejs server.
await alice.set('test:profile', { name: 'Alice' }) throws the following error:

(node:73004) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'change' of null
    at mutation (/Users/jonathanschwartz/Documents/owl/streams/js-idx/dist/idx.cjs.development.js:161:38)
    at /Users/jonathanschwartz/Documents/owl/streams/js-idx/dist/idx.cjs.development.js:127:36
    at _catch (/Users/jonathanschwartz/Documents/owl/streams/js-idx/dist/idx.cjs.development.js:88:16)
    at Object.run (/Users/jonathanschwartz/Documents/owl/streams/js-idx/dist/idx.cjs.development.js:126:24)
    at DoctypeProxy._next (/Users/jonathanschwartz/Documents/owl/streams/js-idx/dist/idx.cjs.development.js:219:17)
    at /Users/jonathanschwartz/Documents/owl/streams/js-idx/dist/idx.cjs.development.js:193:18
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

my ceramicApi instance is the @ceramicnetwork/[email protected] and [email protected]. I'm running a ceramic daemon commit e34b1c5eb783a2df0e82170810c4df35c5196f62 locally on port 7007.

When I do some logging, it looks like I end up in this conditional block

idx.getMultiple(name: string, dids: Array<string>): Promise<Record<string,unknown>>

Arguments:

• name - the alias or IndexKey
• dids - an array of DIDs or Blockchain AccountIds

Returns: Promise<Record<string,unknown>>

A map from the strings in the dids array to their corresponding records.

Implementation

If the array contains any caip10 accountIds we first make a multiquery to resolve the associated DID from these accounts. Pseudocode to create a multiquery for a caip10-link (the full multiquery is just an array of these):

const genesis = Caip10Link.makeGenesis(ceramic, accountId)
const streamId = await StreamID.fromGenesis('caip10-link', genesis)
const query = {
  streamId,
  genesis
}

Now create a multiquery for all DIDs to load the specific definition on each of them. Pseudocode to create a multiquery for a record (the full multiquery is just an array of these):

const genesis = TileDocument.makeGenesis({}, null, {
  controller: [did],
  family: 'IDX'
})
const streamId = await StreamID.fromGenesis('tile', genesis)
const query = {
  streamId,
  genesis,
  paths: ['<definitionId>']
}

The above multiquery will make sure we load both the IDX document as well as the record document by specifying the paths option.

idx.getMultiple(name: string, dids: Array<string>): Promise<Record<string,unknown>>

Arguments:

• name - the alias or IndexKey
• dids - an array of DIDs or Blockchain AccountIds

Returns: Promise<Record<string,unknown>>

A map from the strings in the dids array to their corresponding records.

Implementation

If the array contains any caip10 accountIds we first make a multiquery to resolve the associated DID from these accounts. Pseudocode to create a multiquery for a caip10-link (the full multiquery is just an array of these):

const genesis = Caip10Link.makeGenesis(ceramic, accountId)
const streamId = await StreamID.fromGenesis('caip10-link', genesis)
const query = {
  streamId,
  genesis
}

Now create a multiquery for all DIDs to load the specific definition on each of them. Pseudocode to create a multiquery for a record (the full multiquery is just an array of these):

const genesis = TileDocument.makeGenesis({}, null, {
  controller: [did],
  family: 'IDX'
})
const streamId = await StreamID.fromGenesis('tile', genesis)
const query = {
  streamId,
  genesis,
  paths: ['<definitionId>']
}

The above multiquery will make sure we load both the IDX document as well as the record document by specifying the paths option.

This line throws an error https://github.com/ceramicstudio/js-idx/blob/master/src/index.ts#L234 when attempting to getDefinition.

My schemas as past in the constructor don't contain a .Definition property (which is confusing to me, because I don't see anywehre in the js-idx-schemas repo that would apply that property?

Should this check be against this._definitions instead? Or am I screwing something up (most likely story)?

One should update the schema for increased security enforced by Ajv

Tron is a widely popular and Ethereum compatible blockchain solution. IDX should support Tron due to lower transaction fees (then Ethereum and wide adoption)

Currently when running glaze did:inspect only information about the definitions are included in the output. However, in most cases when inspecting the index learning the streamId of the users record is important.

Current output

glaze did:inspect -c 'https://gateway.ceramic.network' --did 'did:3:kjzl6cwe1jw146xrrmtg4hn5suq4b3lopaxt7qmnr52g6s8uaoj47dfnkial7jv'
✔ Index stream loaded
  ✔ kjzl6cwe1jw14a50gupo0d433e9ojgmj9rd9ejxkc8vq6lw0fznsoohwzmejqs8
    ✔ Definition: 3ID Keychain (Key data for 3ID)
    ✔ Schema: ceramic://k3y52l7qbv1frxiodfo6f25wocb8zz60ywqw4sqcprs26qx1qx467l4ybxplybvgg
  ✔ kjzl6cwe1jw145cjbeko9kil8g9bxszjhyde21ob8epxuxkaon1izyqsu8wgcic
    ✔ Definition: Basic Profile (Basic profile information for a DID)
    ✔ Schema: ceramic://k3y52l7qbv1frxt706gqfzmq6cbqdkptzk8uudaryhlkf6ly9vx21hqu4r6k1jqio
  ✔ kjzl6cwe1jw146zfmqa10a5x1vry6au3t362p44uttz4l0k4hi88o41zplhmxnf
    ✔ Definition: Also Known As (Also Known As is a data set that stores a list of accounts that are publicly linked to the users DID)
    ✔ Schema: ceramic://k3y52l7qbv1fryojt8n8cw2k04p9wp67ly59iwqs65dejso566fij5wsdrb871yio
  ✔ kjzl6cwe1jw149z4rvwzi56mjjukafta30kojzktd9dsrgqdgz4wlnceu59f95f
    ✔ Definition: Crypto Accounts (Crypto accounts linked to your DID)
    ✔ Schema: ceramic://k3y52l7qbv1frypussjburqg4fykyyycfu0p9znc75lv2t5cg4xaslhagkd7h7mkg

We could simply add another property like this:

glaze did:inspect -c 'https://gateway.ceramic.network' --did 'did:3:kjzl6cwe1jw146xrrmtg4hn5suq4b3lopaxt7qmnr52g6s8uaoj47dfnkial7jv'
✔ Index stream loaded
  ✔ kjzl6cwe1jw14a50gupo0d433e9ojgmj9rd9ejxkc8vq6lw0fznsoohwzmejqs8
    ✔ Definition: 3ID Keychain (Key data for 3ID)
    ✔ Schema: ceramic://k3y52l7qbv1frxiodfo6f25wocb8zz60ywqw4sqcprs26qx1qx467l4ybxplybvgg
    ✔ Record: ceramic://k...
  ✔ kjzl6cwe1jw145cjbeko9kil8g9bxszjhyde21ob8epxuxkaon1izyqsu8wgcic
    ✔ Definition: Basic Profile (Basic profile information for a DID)
    ✔ Schema: ceramic://k3y52l7qbv1frxt706gqfzmq6cbqdkptzk8uudaryhlkf6ly9vx21hqu4r6k1jqio
    ✔ Record: ceramic://k...
  ✔ kjzl6cwe1jw146zfmqa10a5x1vry6au3t362p44uttz4l0k4hi88o41zplhmxnf
    ✔ Definition: Also Known As (Also Known As is a data set that stores a list of accounts that are publicly linked to the users DID)
    ✔ Schema: ceramic://k3y52l7qbv1fryojt8n8cw2k04p9wp67ly59iwqs65dejso566fij5wsdrb871yio
    ✔ Record: ceramic://k...
  ✔ kjzl6cwe1jw149z4rvwzi56mjjukafta30kojzktd9dsrgqdgz4wlnceu59f95f
    ✔ Definition: Crypto Accounts (Crypto accounts linked to your DID)
    ✔ Schema: ceramic://k3y52l7qbv1frypussjburqg4fykyyycfu0p9znc75lv2t5cg4xaslhagkd7h7mkg
    ✔ Record: ceramic://k...

Description

It seems like the bundled code does not work.

Technical Information

An error is thrown when trying to import DataModel:

import { DataModel } from '@glazed/datamodel'

Throws this error

|   getDefinitionID(e) {
>     return r(this, n).definitions[e] ?? null;
|   }
|

To replicate:

1. Install dependency

npm install @glazed/datamodel

2. Import package

import { DataModel } from '@glazed/datamodel'

3. Run project

npm start

This error gets throw when idx.set a custom definition that has been published to my local ceramic daemon.

To publish the alias, I used: createDefinition from idx-tools v 0.4.2

Features to be Evaluated:

• Show
• Watch
• State
• Commits
• Schema
• Pin
• Create (specifically, is this different enough to exist alongside Glaze CLI)
• Update (see above)
• Show (see above)

When trying to query the IDX index of a DID using getIndex that is not the currently authenticated one, I always receive no content.

It appears this will always be the case due to _getIDXDoc always returning null here due to it calling _createIDXDoc which appears to always return a new stream that has no content and no metadata.schema.

Am I interpreting this correctly? Or is there maybe another way to query the index of an unauthenticated user?

Hi all,

I've just run the quick start for IDx developers (here) and got an error after the step 1.
Here are the commands I've run:

npm install -g @ceramicnetwork/cli @ceramicstudio/idx-cli
ceramic daemon
idx bootstrap

And here is the error I got:

ReferenceError: TextDecoder is not defined

This issue is probably a classic one. Sorry for the noise if the issue is on my side.

The goal is to have 50% coverage at a minimum.

Example:

glaze did:get -c 'https://gateway.ceramic.network' --did 'did:3:k...l7jv' identity-accounts-crypto a
✖ HTTP request to 'https://gateway.ceramic.network/api/v0/pins/kjzl6cwe1jw14amy1imkbql1d61u00q9cbvhy5c3jtv3nz552fshl013530rauh' failed with status 'Bad Request': {"error":"Method not supported by read only Ceramic Gateway"}

Description

Provide a 2-3 sentence overview of the work to be done.

Technical Information

Provide an explanation of the technical work to be done.

The documentation has a // optional comment by the definition description. The schema in idx-tools has it marked as required.

This DID (did:3:kjzl6cwe1jw147ko5yhnbxk53v2sc8wjrlgirai2tfujdwyw7vjvsej1zbs60gw) has accounts in the AlsoKnownAs schema as undefined, but in the IDX type definition it should always be an array. The schema and the type definition should match. At this point, it would probably make more sense to change the type definition rather than the schema so migration doesn't need to happen.

Current logic attempts to pin any index stream getting created, which doesn't work when connecting to a read-only gateway.
This logic could likely be changed to only attempt to pin when the owner of the index stream creates it.

I'm not sure if this is something I'm doing wrong, or if there's a bug somewhere...

Basically I have a test suite that cleans up after itself.

At the beginning of the tests, we do something that looks like this:

    await idx.set("permissions", { 
      permissions: {}
    });

At the end of the tests I do something like this:

await this._idx.remove("permissions");
const resetPerms = await this._idx.get('permissions')
// returns null as expected

No errors get thrown.

Yet after setting anything on permissions alias, the old state returns:

await idx.set("permissions", { 
   permissions: {}
});

const perms = await idx.get('permissions')
/*
returns old state from last round of tests:
permissions: {
   qmz123: ['*'],
   qmz456: ['READ', 'WRITE']
}
*/

Even manually idx.set('permissions', null) won't do the trick for me either - seems to just not do anything.

PS:

I know it's a bit weird that I have a permissions object as a nested layer inside the permissions collection in the IDX root - the idea here is that (eventually) I'll replace permissions with some DocID, so that if I want to add/remove permissions from this list, I can just update that associated doc without touching the IDX root. I think this will just make the migration a bit smoother and help avoid breaking changes.