integreat-io / integreat Goto Github PK

Mapping should use default values defined on types, both when mapping from and to a source.

This involves removing the defaultTo attribute on attributes mappers of source items, as it makes no sense to provide a different default value depending on which way we map. The effect of this differentiation, would lead to unpredictability, as a missing attributed mapped from to source and back again would get a different default value than a missing attribute mapped only to the source.

The attributes would be defined like number(8), in a function kind of format.

This involves:

• Retrieving the lastSyncedAt metadata for the source
• Passing the updatedAfter param to GET on retrieval, so that the source may retrieve only items updated after this date
• Filtering out items with updatedAt dates before the last synced date, as a last measurement

The not mapper will simply turn a true into a false, and opposite. This will be a two-way mapper, and reverse the operation on the way back to the source.

The method getAuthHeaders returns valid header key/value pairs in an object, ready for including with a http request.

In many cases this object could also serve as an object in other cases, e.g. as an option object for a mediating request module, but to make the difference clearer, there should also be a getAuthObject to allow these to be different when that makes sense.

Params in path should be required, params in query strings should be optional. Or should there be a way to specify which one is required?

This is necessary for retrieving new items since last sync.

Open question: Where to store this date?

Write an adapter that will return dummy data of the right type. Allows for testing before the real sources are hooked up.

Simplify base functionality to focus on access to sources only.

A source may be reached by dispatching an action, and the action is routed to the right source by its type.

const action = {type: 'GET', payload: {id: 'ent1', type: 'entry'}}
const ret = await great.dispatch(action)

With this, there's no longer any need for the connect interface. Other fronts, like REST or GraphQL, may just use the dispatch interface directly.

Consider using an asterisk item to signal a catch all source. Possible alias catchAll flag on source.

Use RxJS stream to store items in database, to control the number of concurrent requests.

The json adapter will currently report every status from http requests except 200 as an error. This is obviously wrong and should be handled correctly.

Implement getExisting on endpoint. This will reference the endpoint to get from, and will be handled by source.send.

First ideas:

// Connect with item type
const entries = great.connect('entry')

// Get one item of connected type
entries.get('ent1').then((item) => { ... })

// Get all items of connected type
entries.forEach((item) => { ... })

// Get items through filter
entries.where('age').gt(18).or('parentsConsent').eq(true).forEach((item) => { ... })

// Set item of connected type
entries.set('ent1', itemObject)

// Remove item of connected type
entries.remove('ent1')

Requires a generalized way of specifying filters to adapters.

Type definitions should describe what data to exchange with the sources. Map definitions describe how to convert from sources to these types and back again.

Example:

{
  id: 'entry',
  source: 'entriesDb',
  attributes: {
    'name': {
      type: 'string',
      default: null
    }
  },
  relationships: {
    author: {
      type: 'person',
      default: 'admin'
    }
  }
}

This is especially useful when mapping back to a source, otherwise there would not be a way to tell if source is expecting an array when the data has only one item.

Suggestion:

path: 'data.items[]'

Catch rejections and log error.

Repository for running examples and for testing the different packages together.

When inserting an item and the database returns a conflict error, the module should warn of this, but continue syncing.

When an action with an id is queued, the id is used as a job id.

An id may be specified in a schedule def, and will be passed on to the resulting action.

Possible syntax:

entries.where('age').gt(18).or('parentsConsent').eq(true).then((items) => { ... })

The error message states that we're trying to access type on null. Everything else seems to be working as expected.

This involves providing an array of types to source.retrieve.

The remaining question here, is how to specify what makes an item expired.

E.g.: items[author='admin'].data and sections[1:4].articles[].

When mapping from or to a source, include all attributes and relationships with default values in type definition.

This is done in actions, and should be the default behavior for GET, GET_ALL, and GET_MANY. To get only values defined in the mapper, pass inn mappedValuesOnly: true in the action payload.

Log error and continue.

The action for getting one item should be named GET_ONE, and the action for getting more items GET.

GET will get from endpoint get, and GET_ONE from getone by default. Both can be overridden with the endpoint property, which specifies the endpoint id to #use.

Both get actions will pass on the params property to the uri generator, along with id and type.

This is done to distribute responsibilities in a more clear way, as the first architecture gave a very wide range of tasks to the integreat class.

The sync module will access sources by dispatching integreat actions, and will place all updates in a queue.

Field definitions will include types ending in square brackets to indicate when they are transformed from an array. E.g. integer[].

Before scheduling jobs, all scheduled jobs in queue should be flushed. This is to prevent duplication of scheduled jobs and jobs not in the current schedule definition to become unmanaged.

Use queue.flushScheduled().

Build as a separat module.

Signature:

(options) => ({
  async push (obj) { ... },
  subscribe (handler) { ...; return subscriptionHandle },
  unsubscribe (subsciptionHandle) { ... }
})

Will remove <type>: from the start of ids coming from the source, and add it back going to the source.

For catch-all mappers, attributes are simply stored as-is going to the source.

Instead, they should be formatted according to their type. This is especially important for dates, as they should be timestamps – not ISO strings.

This should be a simple memory queue for testing purposes, and don't justify an extra dependency.

Should trigger an dispatch event on each dispatch. Listeners will receive the action.

After the dispatch, a dispatched event is triggered. In addition to the action, listeners also receives the result returned from the action handler.