equinor / data-modelling-tool Goto Github PK

A blueprint should be able to reference another blueprint.

For the time being, this will be done in entrypoint.sh.

• BDD
• Unit Tests
  ++

When creating attributes for a blueprint, we should be able to specify dimensions.

Only keep Radix as CD

• Possible issue; Can only trigger Radix CD from Github commit. The same commit that starts the CI.
  might need some branch trickery

To avoid duplicate code we should move CRUD operations to the DmtAPI.

Not only use this to get URL, but it should include get/put/post/del operations also.

This will make the rest of the code more readable and clean.

• Extends
• Annotations

The web application creates basic json-objects from a template/blueprint. These needs to be translated into the json-schema standard for storage and future use.

This could be done on a new endpoint in the API etc. /api/generate-schema

The workspace would use local storage to store blueprints and entities.

This "database" should always be available to the user, without any authentication.

Show not edit mode when clicking a blueprint, show view mode.

Later: We need to be able to create default (maybe several) UI schema for blueprint.

Required for production use.

The team decided to change the core structure of treenodes (with more metadata) and move the treeview generation to api. The api has also had mayor changes due to supporting different datasources which are now implemented.

Some issues are not finished:

• set form to viewmode after submit in edit mode
• datasource create
• datasource update state.datasources with response (blocked by api)
• edit package
• create package
• create blueprint
• create subpackage
• edit subpackage
• fix search open rootnode (UPDATE: replaced by #107

Options.

1. Library
2. Code generation

Library

Kind of library where several jsonschemas are given upfront, in this case before build or start steps in scripts. The same code is used in all apps generated. The fields in the jsonschemas decide which path the app will go. Solved by lots of conditionals and switches.

• Pro: easy to get create a proof of concept.
• Cons: will be hard to add functionality after the generator is done. Need to know how the "library" works.

Code generator

Code is generated by templates where output is exactly one path.
Extending create react app can be used as a starter. The example does not support branches of "templates".

• Pros: the project can be used after "final release" by changing the generated code directly.
  The generated code will be like a create react app, but with forms, charts, api communication.
• Cons: Need to solve how to use variables in template. May be complex and time consuming. Probably requires a large number of templates with lots of logic.

Suggest building a small poc of both.

TODO: Still need to address a codegenerator for api and storage.

We need to translate our JSON files to react-jsonschema-form structure.

For file system, we need to get files local in client before sending it to the API for translation.

The endpoint?
/api/transformer/json-schema?

Should handle getting package from client and insert into DB at selected data source

Output should be same as output of index endpoint. input is same as input_import_collection('blueprint' || 'entities')

the input should be stored to database, just create an index which will be used to display the uploaded files in the treeview. A separate action must be taken by the user to actually store the uploaded files. Not sure about naming here but the file-upload is upload to client side, not to persisting to the database.

New REST endpoint to GET a schema/form from a specific data-source.
Optional transform to json-schema

To be able to use our json-schema templates as validators for form-data, they should contain stricter validation keywords.

Store templates and blueprints in a central database. e.g maf

• Select blueprint

• Select or create db from gui in entities page. (needed to populate treeview with rootnodes, other entities from same blueprint)

• Type title of new entity, (needed to add it as root node in the treeView)

• Treeview:
  one root element, click on to edit title name etc of entity. (root-blueprint)
  render blueprint children in treeview. Click on a children to fill in entity values.
  show other entities of selected blueprint as rootnodes.

• Submit: use server and db name in endpoint?

api/services/database.py
create db runtime from a list of uri, username, password.

expose a db instance by invoking a method on database.py instead of exposing the actual db client variable.

Look into azure services of password manager. (fetch these compile time or runtime)

Not needed until a user need to use a database from a new resource group.

#42

Clean code + will be easier to test API without needing to connect to database.

This is not probably not needed in schema for attributes, just use dimensions for single type also.

Should be able to limit what blueprints to be shown when adding a data source. This is to avoid showing to many blueprints in tree view.

Default should be an attribute. This will be the default value for an attribute.

Use UI schema?

ref meeting 2.9

One standard type of index nodes and tree nodes: file | folder
How metadata is stored to mongo is abstracted by api, then other datasources, (file, postgres) can store this meta data however it wants. For instance, file storage can store this metadata in paths/filename uri.

Tasks Api:

• remove documentType from templates
• add type file | folder and isRoot as arguments to api post/put package
• update index test in api to return type file | folder and isRoot
• keep documentType in mongodb package.json files. root-package | sub-package
  TODO: move mongo specific logic to new plugin used by datasource type = mongodb

Tasks Web:

• update client to use type file | folder

Usecase 1: create blueprint (Domain Developer) støtter også create templates (admin)

Krav:

• lage rootpackage, subpackage, template
• select mulitple templates and edit each one.
• legge til template som property i blueprint. denne dukker da opp i 2. og kan editeres, vil fungere rekursivt.

Ha tilstand for state i page rot komponent. Følgende states trengs:

1. TreeViewExisting state.
   • add package i modal (DD)
   • add package i modal (DD)
   • create blueprint template? modal (admin)
2. Blueprint state.
   • callback fra 1. (Action.addTemplateToBluePrint)
   • action for å editere en blueprint.
3. ingen state, bruker 2 sin. Kan legge til value i blueprint state.
   • action for å legge in verdier fra form.
   • action axios post (endepunkt tar imot ny blueprint.) onSuccess gjør en addBlueprint til 1. Bruker bør eksplisitt viske ut blueprint state.
4. Ingen state, bruker 3. generer blueprint jsonschema fra 2. Merge state i 2. Hvordan håndtere override? mulig løsning, legge til order i blueprint.

Søk: lokal state i Treeview.

Usecase 2: create entities (Domain Expert).

• Samme skisse som usecase 1, 2 blir entity. Og 1 vil kun inneholde blueprints. Litt andre valg i context menu.

Create new API endpoint to store data-sources.

How to get/post/put data sources:
/api/data-sources

How to get blueprint:
/api/blueprints?dataSourceId=,blueprintId=
/api/data-sources/:id/blueprints/:id

Tables that needs to be created:

• Data source
  • username
  • password
  • hostname

show notification on submit form in both modal and BlueprintForm

Api
postBlueprint / postDocument data = formData.

ny id blir parent path som angitt i og <title> .json

response:

{ <parent-id>: {
  children: [....children, ....[...<id på ny blueprint>]]
  },
  {
    <new-id>: {
       nodeType: "file",
      _id: <new-id>,
      ....
   }
  },
}

dermed kan vi merge nodes med disse 2 og få ny index som er riktig.

Maybe something like this would work:

{
  "title": "DatabaseDataSourceTemplate",
  "description": "",
  "type": "object",
  "required": [
    "firstName",
    "lastName"
  ],
  "properties": {
    "name": {
      "type": "string",
      "title": "Name",
      "default": ""
    },
    "host": {
      "type": "string",
      "title": "Hostname",
      "default": ""
    },
    "username": {
      "type": "string",
      "title": "Username"
    },
    "password": {
      "type": "string",
      "title": "Password",
      "minLength": 10
    }
  }
}

Put it under /api/schemas/templates (maybe own folder called data-sources)

• Create package.json files
• Use the package.json files to create a file index

We need to change "create package" to "add data source". And then we need to move "create package" to underneath data sources.

What's the desired behavior here?

Suggest using an alert to notify the user. Overwriting a existing package will most likely surprise the user.