Giter Club home page Giter Club logo

mylims's Introduction

myLIMS

Electronic lab notebook for open science

Installation

The following commands will respectively install the dependencies and build the core app.

npm install

Configuration

Edit it as needed. Example: auth config for ldap provider.

Run

Development

The following commands will start mongodb and ldap (to test the ldap provider) containers, setup a replica set on the mongodb server (to support migrations) and start the development server.

docker-compose up -d
node scripts/init-db.mjs
npm run dev

If it's the first time, don't forget to copy or create the .env file and is necessary to run the migration command

cp .env.example .env
npm i
node ace mongodb:migration:run

Configuration

Is necessary to have running the frontend server in order con administrate de system. For this in the localhost:3333/admin server configuration is possible to administrate the addons, including the login methods. After is required to have created an user:

node ace user:create

Production

cp .env.example .env.prod
docker-compose -f ./docker-compose.prod.yml build
docker-compose -f ./docker-compose.prod.yml up -d
docker-compose -f ./docker-compose.prod.yml exec backend node ace user:create

Design

The new ELN is meant to be highly modular, allowing developers to add new functionalities by creating addons.

Core

The core part is built on top of AdonisJS framework. The core provides commons parts such as addons loading, user model and ELN routes.

New ELN uses MongoDB as the default database system using the adonis-mongodb provider.

Addons

Addons can provide functionalities to the platform by exposing the following components:

  • API routes in routes.ts
  • Adonis providers providers/*
  • Migrations migrations/*

An addons should be stored as a subfolder in the addons folder. It can be named as anything valid for the filesystem.

Models

classDiagram
  class User {
    +ObjectId _id
    +String firstName
    +String lastName
    +String[] usernames
    +String[] emails
    +String role
    +AuthMethods authMethods
  }

  class File {
    +String _id
    +String filename
    +Number size
    +String collection
  }

  class SampleKind {
    +String _id
    +String name
    +String description
    +String color
    +JSON schema

    +fromInput(sampleKind ,input)
  }

  class Sample {
    +ObjectId _id
    +String[] sampleCode
    +String uuid10
    +ObjectId userId
    +String kind
    +String[] labels
    +String project
    +JSON meta
    +String title
    +String description
    +String comment
    +MeasurementLink[] measurements
    +SampleAttachment[] attachments
    +ObjectId[] parents

    +fromInput(sample, input)
  }
  Sample "1" --> "*" Sample
  Sample "*" --> "1" SampleKind
  Sample "*" --> "1" File

  class Measurement {
    +ObjectId _id
    +String username
    +ObjectId sampleId
    +String title
    +String fileId
    +ObjectId eventId
    +String comment
    +ObjectId createdBy
    +String description
  }
  Measurement "*" --> "*" User
  Measurement "*" --> "1" Sample
  Measurement "*" --> "1" Event

  class Notebook {
    +ObjectId _id
    +String title
    +String description
    +ObjectId userId
    +String[] labels
    +ObjectId[] samples
    +MeasurementLink[] measurements
    +String project
    +String content
  }
  Notebook "*" --> "*" Measurement
  Notebook "*" --> "*" Sample
  Notebook "*" --> "1" User
  Sample "*" --> "1" User

  class Event {
    +ObjectId _id
    +String topic
    +EventData data
    +EventProcessor[] processors
  }
  class EventProcessor {
    +String processorId
    +EventHistory[] history
  }
  class EventHistory {
    +String processId
    +String status
    +Date date
    +String message
  }
  class EventData {
    +String type
    +String fileId
  }
  Event --* EventData
  Event --* EventProcessor
  EventProcessor --* EventHistory
  EventData --> File

mylims's People

Contributors

maasencioh avatar stropitek avatar targos avatar xstoudi avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar

mylims's Issues

Create a Docker image for the core

In one of the steps, we need to update the "providers" list in .adonisrc.json (remove providers starting with ./addons/). This allows us to develop with all the addons locally.

Samples tree

  • Relation between samples
  • Relation with users
  • Relation with organizations

Addon metadata

I'm thinking we should decide on a metadata file that add-ons should implement.
Then, instead of having magical files loaded automatically (like routes.ts), the metadata would contain that information.
It would also contain user-friendly name and description for the addon, which can be used in the user interface.

Logging

Where do we store logs?
Do we allow to read the logs from the application?
What are the rules about logging within the application?

Deployment needs

  • Local server with Docker
  • Files in local drive
  • Files in S3-like system
  • Deploy in GCP
  • Kubernetes?

Implement a "master password" system

This will allow to change the main settings of the application, like MongoDB connection, ... (things that are in the config directory)
Enable and disable addons.

Implement authentication correctly

I've been looking into why the session initiated from LDAP login doesn't work, and actually we're doing it wrong.
There should be only one guard for all possible user sources.

filter by labels

  • search flat with filters (status, creation date, update at), pagination
  • sort by default is revisions.0.date
  • github graphql inspiration for sortBy and filterby, pagination (skip, limit)
  • add the modification date column

Name customization for samples

  • Organization defines what is the hierarchy of samples
  • Possible restrictions of hierarchy of samples (form sample you only can have chip, not the contrary for example)

query for files needs levels

  • now everything is queried, the idea will be only query the specific level
  • add depth to fileSync
  • escape regular expression for the query
  • escape-string-regexp

Events

  • track of latest topic addressed
  • keep historical of status
  • trigger of a push on event to a processor

IMG_20210728_133039.jpg

Rework how authentication manages application users

When a someone logs in, a User is searched in the users DB using 3 parameters

  • provider name (local or ldap for now)
  • provider unique id (MongoDB _id for local, uid for ldap)
  • email
  1. Search with provider name + provider unique id
  2. If found, return user
  3. If not found, search with email
  4. If found, update provider info and return user
  5. If not found, create and return new user

Ready check and file import

Todo:

  • Use an enum to describe the status of a revision
  • Fix error on panel admin (hostUrl undefined)
  • Rename "files" collection with a prefix
  • "Check" command that works on "files" collection with same options as "Sync" command
  • Create a new collection that store imported files
  • It may be possible to config the storage location on admin panel

Imported files naming strategy

When imported, files must be organized like this:

  • Filename: random UUID
  • Extension: original file extension
  • Location: subfolderA/subfolderB/filename.extension
    With subfolderA defined as the two first characters of the UUID and subfolderB defined as the two next characters of the UUID.

Check command workflow

for each file related to the given file_sync_option with last revision set to "pending" state
    if ready checks are okay
        set state to "importing"
        copy the file on our system
        generate a new subfolder to store the imported file
        set state to "imported"
    else
        test next file

use formik and yup

  • Validate max is bigger than min
  • validate that size is convertible to bytesize

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.