https://github.com/taylortom/adapt-authoring-core/blob/master/lib/moduleloader.js#L47

I'm starting to think this is an abstraction for abstraction's sake; not sure it's really useful in the long-run. Is it useful to have a query class that can be used for all 3? Queries to different datastores are probably going to vary enough to make render this a bit pointless. It's also probably a bit more dev-friendly + transparent to expose datastore-specific APIs.

Example DataStores

• MongoDB
• xAPI
• Google Analytics

Would be nice to specify the config for a module in the config/package.json for the module, with a default option if not configured.

Example:

• Main app config specifies mongodb as storage
• Users module doesn't specify a storage type
• An 'analytics' module specifies googleanalytics as its storage
• Module#storage function gets the storage module according to the module's config:
  • mongodb for users
  • googleanalytics for analytics

Assuming two modules are on the same server, it's overkill to have to create http requests to use functionality across each module. What is the best way to bridge this gap?

Requirements:

• Make sure we don't unnecessarily repeat any code (e.g. input validation).
• May need to reassess how middleware's used (to make sure any middleware applies only to functionality related specifically to HTTP requests).

Possible solutions:

• Expose relevant functions on AbtractModule export

https://github.com/taylortom/adapt-authoring-mongodb/blob/master/lib/module.js#L68

Requirements

• Ability to set app-wide language
• Ability for individual users to set front-end language (would override default setting)
• Ability to add lang strings in any language in each module
• Ability to override all strings from a single location (i.e. a single file), to open up the possibility of creating 'language pack' modules
• API endpoint for front-end access

Tasks

• Investigate localisation
• Working localisation prototype for one module
• Write any docs necessary to aid future implementation
• Investigate how to document lang strings

Probably makes sense to base module around Polyglot.js again

Original isn't actively maintained, so may be better to switch to one of the forks:

• ESDoc2

See https://github.com/taylortom/adapt-authoring-server/blob/master/lib/module.js#L54

On a related note: should we expose the express app to allow for direct access to the express Application? Might be useful.

Very generic one, but I've not done a lot of async/await in the current prototype, so could do with identifying any areas that would be a good use of it.

Original problem here was that any errors occurring before logger has initialised use console log & were therefore ugly. I've refactored this so that the logging works as expected, but it needs another rewrite as feels a bit hacky.

What we need:

• Utility-level errors need to use the logger (with a console fallback in the case that logger has failed)
• Module-level errors should use logger, as the app should fail if any utilities haven't loaded correctly (check this happens)

In the current incarnation, courseasset records are used to flag individual asset instances.

• Course build: to determine which assets to include
• Asset deletion: to determine if assets are used in any course

Possible solutions:

• ???

We need to think about how unit tests will be implemented and run.

• Research running tests across the module set
• Research integration with CI on GitHub
• Write any docs necessary to aid future implementation

Goal

Allow easy user configuration with as little input from the user as possible.

Requirements

• Support multiple environments/sets of configurations
• Remove (or greatly reduce) the need for defensive coding when accessing config values (e.g. if(!configValue) ...).
• Allow devs to specify constraints for their config values (see Config schemas)
• Singleton?
• Should listen to module create event
• Modules need to define a config file (or default to config.JSON?)
• Should probably namespace config options (e.g. config.mongodb.server_port)
  • How will this work with CMD args?

Notes

• Major performance hit when accessing process.env, so make sure this is read once and cached

Config schemas

For consistency, these should mirror the format we use for other schemas throughout the app.

Allow devs to include schemas with any modules which allow them to specify what config options are needed, and what values are acceptable:

• Whether the option is required
• Default value (if not required)
• Type of value
• Constraints/validation to further restrict what is provided here (this could be replace type, or compliment it)
• Whether the option is hidden from the front-end app
• Description of the value

These schemas could then be processed during the app boot process to:

• Check all required config options have been set by the user (and warn of any missing config options, possibly stop loading process of module or entire app)
• Provide default values for any non-required options not present in the app config file

Schemas would also be useful for various automated processes:

• Current config/config.json route requires a hard-coded list of whitelisted attributes, schemas could expose a value which would set this (e.g. hidden)
• Install process could use schemas to greatly reduce the code in a specific install script.
  This would make it easy to automate the install process & config/config.json API route.

Would be nice to be able to include generic docs which don't necessarily fit with a specific module here (e.g. installation steps).

See /bin/cli.js#L85

Some questions:

• How (and where) do we define the 'interfaces' (i.e. which functions should be available etc.)
• How much should the code work without them an 'essential' API (probably not at all)
• Some functions are a bit verbose (e.g. this.app.lang.t, this.app.logger.log) -- would nice to add dev-friendly shortcuts to these functions (e.g. this.t, this.log), but how does this fit into the above?

This is likely to be a particular problem with the unit tests. The most common example being the lang module (and translating error strings etc.) - should the app continue to function if this isn't present (we can obviously mock the translation module to get around this in tests).

https://github.com/taylortom/adapt-authoring-users/blob/master/lib/middleware.js#L12

From npmjs.com:

bcrypt-nodejs is no longer actively maintained. Please use bcrypt or bcryptjs. See https://github.com/kelektiv/node.bcrypt.js/wiki/bcrypt-vs-brypt.js to learn more about these two options

bcrypt (285k weekly D/Ls, updated 14/04/19)
C++, faster, non-blocking.

bcryptjs (446k weekly D/Ls, updated 04/01//17)
Pure JS, slower, blocking.

Leaning towards bcryptjs because of the hit on install of the native stuff.

i.e. which defined strings are used, any duplicates etc.

Possible libs:

• i18n-extract

https://github.com/taylortom/adapt-authoring-mongodb/blob/master/lib/mongoDBModule.js#L79

Considering that writing APIs will be the main task when extending the authoring tool, we could do with writing some reusable bits to make API creation less code-heavy (and remove duplication).

Things you want to define when creating an API:

• Main mount-point
• Sub-routes
• Route handlers
• Middleware
• DB models/schemas

Other points:

• Skeleton Module class which can be extended (inits the API, controller, data models etc.)
• API/route template with the standard methods: POST, GET, UPDATE, DELETE
  • Should also be able to restrict to a subset of the above
• Generic controller to handle DB storage
• Generic middleware (e.g. to check POST requests have a body)
• Possibly set default paths for the required files (controller.js, api.js etc.), so you can omit boilerplate loading code if using these standard paths
• Consistent querying
• Consistent data responses (e.g. pagination)
• Consistent error handling
• Ways to easily customise/override all of the above
  • Router/per-route middleware
  • Disable specific HTTP methods
  • Override individual handlers (i.e. not all or none)
• Should it be possible to define a new API without any code?? (i.e. just using package.json...and probably schema)

• Document API def
• How to write a module
• Writing unit tests
• Running unit tests
• Writing a config.schema.js

i.e. should it be possible to connect to multiple mongo instances in a single app.

e.g.

Person.
  find({ occupation: /host/ }).
  where('name.last').equals('Ghost').
  where('age').gt(17).lt(66).
  where('likes').in(['vaporizing', 'talking']).
  limit(10).
  sort('-occupation').
  select('name occupation').
  exec(callback);

See https://mongoosejs.com/docs/api/query.html

Need to highlight required somehow

This issue covers the basic implementation of authentication.

• User passwords should be encrypted
• New auth module should setup passport with local strategy
• Use token based authentication
• The auth module should have a way to run necessary auth code on server boot
• The user delete route should require authentication
• Merge changes into main repo(s)

Think it'd be good to encapsulate the install/upgrade process in a separate module to allow for easy modification/replacement later. This would also allow for easier testing + interaction between the installer and other parts of the application.

If the installer is an Adapt Module, it'd also be easy to replicate the version check + log output that we have currently.

Main requirement with this is to be able to allow modules to interact with key parts of the application without needing to know about the implementation.

Main areas I'm thinking of with this:

• Logger
• Config
• Lang

Requirements:

• Provide a class/interface to inherit from/implement
• Will probably need to ensure at least one module has been registered for each interface
• Allow modules to register themselves

Can parse the schemas to generate this.

Write a core module specification/definition guide for developers. This should be used in conjunction with https://github.com/taylortom/adapt-authoring/issues/28.

Things to include:

• Naming conventions
• How-tos

Spec:

• Use API module

For reference: Middleware#sanitiseInput

The middleware nullifies body/params/query on the request if they're empty objects. The original intention was to make presence checking a bit cleaner by using truthy values (e.g. if(req.body) {} rather than checking keys etc. each time).

HOWEVER, this now leads to extra defensive code if you need to use properties of the above (e.g. if(req.params && req.params._id) {} which is annoying...).

Possible solutions:

• Revert back to original functionality, which may reintroduce the need to have existence checks
• Amend middleware to leave original body/params/query objects be, and maybe store a separate boolean value on the request (e.g. req.hasBody, req.hasParams etc.)

It would be nice to be able to update the list of local dependencies and still be able to update via git without losing the local changes (as would happen if package.json was edited). Any solution needs to make sure npm install etc. still work as expected.

List of modules that are needed for the restructure. These will be a mixture of the existing /lib and /plugins files, as well as some necessary extras now we're using a modular architecture.

• Existing API
• Existing module exports

TODO

• Identify modules which will need to access other modules & solutions

'Module' column refers to whether the module is an Adapt authoring module (rather than just a standard Node module).

Name	Module	Description	GitHub
adapt-authoring-adaptoutput	true	Handles the transformation of DB data + local files into an Adapt course.
adapt-authoring-assets	true	Asset handling.
adapt-authoring-authentication-local	true	May be combined with authorisation.
adapt-authoring-authorisation-???	true	May be combined with authentication.
adapt-authoring-clipboard	true	Server-side copy/pasting.
adapt-authoring-content	false
adapt-authoring-content-article (adapt-authoring-content)	true
adapt-authoring-content-block (adapt-authoring-content)	true
adapt-authoring-content-component (adapt-authoring-content)	true
adapt-authoring-content-contentobject (adapt-authoring-content)	true
adapt-authoring-core	false
adapt-authoring-docs	false
adapt-authoring-logger	false
adapt-authoring-middleware	true
adapt-authoring-missingroute	true
adapt-authoring-mongodb	true
adapt-authoring-plugin	false
adapt-authoring-plugin-component (adapt-authoring-plugin)	true
adapt-authoring-plugin-extension (adapt-authoring-plugin)	true
adapt-authoring-plugin-menu (adapt-authoring-plugin)	true
adapt-authoring-plugin-theme (adapt-authoring-plugin)	true
adapt-authoring-roles	true
adapt-authoring-server	true
adapt-authoring-tags	true
adapt-authoring-ui	true
adapt-authoring-users	true

See this Google Doc for more notes.

Below list is subject to change once discussed.

• Investigate scope based roles
• Investigate how to define roles
• Investigate how auth should be applied (i.e. should it be possible to enable/disable at an individual API/API route level, or as an entire application)
• Suggest best way to document using ESDocs
• Think about how to cater for individual resources (see notes)

Questions

Does it matter where the auth comes in the stack (i.e. should other middleware be able to run before auth middleware)
Probably not, but for performance reasons it would be good to be able to stop any unnecessary middleware running if user doesn't have access to the system.

We need a module to allow storage to both local and remote sources (by default, we only need to support local file storage).

Would be nice to allow the following:

• temp files (e.g. build files, can be deleted)
• app files (created during installation etc., app breaks if these are missing)
• assets both local and remote

Possible API example:

await filestore.writeFile('http://drive.google.com/myFile.txt', 'googledrive');
const file = await filestore.readFile('http://drive.google.com/myFile.txt', 'googledrive');
// no param defaults to config 'defaultStore' option or similar
const file = await filestore.readFile('/path/to/myFile.txt');

• Breakdown of 'content' into separate modules + inheritance
  • Support framework schemas
• Investigate splitting schemas:
  • Framework/content schemas
  • Editor schemas (extra editor settings not needed in content) example attributes: course hero image, course tags, course is shared, block layout options, component color label
• Investigate building a course without the need for temp files (e.g. streaming?)
• Investigate 'quick' build methods (e.g. precompiling of core/plugin code)

There's currently no mechanism to document the external API endpoints.

• OpenAPI-based? (Swagger, ReDoc)
• Integrate with ESDoc?

https://github.com/taylortom/adapt-authoring-core/blob/master/lib/loadable.js

To be called in Module constructor.

See Code etiquette page for a starting point.

Might be nice to be able to 'register' errors to the app, to allow easy error type reuse across all modules in the app.

Note: not saying the below is the best way to implement, but gets my point across

e.g. Say a HelloWorld module defines a HelloWorldError:

class HelloWorld {
  constructor() {
    this.app.registerError(HelloWorldError);
  }
}

Some OtherModule can then use this error like so:

class OtherModule {
  someFunc() {
    throw this.app.ErrorFactory('HellowWorldError', { option: 1, option: 'two' });
  }
}

If the specific error hasn't been defined, a standard Error can be returned.

Maybe Router._expressRouter

Think it's useful to have this secretly accessible in case you need access to any Express API not surfaced by the AAT code.

• Write up local module npm install stuff
• Investigate levelling up the search function to include the manual pages
• Look into switching to an ESDoc fork, as original isn't actively maintained
• Document APIs exposed by AAT modules:
  • Probably OpenAPI-based? Swagger, ReDoc
  • Integrate with ESDoc?

...to check required data exists, and that it's safe.

Note: we can't check req.params until the request passes through the router that defines them.

It might be useful to include middleware in a central 'pool' so they can be reused by other modules.

Input isn't currently validated prior to adding it to the routes list. We should probably add something here to check the route has the required structure.

See adapt-authoring-server/lib/router.js.