kuzzleio / kuzzle-plugin-cluster Goto Github PK

We shall build a system to expose Kuzzle stats for each cluster node.
This can be achieved for instance on AWS by setting up a lambda which will plug itself to redis to fetch the nodes list and retrieve the stats for each node.

The lambda can then be triggered via the API manager.

Ideally, the output format should be modularized to allow standard output formats, such as prometheus one for instance.
An alert mechanism should also be included based on the retrieved activity.

Garbage collector run should be launched only on the master node

is this supported ?

The cluster should be able to detect a non-working node and exclude it.

When running Kuzzle in clsuter mode with the MQTT protocol enabled, you get:

kuzzle_2              |You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
kuzzle_2              | PluginImplementationError: this.redis.clusterSubOn(...).then(...).then(...).then(...).then(...).finally is not a function
kuzzle_2              | This is probably not a Kuzzle error, but a problem with a plugin implementation.
kuzzle_2              |     at KuzzleCluster.subscriptionAdded (/var/app/plugins/enabled/cluster/lib/index.js:406:15)

At a first glance, I would suspect the MQTT response room to trigger the event but this requires some further investigation (and fix).

The proxy should be able to detect a non-working node and ban it.

issue is coming from the binding if not resolved.

For instance, [idontexist.ipv4] won't be resolved and the string is kept as is.
As all nodes then subscribe to the same private room, they are told confusing information.

See kuzzleio/kuzzle#569 for more details.

Roles and Profiles are kept in cache in memory.

When a node updates a role or a profile, the information needs to be propagated to the other nodes of the cluster.

while installing npm dependencies without zeromq library, the process fails with following error:
../binding.cc:28:17: fatal error: zmq.h: No such file or directory
and we cannot use the plugin.

The readme need to explain this requirements and how to install the needed library (see https://www.npmjs.com/package/zmq#installation )

Idea: expose to plugins a method to trigger an event that will be propagated to other cluster nodes.

example :

class MyPlugin {
  // Hook configuration:
  constructor () {
    this.hooks = {
      'plugin-my-plugin:my-custom-event': 'myFunction'
    };
  }
  myFunction(payload) {
    this.doSomething(payload);
  }

  
  // broadcast an event to other nodes:
  myCustomMethod() {
    // (....)
    // if the plugin-cluster is not enabled => the `broadcast` method will do nothing
    // if it is enabled, it will encapsulate the event, send it to other cluster nodes,
    //                         and then in each node, trigger a `plugin-my-plugin:my-custom-event` hook:
    context.accessors.broadcast('my-custom-event', payload);
  }

see also comments here: kuzzleio/documentation#339 (comment)

We already expose the cluster topology.
We are missing some way to check the cluster health, as seen both by the proxy and maybe the cluster itself.

• create a I'm OK route on Kuzzle kuzzleio/kuzzle#580
• Implement some mechanism in the proxy query each Kuzzle health and report the info