A spin-off of eris-sharder and megane with services and configurable logging.

• Clustering across cores
• Sharding
• Customizable logging
• Services (non-eris workers)
• IPC to communicate between clusters, other clusters, and services
• Detailed stats collection
• Use a modified version of eris
• Soft cluster and service restarts where the old worker is killed after the new one is ready
• Graceful shutdowns

Run npm install eris-fleet or with yarn: yarn add eris-fleet

Some working examples are in test/.

To get started, you will need at least 2 files:

1. Your file which will create the fleet. This will be called "index.js" for now.
2. Your file containing your bot code. This will be called "bot.js" for now. This file will extend BaseClusterWorker

In the example below, the variable options is passed to the admiral. Read below for what options you can pass.

Here is an example of index.js:

const { isMaster } = require('cluster');
const { Fleet } = require('eris-fleet');
const path = require('path');
const { inspect } = require('util');

require('dotenv').config();

const options = {
    path: path.join(__dirname, "./bot.js"),
    token: process.env.token
}

const Admiral = new Fleet(options);

if (isMaster) {
    // Code to only run for your master process
    Admiral.on('log', m => console.log(m));
    Admiral.on('debug', m => console.debug(m));
    Admiral.on('warn', m => console.warn(m));
    Admiral.on('error', m => console.error(inspect(m)));

    // Logs stats when they arrive
    Admiral.on('stats', m => console.log(m));
}

This creates a new Admiral that will manage bot.js running in other processes.

The following is an example of bot.js. Read below for what you can access and do with clusters.

const { BaseClusterWorker } = require('eris-fleet');

module.exports = class BotWorker extends BaseClusterWorker {
    constructor(setup) {
        // Do not delete this super.
        super(setup);

        this.bot.on('messageCreate', this.handleMessage.bind(this));

        // Demonstration of the properties the cluster has (Keep reading for info on IPC):
        console.log(this.workerID); // ID of the worker
        console.log(this.clusterID); // The ID of the cluster
    }

    async handleMessage(msg) {
        if (msg.content === "!ping" && !msg.author.bot) {
            this.bot.createMessage(msg.channel.id, "Pong!");
        }
    }

    shutdown(done) {
        // Optional function to gracefully shutdown things if you need to.
        done(); // Use this function when you are done gracefully shutting down.
    }
}

The bot above will respond with "Pong!" when it recieves the command "!ping". Make sure your bot file extends BaseClusterWorker!

You can create services for your bot. Services are workers which do not interact directly with Eris. Services are useful for processing tasks, a central location to get the latest version of languages for your bot, custom statistics, and more! Read below for what you can access and do with services. Note that services always start before the clusters. Clusters will only start after all the services have started. To add a service, add the following to the options you pass to the fleet:

const options = {
    // Your other options...
    services: [{name: "myService", path: path.join(__dirname, "./service.js")}]
}

Add a new array element for each service you want to register. Make sure each service has a unique name or else the fleet will crash.

Here is an example of service.js:

const { BaseServiceWorker } = require('eris-fleet');

module.exports = class ServiceWorker extends BaseServiceWorker {
    constructor(setup) {
        // Do not delete this super.
        super(setup);

        // Run this function when your service is ready for use. This MUST be run for the worker spawning to continue.
        this.serviceReady();

        // Demonstration of the properties the service has (Keep reading for info on IPC):
        console.log(this.workerID); // ID of the worker
        console.log(this.serviceName); // The name of the service

    }
    // This is the function which will handle commands
    async handleCommand(dataSentInCommand) {
        // Return a response if you want to respond
        return dataSentInCommand.smileyFace;
    }

    shutdown(done) {
        // Optional function to gracefully shutdown things if you need to.
        done(); // Use this function when you are done gracefully shutting down.
    }
}

Make sure your bot file extends BaseServiceWorker! This service will simply return a value within an object sent to it within the command message called "smileyFace". Services can be used for much more than this though. To send a command to this service, you could use this:

const reply = await this.ipc.command("myService", {smileyFace: ":)"}, true);
this.bot.createMessage(msg.channel.id, reply);

This command is being sent using the IPC. In this command, the first argument is the name of the service to send the command to, the second argument is the message to send it (in this case a simple object), and the third argument is whether you want a response (this will default to false unless you specify "true"). If you want a response, you must await the command or use .then().

If you encounter an error while starting your service, run this.serviceStartingError('error here') instead of this.serviceReady(). Using this will report an error and restart the worker. Note that services always start before the clusters, so if your service keeps having starting errors your bot will be stuck in a loop. This issue may be fixed in the future from some sort of maxRestarts option, but this is currently not a functionality.

If you encounter an error when processing a command within your service, you can do the following to reject the promise:

// handleCommand function within the ServiceWorker class
async handleCommand(dataSentInCommand) {
    // Rejects the promise
    return {err: "Uh oh.. an error!"};
}

When sending the command, you can do the following to deal with the error:

this.ipc.command("myService", {smileyFace: ":)"}, true).then((reply) => {
    // A successful response
    this.bot.createMessage(msg.channel.id, reply);
}).catch((e) => {
    // Do whatever you want with the error
    console.error(e);
});

Below is more in-depth documentation.

Here is a complete list of options you can pass to the Admiral through the Fleet constructor.

You can choose what to log by using the whatToLog property in the options object. You can choose either a whitelist or a blacklist of what to log. You can select what to log by using an array. To possible array elements are ['gateway_shards', 'admiral_start', 'shards_spread', 'stats_update', 'all_clusters_launched', 'all_services_launched', 'cluster_launch', 'service_launch', 'cluster_start', 'service_start', 'service_ready', 'cluster_ready', 'shard_connect', 'shard_ready', 'shard_disconnect', 'shard_resume', 'service_restart', 'cluster_restart', 'service_shutdown', 'cluster_shutdown', 'total_shutdown']. Here is an example of choosing what to log:

const options = {
    // Your other options
    whatToLog: {
        // This will only log when the admiral starts, when clusters are ready, and when services are ready.
        whitelist: ['admiral_start', 'cluster_ready', 'service_ready']
    }
};

Change whitelist to blacklist if you want to use a blacklist. Change the array as you wish. Errors and warnings will always be sent.

Clusters and services can use IPC to interact with other clusters, the Admiral, and services. Here are all the IPC uses:

You can also add a custom source to the log if objectLogging is set to true in the options. Here is an example: process.send({op: "log", msg: "hello!", source: "a cool place"})

If you notice your logs ending up as { }, try doing the following to your error:

const { inspect } = require('util'); // No need to install a package, this is included in node
let yourError = Error; // Whatever error you need to log
console.error(inspect(yourError)); // Logs the error

To restart clusters, you can do the following in your bot.js file. 0 is a placeholder for the ID of the cluster you wish to restart.

this.ipc.restartCluster(0);

The above code will restart the cluster while avoiding downtime, meaning it will only kill the original worker after the new worker is ready. If you want to kill the worker immediately, use this.ipc.restartCluster(0, true). The second argument is whether you want to preform a hard restart. This is false by default.

You can also restart all the clusters. You can do this by using

this.ipc.restartAllClusters();

If you want to preform a hard restart, use this.ipc.restartAllClusters(true).

You can shutdown clusters. Use the following to shutdown clusters:

this.ipc.shutdownCluster(0);

The above code will shutdown the cluster gracefully. If you would like to kill the worker immediately, use this.ipc.shutdownCluster(0, true). You cannot restart a cluster after shutting it down. A future update may allow this.

To restart services, you can do the following in your bot.js file. "myService" is a placeholder for the name of the service you wish to restart.

this.ipc.restartService("myService");

The above code will restart the service while avoiding downtime, meaning it will only kill the original worker after the new worker is ready. All commands will be sent to the original worker until the new worker is ready. If you want to kill the worker immediately, use this.ipc.restartService("myService", true). The second argument is whether you want to preform a hard restart. This is false by default.

You can also restart all the clusters. You can do this by using

this.ipc.restartAllServices();

If you want to preform a hard restart, use this.ipc.restartAllClusters(true).

You can shutdown services. Use the following to shutdown services:

this.ipc.shutdownService("myService");

The above code will shutdown the service gracefully. If you would like to kill the worker immediately, use this.ipc.shutdownService("myService", true). You cannot restart a service after shutting it down. A future update may allow this.

You can totally shutdown your fleet using the following:

this.ipc.totalShutdown();

The above code will shutdown the service gracefully. If you would like to kill the worker immediately, use this.ipc.totalShutdown(true). A total shutdown exits all processes, including the master process.

You can register certain events to a callback. This can recieve broadcasts and stats. The object sent in the callback is {op: "the event's name", msg: "the message"}. Here is an example of registering an event:

this.ipc.register("stats", (message) => {
  // Do stuff
  console.log(message.msg);
});

You can unregister events you registered above.

this.ipc.unregister("stats");

You can broadcast events that other clusters can recieve by registering with the event. The first argument is the name of the event you are broadcasting (this should match the name of the event other clusters are registered to). The second argument is optional and is the the message you want to send. Note that the cluster sending this will also recieve the broadcast since this broadcasts to all clusters.

this.ipc.broadcast("hello clusters!", "Want to chat?");

You can send a message from one cluster to another specific cluster based on the cluster ID. The first argument is the ID of the cluster to send the message to. The second argument is the name of the event the other cluster should be registered to. The third argument is optional and is the message to send.

this.ipc.sendTo(1, "Hello cluster 1!", "Squad up?");

Fetches a user from another cluster. Not much to explain here. The only argument used should be the ID of the user. Be sure to await this or use .then()

await this.ipc.fetchUser(123456789);

Fetches a guild from another cluster. The only argument used should be the ID of the guild. Be sure to await this or use .then()

await this.ipc.fetchGuild(123456789);

Fetches a channel from another cluster. The only argument used should be the ID of the channel. Be sure to await this or use .then()

await this.ipc.fetchChannel(123456789);

Fetches a member from another cluster. The first argument should be the ID of the member. The second argument should the be ID of the guild the member is in. Be sure to await this or use .then()

await this.ipc.fetchMember(123456789, 987654321); 

Send a command to a service. The arguments are as follows:

await this.ipc.command("ServiceName", "hello service!", true); 

Gets the latest stats. This is an alternative to registering the "stats" event. Be sure to use await or .then()

await this.ipc.getStats();

Stats are given in the following object format:

{
    guilds: Number, // # of guilds the bot is in
    users: Number, // # of users the bot has cached
    clustersRam: Number, // Total RAM used by all clusters in MB
    servicesRam: Number, // Total RAM used by all services in MB
    masterRam: Number, // Total RAM used by the master process in MB
    totalRam: Number, // The real total rss.
    voice: Number, // # of voice connection the bot is in
    largeGuilds: Number, // # of "large" guilds the bot is in
    shardCount: Number, // # of shards
    clusters: Array<{ // Array of stats for clusters (you can use the length property of this to get the cluster count)
        id: Number, // ID of the cluster
        guilds: Number, // # of guilds the cluster is in
        users: Number, // # of users the cluster has cached
        uptime: Number, // Uptime of the cluster
        voice: Number, // # of voice connections the cluster is in
        largeGuilds: Number, // # of "large" guilds the cluster is in
        ram: Number, // RAM the cluster's process is using
        shardStats: Array<{ // Array of stats for the shards in the cluster (you can use the length property of this to get the shard count of the cluster)
            latency: Number, // Latency of the shard
            id: Number, // ID of the shard
            ready: Boolean, // Whether the shard is ready
            stats: 'disconnected' | 'connecting' | 'handshaking' | 'ready', // The status of the shard,
            guilds: Number, // # of guilds the shard services
            users: Number // # of members the shard services
        }>
    }>,
    services: Array<{
        name: String, // Name of the service
        ram: Number // RAM the service's process is using
    }> // Array of stats for services
}

Eris-fleet is able to use packages such as eris-additions if you desire. To do so, modify your bot file to match the following template:

const { BaseClusterWorker } = require('eris-fleet');

// Example using eris-additions
const Eris = require("eris-additions")(require("eris"));

class BotWorker extends BaseClusterWorker {
    constructor(setup) {
        super(setup);
        // Your cool stuff
    }
}

// This export is needed for this to work.
module.exports = {BotWorker, Eris};