For version 4.0.0 alpha version check ClusterWS Website (under work)
Lightweight, fast and powerful framework for building scalable WebSocket applications in Node.js. ClusterWS uses cWS a fork of uWebSockets.
:boom: Lightweight, fast and powerful framework for building scalable WebSocket applications in Node.js
For version 4.0.0 alpha version check ClusterWS Website (under work)
Lightweight, fast and powerful framework for building scalable WebSocket applications in Node.js. ClusterWS uses cWS a fork of uWebSockets.
I am using the latest version of clusterWS on Windows, but my target system is Ubuntu.
From the documentation on https://github.com/ClusterWS/ClusterWS/wiki/Pub-Sub-System I see that my server can push events to any channel. It can also talk 1on1 to any client that connects to that worker.
The application I am working on is a matchmaking browser. For this I need to subscribe to an incoming channel, not just a single socket (which could end up on the wrong worker), but this does not seem to be possible.
What is the correct way to go about this? Should I find a way to not listen to a channel, should I import the client library and connect to my own server or is there something I am missing?
In my project (writed in TypeScript) I extend the ClusterWS class like this:
class Server extends ClusterWS {
public app: express.Application;
/**
* Constructor.
*
* @class Server
* @constructor
*/
constructor() {
super(new INUserOption());
// create expressjs application
this.app = App;
}
}
now ho I can do this in 1.5.0 version?
What I want to do:
For my application I have multiple classes that each get created with access to the wss object. I want them to handle different sockets, but to avoid having to modify my main files, I have a connection handler in each of the classes to filter out the sockets they want.
Expected behaviour:
When I have multiple event handlers listening to the same event I would expect them all to get executed with the same data.
Observed behaviour:
Only 1 wss.on('connection')
handler gets executed. The rest are ignored.
Workaround:
Have 1 connection handler in the main thread and loop over all my socket handlers.
Hello everyone,
We are looking for the help with testing ClusterWS load and performance. If any one is interested to run stress tests on the ClusterWS and provide us with results, would be really cool. Currently we have repository with Benchmark but it is not finished and uses old version of ClusterWS, plus complication is that we can not open more then 16000 connections from our testing machine. The way to test is Set Up good server some where for example Amazon and create thousands of connection from different machines to that server then check the CPU, Ram, amount of users connected per seconds and other useful things.
Thank you very much.
I am trying to use ClusterWS with pm2.
Starting server with node server.js all works fine.
But starting with pm2 start server.js and checking with pm2 monit what's going on, I get never ending log of:
server > Broker PID undefined has been restarted
server > Worker PID undefined has been restarted
server > Broker PID undefined has been restarted
server > Worker PID undefined has been restarted
server > Broker PID undefined has been restarted
server > Worker PID undefined has been restarted
Hi, I need to handle some events like work websocket (https://www.npmjs.com/package/websocket).
With websocket I handle the event "wsServer.on('request', function(request) { })" to manage and validate the client login and origin.
There is a similar method with ClusterWS?
Please can you share what are all emitted events of SocketServer" and "Socket". Thank you.
We are going to start developing 2.0 version soon, currently, in our plans, we have few things to change:
brokers: number
)If you have some other interesting suggestions post them down to this thread. We will think about all ideas😄
In our new version of ClusterWS we have added/will add:
Machine Scaling
scale WebSocket apps across different serverTypeScript
supportUpdate will be rolled soon, just need to make some additions changes . :) Happy codding!
Maybe rename it to something like heartbeat
considering keepAlive
is part of the http
protocol :D
Do you have intention, in the future, to make a Redis support?
I want to use ClusterWS for a large scale app with many connected clients and with thousands of channels. I think that is a good way to use a dedicated memory database to handle over 100k clients...or not?
Is it possible to use this with restify? I see an example for KOA and Express. However, when I try a similar passing style with restify.createServer() it looks like it's unable to link ClusterWS and restify.
Do you have any plans to include additional middleware listeners, for example:
Unsubscribe Use Case
Upon a authenticated client subscribing to a channel, my worker will consume a RabbitMQ queue. When a particular client unsubscribes because the they changed the view in the web app, I would like to be able to hook in and stop consuming the appropriate queue.
channelClose Use Case
Upon a client subscribing to a channel, my worker will consume a RabbitMQ queue. When the channel is closed or the number of subscribers is 0 I would like to be able to hook in and stop consuming the appropriate queue.
onRecieve Use Case
Upon a client submitting data to the server, intercept requests and perform basic some basic/global security checks and functions (strip certain characters before processing).
In new version 1.2.0
Also planning, may come a bit later:
webpage-client.js
socket = new ClusterWS({url:'ws://myserver.com:80'});
master-server.js
new ClusterWS({
port: 80,
worker: Worker,
brokersPorts: [8002],
workers: 1,
horizontalScaleOptions: {
masterOptions: {
port: 8000
},
brokersUrls: []
}
});
>>> Master on: 80, PID: 16409
>>> Broker on: 8002, PID 16421
Worker: 0, PID 16427
slave-server.js
new ClusterWS({
port: 8003,
worker: Worker,
brokersPorts: [8004],
workers: 5,
// Scale configuration
horizontalScaleOptions: {
brokersUrls: ['ws://myserver.com:8000']
}
});
>>> Master on: 8003, PID: 9486
>>> Broker on: 8004, PID 9492
Worker: 0, PID 9499
Worker: 1, PID 9505
Worker: 2, PID 9511
Worker: 3, PID 9517
Worker: 4, PID 9523
The slaves detects the master just fine, since if I shut the master down, the slave will alert me about it and then alert again if the master comes back online:
Broker has disconnected, system is trying to reconnect to ws://myserver.com:8000
Can not connect to the Broker ws://myserver.com:8000. System in reconnection state please check your Broker and URL
Broker has been connected to ws://myserver.com:8000
Both master and slaves run the same Worker function. The goal here is to have the connections split between both master workers and slave workers.
Still, wss.on('connection') seems to never triggers on the slave workers. They're all connecting to the master server, so the slave is doing nothing at all. I got one worker on the master with 4 connections and 5 workers on the slave totally idle, doing nothing.
Am I doing something wrong? Or did I understand wrong how horizontal scale actually works here?
Hi, please can you help me to use ClusterWS with my PM2 config?
{
"apps": [
{
"name": "Server",
"script": "bin/server.js",
"watch": true,
"ignore_watch": [
"tmp",
"log",
"node_modules"
],
"watch_options": {
"persistent": true,
"ignoreInitial": true
},
"instances" : 4,
"exec_mode" : "cluster"
}
]
}
This start 4 instance of my app and I'm not sure that ClusterWS can work with this config.
I want to know how to get client ip?
I have the following code for counting the connected clients:
wss.on('connection', (ws) => { clients.push(ws); });
I have also 4 numbers of workers in my server configuration, it always returns 1 while I need to count the total number of clients which is: clients.length
Please save my day!
Last line. "Happy codding" 🐟
Could you guys please provide a LICENSE for this Repository. Is it licensed under the MIT License ?
What is the latest version or even the stablest version in order to add in my build.gradle file. I have added the following line but several commands are not available:
implementation 'com.github.ClusterWS:ClusterWS-Client-Java:1.1.1'
Please help!
Version : 3.1.1
OS: Mac OS 10.13.3
I trying https://github.com/ClusterWS/ClusterWS/wiki/Horizontal-%28Machine%29-Scale this tutorial and I have the master and 2 child nodes ready as shown in the tutorial.
When i run the child nodes i get a warning like this
Can not connect to the Broker ws://127.0.0.1:8000/?token=undefined. System in reconnection please check your Broker and Token
I am not even sure if its a warning or error (some indication to that would be nice).
Now My question is how should i connect to the cluster ?
I tried connecting to the master's port. then all the requests are handled by the master not the child nodes. Does this have something to do with the warning that i received earlier ?
And If i connect to the child's port then all the request is handled by the child (as expected).
But i would expect the child to handle the request when i request the master and the master to act as a load balancer. I might be wrong. It would be awesome to have a documentation which covered all of this.
And for now please let me know in which node the client should connect and which port.
Thanks
Few releases back I was able to npm i clusterws
and import your incredible rewrite of uws
implementation. Would you be willing to expose these rewrites or publish them to a separate package? Amazing work!
[root@baixou-d1-new pushextensao]# node pushextensao.js
/home/services/pushextensao/node_modules/clusterws/index.js:57
throw new Error("Could not run µWebSockets bindings");
^
Error: Could not run µWebSockets bindings
at /home/services/pushextensao/node_modules/clusterws/index.js:57:15
at Object.<anonymous> (/home/services/pushextensao/node_modules/clusterws/index.js:59:2)
at Module._compile (module.js:649:30)
at Object.Module._extensions..js (module.js:660:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:501:12)
at Function.Module._load (module.js:493:3)
at Module.require (module.js:593:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/services/pushextensao/pushextensao.js:120:20)
I already tried removing all modules from your node_modules directory and run npm cache clean and then npm install from scratch... but it didn't work.
Any suggestion?
Would love the ability to pass custom environment variables down into the worker processes
ClusterWS 3.1.1
ClusterWS-Client-JS 3.0.5
CentOS Linux release 7.4.1708
On server side I use official example with TLS and express. Page loads fine and browser handles certs fine.
I get websocket connection working if I do it without tlsOptions and via ws://
Also wss.on('connection', (socket) => {})
on server is fired..
But I get this on browser(Chrome) console and websocket connection fails:
WebSocket connection to 'wss://mydomain.com:8123/' failed: Error during WebSocket handshake: net::ERR_SSL_PROTOCOL_ERROR
On safari it gives error:
WebSocket network error: The operation couldn’t be completed. (OSStatus error -9847.)
Found via google that code -9847 is probably something to do with errSSLRecordOverflow. ( eclipse-vertx/vert.x#269)
I've a problem when I try to upgrade from 1.4.0 to 1.5.0 with verifyConnection method.
This is my configuration (in 1.4.0):
const verifyConnection = (info: IObject, callback: TListener) => {
if (!originAllowed(info.origin)) callback(false); // socketServer.on('connection') won't be called
isAuthenticatedApi(info.request, undefined, (err: Error) => {
if (err) {
callback(false);
console.log(new Date() + " Rejected Unauthenticated WebSocket from origin " + info.origin + ".");
} else
callback(true);
});
};
socketServer.on("verifyConnection", verifyConnection);
Now how I can convert it to work with 1.5.0?
Would love to have "dev" mode options for speed up node watch auto-reload stuff in development mode.
new ClusterWS({
port: PORT, // specify port of the application
worker: Worker, // Worker function must be provided
port: 3000,
dev: true
});
Expected behavior:
I just explore Sapper, looks great, and create a boilerplate you can try:
https://github.com/coderbuzz/sapper-clusterws
In Sapper dev environment, each time server OR client code changes it restart node, (and also ClusterWS cluster's master-worker), restarting node each time client or server code changes is too costly because ClusterWS running in Cluster mode.
Make middleware accessible from js Object instead of typing string by yourself (that would reduce typos).
My envs:
macOS: 10.13.4
node: 10.2.1
npm: 5.6.0
clusterws: 3.0.5
The latest clusterws depends on [email protected] and not found uws_darwin_64.node
.
λ /V/W/D/p/e/p/n/clusterws git:(𝘮 ← 596 dev) 3M 1A > ls -la node_modules/uws/
total 28112
drwxr-xr-x 26 fundon staff 884 May 26 11:31 .
drwxr-xr-x 3 fundon staff 102 May 26 11:31 ..
-rw-r--r-- 1 fundon staff 869 Apr 23 03:28 LICENSE
-rw-r--r-- 1 fundon staff 2452 Apr 23 03:28 README.md
-rw-r--r-- 1 fundon staff 2372 Apr 15 07:14 binding.gyp
drwxr-xr-x 9 fundon staff 306 May 26 11:31 build
-rw-r--r-- 1 fundon staff 27964 May 26 11:31 build_log.txt
-rw-r--r-- 1 fundon staff 1411 May 26 11:31 package.json
drwxr-xr-x 28 fundon staff 952 May 26 11:31 src
-rw-r--r-- 1 fundon staff 18224 Apr 23 03:28 uws.js
-rwxr-xr-x 1 fundon staff 388876 Apr 22 20:07 uws_darwin_46.node
-rwxr-xr-x 1 fundon staff 388876 Apr 22 20:08 uws_darwin_47.node
-rwxr-xr-x 1 fundon staff 388868 Apr 22 20:08 uws_darwin_48.node
-rwxr-xr-x 1 fundon staff 388916 Apr 22 20:08 uws_darwin_51.node
-rwxr-xr-x 1 fundon staff 388916 Apr 22 20:08 uws_darwin_57.node
-rwxr-xr-x 1 fundon staff 388916 Apr 22 20:08 uws_darwin_59.node
-rwxr-xr-x 1 fundon staff 1580216 Apr 23 03:27 uws_linux_46.node
-rwxr-xr-x 1 fundon staff 1580216 Apr 23 03:27 uws_linux_47.node
-rwxr-xr-x 1 fundon staff 1580216 Apr 23 03:27 uws_linux_48.node
-rwxr-xr-x 1 fundon staff 1580216 Apr 23 03:28 uws_linux_51.node
-rwxr-xr-x 1 fundon staff 1580216 Apr 23 03:28 uws_linux_57.node
-rwxr-xr-x 1 fundon staff 1580216 Apr 23 03:28 uws_linux_59.node
-rwxr-xr-x 1 fundon staff 609280 Apr 23 05:24 uws_win32_48.node
-rwxr-xr-x 1 fundon staff 609792 Apr 23 05:24 uws_win32_51.node
-rwxr-xr-x 1 fundon staff 609792 Apr 23 05:24 uws_win32_57.node
-rwxr-xr-x 1 fundon staff 609792 Apr 23 05:25 uws_win32_59.node
λ /V/W/D/p/e/p/n/clusterws git:(𝘮 ← 596 dev) 3M 1A > ls -la node_modules/uws/
total 15312
drwxr-xr-x 19 fundon staff 646 May 26 11:32 .
drwxr-xr-x 3 fundon staff 102 May 26 11:32 ..
-rw-r--r-- 1 fundon staff 869 May 11 01:34 LICENSE
-rw-r--r-- 1 fundon staff 2452 May 11 01:34 README.md
-rw-r--r-- 1 fundon staff 2372 Apr 15 07:14 binding.gyp
drwxr-xr-x 9 fundon staff 306 May 26 11:32 build
-rw-r--r-- 1 fundon staff 15983 May 26 11:32 build_log.txt
-rw-r--r-- 1 fundon staff 1402 May 26 11:32 package.json
drwxr-xr-x 28 fundon staff 952 May 26 11:32 src
-rw-r--r-- 1 fundon staff 18703 May 11 01:34 uws.js
-rwxr-xr-x 1 fundon staff 389124 May 10 18:09 uws_darwin_57.node
-rwxr-xr-x 1 fundon staff 389124 May 10 18:10 uws_darwin_59.node
-rwxr-xr-x 1 fundon staff 389356 May 26 11:32 uws_darwin_64.node
-rwxr-xr-x 1 fundon staff 1580216 May 11 01:34 uws_linux_57.node
-rwxr-xr-x 1 fundon staff 1580216 May 11 01:34 uws_linux_59.node
-rwxr-xr-x 1 fundon staff 1580248 May 11 01:34 uws_linux_64.node
-rwxr-xr-x 1 fundon staff 610304 May 11 03:26 uws_win32_57.node
-rwxr-xr-x 1 fundon staff 610304 May 11 03:26 uws_win32_59.node
-rwxr-xr-x 1 fundon staff 610816 May 11 03:26 uws_win32_64.node
I have started working with the module to be able to scale WebSocket between machines, the module should be ready soon with new update for ClusterWS. Really hope you will like it 😄
To make sure that i am not adding a lot of unneeded stuff to the library and keep it minimalist, Machine Scaling module is developing in the separate repositor (because not everyone wants to use it). Thank you.
Do you have any plans to add https://flutter.io client library?
Hi, I got always this error when I try to send a message from my Client, how can I check when this exception was raised?
I would like to ask you if it is possible that you add host
to ClusterWS configurations/options. For me to make use of ClusterWS, I need to pass host IP to workers http server listen method.
I prepared diff below. However, it is just off the top of my head, I could not run the ts-builder, therefore please make sure it does not break anything.
diff --git a/src/index.ts b/src/index.ts
index 6d80a4b..36f8e7e 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -13,6 +13,7 @@ export default class ClusterWS {
const options: Options = {
port: configurations.port || (configurations.tlsOptions ? 443 : 80),
+ host: configurations.host || null,
worker: configurations.worker,
workers: configurations.workers || 1,
brokers: configurations.brokers || 1,
diff --git a/src/modules/worker/worker.ts b/src/modules/worker/worker.ts
index 73ed441..d4062f3 100644
--- a/src/modules/worker/worker.ts
+++ b/src/modules/worker/worker.ts
@@ -28,7 +28,7 @@ export class Worker {
this.wss.middleware.verifyConnection.call(null, info, callback) : callback(true)
}).on('connection', (socket: WebSocket) => this.wss.emit('connection', new Socket(this, socket)))
- this.server.listen(this.options.port, (): void => {
+ this.server.listen(this.options.port, this.options.host, (): void => {
this.options.worker.call(this)
process.send({ event: 'READY', pid: process.pid })
})
diff --git a/src/utils/interfaces.ts b/src/utils/interfaces.ts
index b678225..97c5ee9 100644
--- a/src/utils/interfaces.ts
+++ b/src/utils/interfaces.ts
@@ -46,7 +46,8 @@ export interface HorizontalScaleOptions {
export interface Configurations {
worker: WorkerFunction
- port?: number
+ port?: number,
+ host?: string,
workers?: number
brokers?: number
useBinary?: boolean
@@ -59,7 +60,8 @@ export interface Configurations {
export interface Options {
worker: WorkerFunction
- port: number
+ port: number,
+ host: string,
workers: number
brokers: number
useBinary: boolean
Would really appreciate if host
option can be added.
And keep up with good work :)
How to work with Koa?
@goriunov Hi!
My case:
For examle i create 4 workers which process game grid sections separately.
How to enable communication via socket between them for sync states,objects, and another info.
Should i connect to master via socket and create private channel for workers?
Thanks!
From the docs it seems that to scale a master instance is necessary. What happens when the master instance goes down?
If this stops the cluster from working, then it can be a reliability issue. As a suggestion, I think, a better architecture will be if the servers elect a master automatically from the available instances.
Thanks for sharing your work.
I am using the latest version of clusterWS and the codes of my server is:
const express = require('express');
const http = require('http');
const app = express();
const server = require('http').Server(app);
const helmet = require('helmet');
const bodyParser = require('body-parser');
const moment = require('moment');
const cluster = require('cluster');
const ClusterWS = require('clusterws');
const WebSocket = require('ws');
//middlewares
app.use(helmet());
app.use(bodyParser.json({
limit: '1mb'
}));
app.use(bodyParser.urlencoded({
extended: true,
limit: '1mb'
}));
if(cluster.isMaster) {
var numberOfCPU = require('os').cpus().length;
for(var i = 0; i < numberOfCPU; i++) {
cluster.fork();
}
}
else {
//routes
require('./controllers/account')(app);
require('./controllers/game')(app);
require('./controllers/contact')(app);
require('./controllers/friend')(app);
//require('./controllers/wall')(app, io);
require('./controllers/server')(app);
require('./controllers/leaderboard')(app);
const clusterws = new ClusterWS({
worker: worker,
port: 4001,
url: 'localhost',
workers: 4
});
server.listen(4001, () => {
console.log('api is running on port 4001');
});
}
function worker() {
const server = this.server;
const wss = this.WebSocket;
server.on('request'.app);
console.log('hi');
require('./helpers/socket/chat/main-wall').mainWall(wss);
}
I did not understand where to place the related codes to get clusterws to work. Your help is much appreciated.
I see that you have changed handling of Ping/Pong from #1/#2 to a 9 and 65 code.
My problem is that my WebSocket client send only in text mode and this is a problem with your implementation. You check is message is !== string type . My sended pong is elaborated like a message and not a pong.
I am planning to start working on version 4.0.0 soon if you have nay suggestions/preference for what would u like to seen in new release we can discuss in this thread.
Main focus on 4.0.0 will be performance and cross communication stability. API may have some changes but will be as close (back compatible) as possible.
Will try to keep this issue updated :)
Hi, is there a solution for listen on 443 and 80 port in same app?
Hello, I've recently created typed-event-emitter library becuase I was annoyed of non strongly typed nature of normal EventEmitter. I tried to connect it with socket.io library but it's to cluttered and poorly written (es5 plus some magic coding pratcicles I don't really like). Then I came across ClusterWS which seems like a really nice alternative to socket.io.
I am not expecting such mature project as ClusterWS to start using such new library as typed-event-emitter
but I would gladly submit a pull requests to both client and server packages to use enum + interface/type
mechanism I used in typed-event-emitter
to accomplish 100% type safety while using socket.on/emit
(and to similar methods).
https://www.npmjs.com/package/uws
As you can see, 17hs ago the package uws went the way of the dodo, now if you try lo install the package you get nothing except a README.md with "I no longer publish to NPM. Find me elsewhere."
FAST FIX: replace dependency "uws": "ˆ10.148.0" for "uws": "10.148.1"
After running server on localhost using nodemon, it keeps printing
Error PID 1419:
TypeError: Cannot read property '2' of undefined
on console.
what is the problem!?
Hi, please can you explain me how can I send a message to a channel or a specified user from server?
I need to send to an user or many users a message or data when an event occur on server.
Any plans or ideas on how this solution could be made to scale across multiple machines ?
Hi, I've a question about debugging with ClusterWS. I've configured my VSCode to debug my TypeScript app that use ClusterWS and all work fine if I add a breakpoint into "index.ts" file or other "master" files. The problem is that if I add some breakpoint into worker process these not works.
Please have you some experience about that?
Hi Everyone, if you are developer and you are developing anything with clusterws
and watching for changes i would recommend you to watch for changes on the client libraries too. if you are JavaScript developer start watching for js library as i am going to improve documentation and other things. Please note that libraries's (JavaScript, Java, Swift) info such as release note, issues and all other things will not be posted to the ClusterWS server repository (this repository) and you may not be up to date with newer versions of libraries and bug fixes.
Thank you for the great package, I want to use it as ws
replacement, I have binary messages that are serialized and deserialized using protobuf3, but now, when I receive a message with socket.on('message', callback)
, I get the error
Error PID 17713:
SyntaxError: Unexpected token � in JSON at position 0
I really don't understand, isn't this a pure weboscket server like any other library or I have to serialze/deserilaize my messages in JSON in order to work?
Hi there (again) !
I have read and use the recent changes and I must say that ClusterWS is really helpful and saved me a lot of time. However, I was wondering if there is a way to handle failure when a worker encounters an uncaught exception.
In our project, each worker deals with its own "instances" and shares the name of these "instances" to others workers by using new middleware functions (so all workers have a list of all available instances among all workers). I want to find a way to say to others workers that instances from the failed worker should be removed from available instances.
Does that make sense for you and do you have a solution for me?
Thank you in advance ;)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.