codeoverflow-org / nodecg-io Goto Github PK

Description

As a user/streamer I am interested in a Art-Net and maybe sACN (streaming ACN) integration for professional lighting hardware.

This enables the control of professional hardware directly via Art-Net or sACN or via a bridge to DMX-512.

I might be able to look into this on my own but need to work my way around the repository first.

References

Art-Net Wikipedia: https://en.wikipedia.org/wiki/Art-Net
sACN Wikipedia: https://en.wikipedia.org/wiki/Architecture_for_Control_Networks
Art-Net npm: https://www.npmjs.com/package/artnet
sACN npm: https://www.npmjs.com/package/sacn

Description

The boilerplate for bundles can be reduced by having a function in the core which gets the core and requires a service.
Also the we should return a object which can register handlers or get the raw handler instead of having to pass all handlers into the requireService function.

Description

Currently the Wiki is just in this repo and should be moved into a separate repository just like the one of ChatOverflow.
It should also be built by a GitHub Actions and be hosted by GitHub Pages.

Progress

• Move docs to new repo
• Create gh actions workflow for automatic deployment
• Create gh actions workflow for automatic generation of dependency tree and services file
• Restructure the wiki
• Add a guide for contributing to the wiki
• Review all existing pages to make sure they are well structured and easy to understand.
  Add clarifications where needed.
• Use extensions (like admonition) to improve the overall style and look
• Make it MINZIG

Description

Currently a bundle has one callback for service client updates:

The bundle always has to check whether the passed client is undefined, which can be forgotten, especially if the developer uses JavaScript, resulting in fatal errors. Instead we should split this into two callbacks, one for when a client is available and one when not. This is also the way that jQuery and the Promise constructor handle this.

Description

During development the step of entering the password can get pretty annoying. In ChatOverflow we had an cli parameter that could be passed with a password which was used to automatically login.
We should have a similar system, either with cli parameters or environment variables.

A problem is that the framework doesn't know when all services are loaded and the PersistenceManager requires that all services have registered them selfs. Either we use a timeout like one second after initial loading or we find a way to know when nodecg has finished loading all bundles.

Obviously other bundles could get the password that is passed to the framework, but this is a feature for development only, same as the one in ChatOverflow.

References

CO cli login flag

Description

Currently all configuration is saved in non-persistant replicants. This is pretty bad because any bundle can access this and therefore can also access configs and credentials.
To fix this we should get of all replicants other than the one that holds the encrypted config. The GUI should then also only use the one with the encrypted config instead of accessing the unencrypted replicants, like it currently does.

Description

Currently when a service instance is deleted there may be still bundles that have that instance assigned to them which creates problems.
If a service instance is deleted all bundles that are assigned to is should be unset.

References

Todo comment:

Description

There currently exists way too much boilerplate in the client implementation.
To do this the implementation should be moved into a class which extends a class provided by the framework. This base class could do things like getting the core using a nodecg instance and printing a log if it is not loaded.

Description

The definition of our "native client" is not pretty clear and needs clarification.
Please refer to #68 (comment).

Description

For local development we need to have access to local modifications of e.g. types and therefore need to link the packages. Currently this is done using this hacky npm postinstall script:

Description

I would like to add mqtt subscriber and mqtt publischer services, since it would grant huge flexibility for connecting to iot devices.

Description

There should be a service of OBS using the https://github.com/Palakis/obs-websocket/ plugin.
Potential node.js library https://www.npmjs.com/package/obs-websocket-js

Description

All services that can use the OAuth2 implicit flow or PCKE for Oauth2 could be authenticated using OAuth in the GUI. It has the advantage that the user doesn't need to get any credentials and can just use the flow which saves time and makes using nodecg-io easier.

For services that need a OAuth2 flow which requires the client secret I would propose that the user has to provide its own application detail like client id and so on and for services that can use the implicit flow or PCKE like Twitch and Spotify we could have a registered application and publish the client id.

This would be implemented into the new GUI (#3) and therefore this is not really relevant to the current development and really in the future of this project.

References

OAuth discussion of ChatOverflow: codeoverflow-org/chatoverflow#168

Description

We should add a Twitch PubSub service to be able to react to e.g. channel points. Full list of supported events: https://dev.twitch.tv/docs/pubsub#topics. As a package we should use twitch-pubsub-client.
I think the currently existing nodecg-io-twitch service should be renamed with this change to nodecg-io-twitch-chat inorder to make it clear that it only handles the chat and not other parts of Twitch.

Description

You have to restart the twitch package every few days. (4-7 days)

How to reproduce

Create a plugin that requires nodecg-io-twitch (example: modwatch)
Let it run for a few days.
The bot doesn't do anything after a few days.

Error Log

 ⚠️  ircv3 WARNING Reconnecting because the last ping took over 10 seconds

Description

Once we have a state where the framework is really usable we should create a wiki like the ChatOverflow Wiki which explains how to install it, develop bundles and use the services. Currently a simple install guide is in the README but this should be in a wiki instead.

Description

Same as in ChatOverflow we should setup CI with GitHub Actions to make sure that our compiles and once we have some tests (#8) those should also be automatically run with the CI.

The How to contribute link (https://github.com/codeoverflow-org/nodecg-io/blob/master/docs/docs/contribute.md) in the readme file is broken and throws an 404 when clicked

Description

We should use a interface that all service clients need to implement. This should contain a function named getNativeClient() that returns the original client. This way we enforce our requirement that all services need to provide access to the underlying client and we also enforce this exact name.

If we decide to add other functions that should also be available on all clients we can also add them to this interface.

Enhancement idea by @sebinside on stream and on discord.

Description

Since #80 bundles use the serviceName to require a Service this will cause confusion since some services are named different in the files as it will be registered. e.g The sample bundle websocket-server registers a dependency to the service websocket-server which in the files is called nodecg-io-ws-server. This was also considered in the discord.

Here the current names + id's

References

Description

Currently there is no way to unregister handlers which a bundle has registered.

Given this scenario:

1. Have the sample twitch bundle loaded with no assigned service instance and a created twitch service instance.
2. Assign the instance to the sample bundle and see that the twitch chat is printed correctly.
3. Unassign the service instance from the bundle (setting it to "none" in the GUI), notice how the chat is still printed.
4. Reassign the service instance and notice how each message is printed twice.

This is because the bundle has registered its handler when assigning the service instance for its first time. When the service instance is unassigned the handler still lives because there is no way for the bundle to unregister its handler. When it is reassigned two handlers are registered causing the above mentioned behavior.

This is obviously fixed when you restart nodecg, but as a small workaround you can also click Save on the service instance triggering a recreation of the service client followed by a registering of the handlers by each bundle which uses this service instance which results in only one attached handler.

Here some ideas how we could fix this:

1. Create one service client for each instance that uses it. This also fixes that a different bundle could do nasty stuff to disturb other bundles like randomly stopping the service client or unregistering all handlers, but on the other side this can create problems with services like Spotify which only allow a limited amount of unique connections.
2. The bundle should supply a callback which removes the registered handlers from the given service instance. Bad because it isn't enforced and bundle developers might forget this and have to remember this when they create a new handler for some event.
3. Have the service provide a function which removes all listeners. Then if the assigned service instance to a bundle changes this function would be called which would unregister all event listeners. This client with no registered handlers would then be passed to all bundles that use this instance again so they can add their listeners again.

I'm not really happy with all of these solutions, but 1 and 3 would be acceptable I guess.

Note: services that don't make use of event callbacks wouldn't be affected. This might be services like OLA that are just HTTP and don't have any sockets and therefore no events.

Description

When currently a service instance is deleted the service client isn't stopped and lives on.
InstanceManager.deleteServiceInstance should also stop the service client if applicable before it removes the service client from the internal list.

Description

Please add a way to control items, scene and stuff on slobs based on events. Also please make it possible to listen for events.
(I want to build a "flow", which enables an element as soon as a game capture finds a target)

Description

The install guide, which is the current content of the README, should be moved to a wiki (#10) and the README should give information about the project structure and so on.

References

CO issue: codeoverflow-org/chatoverflow#123

I think enabeling the midi-input and midi-output services to deliver a list of available devices would be beneficial since it would enable at least an easier configuration.

Description

When a RCON instance exists and the server can't be reached the RCON service lets NodeCG crash because of an exception even though there is a try catch block in the createClient function of the RCON Service:

How to reproduce

1. Create a RCON Service instance while the corresponding server is running.
2. Exit NodeCG and (in my case) stop the server.
3. Run NodeCG and try to login, the error will be logged.

Error Log

UNCAUGHT EXCEPTION! NodeCG will now exit.
{ Error: connect ECONNREFUSED 127.0.0.1:25575
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1083:14)
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 25575 }

References

RCON PR: #16

Description

The IRC service just hangs when trying to create a client with a wrong password. There should be atleast some handling like a auth timeout so we can show a error message.

How to reproduce

1. Create a irc client.
2. Set any config with a wrong password.
3. Click save
4. Observe it not being saved and no error in the GUI nor the console

Error Log

None, validateConfig of the IRC Client just never returns.

References

https://discordapp.com/channels/577412066994946060/581828193196179457/757300592166043821

Description

Create a simple SLOBS API.

References

Request: #98
API document (don't know if it's the current implementation): https://stream-labs.github.io/streamlabs-obs-api-docs/docs/index.html

Edit:
This should be a low priority api. This service is only for a specific subset of OBS Users.
If the normal OBS Websocket Service works, this will be closed (cause I can't see any differences between the websocket api and Streamlabs own implementation)

Description

This Platform could need a service that can execute commands on receiving an event.
I think there should be a service that can take A structure defining the parameters and a pattern for one or more commands. These can be executed from within any bundle by sending an event with the missing information.
This should be best implemented by having one service for all commands

Description

Since commit Lint Everything when I restart nodecg everything is gone except the password. So when I enter a wrong password it says that the password is wrong. If I enter my correct password the nodecg-io dashboard opens but all services that I had configured previously are gone.

How to reproduce

Start nodecg, set up the twitch service and restart nodecg. The log into nodecg-io again.

Error Log

The only thing it says is that the configuration gets decrypted and loaded

info: [nodecg/lib/server] NodeCG running on http://localhost:9090
info: [nodecg-io-core] Decrypting and loading saved configuration.

References

Commit since it does not work: 140b14cf28d5b13696e3c673b849b7c8d0cc9ea0

How to solve this

When I remove the prototype?. before hasOwnProperty in file persistenceManager.ts (line 128 and 172) that were added by the Lint Everything commit it works.

Description

A nodecg-io installation currently uses up to 2 GB because each bundle has a copy of nodecg in node_modules for types. Also typescript is used everywhere. Therefore we should use hoisting to only have these common used dependencies in the root node_modules.
Lerna has two modes to do this. Either lerna bootstrap --hoist or lerna link convert, the first is mentioned in the official guide. Also the first seems to be more developer firendly to me as it doesn't change the dependencies to file links and stuff like this.
@Tim-Tech-Dev might have more experience with lerna and you had added hoisting in #18. Maybe you can tell me as a lerna noob whats the difference and what advantages the one has over the other.

References

PR which once contained hoisting #18
Official lerna hoisting guide: https://github.com/lerna/lerna/blob/master/doc/hoist.md

Description

It is good practice to have a linter set up that ensures that the code quality is not terrible. Maybe even add a git commit hook, but I when you just want to commit a simple typo fix for example and then you can't commit because of some unrelated linting problem I personally get pretty annoyed.
I think I saw that GitHub Actions can make automatic commit comments of the linting errors so that might also be an option, because no automatic linting will result in it not being used at all.
Running prettier on as a git commit hook might be a good idea, as it shouldn't fail and don't distract the developers and it allows us to enforce a code style in the repo.

Description

Adding a Mastodon service could be a nice addition to whom that are using federated social networks.

References

You probably could use this package.

~ Lars

Description

MessageManager has a lot of redundant code, that should be extracted into helper function to make it cleaner. Most code duplication seems to come from the conversion of our internal Result type into the two parameters that the callback requires and checking if the callback has been set, so this can be an extra function for sure.

Description

Lumia Stream connects smart home lighting and livestream chats. It follows the same principle as nodecg-io: Connecting input events (e.g. sub events, chat messages) to output triggers (e.g. changing a light's color). Lumia streams supports several services (e.g. Phillips Hue and Nanoleaf) and has several neat functions. However, it's limited to controlling light, closed source and the premium version costs $4 / month. Thus, it's probably a good idea to learn from it's potential and integrate the best of the software into nodecg-io.

It shall be examined which features of Lumia Stream represent good services or sample bundles. This might yield new issues or pull reqeuests. Additionally, Lumia Stream offers an API. Integrating this as service is also one possible solution. However, I would favor alternative approaches since nodecg-io should try to not rely on similar software products.

Last, I want to highlight that it's not my intention to steal functionality from Lumia Stream but rather to get inspired and - as the title states - learn from the realization since it can be considered related work to nodecg-io.

References

Lumia Stream: https://lumiastream.com/
Lumia Stream API: https://lumiastream.com/dev/overview

Description

Currently service clients just get killed when stopping NodeCG.
Instead all clients should be gracefully stopped when NodeCG is stopping by using the stopClient function of the corresponding services.

@sebinside wanted a elgato keylight integation for nodecg-io.

I found this npm package.

I could implement it, but I can't test it ('cause I don't have a keylight)

Description

To make sure that the user provided config actually has the correct fields and that these fields have the correct type we should validate the config against the json schema that is provided by the service.

This is needed so we can cast the config of type unknown to the config type of the service with knowing that it is actually compatible and that the service can rely on the types of its config.

First it was intended to have a replicant for each service instance and use the Replicant validation from nodecg to do this validation. But this plan has failed for various reasons (read the commit message of ff16cf3 for more information) and wasn't that great when it comes to persisting it.

So now we have to validate the config against the json schema ourself. ajv-validator/ajv seems to fit our needs here, so I'll probably use that to validate the json schema.

References

Todo comment:

Description

Since Overlays and alerts are a pretty common task when streaming nodecg-io should provide a service to make the simpler ones trivial to use.
I think the following is a good basic feature set to be versatile, pretty simple to implement, but somewhat powerful:

• It should allow configuring message topics that other services can send to trigger alerts and animations(e.g. sub alerts, ...)
• It should allow configuring replicants to display values permanently (e.g. latest follower, currently playing track)
• It should be able to have custom HTML snippets for the visual things
• It should allow triggering the alerts or manually changing the replicants via a dashboard for ease of testing

Maybe some way to display the chat could be useful, but I am not really shure if that should be part of this service.

Description

Replicants are a powerfull tool of NodeCG. They allow the sharing of data between multiple bundles and between backend and frontend code without requireing much boilerplate code.

This functionallity is probably useful for many service bundles, e.g. sharing the currently played song or sharing the latest subscriber. However, it shall be examined how we can use Replicants in the most efficient way. This includes the question if some functionality can be generalized and encapsulated by the nodecg-io-core.

This issue is probably also interesting for those who already used NodeCG.

A never ending discussion of @daniel0611 and @sebinside ...

References

More on replicants: https://nodecg.com/docs/classes/replicant
Read replicants: https://nodecg.com/docs/classes/readReplicant

Description

Currently a bundle can have access to only one service instance per service type. We need to discuss if we want to support that a bundle could use multiple instances of a single service type. This is needed if a bundle needs access to e.g. multiple twitch accounts or multiple ftp servers.

References

Todo comment:

Description

Unit test should be added at least for the core framework. Service could also use unit tests, but the core framework is more important for now I guess.

References

NodeCG mocks: nodecg/mock-nodecg

Description

As seen in #28 (comment) developers it is not really clear to developers how exactly the bundle name should be entered.

To fix this developer UX problem we should just require the nodecg instance to be passed to requireService instead of the bundle name. The nodecg instance provides a variable to get the bundleName which can then be used.

This also has the advantage that if the bundle gets renamed these references to the name get also updated automatically.
Malicious bundles can could fake the name by providing a dummy object, but that was also possibile before and way more tempting.

References

NodeCG bundleName variable

Description

Currently all Services have a try catch block in their validateConfig() and createClient() functions.
These should be pulled into the Framework to the functions which call these to reduce code completion and to prevent Services from being able to crash NodeCG if they don't handle exceptions or maybe forget.
Also this lowers the amount of work that is required to create a Service.

Description

Reacting to certain stats or information about the system can be interesting. I think it would be great to provide some or all of the following data points about the system and maybe control some that make sense:

• CPU usage
• CPU temperature
• RAM usage
• GPU usage
• GPU temperature
• Currently focussed window
• All open windows
• Process stats for an application
• Process list /application list
• The computer's uptime
• Network load for a network device
• Size of configured directories

I think the following things would make sense to provided as replicants and send events/notifications. Maybe some of the events could be requested by the other services:

• Focus change
• Certain processes starting/stopping
• Windows openings/closing

I think it should allow to change the focussed window and maybe maximize or minimize windows.
Maybe splitting it into two services could be a good idea depending on the Maintainers

Description

Make the services consistent and add abstraction funcions, where they are not present atm.
The following services do not provide abstraction:

• nodecg-io-discord
• nodecg-io-midi-input
• nodecg-io-midi-output
• nodecg-io-obs
• nodecg-io-philipshue
• nodecg-io-sacn-receiver
• nodecg-io-sacn-sender
• nodecg-io-slack
• nodecg-io-spotify
• nodecg-io-streamdeck
• nodecg-io-telegram
• nodecg-io-twitch
• nodecg-io-twitter
• nodecg-io-websocket-server

Description

The current GUI, which is written in plain html/css/ts is intented to make the project usable, but is not very user friendly, similar to the Better-REPL-GUI of ChatOverflow. As a more long term solution a new GUI in Vue should be developed that also focuses at UX.

Description

It would be nice if we could have for each service at least a very simple sample bundle.

Services with currently no sample at all:

• IRC
• RCON
• sACN sender
• Spotify
• Websocket client

Description

Currently clients aren't stopped after they are used. This results in still living handlers if a client is updated, the bundle registers its callbacks on the new client and the old client is still active, include those old callbacks.

References

Todo comment:

Description

Since devices like Blackmagic Designs Atem mini are rather popular. I think we should support Blackmagic Designs open protocol to control the atem devices.