Comments (7)
Service discovery tool: https://www.consul.io/discovery.html
from artemis.
And the Github repo: https://github.com/hashicorp/consul
from artemis.
graph TD
A[Backend Service Manager]-->B[XMLRPC SuperVisor Endpoint #1]
A-->C[...]
A-->D[XMLRPC SuperVisor Endpoint #N]
B-->E[Container Micro Services #1]
C-->F[...]
D-->G[Container Micro Services #N]
E-->H[Backend Service Discovery]
F-->H
G-->H
H-->A
The service manager communicates with a DNS-based service discovery module (consul) to register the microservices for the different containers. The discovery happens on microservice boot time. Then it (the manager) can be instructed by the frontend or other modules to control (start/stop/restart) the disaggregated microservices by addressing their Supervisor Endpoints via XMLRPC. Note that the user does not change anything in its custom supervisor configuration; simply the 5 detectors (for example) will run in their own separate container. If this is fine as an approach, we can start thinking of disaggregating more and more microservices in stages (e.g., detection, then db, etc.).
Btw, all microservices should be registered to the rmq bus (not shown on the figure for simplicity), including the service manager and discovery modules.
from artemis.
Note that since with this arch we will have a supervisor running per container, we need to break down the initial sup conf into several files and map them to the different containers. So we will need a parser and conf generator for sup confs.
from artemis.
After discussion with @slowr @pgigis , the following steps need to be made:
- Create dedicated supervisor files per microservice (e.g., monitor, detection). The user will no longer edit the services.d/supervisor.conf file, but will simply set scaling numbers in docker-compose.
- Pass the basic name of the microservice in the respective container as a variable in the docker-compose.yaml
- When a container gets up, it sends the following string to redis:
<container_name>_<hash_from_/etc/hostname>
Note that the hash from /etc/hostname
is pingable by other containers.
4) The redis registers the new service (e.g., in a set indexed by the basic name of the service)
5) When the user needs to control services using the UI, redis is consulted and the corresponding services are controlled via the supervisor endpoint (answering in the respective hash names). Note that if a certain service does not respond it can be considered as "down" and be removed from redis.
@slowr if possible check if you can create a prototype of this discussed workflow for a single container (e.g., by isolating/detaching detection or monitor; you can start with monitor).
from artemis.
@slowr Since the first part is done, but the redis queries are pending, I am moving this to 1.4.0 for completion.
from artemis.
I am going to close this ticket as the first iteration already happened. The split of the services to a separate container needs a bigger core change.
from artemis.
Related Issues (20)
- Re-evaluate the RPKI state when new routing advertisements come in HOT 2
- Unable to activate microservices in microk8s cluster HOT 4
- Remove process parallelism from re-configuration task
- No Mail sending HOT 4
- ARTEMIS Lite design and PoC HOT 5
- Don't load config file or modulies HOT 5
- Use with proxy HOT 4
- pg-amqp-bridge docker image HOT 2
- PostgreSQL Version Update HOT 1
- Disk full after some time HOT 2
- Routinator Fails to Start HOT 1
- Standard SYSLOG Handler Causes Multiple Containers to Fail to Startu (Ubuntu 20.04) HOT 4
- Writing Hijacks to SYSLOG to be consumed by Splunk (etc) HOT 1
- Not working after update HOT 2
- artemis-database fails on a fresh install in kubernetes HOT 1
- Running on k8s with tight security results in configuration container crash HOT 2
- non-root user enabled images and k8s securityContext additions
- incompatible with podman-compose - alpine musl gethostbyname and gethostbyaddr only return single ip HOT 2
- ARM64 Debian Linux 11 docker compose operation fails
- [BUG] Use the default password to continuously display unauthorized login failure
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from artemis.