sometimes we might need something that manages "jobs", something like sidekiq or kue

dump ideas on how it should work, and maybe create a PoC project

There are a few configuration options that probably every microservice should have.

Below is a list:

• PORT = What port to listen on. Note that it could differ from the LOCATION variable, simply because some middleware (docker, nginx, reverse proxy, firewall, router, etc.) could be changing the real port the service should be communicated with.
• LOCATION = How to communicate with the microservice, if someone were to connect to it. Ref: #1

One thing that I do a lot is overcomplicate the system, making it overly complex, and ultimately unwieldy, or never finished. This is a design step that needs some thinking: How do we decide when we have a good MVP to start with?

One of my goals here is to provide a simple, standardized flowchart (?) of how to put the various pieces of microservices together. One size never fits all well, and because of that, we want to give people options on doing things, e.g. having a "simple but limited" way of doing things, with a "complex but powerful" way of doing things, and so forth, as well as providing ideas on how to "upgrade" to the next complexity stage fairly smoothly.

Another goal is language agnosticism, so only provide libraries as examples only.

Considering how I am working on XBPF at my full-time job, I am working on designing and then implementing a microservice system over at my job.

Currently, it's in some weird transitional state, but it's definitely something to keep in mind when braindumping ideas into this repo: Providing my experiences with implementing such a system.

this isn't really related, but i'm doing a quick braindump from a piece of paper i had:

• ok: 200 (OK) -> resource actions?
• security/validation:
  • 400 (Bad request) [bad/malformed data sent]
  • 413 (Too Large) [bigger payload than allowed]
  • 429 (Too many requests)
• routing:
  • 404 (Route not found) [route vs resource?]
  • 501 (Will implement, however)
• error/internal:
  • 500 (Server error) [generic]
  • 503 (Unavailable) [covers 502]
  • ??? (Client error) [generic] -> Resource not found? Treated like OK?
• security/auth:
  • 403 Forbidden
  • 401 Unauthorized (?, less secure?)

There are sooooooo many ways to configure microservices. I'm currently thinking of the "default" way to configure them.

For starters, I really like environment variables, simply because of how flexible they are, but they definitely are inflexible. Maybe using a combination?

I don't believe you should build a system with microservices right off the bat. I feel you should start with a monolith, and then break it down more and more when you know what you will use microservices for for sure.

The question here is: How do you make it so you can write a monolithic app that can be relatively easily decomposed later? Are there any programming patterns to follow?

Eventually, you'd want to stabilize a service's API, so it doesn't change often.

The problem here is that I get stuck with API design: Is it good enough? Will it cover the cases I want it to cover? How will I be able to easily change it? I get overloaded and just locked up with those thoughts, sometimes for days on end.

So there needs to be a development process where you can just write bad code and a messy for a while to get things done, and then figure out when you want to clean it all up, and stabilize it into a clean API with good code.

This is the current process I currently thought up off the top of my head:

1. basic planning (think of what it should do, what you'll need)
2. implementing (dirty, don't care about code quality or tests)
3. integrating/testing (play with it some, fix any big bugs and figure out any problems you'll need to solve in the future)
4. deciding stability (wait for a bit, fully integrate it into the system as a "beta" service, see how well it works)
5. go back to step 1 if necessary (needs a full rewrite? needs a full set of features?)
6. advanced planning (fine-grain design of the code and API, and how it effectively works; DRY/KISS, etc.)
7. rewrite, including writing tests
8. reintegration/testing (ironing out the small problems)
9. stabilization (v1.0! v2.0! v3.0!)

http status codes are annoying and there are too many of them and they're bloated and they suck
and the rest are too generic

there should be fewer, more understandable codes that specify the general (common) situation, and then let the user specify a sub-code if they want to manually handle some specific case

• What do we use?
• How do we use it?

More questions later.

with a distributed system, it's much harder to have atomic operations, so you have to manage atomics on a more fine grained level, but also be able to consistently distribute events so the whole system is at an eventually consistent state, which includes being able to gracefully handle any error at any point in time

think about:

• eventual consistency in an event-based system
• atomic event groups? (event bundling?)
• duplication checking, and knowing when duplication is ok
• absolute operations (don't "increment by 5", but "set to 10")

There are two main patterns for service discovery: Registration (and maintenance) by the service itself, and registration by a third party that pretty much monitors the service, and watches its health.

Currently, I am leaning towards service's own registration, but having the "health" part be maintained by a third party. It's kind of a mix of both, and it could be messy.

One thing that's been on my mind is: How do we do zero-downtime deployments with databases? How will the services handle migrations? How should migrations be kept track of and run?

Another thing to keep in note: How do we version the changes? What do we do in case of a breaking change? We do we do in case of a migration that needs to be run atomically, otherwise it will be in a consistent state while it's running?