Giter Club home page Giter Club logo

vardius / go-api-boilerplate Goto Github PK

View Code? Open in Web Editor NEW
922.0 922.0 135.0 9.93 MB

Go Server/API boilerplate using best practices DDD CQRS ES gRPC

Home Page: https://go-api-boilerplate.local

License: MIT License

Makefile 1.13% Go 76.97% Dockerfile 0.68% HTML 0.36% TypeScript 15.74% SCSS 0.84% Mustache 0.23% HCL 3.54% Smarty 0.52%
api best-practices boilerplate bootstrap cqrs ddd docker event-sourcing golang grpc helm helm-charts kubernetes microservices oauth2 oauth2-server restful rpc starter-kit telepresence

go-api-boilerplate's Introduction

Hi there πŸ‘‹

why ?

I develop out of love ❀️ to code and desire to constantly improve myself as a software developer and architect. I believe that open source projects are the future, allowing other people to contribute and improve technology by covering more edge cases on a wider spectrum of a problems.

In modern times where technology drives our lives and big IT giants spit lots of software out trying to gain monopoly in every area, open source developers are more than needed to redirect the traffic back to people and focus on world's needs rather than tracking and collecting information for money.

who am I ?

I am a Senior Software Engineer at Autopilot, a graduate of Computer Science at the Faculty of Electronics, WrocΕ‚aw University of Science and Technology where I did both my Bachelor and Masters degrees. Full stack software developer with lot of experience. Knowing the highest programming method and cutting edge technologies. I am constantly striving to learn new technologies and looking for ways to better myself in this rapidly changing industry. Beside working full time I am doing open source projects and develop my own applications. I am dedicated to my craft and I am proud of my code. I am trying to write beautiful, efficient code to a high standard, in a timely and scalable way that improves the code-base of products in meaningful ways.

my work

I am interested in DDD, CQRS, ES, gRPC, developing tools to help software work better in production, solving most common problems without overthinking it. Simple is the best but performance matters. Software I develop is often a result of the need to build solutions for problems I encounter at work. I like to explore different ways of problem solving and trying different methods to address any issues.

If you follow the same principles you are more then welcome to have a look at my work. If anything (and I hope more than one), seems interesting or useful for you, don't hesitate to contribute!

go-api-boilerplate's People

Contributors

beautyfree avatar dependabot[bot] avatar fossabot avatar hippeus avatar mar1n3r0 avatar vardius avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-api-boilerplate's Issues

panic: interface conversion

panic: interface conversion: interface {} is user.WasRegisteredWithEmail, not *user.WasRegisteredWithEmail

goroutine 65 [running]:
github.com/vardius/go-api-boilerplate/cmd/user/internal/application/eventhandler.WhenUserWasRegisteredWithEmail.func1({0xada780, 0x4000283470}, 0x4000200880)
/app/cmd/user/internal/application/eventhandler/when_user_was_registered_with_email.go:22 +0x374
github.com/vardius/go-api-boilerplate/pkg/eventbus/memory.(*eventBus).Subscribe.func1({0xada780, 0x4000283470}, 0x4000200880, 0x40002d6840)
/app/pkg/eventbus/memory/event_bus.go:107 +0xe4
reflect.Value.call({0x7a6480, 0x400045d200, 0x13}, {0x8836e1, 0x4}, {0x400011f3e0, 0x3, 0x4})
/usr/local/go/src/reflect/value.go:543 +0x590
reflect.Value.Call({0x7a6480, 0x400045d200, 0x13}, {0x400011f3e0, 0x3, 0x4})
/usr/local/go/src/reflect/value.go:339 +0x8c
github.com/vardius/message-bus.(*messageBus).Subscribe.func1(0x400045d220)
/go/pkg/mod/github.com/vardius/[email protected]/bus.go:60 +0x74
created by github.com/vardius/message-bus.(*messageBus).Subscribe
/go/pkg/mod/github.com/vardius/[email protected]/bus.go:58 +0x278

docker-compose grpc error

gRPC client-server connection is broken when running with docker-compose
When a client tries to dial server TLS handshake error from 172.22.0.1:34824: tls: oversized record received with length 21536

we need to fix containers setup

rpc error: code = Unimplemented desc = unknown service

Looking for advice... I am getting this when trying to make a gRPC call to a new service cloned from an existing one.
Wondering how could this be debugged as readiness and health checks are reporting all ok.

As far as I understand the gRPC server hasn't registered the service but no errors were reported at the time of starting.

Suspecting specifically this part:

userproto.RegisterUserServiceServer(adapter.server, adapter.userServer)

Curious to know how can we check that it was successfully registered.

If the service is added to the grpcConnectionMap of a dependent service the output is:
gRPC connection is not serving

It makes me think that there is a network issue on my side since the internal readiness probe works.

P.S. The code generated in pb.go is identical and complete.

Documentation

If

  1. You have some questions
  2. You had a problem with setup
  3. You had a problem with configuration
  4. You don't know how to do something
  5. Have ideas or propositions
  6. Think something could be explained better/in more details

Let me know so I can update documentation, allowing other ppl get over same issues faster.
Please comment here about your suggestion what should/could be added/updated in the documentation. And I will try to explain or to do it as soon as possible.

404 page not found/503 Service Temporary Unavailable

Followed that setup:
https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/

Browser request:
https://go-api-boilerplate.local/users?page=1&limit=10

Browser response:
404 page not found/503 Service Temporary Unavailable

HTTP response:

curl go-api-boilerplate.local
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>openresty/1.15.8.2</center>
</body>
</html>

HTTPS response:

curl https://go-api-boilerplate.local/users?page=1&limit=10
[1] 9013
[mar1n3r0@marin-airhead go-api-boilerplate]$ curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

kubernetes-dashboard service fails to start

The kubernetes-dashboard service also fails to start with the following message:

2019/12/21 14:14:56 Storing encryption key in a secret panic: secrets is forbidden: User "system:serviceaccount:go-api-boilerplate:kubernetes-dashboard" cannot create resource "secrets" in API group "" in the namespace "kube-system"

Originally posted by @mar1n3r0 in #23 (comment)

register-user-with-email - Unauthorized

Looking at the router code it seems to be a public route just like list-users and get-user yet it's returning unauthorized just like me and change-email-address which are behind the firewall.

That seems like a fix: #38
I get HTTP status 201 Created, it's not saved to the database though.

DEBUG: [CommandBus|Publish]: user.RegisterWithEmail {Email:[email protected]}
2019/12/25 21:12:57.910400 DEBUG: [EventBus|Publish]: user.WasRegisteredWithEmail {"id":"9c79bb8e-30e7-4695-9bbf-dc7b1950e856","metadata":{"type":"user.WasRegisteredWithEmail","stream_id":"af530f9c-98c6-474a-b0b4-cb6d3ed1017c","stream_name":"user.User","stream_version":0,"occurred_at":"2019-12-25T21:12:57.893211786Z"},"payload":{"id":"af530f9c-98c6-474a-b0b4-cb6d3ed1017c","email":"[email protected]"}}

The event was published but it seems like the event handler never subscribed.

Edit: The pubsub gRPC server didn't have the setServingStatus to serving. Not sure if the user service is the right place for it but it did the trick. The second commit should have fixed it.

the connection to userserver is down for some reason

When client tries to dial a server we get following error:

"Error": {
    "code": 14,
    "message": "all SubConns are in TransientFailure"
},

This error means the connection to userserver is down for some reason, beside the fact the server tells us it is running without error

2018/02/08 07:03:37.603287 INFO: [userserver] running at localhost:3001

Input validation, sanitizing and form/multiform/files support, OAuth2 logic

Continuing the topic here.

Because we are using CQRS patter, the way I think validation should be done is for value objects to validate incoming data.

So technically for example here:

// ChangeEmailAddress command
type ChangeEmailAddress struct {
	ID    uuid.UUID `json:"id"`
	Email string    `json:"email"`
}

each property should be a value object (VO), that validates data

// ChangeEmailAddress command
type ChangeEmailAddress struct {
	ID    uuid.UUID `json:"id"`
	Email vo.Email  `json:"email"`
}

this way you ensures that data passed to the events always conforms to the rules, i dont mind having different validation system that we could for example use in our VOs

but anyway lets first fix/merge current open prs and then we will consider other futures.
scopes are good idea, i am not sure but i dont think we should need to do much here, it should work out of box with current oauth2 implementation

Originally posted by @vardius in #39 (comment)

Maybe it's better to do the sanitizing and validation one step before.
If we ensure this happens in BuildCommandDispatchHandler where we still have the request instead of doing it later in the NewCommandFromPayload then the CQRS structs will always get the sanitized/validated data. The earlier the better, less data moving down the pipe.

This has the advantage that we can reuse a mature open-source package which has additional benefits like adding form/multiform/files support.

What do you think @vardius ?

Edit: That looks like a great candidate for both json, text/plain and form data altogether:

govalidator

So far I got my eyes on: albrow/forms as the validation happens on the http.Request level.

But since last update was 2 years ago and it's not migrated to using modules we can take a look at other options as well: awesome-go forms

The previously mentioned ozzo-validation seems to be tailored to validation of value objects level so not really suited for use with the http package directly. This kind of takes it out of the equation for now I guess.

[question] domain pollution

Hello @vardius

Thank you for sharing this library, it is very helpful in many regards!

My question is about your choice of appending the metadata in the aggregate itself. Eg.

func (u *User) trackChange(ctx context.Context, event *domain.Event) error {
	var meta domain.EventMetadata
	if i, hasIdentity := identity.FromContext(ctx); hasIdentity {
		meta.Identity = i
	}
	if m, ok := metadata.FromContext(ctx); ok {
		meta.IPAddress = m.IPAddress
		meta.UserAgent = m.UserAgent
		meta.Referer = m.Referer
	}
	if !meta.IsEmpty() {
		event.WithMetadata(&meta)
	}

	u.changes = append(u.changes, event)
	u.version++

	return nil
}

Seeing the UserAgent or IP exposed in the aggregate feels a bit like pollution of a domain with application/infra data.
I understand that the aggregate essentially proxies this data from the command handler, which usually is acceptable practice, as aggregate is not making use of that data.

I am wondering if you would not mind sharing your thoughts as to why you chose to enrich the event in the aggregate as opposed to eg. in the aggregate repository implementation.

Thanks again for sharing!

Further docs needed

Hi,

Thank you for the great repo. I think it has a lot of potential but I think it has a steep learning curve.

I would like to ask for an architectural documents, one that shows the components and one for the data flow. Also some docs about AWS ECR would be nice. I was able to generate images, tag and upload them to ECR but couldn't make k8s pull the images from ECR, I am guessing due to some docker-secret thing.

Please keep up the great work

request-access-token - oauth2: cannot fetch token: 404 Not Found/400 bad request

DEBUG: [EventBus|Subscribe]: user.AccessTokenWasRequested {"id":"f610c09b-d7c6-4311-9573-cd841dd4a819","email":"[email protected]"}
2019/12/27 16:43:57.982291 DEBUG: [EventBus|Subscribe]: user.AccessTokenWasRequested {"id":"f610c09b-d7c6-4311-9573-cd841dd4a819","email":"[email protected]"}
2019/12/27 16:43:57 [EventHandler] {"id":"f610c09b-d7c6-4311-9573-cd841dd4a819","email":"[email protected]"}
2019/12/27 16:43:57.991235 DEBUG: [EventBus|Subscribe]: user.AccessTokenWasRequested {"id":"f610c09b-d7c6-4311-9573-cd841dd4a819","email":"[email protected]"}
2019/12/27 16:43:58 [EventHandler] Error: oauth2: cannot fetch token: 404 Not Found
Response: 404 page not found

That was fixed by #43 Add /v1 prefix to oauth2Config

The next one happening is:

user_logs:

2019/12/31 20:44:40.629628 DEBUG: [EventBus|Publish]: user.AccessTokenWasRequested {"id":"d53836de-23d2-45a6-abb0-87f5abea8a2f","metadata":{"type":"user.AccessTokenWasRequested","stream_id":"00000000-0000-0000-0000-000000000000","stream_name":"user.User","stream_version":0,"occurred_at":"2019-12-31T20:44:40.628333849Z"},"payload":{"id":"00000000-0000-0000-0000-000000000000","email":""}}
2019/12/31 20:44:40.645071 INFO: [Request|End]: 7d0d9965-6268-45d7-9ceb-79e98b284ee6 : (0) : POST /v1/dispatch/request-user-access-token -> 172.17.0.8:45394 (23.73608ms)
2019/12/31 20:44:40.646782 DEBUG: [EventBus|Subscribe]: user.AccessTokenWasRequested {"id":"00000000-0000-0000-0000-000000000000","email":""}
2019/12/31 20:44:40 [EventHandler] {"id":"00000000-0000-0000-0000-000000000000","email":""}
2019/12/31 20:44:40 [EventHandler] Error: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_request","error_description":"The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed"}

auth_logs:

2019/12/31 20:57:15.329019 ERROR: oAuth2 Server response error: invalid_client
&{invalid_client 0 Client authentication failed  401 map[]}
2019/12/31 20:57:15.331306 INFO: [Request|End]: 4a8f5593-ba15-444e-844e-6a7c381bedd1 : (0) : POST /v1/token -> 172.17.0.15:56618 (5.698101ms)
2019/12/31 20:57:15.332723 INFO: [Request|Start]: 490cbe9b-dd4c-43de-92eb-5abbb59cea66 : (0) : POST /v1/token -> 172.17.0.15:56618
2019/12/31 20:57:15.333062 ERROR: oAuth2 Server response error: invalid_request
&{invalid_request 0 The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed  400 map[]}

mixed data/state between services

This actually happens very rarely and usually upon first launch of the cluster and goes back to normal after deletion of the pod(restart).
Can you please clarify if the user service is supposed to check by design the health of the auth service ?
user service logs:

RespondJSONError: <internal> gRPC connection auth is not serving
        /app/cmd/user/internal/interfaces/http/handlers/health.go:33

cert-manager - unknown field "acme" in io.cert-manager.v1alpha2.Certificate.spec

On fresh clone and install CRDs installed, happens right after make helm-install:

Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1", error validating "": error validating data: ValidationError(Certificate.spec): unknown field "acme" in io.cert-manager.v1alpha2.Certificate.spec]

Invalid `spec.hostPath.path`

Error: failed to create resource: PersistentVolume "go-api-boilerplate-app" is invalid: spec.hostPath.path: Required value

Please default to self-signed issuer with cert-manager

Hi,

I'm an engineer at Let's Encrypt. I've noticed we are getting spammed by clients using the default config at https://github.com/vardius/go-api-boilerplate/blob/ed45dac1f313c236f293575c98feff1e0da169f6/helm/app/values.yaml to repeatedly attempt issuance for "go-api-boilerplate.local" (invalid name) with an email address of "[email protected]" (also invalid).

I'd like to request that you switch to a self-signed issuer for the default config, knowing that some people will set it up without knowing that they need to customize the Let's Encrypt section of the config.

Also, it would be better to change the default email address to "[email protected]". The example.com domain name is reserved for examples, and also will be correctly rejected by Let's Encrypt if someone forgets to customize their config before requesting a certificate.

Thanks,
Jacob

Kubernetes 1.16 API breaking changes

Hello and thanks for the fantastic boilerplate. I am using it for a starter point for my own project and currently testing with minikube locally.
I will try to summarize my discoveries so far:

  1. After the breaking changes with apiVersion with kubernetes 1.16 onwards some of the helm charts don't work for me since they were not updated according to the guidelines.
    See: https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/
    Example after running helm make-install originally:
helm install --name go-api-boilerplate --namespace go-api-boilerplate helm/app/
Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"
make: *** [Makefile:64: helm-install] Error 1

As you can see with the current script it's impossible to see which chart specifically failed. Do you observe the same behavior by any chance ?
2. Because of that I have heavily modified the helm scripts to actually get latest packages from helm hub rather than use the archives included in the repo.
3. Even some of the latest stable charts are not updated as of yet to 1.16 guidelines. Namely magic-namespace and heapster so far. Due to this I have disabled those for the time being
4. Since I am installing the charts one by one and not as one whole package like the original I am facing some unexpected issues. Mostly some kubernetes services names don't match nginx ingress definitions. For example: go-api-boilerplate.user, go-api-boilerplate.auth etc... I see them as microservice-user, microservice-auth in my services list.

As soon as I have some decent workaround for the hardcoded charts I can create a PR, meanwhile I would appreciate if you had some ideas how to make those dependencies more flexible instead of tgz packages in the repo.

Originally posted by @mar1n3r0 in #15 (comment)

Domain validations

This seems like a really nicely structured repo for using go in microservices. However from reading around on ddd - domains layer I understand that some business logic (validation/business logic) should be applied on them and it should stick inside domain/aggregate itself and I can't see any examples of domain validation in this project/structure. Lets take as example domain job with "deadline", so user cant post a job if deadline is before specific date (ie from today + 2 weeks forward). This should be "validated" in domain layer itself (not inside command handlers) as I understand or I am wrong? If yes how would we handle/return errors on this one in domain with current structure?

Create command from handler

That's a curious but common use case where you have a webhook endpoint which is waiting on external events and is supposed to create commands once the expected event arrives. Since the commandDispatchHandler is tied to the router, could it be used decoupled from the HTTP request and fed params from a handler method?

External gRPC client - subscribe to events

Is it possible that we can connect an external gRPC client and subscribe it to event types the same way internal event handlers are subscribed via the eventbus ?

I saw the example with authClient being called from user event handler but this is injected and used from the service container.

https://github.com/vardius/go-api-boilerplate/blob/master/cmd/user/internal/application/eventhandler/when_user_access_token_was_requested.go

if _, err := authClient.DispatchTokenCommand(identity.ContextWithIdentity(ctx, &i), &proto.DispatchAuthCommandRequest{
			Name:    createTokenCommandName,
			Payload: nil,
		}); err != nil {
			return apperrors.Wrap(err)
		}

https://github.com/vardius/go-api-boilerplate/blob/9eb29c7fde062c9e99ec7dfcb36b8c536b1e2e8c/cmd/user/internal/domain/init.go#L65

if err := container.EventBus.Subscribe(ctx, user.AccessTokenWasRequestedType, eventhandler.WhenUserAccessTokenWasRequested(cfg, jwt.SigningMethodHS512, container.Authenticator, container.UserPersistenceRepository, container.AuthClient)); err != nil {
		return apperrors.Wrap(err)
	}

Can we connect to external client in place where container.AuthClient is ?

Other database providers

Id love to use this with mongodb it would be great if there was an example or some db abstraction to make it easy to plug in another type

CORS HandlePreflight not generating any headers

Stumbled upon this one:
cors-preflight

router.HandleFunc("/profile/tweets", ProfileTweets).Methods("GET","OPTIONS")

c := cors.New(cors.Options{
    AllowedMethods: []string{"GET","POST", "OPTIONS"},
    AllowedOrigins: []string{"*"},
    AllowCredentials: true,
    AllowedHeaders: []string{"Content-Type","Bearer","Bearer ","content-type","Origin","Accept"},
    OptionsPassthrough: true,
})

Tried to replicate the solution with gorouter:

	// Public User routes
	router.POST("/v1/dispatch/{command}", commandDispatchHandler)
	// Handle preflight
	router.OPTIONS("/v1/dispatch/{command}", commandDispatchHandler)

Is there an option we can actually assign 2 methods to a single route at the same time ?

terraform apply

I tried make terraform-install and result is

cd k8s \
  kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.crds.yaml \
terraform init \
terraform apply \
  terraform output -raw templates | kubectl apply -f -
error: no objects passed to apply
make: *** [Makefile:66: terraform-install] Error 1

when I try command by comand then by kubectl apply is result this...


Error: Post "http://localhost/api/v1/namespaces": dial tcp 127.0.0.1:80: connect: connection refused
β”‚ 
β”‚   with kubernetes_namespace.main,
β”‚   on k8s-namespace.tf line 1, in resource "kubernetes_namespace" "main":
β”‚    1: resource "kubernetes_namespace" "main" {
β”‚ 
β•΅
β•·
β”‚ Error: Post "http://localhost/apis/storage.k8s.io/v1/storageclasses": dial tcp 127.0.0.1:80: connect: connection refused
β”‚ 
β”‚   with kubernetes_storage_class.mongodb,
β”‚   on mongodb.tf line 28, in resource "kubernetes_storage_class" "mongodb":
β”‚   28: resource "kubernetes_storage_class" "mongodb" {

Can you help me? Please

Deleting namespaces

Deleting namespace will stuck at Terminating state. More info here: kubernetes/kubernetes#60807

This happens only with CRDs. By deleting the CRD object, it gets in state were the objects owned by it were not deleted.
Temporary solution is to edit namespaces and empty finalisers.

Perform following:

  1. Delete namespace
βœ— make helm-delet
  1. Dump namespace to json
kubectl get namespace go-api-boilerplate -o json > tmp.json
  1. Edit tmp.json and remove"kubernetes" from finalisers
  2. Run proxy
kubectl proxy
  1. Upload edited json
curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/\go-api-boilerplate/finalize

If you are running newest version of k8s instead of doing all steps you can just run one command as follow:

kubectl get namespace go-api-boilerplate -o json \
            | tr -d "\n" | sed "s/\"finalizers\": \[[^]]\+\]/\"finalizers\": []/" \
            | kubectl replace --raw /api/v1/namespaces/go-api-boilerplate/finalize -f -

This way you will not have to spawn a kubectl proxy process and avoid dependency with curl. kubernetes/kubernetes#60807 (comment)

pubsub not subscribing to all topics but no errors

Edit that fixes it I think: #44

Reference: using-goroutines-on-loop-iterator-variables

Tested with register-with-user-email so far:

DEBUG: [CommandBus|Publish]: user.RegisterWithEmail {Email:[email protected]}
2019/12/28 15:21:35.993717 DEBUG: [EventBus|Publish]: user.WasRegisteredWithEmail {"id":"9b8d95a1-e9dc-4ef7-9c88-428898b0d0a3","metadata":{"type":"user.WasRegisteredWithEmail","stream_id":"cd8d6723-75d4-4d5a-bb1f-904cde2c45e2","stream_name":"user.User","stream_version":0,"occurred_at":"2019-12-28T15:21:35.9931463Z"},"payload":{"id":"cd8d6723-75d4-4d5a-bb1f-904cde2c45e2","email":"[email protected]"}}

Command and event buses publish but the event handlers are a hit or miss to be registered and subscribed for topics. Can't seem to find a pattern yet as to when it works and when not.
The easiest way to reproduce it is to open a tab with list users, after I create a user it doesn't appear in the list 9 out of 10 restarts of the system.

Replicated locally, the issue persists, seems like event handlers are not registering for all topics.

Narrowed it down to this part:
cmd/user/internal/application/eventhandler/register.go

// Register registers event handlers for topics
// will panic after timeout if unable to register handlers
func Register(conn *grpc.ClientConn, eventBus eventbus.EventBus, topicToHandlerMap map[string]eventbus.EventHandler, timeout time.Duration) {
	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()
	fmt.Printf("topicToHandlerMap: %v \n", topicToHandlerMap)
	connName := "pubsub"

	// Will retry infinitely until timeouts by context (after 5 seconds)
	_, err := gollback.New(ctx).Retry(0, func(ctx context.Context) (interface{}, error) {
		//fmt.Printf("conn: %v", conn)
		fmt.Printf("isConnectionServing %s: %t \n", connName, grpc_utils.IsConnectionServing(connName, conn))
		if grpc_utils.IsConnectionServing(connName, conn) {

			for topic, handler := range topicToHandlerMap {
				fmt.Printf("for topic: %v \n", topic)
				// Will resubscribe to handler on error infinitely
				go gollback.New(context.Background()).Retry(0, func(ctx context.Context) (interface{}, error) {
					fmt.Printf("gollback topic: %v \n", topic)
					err := eventBus.Subscribe(ctx, topic, handler)
					fmt.Printf("err: %v \n", err)
					return nil, errors.Newf(errors.INTERNAL, "EventHandler %s unsubscribed (%v)", topic, err)
				})
			}

			return nil, nil
		}

		return nil, errors.Newf(" %s gRPC connection is not serving", connName)
	})

	if err != nil {
		panic(err)
	}
}

user logs:

isConnectionServing pubsub: true 
for topic: user.AccessTokenWasRequested 
for topic: user.WasRegisteredWithEmail 
for topic: user.WasRegisteredWithGoogle 
for topic: user.WasRegisteredWithFacebook 
for topic: user.EmailAddressWasChanged 
gollback topic: user.EmailAddressWasChanged 
gollback topic: user.EmailAddressWasChanged 
gollback topic: user.EmailAddressWasChanged 
gollback topic: user.EmailAddressWasChanged 
2019/12/29 16:25:00.862596 INFO: [EventBus|Subscribe]: user.EmailAddressWasChanged
2019/12/29 16:25:00.862612 INFO: [EventBus|Subscribe]: user.EmailAddressWasChanged
gollback topic: user.EmailAddressWasChanged 
2019/12/29 16:25:00.862598 INFO: [EventBus|Subscribe]: user.EmailAddressWasChanged
2019/12/29 16:25:00.862609 INFO: [EventBus|Subscribe]: user.EmailAddressWasChanged
2019/12/29 16:25:00.862665 INFO: [EventBus|Subscribe]: user.EmailAddressWasChanged

The async nature of gollback is somehow producing unexpected results with the sync for loop as it is repeating some topics and skipping others.
pubsub logs:

2019/12/29 15:34:23.166428 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:34:23.166430 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:34:23.166434 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:34:23.166456 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:34:23.167126 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:35:43.166624 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:35:43.166624 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:35:43.166651 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:35:43.166651 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:35:43.166775 INFO: gRPC Server|Subscribe] user.WasRegisteredWithGoogle
2019/12/29 15:37:53.446528 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:37:53.446543 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:37:53.446530 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:37:53.446547 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:37:53.446584 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:39:43.447461 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:39:43.447473 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:39:43.447467 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:39:43.447487 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:39:43.447472 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:42:23.449741 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:42:23.449757 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:42:23.449766 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:42:23.449782 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested
2019/12/29 15:42:23.449789 INFO: gRPC Server|Subscribe] user.AccessTokenWasRequested

Remove deprecated magic-namespace

magic-namespace deprecation notice

Note that this chart manages namespaces and also Helm 2 Tiller instances within those namespaces. Helm 3 releases are scoped to namespaces and therefore cannot create namespaces. Helm 3 also does not utilize the Tiller component. Helm 3, therefore, is both incapable of installing this chart and has no practical need for this chart to begin with. With this being the case, maintainers have no intention of remediating this chart's incompatibilities with Helm 3, as that would undermine the chart's ongoing usefulness to Helm 2 users without deriving any benefit to Helm 3 users.

In conclusion, consider this chart to be deprecated, with updates and support ending on the same schedule as Helm 2 itself.

I guess with that in mind we can completely remove magic-namespace.

Originally posted by @mar1n3r0 in #46 (comment)

Returning data through the command bus

Hey brother! I want to first thank you for this awesome boilerplate. It's helped me immensely in starting my ES project. You are awesome!

I hope this isn't too naive a question (I've been working with golang for maybe 5 months or so), but I'm trying to return data to the user upon successful command executions (particularly an EventStoreDB commit position and sometimes the ID of a new aggregate root), and I can't seem to figure out how to return data to the user once it's been published to the command bus through the dispatch handler. I can figure out how to send errors, but if there's no error, how can I return a 200 with something like

{
  "id":5849
}

Perhaps I'm just missing something..

Using external gRPC clients

Went quickly through the pubsub and pushpull setup. What do you think will be required to have event sourcing working with external gRPC clients ?

Also do you believe using third-party clients is conflicting with CQRS as a pattern since write and read would happen over gPRC?

Init containers migration failing

I have locked migrate/migrate to v4.7.1 and managed to get the specific errors from the init containers:

error: open /migrations/auth: no such file or directory
error: open /migrations/user: no such file or directory

The migrations directory is not copied to the final image. Expected migration files for example auth service

- 'file:///migrations/auth'

Copied files are:
COPY --from=buildenv /go/bin/app /go/bin/app

Doing so should solve the issue, with migrations

FROM scratch
COPY --from=buildenv /go/bin/app /go/bin/app
+ COPY --from=buildenv /app/migrations/"$BIN" /migrations/"$BIN"
ENTRYPOINT ["/go/bin/app"]

This needs to be done for both auth and user service.

Originally posted by @vardius in #22 (comment)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.