Reusability - grpc about blast HOT 13 CLOSED

Hi @joeblew99 ,
Sounds good.:)
I'll replace to grpc-gateway from the current HTTP server.

from blast.

I had a look at the GRPC code you have now.
Wondering if you knew about GRPC Gateway back then or not ?

What do you think of this approach ?

1. Client (some MicroService written in any language).

• They could use swagger ( from GRPC gateway ) to codegen their own code.
• Or they can take the GRPC and codegen their own code.
• Or they can just make raw HTTP calls, which would be pretty aweful because the API and types of Blast are not that simple. Types especially are quite complex and deeply needed.
• All 3 options above use HTTP. Websockets are not needed.

2. Blast Server

• Runs GRPC and GRPC Gateway, so that client can use either.

About the CLI.
I saw some golang lib that can take a GRPC API and code gen the golang CLI with the viper code inside. Its really code generating a CLI can then has a GRPC Client that calls the GRP Server. DO you think this is a good idea ? I like it because it makes it much easier to refactor code when needed.

Security.
If someone wanted to use Blast in a Multi-Tenant setup, i wonder if they can use caddy to restrict access to different parts of the Data Blast holds ?
I cant actually think of a way except to run different Instances of blast and just lets Caddy on top and restrict by Domain / IP address.

from blast.

@joeblew99
I released v0.8.0, rewritten to grpc-gateway.
I'm not familiar with swagger, but it creates swagger.json. However, expected swagger.json could not be generated because Bleve's SearchRequest and SearchResult use Protocol buffers's any.proto. I will continue to investigate it.
Currently, the user will need to write a raw HTTP request directly.

Can you tell me a few about the CLI library? I'll try it out.

I agree with your opinion about multi-tenancy. Blast does not plan for multi-tenant support yet.

Thanks,

from blast.

@mosuka what was the protoc command you used? Probably an extra param for proto_path is needed.

from blast.

hey @mosuka awesome stuff. I will have a look soon.

About the GRPC tools.
There is a general list here:
https://github.com/grpc-ecosystem/awesome-grpc#tools-cli

May Favs:

CLI for GRPC
https://github.com/fullstorydev/grpcurl

• pretty cool

Web GUI for GRPC
https://github.com/fullstorydev/grpcui
OR
https://github.com/gogo/letmegrpc

Proxy
https://github.com/improbable-eng/grpc-web

Just ask if this is not enough....

• helps cognitively.

from blast.

Hi @l4u @joeblew99 ,
swagger.json has been generated.
https://github.com/mosuka/blast/blob/master/protobuf/index/index.swagger.json

However, the response specification is not what I expected. For example, the response body specification of GetResponse will be any.proto.
This is because google.protobuf.Any is used for fields in GetResponse.

I would like to hear your opinion. Which is the preferred response body?

1. Current response body using custom json marshaler:

{
  "fields": {
    "_type": "enwiki",
    "text_en": "A search engine is an ...",
    "timestamp": "2018-07-04T...",
    "title_en": "Search engine..."
  }
}

2. the response body of the original grpc-gateway:

{
  "value": {
    "type_url": "map[string]interface {}",
    "value": " ... base64 encoded string ... "
  }
}

I have never used swagger so I don't know which one is better.

Thanks,

from blast.

@mosuka

do you have to use google.protobuf.Any ? What is the reason for it ?

Also i noticed your not usin the "openapiv2" options. See:
https://github.com/johanbrandhorst/grpc-gateway-boilerplate/blob/master/proto/example.proto#L13

Here is boilerplate Project BTW.
https://github.com/johanbrandhorst/grpc-gateway-boilerplate/blob/master/proto/example.proto

from blast.

@joeblew99

The index mapping specified at startup determines the field name and type of the document. Therefore, since it is difficult to handle as Struct, Blast handles the document and as map[string]interface{}.

In Protocol Buffers, it is difficult to express map[string]interface{}, bleve.SearchRequest and bleve.SearchResult, so I use the Any message type.

from blast.

@mosuka

Sorry i got side tracked on something else.
It seems you did the commit anyway ?

from blast.

@joeblew99
I had implement grpc-gateway in v0.8.0. However swagger is still incomplete.

from blast.

I found a much easier solution to the grpc aspects.

Started to use envoy to run the grpc-web layer.
It is very recommend.

Gloo from solo makes it easy.

We just run it with standard kubernetes.

See here : https://github.com/getcouragenow/packages/tree/master/maintemplate/server/deployments/helm/templates

Blast and cete are then much simpler.

We plan to stand up with in our cluster.
I can send you you the PR when we are done with the line manifests to run it if you want ??

from blast.

@gedw99
Sure. That sounds great.
PR for the grpc-web layer is welcome. :)

from blast.

from blast.