me-box / core-export-service Goto Github PK

Hi

The image for the export service is over 600 MB is their anything we can do to make it smaller?

Cheers

Tosh

As this has been metioned since plenary meeting, and also related to recent manifest v2 migration
so open an issue here to provide a place of discussion and progress tracking

Hello,
When I built the core-export-service ,using command docker build -t export-service .
The following error occured.

Please could you tell me the solution?

see https://hub.docker.com/r/databoxsystems/databox-export-service/builds/bjzgvygaqgtqhbsdxd2gu36/

Hi,

The export service currently can not communicate with the arbiter. Reported here me-box/databox#78 (comment) and confirmed on my dev machine.

Toshs-MacBook$docker ps
CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS 
6429cf83a5d8        export-service:latest      "./service"              1 hours ago        Up 1 hours         8080/tcp                                                             
da1ccd67e12b        arbiter:latest             "npm start"              1 hours ago        Up 1 hours         8080/tcp   

Toshs-MacBook$ d exec -t 6429cf83a5d8 cat /run/secrets/DATABOX_EXPORT_SERVICE_KEY
p5BDO3XZ4YYHclfg8ZMDdU8sbc0Ulw/lAnkChhfCGLk=
Toshs-MacBook$ d exec -t da1ccd67e12b cat /run/secrets/DATABOX_EXPORT_SERVICE_KEY
p5BDO3XZ4YYHclfg8ZMDdU8sbc0Ulw/lAnkChhfCGLk=

Toshs-MacBook-$ docker service logs databox_export-service
databox_export-service.1.qsiq8cerhtye@moby    | service: [ERROR] [macaroon] no secret from arbiter: Invalid API key

@sevenEng @yousefamar can you take a look.

Anil (@avsm) maintains a large set of OCaml Dockerfiles (https://github.com/ocaml/ocaml-dockerfiles) that may be useful for Dockerising this component. I think the alpine branch would work out well.

Error from logs:

databox_databox-export-service.1.m1x4rfzfez8b@moby | service: internal error, uncaught exception:
databox_databox-export-service.1.m1x4rfzfez8b@moby | (Failure "Client connection was closed")

We are moving to swarm mode (me-box/databox#42). Hence, the way in which secrets are passed around has changed.

This component needs updating to work with the new method. secrets are now in files in /run/secrets

please create a fet/swarm branch with the required changes - this can then be merged into master when the development branch on me-box/databox is merged into master.

see here for the secret names https://github.com/me-box/databox/blob/development/docker-compose.yaml

I'm currently writing "front-end" export-service code for the Databox libs, and was just wondering a few things.

• Is polling for responses the final way it's going to be? Or will we at some point also allow databox-store-blob-like WebSocket subscription events? During the platform talk, I think somebody mentioned that there may be a store sitting in front the export service to hold responses. I personally think it's better if the export service itself is sort of a pseudostore, in the sense that it may replicate store APIs (e.g. WebSocket subscription events). Seems like there are at least 3 ways we're envisioning this, so I just wanted to make sure what we're aiming at.

• Is it safe to allow apps to set IDs for export requests? (e.g. what if one app sets stupid IDs and collides with another, or a different app starts enumerating IDs to try and steal another app's responses?) I think having the response to an export request include an unguessable UUID — which you already have — is an elegant way to do it, and that that is more than enough, so allowing apps to set IDs themselves wouldn't be necessary. (Btw, it's also possible to have app "sessions" by using the macaroons kind of like cookies since they each also contain unique IDs).

• Assuming the answer to the first point is yes, and the answer to the second is no, don't we need to have export requests be different from response polling requests? (i.e. two different API endpoints?) E.g. if an app wants to export two payloads to the same destination, and by the time it wants to export the second, the first hasn't cleared the queue yet, then it would have no way to push the second payload since the export service would interpret the request as a response poll for the first payload, right? I can see how giving each payload export its own ID would solve this, but then see point 2.

All thoughts and input appreciated a usual!

Hi

the export service is currently exiting at start-up with the following error

 Fatal error: exception (Failure "TLS to non-TCP currently unsupported: host=databox-arbiter endp=(Unknown \"name resolution failed\")")

I had to merge this sooner than I would have liked me-box/databox#64

There is now a pem formatted cert in /run/secrets/DATABOX_EXPORT_SERVICE.pem

This should save you having to juggle the json into files ;-)

Can you move over to the new secrets so I can remove the old one?

Cheers

Tosh