Discussion: Media Upload about pyro-api HOT 7 CLOSED

Selection of Cloud service provider
Not sure I understand your question. I think we have all the required layers. The rest are nice-to-have.
On AWS I know there are regular historical dumps of the Data. That could be useful in case the service goes down. We also don't need load balancers I think as the number of queries will always be quite small (I assume).

Another nice-to-have would be a monitoring tool :

• For the API; e.g to know which devices are queried the most or to detect a surge in requests.
• For the DB to have ready-made queries (using metabase for example although I don't know how easy It is to develop).
  Aditionnaly, a throttling component could be useful to limit malicious attempts to overload the API.

Client-side API
I created the discussion #24

from pyro-api.

Shall we close it as the linked PR ha been merged? I agree the topic is not fully closed though as we don't have a proper service for now.

from pyro-api.

Hi @florianriche,
Thanks for mentioning this topic!

Selection of Cloud service provider
Three things to consider here: ease of usage, pricing, ethics.
AWS is quite easy to use, and I wouldn't have much issue going with that option but we have to discuss this more largely as this is a big decision (cf. cc).
Now whatever we choose, I believe it's best to make sure we separate the CSP interaction parts of the code from the rest (in case we intend to change the provider later).

Credential management
If you are referring to secrets & co, perhaps we should use an .env file?

Client-side API
I suggest we move forward with a client-side python package, so that we can easily install it on multiple environments and it can plug in with other python modules.

cc @x0s @MateoLostanlen @2ndcouteau @blenzi @Akilditu @fe51 @jeanpasquier75

from pyro-api.

Selection of Cloud service provider
Now that I think of it, I wonder if it would be easier to convince public institutions to use our solutions if we rely on french providers (OVH, Scaleway...). That would be good PR- wise.
Agree for the independence regarding the rest of the code. In the implementation of the linked PR, I tried to abstract the Cloud serviceProvider.

Credential management
I was more talking about the way AWS handles credentials, there are several ways, it can be hardcoded in a config file somewhere on the hard drive, there can be public/private keys... but a simple .env file seems indeed the GOTO solution for a first step.

Client-side API
Shall we open a new Github repository to tackle that subject or just add some code to the CV repo?

from pyro-api.

Selection of Cloud service provider
I mentioned this, because we've just had very recently an opportunity with a CSP that is using servers to provide heating to offices. However we're still unsure about the timeline here. @florianriche in your opinion, on the short-term, what is our prime requirement in regards to CSP that are not filled currently? (DB persistency?)

Client-side API
Perhaps, let's first open an issue in this repo to draft a design doc / specs for it, to make sure we don't created extra repos that are not required.

from pyro-api.

Thanks for creating all the other issues!
Back to the main topic here, whatever the choice of CSP, we need to:

• add a key/path field to the media table that will locate the content file once uploaded (by default None = not uploaded)
• add a route for uploading that will be called by the client-side (registered device)

The second one depends on the CSP, but perhaps for now, we can make a "mock" upload and still design the key/path update?

from pyro-api.

Good thinking, I agree!

from pyro-api.