Load API proxy info from config file about microgateway-core HOT 6 CLOSED

If we're going to fix this... if we're going to change the way EM determines which proxies to "deploy" and listen for, then we ought to work toward a real administrative model. That means:

• no more relying on the magical edgemicro_ name prefix
• a proper treatment in the Edge Admin API and Admin UI. "Micro Proxy" is a new concept distinct from the current "Apiproxy". It should have it's own resource in the Edge Admnin API, like /v1/o/microproxies . The behavior for the micro-proxy behavior (key or token verification, quota enforcement, etc) should be specified in Edge, not remotely. (micro-proxy may or may not be the right term)
• "deployment" is not the right verb for activating micro-proxies. They should be "enabled" or "disabled". It's the responsibility of the micro-gateway to phone home and pick up the micro-proxies.
• the "enable/disable" should be relevant to a named entity. Like an environment, but specific to edge-micro gateways. (Or remotely managed gateways). For the purposes of this discussion, let's call the new thing a "zone", but the name can be negotiated. BTW, I'm not sure how this intersects with Edge-X. So an Admin could introduce a named zone like "zone1" and enable a microproxy on that zone. Then configure an edge-micro gateway into zone1. On startup, EM phones home, asks for microproxies enabled on "zone1", and then downloads and runs all those microproxies. "zone" is just a way to map microproxies into gateways. If we think it's helpful, we could also add in the API Product concept..., so that EM gateways could get API product info.

I dislike the current model because it overloads the meaning of "apiproxy".
I dislike other approaches that continue to overload that concept, and also overload other existing concepts. Eg, "map a proxy into a product, and have the Edge Micro gateway deploy only proxies in a given product." A product already HAS a meaning, and it's not THIS. The apiproxy entity already HAS a meaning, and it is different than the thing Edge-Micro needs. Using that apiproxy entity to model the thing that Edge-Micro needs to download is confusing.

I feel that it's time to do it right, and that means not overloading existing concepts inappropriately.

from microgateway-core.

i really don't like the way - edgemicro is now plugged - proxies starting with ""edgemicro_" - i guess, if we find a way to fix it - all these issues will automagically solve by itself

ok - so lets keep @srinandan's proposal as

• Proposal1:

proxies:

name: edgemicro_httpbin revision: 1
name: edgemicro_twoway revision: 1

so here are other proposals

• Proposal2
  group one or more of the edgemicro into an environment, so micro is always tied to an environment and deploys only proxies deployed to an environment
  Pros - this feels logical, also starts as a way for us to manage and discover micro instances - no additional configuration
  Cons - may need some edge changes to accomodate env's with micro, will this become unmanageable?
• Proposal3
  tag an edgemicro to an product, so it proxies only the apis in the product.
  Pros: provides more granularity than env, now there is clear way for SDLC [env] and provisioning
  Cons: will miss discovery? [do we care?], new config

apiproducts: name: product1

from microgateway-core.

good points @DinoChiesa

• "no more relying on the magical edgemicro_ name prefix "-- 100% agree, we need to get away from it sooner
• "Micro Proxy" is a new concept -- Why do we need one? all proxies are just one and the same, the only difference is the runtime capabilities. In future, we might have runtimes that have feature parity, we want to get there. I understand this is confusing today, but we are in that path where we need not differentiate
• "deployment" is not the right verb for activating micro-proxies. -- if we want to just use 'apiproxy', then i would like to keep it as 'deployment'. But you are right - this has different meanings and not applicable for all micro usecases/patterns we are seeing. In some cases, i think even enable/disable may not make sense [i will try listing down the patterns and why some names are not applicable]. 'deployment', i think is well understood in the apigee realm, it might be good to use it as-is
• "Zones" - we are going to run into this challenge more with hybrid and multi-cloud deployments, we need a first-class qualifier. But what we need to think thro is - where does this new data model fit in, is it just a runtime isolation - in that case, /environments/{e}/zones or more of a project level isolation, in which case, /organizations/{o}/zones|projects/{id}/environments - I would prefer the latter.
  • "I dislike the current model because it overloads the meaning of "apiproxy" - Not sure dino, i think it still is a API proxy - just a different runtime. But as i mentioned earlier - it is still catching up on the feature parity

I think the right way to do this

1. Proposal 2 +
2. Zones/Project /organizations/{o}/zones|projects/{id}/environments +
3. micro runtime honoring the policy definitions configured in the 'apiproxy' - eventually this concept will not be overloaded

and we should be able to do 'proposal 2' today easily - and when Edge starts support for 'zone/project' - it would still continue to work only better - what do you think?

from microgateway-core.

re: "Why a new concept called 'microproxy'?" The answer is in your question. "all proxies are just one and the same, the only difference is the runtime capabilities." Right - and the EMG runtime is different enough that re-using the existing apiproxy concept is confusing. If I place an LDAP policy or an XMLToJSON policy in the apiproxy... EMG does nothing with that. Confusing! The EMG silently ignores many properties of the proxy. This is frustrating and misleading. Hence, we need a new entity type. You said, "[EMG] is still catching up on the feature parity". But I don't think there is a plan to make EMG replicate the behavior of the existing Java-based Edge MP. Edge-X is a different thing, distinct from EMG. Edge-X will use apiproxy. EMG (and maybe a future nginx-based or golang-based gateway, "EMGnext") should use this new simplified config model with its dedicated entity type. If we were Microsoft engineers , we'd call the entity "apiproxy2".

It would be OK, to me, if we expanded the existing apiproxy entity to include a type="micro" attribute. But this would also imply a totally different, and much simpler object schema than the existing apiproxy. And an admin client would need to be able to query for JUST the proxies that had type='micro' set.

re: deployment vs enablement. you make a good point. I concur - "deploy" is a more sensible verb.

re: zones/projects. This needs further discussion. The zone(project), in my mind is exactly analogous to "environment", except it will apply only to microproxies, and it will be lighter weight, and admins will be able to CRUD them at will. They are just "names". Just a name for a collection of proxies. At runtime, an EMG just looks for a specific zone (or project) and "runs" the proxies listed as "deployed" for that zone. No need for environments. In my thinking, environment:apiproxy::zone:microproxy . The microproxy never gets deployed to an environment, and in fact the two concepts have no relation. And the project / zone does not intersect with the existing apiproxy entity. But I may be missing something.

Maybe the model ought to be to centrally configure a set of EMGs in each zone (identified by IP address? by token?) as well, so that when any EMG phones home, it does not need to "know" which zone it is in. Edge will determine it based on the identity the EMG presents. EMG asks Edge MS, "what should I do?" and MS server says: "you are configured in zone1, and here are the proxies you need for that zone".

from microgateway-core.

right dino, i wasn't clear when i said about feature parity. the one with feature parity could be a different kind of gateway, not this EMG

ok - here is the summary of discussions we had yesterday with the team -
to support hybrid/multi-cloud deployments - we might end up having different kinds of gateway -

• one's that has feature parity and understands 'apiproxy' - lets say an Enterprise gateway

• one's that doesn't have feature parity and probably a different config model than 'apiproxy' - this is the current MG

  Lets talk about the MG, one of the argument we could make is

  if it uses a different config, why do we need to ask devs to create the edgemicro_* apiproxy. Instead, why don’t we just ask them to create config files, which MG understands

So the developer experience would look like this - only pre-req is he has a apigee account and have access to an apigee org

npm install -g edgemicro
edgemicro login
edgemicro start -f config.yaml

thats it,
But this is still going to abuse the 'apiproxy' concept in edge [maybe just temporarily], by automatically creating a 'apiproxy' which is basically a surrogate for this configuration
The reason we need this is - MG still needs all the other goodness of Edge -- [products, developers, apps, analytics] for all of which an 'apiproxy' is a pre-requisite

Since the apiproxy is just a surrogate - we might need to made them 'readonly' in the UI - based on a attribute or based on a 'environment' a proxy is deployed

So - we are not creating 'microproxy' yet - but probably just going to set an attribute to an apiproxy

The second problem we need to solve is - we need to hook up these federated runtimes/gateways into edge data model, i think this needs to be associated with environment [or an env within a zone]. Maybe we need to characterize it by explicitly saying it as a federated environment
the reason for that would be a consistent API, for eg, like this
For eg, /o/:o/e/:e/deployments gives deployments

we should talk about zones, i think we still need them and important - probably in a different thread

from microgateway-core.

There was some previous discussion about having the local configuration of edgemicro determine what API Proxies would be used. I think that could be a step backward depending on how edgemicro is being installed and managed for different customers.

We'll be deploying edgemicro for many customers on-premise with limited ability to change the "on disk" configuration after installation. The ability for edgemicro to pull down updated configuration from an Org+Env configuration within Edge Cloud is super valuable so we can centralize control of what's running. Once the new feature is added so you don't need to restart each edgemicro instance to pick up that updated configuration, that's a powerful feature. We're running homogeneous servers in most of those scenarios, which makes that approach quite straight forward.

We'll have some other cases where edgemicro is installed using puppet in environments totally under our control, so configuration of API proxies as part of the edgemicro "on disk" configuration could be managed there relatively easily.

Not sure I follow the conversation about edgemicro not respecting the Org+Env configuration that is baked into Edge Cloud. Our deployments will have those environment concepts and we'd hope all the elements of Applications and Products assigned to those environments will also be respected in edgemicro.

Totally agree that an edgemicro_ API Proxy configuration in Edge Cloud should scream out with warnings if there is anything in the configuration that won't be respected by edgemicro.

from microgateway-core.