saarasio / enroute Goto Github PK
View Code? Open in Web Editor NEWEnRoute Ingress API Gateway - Kubernetes-native, Ingress API Gateway Powered using Envoy
Home Page: https://getenroute.io
License: Apache License 2.0
EnRoute Ingress API Gateway - Kubernetes-native, Ingress API Gateway Powered using Envoy
Home Page: https://getenroute.io
License: Apache License 2.0
Currently, the following upstream fields read from the cloud are not programmed for Envoy. Need to add support for these fields.
"upstream_strategy": null,
"upstream_validation_cacertificate": null,
"upstream_validation_subjectname": null,
Right now we
cd enroute-cp && go run webapp.go
This is related to #2 . When an image is created, this should also be run
Note that when upstream_ip is not provided, the error message is - "No JSON object could be decoded". This is not very helpful.
[ec2-user@ip-172-31-18-10 enroute-cp]$ curl -s -X POST localhost:1323/upstream -H "Authorization: Bearer treehugger" -H "Content-Type: application/json" -d '{"Upstream_name":"db_serve_process", "upstream_hc_host":"127.0.0.1", "upstream_hc_path":"/v1/version", "upstream_port":"8081"}' | python -m json.tool
No JSON object could be decoded
[ec2-user@ip-172-31-18-10 enroute-cp]$ curl -s -X POST localhost:1323/upstream -H "Authorization: Bearer treehugger" -H "Content-Type: application/json" -d '{"Upstream_name":"db_serve_process", "upstream_hc_host":"127.0.0.1", "upstream_hc_path":"/v1/version", "upstream_port":"8081"}' | python -m json.tool
No JSON object could be decoded
[ec2-user@ip-172-31-18-10 enroute-cp]$ curl -s -X POST localhost:1323/upstream -H "Authorization: Bearer treehugger" -H "Content-Type: application/json" -d '{"Upstream_name":"db_serve_process", "upstream_ip":"127.0.0.1", "upstream_hc_path":"/v1/version", "upstream_port":"8081"}' | python -m json.tool
{
"Error": "Please provide a valid weight value."
}
[ec2-user@ip-172-31-18-10 enroute-cp]$ curl -s -X POST localhost:1323/upstream -H "Authorization: Bearer treehugger" -H "Content-Type: application/json" -d '{"Upstream_name":"db_serve_process", "upstream_ip":"127.0.0.1", "upstream_hc_path":"/v1/version", "upstream_port":"8081", "upstream_weight":"98"}' | python -m json.tool
{
"data": {
"insert_saaras_db_upstream": {
"affected_rows": 1
}
}
}
The error message below should say "Service_name" and not "Name"
[ec2-user@ip-172-31-18-10 enroute-cp]$ curl -s -X POST localhost:1323/service -H "Authorization: Bearer treehugger" -H "Content-Type: application/json" -d '{"Name":"adminservice"}' | python -m'' json.tool
{
"Error": "Please provide service name using Name field"
}
[ec2-user@ip-172-31-18-10 enroute-cp]$ curl -s -X POST localhost:1323/service -H "Authorization: Bearer treehugger" -H "Content-Type: application/json" -d '{"Service_Name":"adminservice"}' | python -m'' json.tool
{
"Error": "Please provide fqdn using Fqdn field"
}
For PATCH calls, Content-Type should be set to application/json when making such API calls. However, when it is missing, updates fail and there is no error message. There should be a way to check for Content-Type and throw an error. Maybe this should be settable at a group of API level.
[ec2-user@ip-172-31-18-10 ~]$ curl -s -X PATCH https://ingresspipe.io:8443/upstream/default -H "Authorization: Bearer treehugger" -d '{"Upstream_ip":"172.17.0.1"}' | python -m json.tool
{
"data": {
"update_saaras_db_upstream": {
"affected_rows": 1
}
}
}
[ec2-user@ip-172-31-18-10 ~]$ curl -s -X PATCH https://ingresspipe.io:8443/upstream/default -H "Authorization: Bearer treehugger" -d '{"Upstream_ip":"172.17.0.1"}' | python -m json.tool
{
"data": {
"update_saaras_db_upstream": {
"affected_rows": 1
}
}
}
[ec2-user@ip-172-31-18-10 ~]$ curl -s -X PATCH https://ingresspipe.io:8443/upstream/default -H "Authorization: Bearer treehugger" -H "Content-Type: application/json" -d '{"Upstream_ip":"172.17.0.1"}' | python -m json.tool
{
"data": {
"update_saaras_db_upstream": {
"affected_rows": 1
}
}
}
v5s-iMac:~ v5$ curl -s -X POST https://ingresspipe.io:8443/service -H "Content-Type: application/json" -d '{"Service_Name":"proxy_admin"}' | python -m json.tool
No JSON object could be decoded
v5s-iMac:~ v5$ curl -s -X POST https://ingresspipe.io:8443/service -H "Content-Type: application/json" -d '{"Service_Name":"proxy_admin", "Fqdn":"blah"}' | python -m json.tool
{
"data": {
"insert_saaras_db_service": {
"affected_rows": 1
}
}
}
Right now upstream ports are set to values provided by contour. They are not configurable. This value needs to be configurable from the webapp API
Build directory structure to follow project -
enroute
├── certs
├── docs
├── enroute-cp
├── enroute-dp
├── packaging
└── scripts
Related to #8
Right now service deletion requires -
(1) Remove route-upstream relationships
(2) Remove route
(3) Delete service
A deep delete operation is needed which avoids this three level operations
We need a way to copy a (service -> route -> upstream). The name of the new entities will have a prefix that will be provided in the REST call.
Related to #8
We need to fix up the webapp to make route a separate object. Besides, we also need to add support for additional route matching.
The image should -
We should be able to download it from docker hub.
When run, the port and listen IP should be configurable
We should be able to define the protocol used to communicate between CP and DP. Externally connecting DP should talk to CP over HTTPS.
Add additional parameter to enroute to take the protocol used to talk to CP.
ubuntu@ip-172-31-17-83:~/enroute$ curl localhost:1323/upstream | python -m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1042 100 1042 0 0 149k 0 --:--:-- --:--:-- --:--:-- 169k
{
"data": {
"saaras_db_upstream": [
{
"create_ts": "2019-09-09T17:38:44.516337+00:00",
"update_ts": "2019-09-09T18:32:55.590527+00:00",
"upstream_hc_healthythresholdcount": null,
"upstream_hc_host": "127.0.0.1",
"upstream_hc_intervalseconds": null,
"upstream_hc_path": "/",
"upstream_hc_timeoutseconds": null,
"upstream_hc_unhealthythresholdcount": null,
"upstream_id": 2,
"upstream_ip": "127.0.0.1",
"upstream_name": "webapp_serve_process",
"upstream_port": 1323,
"upstream_strategy": null,
"upstream_validation_cacertificate": null,
"upstream_validation_subjectname": null
},
{
"create_ts": "2019-09-09T18:28:18.684243+00:00",
"update_ts": "2019-09-10T00:06:01.290403+00:00",
"upstream_hc_healthythresholdcount": 0,
"upstream_hc_host": "127.0.0.1",
"upstream_hc_intervalseconds": 0,
"upstream_hc_path": "/v1/version",
"upstream_hc_timeoutseconds": 0,
"upstream_hc_unhealthythresholdcount": 0,
"upstream_id": 3,
"upstream_ip": "127.0.0.1",
"upstream_name": "db_serve_process",
"upstream_port": 8081,
"upstream_strategy": "",
"upstream_validation_cacertificate": "",
"upstream_validation_subjectname": ""
}
]
}
}
Related to #8
Right now demo and admin are both running on ingresspipe.io. Break up and reorganize so that only demo pieces are running on ingresspipe.io
Should be able to control logging using the webapp API. Should be configurable per application
We have the domain name and we can try and automatically generate a let's encrypt certificate
gRPC is something that is causing a lot of people to use Envoy. The gRPC configuration also makes a lot of people struggle with config.
If service is absent, a default value is created with uninitialized values. When we attach a route to a service, we need to add validation to check if the service is indeed present.
Should be able to control log level using the webapp API. Also logging should be per application
[2019-09-05 04:11:28.823][15][debug][upstream] [source/common/upstream/cds_api_impl.cc:78] cds: add/update cluster 'test/test/9001/42099b4af0' [1924/3125]
[2019-09-05 04:11:28.823][15][debug][misc] [source/common/protobuf/utility.cc:88] Proto validation error; throwing Proto constraint validation failed (ClusterValidationError.HealthChecks[i]: ["embedded message failed validation"] | caused by HealthCheckValidationError.HttpHealthCheck: ["embedded message failed validation"] | cau
sed by HttpHealthCheckValidationError.Path: ["value length must be at least " '\x01' " bytes"]): name: "test/test2/9001/da39a3ee5e"
type: EDS
eds_cluster_config {
eds_config {
api_config_source {
api_type: GRPC
grpc_services {
envoy_grpc {
cluster_name: "contour"
}
}
}
}
service_name: "test/test2"
}
connect_timeout {
nanos: 250000000
}
health_checks {
timeout {
seconds: 2
}
interval {
seconds: 10
}
unhealthy_threshold {
value: 3
}
healthy_threshold {
value: 2
}
http_health_check {
host: "contour-envoy-healthcheck"
}
}
common_lb_config {
healthy_panic_threshold {
}
}
alt_stat_name: "test_test2_9001"
drain_connections_on_host_removal: true
[2019-09-05 04:11:28.823][15][warning][config] [source/common/config/grpc_mux_subscription_impl.cc:72] gRPC config for type.googleapis.com/envoy.api.v2.Cluster rejected: Error adding/updating cluster(s) test/test2/9001/da39a3ee5e: Proto constraint validation failed (ClusterValidationError.HealthChecks[i]: ["embedded message fail
ed validation"] | caused by HealthCheckValidationError.HttpHealthCheck: ["embedded message failed validation"] | caused by HttpHealthCheckValidationError.Path: ["value length must be at least " '\x01' " bytes"]): name: "test/test2/9001/da39a3ee5e"
type: EDS
eds_cluster_config {
eds_config {
api_config_source {
api_type: GRPC
grpc_services {
envoy_grpc {
cluster_name: "contour"
}
}
}
}
service_name: "test/test2"
}
connect_timeout {
nanos: 250000000
}
health_checks {
timeout {
seconds: 2
}
interval {
seconds: 10
}
unhealthy_threshold {
value: 3
}
healthy_threshold {
value: 2
}
http_health_check {
host: "contour-envoy-healthcheck"
}
}
common_lb_config {
healthy_panic_threshold {
}
}
alt_stat_name: "test_test2_9001"
drain_connections_on_host_removal: true
[2019-09-05 04:11:28.823][15][trace][config] [source/common/config/grpc_mux_impl.cc:60] Sending DiscoveryRequest for type.googleapis.com/envoy.api.v2.Cluster: version_info: "7"
node {
id: "service-node"
cluster: "test"
build_version: "e349fb6139e4b7a59a9a359be0ea45dd61e589c5/1.11.1/Clean/RELEASE/BoringSSL"
}
type_url: "type.googleapis.com/envoy.api.v2.Cluster"
response_nonce: "8"
error_detail {
code: 13
message: "Error adding/updating cluster(s) test/test2/9001/da39a3ee5e: Proto constraint validation failed (ClusterValidationError.HealthChecks[i]: [\"embedded message failed validation\"] | caused by HealthCheckValidationError.HttpHealthCheck: [\"embedded message failed validation\"] | caused by HttpHealthCheckValidationError$
Path: [\"value length must be at least \" \'\\x01\' \" bytes\"]): name: \"test/test2/9001/da39a3ee5e\"\ntype: EDS\neds_cluster_config {\n eds_config {\n api_config_source {\n api_type: GRPC\n grpc_services {\n envoy_grpc {\n cluster_name: \"contour\"\n }\n }\n }\n }\n service_name:
\"test/test2\"\n}\nconnect_timeout {\n nanos: 250000000\n}\nhealth_checks {\n timeout {\n seconds: 2\n }\n interval {\n seconds: 10\n }\n unhealthy_threshold {\n value: 3\n }\n healthy_threshold {\n value: 2\n }\n http_health_check {\n host: \"contour-envoy-healthcheck\"\n }\n}\ncommon_lb_config {\n h$
althy_panic_threshold {\n }\n}\nalt_stat_name: \"test_test2_9001\"\ndrain_connections_on_host_removal: true\n"
}
This call fails -
v5s-iMac:~ v5$ curl -s -X GET https://ingresspipe.io:8443/proxy/adminproxy | python -m json.tool
{
"message": "Not Found"
}
Route id is a foreign key in route_upstream relationship table. Entry in this table should be deleted before the route is deleted.
When a route delete call is made, we could first delete the entries from route_upstream table.
9b1dd9:~ v3$ curl -s -X DELETE https://ingresspipe.io:8443/service/adminservice2/route/db_requests -H "Authorization: Bearer treehugger" | python -m json.tool
{
"errors": [
{
"extensions": {
"code": "constraint-violation",
"path": "$"
},
"message": "Foreign key violation. update or delete on table \"route\" violates foreign key constraint \"route_upstream_route_id_fkey\" on table \"route_upstream\""
}
]
}
Webhttp is a client to DB. It talks to DB over IP:Port. Make this IP:Port configurable
"saaras_db_proxy": [ [21/903]
{
"create_ts": "2019-09-09T17:23:26.540791+00:00",
"proxy_id": 1,
"proxy_name": "adminproxy",
"proxy_services": [
{
"service": {
"create_ts": "2019-09-09T17:31:56.327328+00:00",
"fqdn": "ingresspipe.io",
"routes": [
{
"create_ts": "2019-09-09T17:34:50.108189+00:00",
"route_id": 1,
"route_name": "dp_requests",
"route_prefix": "/v1/graphql",
"route_upstreams": [],
"update_ts": "2019-09-09T17:34:50.108189+00:00"
},
{
"create_ts": "2019-09-09T17:35:05.120026+00:00",
"route_id": 2,
"route_name": "webapp_requests",
"route_prefix": "/",
"route_upstreams": [],
"update_ts": "2019-09-09T18:32:55.590527+00:00"
},
{
"create_ts": "2019-09-09T18:28:18.684243+00:00",
"route_id": 3,
"route_name": "db_requests",
"route_prefix": null,
"route_upstreams": [
{
"upstream": {
"upstream_hc_healthythresholdcount": null,
"upstream_hc_host": null,
"upstream_hc_intervalseconds": null,
"upstream_hc_path": null,
"upstream_hc_timeoutseconds": null,
"upstream_hc_unhealthythresholdcount": null,
"upstream_id": 3,
"upstream_ip": null,
"upstream_name": "db_serve_process",
"upstream_port": null,
"upstream_strategy": null,
"upstream_validation_cacertificate": null,
"upstream_validation_subjectname": null,
"upstream_weight": 100
}
}
],
"update_ts": "2019-09-09T18:33:56.980135+00:00"
}
],
"service_id": 1,
"service_name": "adminservice",
For the above config, a listener is not configured. It is probably because health check is not programmed for the upstream. Check if this is expected behavior? The upstream here is hasura and doesn't necessarily have a health check. Should we program a listener in absence of a health check?
A work around is to setup a health check for path /v1/version right now, but this may not be the case in production.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.