Giter Club home page Giter Club logo

privacy-gateway-server-go's Introduction

Privacy Gateway Application Server

This project contains a gateway implementation for an Oblivious HTTP (OHTTP) gateway in Go.

Overview

This gateway implements a simple version of the Oblivious Gateway Resource as described in the specification. In particular, it accepts encapsulated Binary HTTP requests and then uses the corresponding HTTP requests to fetch a Target resource. The response from this Target is encapsulated back to the original client of the encapsulated request.

By default, the gateway exposes the following API endpoints:

  • "/gateway": An endpoint that will accept OHTTP requests, fetch the corresponding target resource, and return an OHTTP response.
  • "/gateway-echo": An endpoint that will echo the contents of the encapsulated OHTTP request back in an OHTTP response.
  • "/ohttp-configs": An endpoint that will provide an encoded KeyConfig.
  • "/health": An endpoint for inspecting the health of the gateway (returns 200 in normal conditions).

The gateway only supports the HPKE ciphersuite based on DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, and AES-128-GCM.

The gateway does not currently support key rotation. This issue tracks adding this feature.

Deployment

This section describes deployment instructions for the gateway.

Configuration Variables

The behavior of the gateway is configurable via a number of environment variables. These are explained below.

  • SEED_SECRET_KEY: This environment variable is a hex-encoded byte array representing a secret seed used to derive the gateway private and public key pair. It MUST be 32 randomly generated bytes produced from a cryptographically secure random number generator, such as /dev/urandom. See this guidance for additional information.
  • ALLOWED_TARGET_ORIGINS: This environment variable contains a comma-separated list of target origin names that the gateway is allowed to access. When configured, the gateway will only attempt to resolve requests to target origins in this list. Any other request will yield a HTTP 403 Forbidden return code.
  • CERT: This environment variable is the name of a file containing the certificate (chain) used to serve TLS connections.
  • KEY: This environment variable is the name of a file containing the private key used to serve TLS connections.

Custom Application Payloads {#custom-config}

The gateway can be configured to service Binary HTTP (BHTTP) messages or custom application payloads. To use custom applciation payloads, you must specify the type of application request and response encodings using the CUSTOM_REQUEST_TYPE and CUSTOM_RESPONSE_TYPE environment variables. For example, if you were using protobuf as the application data encoding, you might set CUSTOM_REQUEST_TYPE="message/protohttp request" and CUSTOM_RESPONSE_TYPE="message/protohttp response". See the OHTTP library and OHTTP standard for additional information about choosing custom content types. This example protobuf file contains an example protobuf encoding of HTTP messages as an alternate to BHTTP.

When specifying a custom application format, it is also required to implement a new handler for the format. This can be done by adding a new ContentType handler that implements the logic for producing an application response for your application request. As an example, if the custom content type corresponded to DNS messages, the handler might resolve the DNS query and produce an encoded DNS response. Alternatively, if using the example protobuf-based HTTP encoding, the ContentType handler might be implemented as follows:

func protobufHandler(binaryRequest []byte) ([]byte, error) {
	request := &Request{}
	if err := proto.Unmarshal(binaryRequest, request); err != nil {
		return nil, err
	}

        // Convert the protohttp Request to a http.Request equivalent value
	targetRequest, err := protoHTTPToRequest(request)
	if err != nil {
		return nil, err
	}

	client := &http.Client{}
	targetResponse, err := client.Do(targetRequest)
	if err != nil {
		return nil, err
	}

	response, err := responseToProtoHTTP(targetResponse)
	if err != nil {
		return nil, err
	}

	return proto.Marshal(response)
}

That's it!

Local development

To deploy the server locally, first acquire a TLS certificate using mkcert as follows:

$ mkcert -key-file key.pem -cert-file cert.pem 127.0.0.1 localhost

Then build and run the server as follows:

$ make all
$ CERT=cert.pem KEY=key.pem PORT=4567 ./gateway

Preconfigured deployments

Deploy deploy to Scalingo

Manual deployment

This server can also be manually deployed on any bare metal machine, or in cloud providers such as GCP. Instructions for both follow.

Bare metal

Deployment on bare metal servers, such as Equinix, can be done following the instructions below. These steps assume that git and go are both installed on the metal.

  1. Configure a certificate on the metal using certbot. Once complete, the output should be something like the following, assuming the server domain name is "example.com":
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/example.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/example.com/privkey.pem

You must configure certbot to renew this certificate periodically. The simplest way to do this is via a cron job:

$ 00 00 1 * 1 certbot renew
  1. Configure two environment variables to reference these files:
$ export CERT=/etc/letsencrypt/live/example.com/fullchain.pem
$ export KEY=/etc/letsencrypt/live/example.com/privkey.pem
  1. Clone and build the server:
$ git clone [email protected]:cloudflare/app-relay-gateway-go.git
$ cd app-relay-gateway-go
$ go build ./...
  1. Run the server:
$ PORT=443 ./gateway &

This will run the server until completion. You must configure the server to restart should it terminate prematurely.

GCP

To deploy, run:

$ gcloud app deploy

To check on its status, run:

$ gcloud app browse

To stream logs when deployed, run

$ gcloud app logs tail -s default

privacy-gateway-server-go's People

Contributors

alexander-chervony avatar bkochendorfer avatar chris-wood avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

privacy-gateway-server-go's Issues

Gateway<>Target ACL configuration

Right now the gateway will act on any valid HTTP request it receives from a client. That means if a client sends an encoded HTTP request for, say, GET github.com/index.html, it will honor it. We should probably allow the gateway to restrict the set of accessible targets. We could configure this type of ACL based on another environment variable.

BHTTP: support truncation for received known-length requests

BHTTP requests with truncated empty content and trailer field sections are not processed as expected according to BHTTP specs.

Example:

Request with truncated empty content and trailer field section (doesn't work):
0004504f5354056874747073186170692e73746167696e672e6f776865616c74682e636f6d1c2f63616c6c2f76312f75736572732f6d652f616e6f6e796d697a6564409c06416363657074106170706c69636174696f6e2f6a736f6e09582d4170692d4b65792466336161646639652d616562632d343061322d383265322d38616362643439346665326611582d4170706c69636174696f6e2d4b65792435646133323532612d323931392d343033622d393035342d3538643965653430656435620c436f6e74656e742d54797065106170706c69636174696f6e2f6a736f6e

Same request with zero-length empty content and trailer field section (works):
0004504f5354056874747073186170692e73746167696e672e6f776865616c74682e636f6d1c2f63616c6c2f76312f75736572732f6d652f616e6f6e796d697a6564409c06416363657074106170706c69636174696f6e2f6a736f6e09582d4170692d4b65792466336161646639652d616562632d343061322d383265322d38616362643439346665326611582d4170706c69636174696f6e2d4b65792435646133323532612d323931392d343033622d393035342d3538643965653430656435620c436f6e74656e742d54797065106170706c69636174696f6e2f6a736f6e0000

Add end-to-end latencies monitoring

It is beneficial to measure OHTTP-added latency before applying the protocol on a big scale.

To reliably trace the time on each step of the OHTTP pipeline in ideal world it would be needed to use same Precision Time Protocol service on all the actors: client, relay, gateway, target server. But it's quite hard to achieve since all of this is on the different networks and require substantial efforts.

But similar metrics can be implemented WITHOUT PTP servers if to be done on RELAY side with just the execution time provided by the gateway. The idea is simple: on the relay in runtime measure total execution time and request start time. Gateway should add 3 extra custom headers into the response to the gateway (along with encapsulated response): 1 gateway code execution duration; 2. Roundtrip to Target server duration; 3. Target server execution time (returned by target server and not from all endpoints but the completeness here is not needed as these network latencies to one Target endpoint or another are pretty similar to the same server/AWS env).
Having all these values from Relay, Gateway and Target the RELAY can summarise them into quite a comprehensive metrics for added latencies on every step and the network latencies in-between.

Encapsulate all errors after request decapsulation

If there's something wrong with the decapsulated request body, that error needs to be generated and returned to the client in an encapsulated response. It should not yield a 4xx response to the client.

Add key rotation support

The key is currently fixed per gateway instance. The gateway should support key rotation for applications and environments that require this feature.

Refactor metrics reporting

should be:

  • v2 prefix;
  • metric labels/tags inferred from error names;
  • custom metric calls removed afterwards

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.