Giter Club home page Giter Club logo

pages-proxy's Introduction

pages-proxy

Proxies traffic from the Cloud.gov Pages (formerly Federalist) S3 bucket to a CDN broker. Ensures HTTPS and adds the proper headers.

Usage

To deploy the app:

$ cf push <app-name> --strategy rolling --vars-file </path/to/vars-file> -f </path/to/manifest>

If the rolling deployment fails for any reason, make sure to clean up by running: $ cf cancel-deployment

Proxying a Site

When a site is added to Pages, it will be available through this proxy at https://<site-bucket>.sites.pages.cloud.gov/site/<owner>/<repo>. When the site is ready to go live, a CloudFront distribution with the proxy URL as its origin can be provisioned. This is done via the Admin UI panel in the '/domains' tab by creating a domain, ensuring correct CNAME configuration, and clicking "Provision".

Headers

This proxy adds the following headers to the response from the S3 bucket:

  • Strict-Transport-Security: max-age=31536000; preload
  • X-Frame-Options: SAMEORIGIN
  • X-Server: Cloud.gov Pages

Unique Site Headers

To support sites with expanded HSTS headers, the proxy uses the {{ INCLUDE_SUBDOMAINS }} environment variable to identify these requests to provide the expanded header Strict-Transport-Security: max-age=31536000; includeSubDomains; preload. If these site domains change for any reason, the {{ INCLUDE_SUBDOMAINS }} variable will need to be updated in the manifest.yml.

Short-term Site Redirects

To support short-term site redirects, the proxy uses an included redirects config which is built during deployment via a credhub json credential named proxy-<env>-site-redirects and set as an environment variable as SITE_REDIRECTS. The site redirects value is an array of objects with the following structure:

Key Required? Default Description
subdomain N/A The site's Pages subdomain
target N/A The target domain for the redirect
path '' An optional path appended to the redirect target
usePreviewPath false An optional boolean to append the site's preview path to redirect target

Local setup

Install Depedencies

  docker-compose run --no-deps --rm app npm install

Running tests against the mock server

  docker-compose run --no-deps --rm app npm run parse
  docker-compose run --rm app npm test

Running tests against s3 buckets

  docker-compose run --no-deps --rm app npm run parse:integration
  docker-compose run --rm app npm run test:integration

Continuous Integration

The proxy uses Concourse CI to run tests and deploy to different environments in the cloud.gov Pages organization. The pipeline is defined in the ci/pipeline.yml file and supporting CI scripts are found in the ci directory. This pipeline is using Concourse's instanced pipeline feature to minimize boilerplate configuration when declaring tasks and resources for each deployment environment.

↓ NOTICE ↓

Proxy Dev deploys the proxy app when a PR is created into the staging branch. This uses a unique pipeline file: ./ci/pipeline-dev.yml

Pipeline instance variables

Two instances of the pipeline are set for the staging and production environments. Instance variables are used to fill in Concourse pipeline parameter variables bearing the same name as the instance variable. See more on Concourse vars. Each instance of the pipeline has two instance variables associated to it: deploy-env & git-branch.

Instance Variable Dev Environment Staging Environment Production Environment
deploy-env dev staging production
git-branch staging staging main

Pipeline credentials

Concourse CI integrates directly with Credhub to provide access to credentials/secrets at job runtime. When a job is started, Concourse will resolve the parameters within the pipeline with the latest credentials using the double parentheses notation (ie. ((<credential-name>))). See more about the credentials lookup rules.

Some credentials in this pipeline are "compound" credentials that use the pipeline's instance variable in conjuction with its parameterized variables to pull the correct Credhub credentials based on the pipeline instance. The following parameters are used in the proxy pipeline:

Parameter Description Is Compound
((((deploy-env))-cf-username)) The deployment environments CloudFoundry deployer username based on the instanced pipeline
((((deploy-env))-cf-username)) The deployment environments CloudFoundry deployer password based on the instanced pipeline
((dedicated-aws-access-key-id)) AWS access key for testing
((dedicated-aws-secret-access-key)) AWS secret key for testing
((slack-channel)) Slack channel
((slack-username)) Slack username
((slack-icon-url)) Slack icon url
((slack-webhook-url)) Slack webhook url
((git-base-url)) The base url to the git server's HTTP endpoint
((proxy-repository-path)) The url path to the repository
((gh-access-token)) The Github access token
((pages-proxy-((deploy-env))-site-redirects)) JSON array of redirect objects
((federalist-proxy-((deploy-env))-site-redirects)) JSON array of redirect objects

Setting up the pipeline

The pipeline and each of it's instances will only need to be set once per instance to create the initial pipeline. After the pipelines are set, updates to the respective git-branch source will automatically set the pipeline with any updates. See the set_pipeline step for more information. Run the following command with the fly CLI to set a pipeline instance:

$ fly -t <Concourse CI Target Name> set-pipeline -p proxy \
  -c ci/pipeline.yml \
  -i git-branch=main \
  -i deploy-env=production

Getting or deleting a pipeline instance from the CLI

To get a pipeline instance's config or destroy a pipeline instance, Run the following command with the fly CLI to set a pipeline:

## Get a pipeline instance config
$ fly -t <Concourse CI Target Name> get-pipeline \
  -p proxy/deploy-env:production,git-branch:main

## Destroy a pipeline
$ fly -t <Concourse CI Target Name> destroy-pipeline \
  -p proxy/deploy-env:production,git-branch:main

Production pages-proxy pipeline transition

We are currently migrating from Federalist to Pages. The migration includes maintaining the former "Federalist" components of the platform to smoothly transition our customers and their sites. The CI configuration for this deployment pipeline can be found in ci/federalist-pipeline.yml. This pipeline will serve to manage the existing federalist-proxy during the transition until it can be decommissioned.

Notes

When making changes

In order for changes to the nginx.conf file or mock server to be reflected when running the tests, the dockers services must be restarted. This can be done by running docker-compose down before the above commands to parse the nginx.conf and run the tests.

Integration tests

Integration tests use the following S3 buckets provisioned in the sandbox space in the gsa-18f-federalist cloud.gov organization:

  • proxy-integration-test-dedicated

Before running the tests, make a copy of the .env.sample file named .env and populate with the credentials from corresponding service keys proxy-integration-test-dedicated-key in the sandbox space. Ex. cf t -s sandbox && cf service-key proxy-integration-test-dedicated proxy-integration-test-dedicated-key.

pages-proxy's People

Contributors

davemcorwin avatar apburnes avatar amirbey avatar jseppi avatar jmhooper avatar drewbo avatar brandocalrissian avatar svenaas avatar hursey013 avatar dependabot[bot] avatar bengerman13 avatar pburkholder avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.