A rising tide lifts all boats. -- United States President, John F. Kennedy (borrowed from the New England Council)

Tide is an automated tool to provide insight into WordPress code and highlight areas to improve the quality of plugins and themes.

We believe the web can be better. With Tide, the code which underpins every website can be more standardized, faster, and more secure. Tide is focused on WordPress, because no other platform has as large an impact on the state of the web. Tide raises the quality of code one plugin or theme at a time, by elevating the importance of code quality in the developer consciousness. Because a rising Tide lifts all boats.

• Introduction
• Dependencies
• Cloning
• Setup
  • Environment Variables
  • Google Cloud SDK
  • Google App Engine
  • Google Cloud Storage for App Engine
  • Service Account
• API
  • API Notes
  • API Settings

• MongoDB
  • MongoDB Settings

• Lighthouse Server
  • Lighthouse Server Notes
  • Lighthouse Server Settings
  • Running Lighthouse audits
• PHPCS Server
  • PHPCS Server Notes
  • PHPCS Server Settings
• Sync Server
  • Sync Server Notes
  • Sync Server Settings
• Deployments to Google Cloud Platform (GCP)
  • GCP Settings
  • Google Cloud SQL (GCSQL)
    • GCSQL API Settings
  • Google App Engine (GAE)
    • GAE API Settings
  • Google Kubernetes Engine (GKE)
    • GKE Lighthouse Server Settings
    • GKE PHPCS Server Settings
    • GKE Sync Server Settings
    • GKE Redis Settings
• Google Cloud Storage (GCS)
  • GCS Settings
• Google Cloud Firestore (GCF)
  • GCF Settings
• Amazon Web Services (AWS)
  • AWS Settings
• Contributing
• Contact Us
• Credits
• License

The main focus of this documentation is to setup a local development environment. Cloud deployments will be covered, though many assumptions will be made about your level of understanding not only with Google Cloud Platform (GCP) but also the Tide project and it's manual setup process in relation to WordPress on App Engine. We will not be going into great detail when it comes to deploying Tide on GCP.

• Install Composer
• Install Docker
• Install Go
• Install Google Cloud SDK & gsutil
• Create a new Cloud Project using the Cloud Console
• Enable Billing on that project
• Enable the Cloud SQL API (optional)

Ensure you're in the directory where you would like to install Tide:

git clone -b develop --recursive https://github.com/wptide/wptide.git tide

Change to Tide working directory:

cd tide

Update submodules:

git submodule update --init --recursive

Tide is a complicated system of services that we've tried to make easy to build and deploy both locally and on GCP. The following directions are by no means exhaustive, but should get you there without too much trouble if you follow along and do each relevant step correctly and in the following order.

Copy the .env.dist file to .env.

cp .env.dist .env

Update the value for each environment variable in your custom .env file. The variables and their descriptions can be found at the end of each relevant section. You must do this before setting up any of the services.

Additionally, you can create an .env.gcp file in the project root. This will make deploying services to GCP a bit easier since the .env.gcp file will override values in the .env file. The .env.gcp is optional for local development. If you see warnings in the console about the file missing when running certain make commands, that's ok. The .env.gcp file will only affect the .tpl files and you should only add overrides for GCP specific resources.

The .tpl files are template files that through variable interpolation are converted and used to deploy your project to GCP and even setup the local API. So these files play a critical role in getting Tide setup. If these files are not generating the correct output, please contact us to troubleshoot and figure out a solution for your OS.

So far we've only tested on OS X with and without the envsubst command available. Other systems may not work correctly and we want to resolve that quickly.

Configure Google Cloud SDK with your account and the appropriate project ID:

$ gcloud init

Create an App Engine application within your new project:

$ gcloud app create

Configure the App Engine default GCS bucket for later use. The default App Engine bucket is named YOUR_PROJECT_ID.appspot.com. Change the default Access Control List (ACL) of that bucket as follows:

$ gsutil defacl ch -u AllUsers:R gs://YOUR_PROJECT_ID.appspot.com

Note: The previous step is optional if you are setting up Tide for local development only.

If you want to upload images to WordPress then you'll need to create a bucket for those images to live (unless you opt to use the local file system). Open the Cloud Storage Browser and click Create Bucket.

Run the following command to change the ACL's of your new bucket:

$ gsutil defacl ch -u AllUsers:R gs://YOUR_BUCKET_NAME

When the API is up and running, log into the WordPress admin and go to the plugin settings page for GCS Upload then add your bucket name to the form field and click Save Settings.

Finally, go to the the Credentials section of your project in the Console. Click 'Create credentials' and then click 'Service account key.' For the Service account, select 'App Engine app default service account.' Then click 'Create' to create and download the JSON service account key to your local machine. Save it as service-account.json in the the projects root directory for use with connecting to both Cloud Storage and Cloud SQL.

First generate the API templates:

$ make api.tpl

Technically this first step can be skipped since the second command will automatically run the first command before installing Composer dependencies.

Next install the dependencies as follows:

$ make api.composer

Then start the API Docker images in isolation:

make api.up

Last run the setup script:

make api.setup

Run the setup script to initialize WordPress for the first time or if you would like a convenient way to update the default values when you change relevant environment variables.

The API implements MySQL, PHP-FPM, and an Nginx web server with WordPress installed serving a theme and a REST API.

The local database is stored in the data/api/mysql directory. If you ever need to start from scratch delete that directory and run make api.setup again. Be sure to stop the API with make api.down or make down and then start it again with make api.up.

Running make down will stop all Docker services.

If you see an error like this on OS X when bringing up the API you need to add the directory to the Preferences -> File Sharing section of the Docker for Mac app.

ERROR: for gotide_api-mysql_1  Cannot start service api-mysql: b'Mounts denied: ...'

For local development you can manually set the API_KEY and API_SECRET for the audit-server user, which will automatically update the user meta values when make api.setup is ran. If you do not set those environment variables, or are running Tide in production, then you can access the auto generated key and secret from the audit-server user profile.

By default MongoDB is used as the local message provider for the Lighthouse/PHPCS Servers and will not be supported for production use. In order for the services to use MongoDB you'll need to ensure the following settings are in you .env.

First build the Lighthouse Server Docker image:

$ make lighthouse.build.image

Next start the Lighthouse Server in isolation:

$ make lighthouse.up

You can combine the previous two steps and simply run:

$ make lighthouse.build.up

Take the isolated Lighthouse Server down:

$ make lighthouse.down

The Lighthouse Server is a Go binary that reads messages from a queue and runs Google Lighthouse reports against themes, then sends the results back to the Tide API.

Details on running Lighthouse audits are available in the Tide wiki.

First build the PHPCS Server Docker image:

$ make phpcs.build.image

Next start the PHPCS Server in isolation:

$ make phpcs.up

You can combine the previous two steps and simply run:

$ make phpcs.build.up

Take the isolated PHPCS Server down:

$ make phpcs.down

The PHPCS Server is a Go binary that reads messages from a queue and runs PHPCS reports against both plugins and themes, then sends the results back to the Tide API.

First build the Sync Server Docker image:

$ make sync.build.image

Next start the Sync Server in isolation:

$ make sync.up

You can combine the previous two steps and simply run:

$ make sync.build.up

Take the isolated Sync Server down:

$ make sync.down

The Sync Server is a Go binary that polls the WordPress.org API's for themes and plugins to process and writes them to a queue.

Deploying to GCP is optional and not required for local development. In this section we've included some of the basic steps required to get setup on GCP.

These three settings are required for both local development and deployments to GCP. Be sure to use the same region you chose during the earlier gcloud app create step.

Deploying a database to Cloud SQL for the WordPress API only requires a bit of configuration to the environment variable and then to run a single make command.

Create the Cloud SQL database:

make api.deploy.sql

This command will create a database instance and failover instance, set the root password, create a database for WordPress, and create a user for that database.

Deploying a Redis instance to Cloud Memory for the WordPress API only requires a bit of configuration to the environment variable and then to run a single make command.

Deploy the Google Cloud Memorystore Redis instance:

make api.deploy.redis

Get metadata, including the internal VPC IP address, for the Google Cloud Memorystore Redis instance:

make api.get.redis

Delete the Google Cloud Memorystore Redis instance:

make api.clean.redis

Deploying the API to App Engine is fairly straight forward. You'll need to provision a database and then deploy the app. With one caveat. The first time you deploy the API you must remove the readiness_check section from your service/api/app.yaml or your app will never become healthy. The section looks like this:

readiness_check:
  path: "/health-check.php"
  timeout_sec: 4
  check_interval_sec: 5
  failure_threshold: 2
  success_threshold: 2
  app_start_timeout_sec: 60

Once you've deployed and installed WordPress add the readiness_check section back to app.yaml and re-deploy. The health check should be working and ensuring the health of your containers on App Engine.

Deploy the API to App Engine:

make api.deploy.app

You will need to activate the plugins, create the necessary API user accounts, and setup permalinks manually.

All the goroutines are deployed with the same basic steps. Push the image to Google Container Registry (GCR), create the Kubernetes cluster, and then create a Kubernetes deployment.

I'll demonstrate with the PHPCS Server. Other than specific make commands for each service, these step are all the same.

Push the image to GCR:

make phpcs.push.image

Create the GKE cluster:

make phpcs.build.cluster

Deploy the GKE cluster:

make phpcs.deploy.cluster

Delete the GKE cluster:

make phpcs.clean.cluster

If you want to upload Tide audit reports to Google Cloud Storage then you'll need to create a bucket for those reports. Open the Cloud Storage Browser and click Create Bucket.

If you want to use Cloud Firestore as the database provider for the Sync Server or as the message provider for the Lighthouse/PHPCS Server you'll need to setup Cloud Firestore for your project by following these steps.

1. If you don't already have a Firebase project, add one in the Firebase console. The Add project dialog also gives you the option to add Firebase to an existing Google Cloud Platform project.

2. Allow read/write access on all documents to any user signed in to the application by replacing the previous security rules with the following in the Cloud Firestore console.

   service cloud.firestore {
       match /databases/{database}/documents {
           match /{document=**} {
               allow read, write: if request.auth.uid != null;
           }
       }
   }
   

3. In order for the message queue to be queryable by the Go servers you will need to add composite indexes for both the GCF_QUEUE_LH and GCF_QUEUE_PHPCS message queues. The following directions are going to be for GCF_QUEUE_PHPCS, but must be repeated for the GCF_QUEUE_LH queue. It's assumed you are using the default queue-phpcs value for GCF_QUEUE_PHPCS.

   1. Open the composite indexes tab in the Firebase console.
   2. Click Add index manually or the Add Index button if you already have created a composite index.
   3. Add queue-phpcs as the collection name.
   4. Add a new field named retry_available and set sort to ascending.
   5. Add a new field named lock and set sort to ascending.
   6. Add a new field named created and set sort to ascending.
   7. Click Create Index and repeat for queue-lighthouse.

Note: If you change the setting values below in your .env then the collection names in the third step should be the same as those values.

@todo add docs for AWS.

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Have questions? Don't open an Issue, come join us in the #tide channel in WordPress Slack. Even though Slack is a chat service, sometimes it takes several hours for community members to respond — please be patient.

Props: Bartek Makoś (@MakiBM), Brendan Woods (@brendanwoods-xwp), Daniel Louw (@danlouw), David Cramer (@davidcramer), David Lonjon (@davidlonjon), Derek Herman (@valendesigns), Jeffrey Paul (@jeffpaul), Justin Kopepasah (@kopepasah), Leo Postovoit (@postphotos), Lubos Kmetko (@luboskmetko), Luke Carbis (@lukecarbis), Miina Sikk (@miina), Mike Crantea (@mehigh), Rheinard Korf (@rheinardkorf), Rob Stinson (@robstino), Sayed Taqui (@sayedtaqui), Utkarsh Patel (@PatelUtkarsh)

Tide utilizes an MIT license.