Crowd-sourced visual interpretation of on-demand satellite imagery

• Java Development Kit (version 17)
• Clojure CLI tools 1.10
• Postgresql (version 12)
• PostGIS (version 3.2+)
• p7zip
• node 10
• Python 3

On startup, the CEO application reads a file called config.edn in the top level directory of this repository, which contains various user-configurable system parameters. A sample file called config.default.edn is provided with this repository to get you started. To begin, simply copy it to config.edn. Then edit it according to the following sections.

Please note that you must replace all values surrounded with angle brackets (e.g., “<api-key>”) with actual values for the application to work correctly. Most of the fields should be self-explanatory.

Postgresql needs to be installed on the machine that will be hosting this website. This installation task is system specific and is beyond the scope of this README, so please follow the instructions for your operating system and Postgresql version. However, please ensure that the database server’s superuser account is named “postgres” and that you know its database connection password before proceeding.

Once the Postgresql database server is running on your machine, you should navigate to the top level directory (i.e., the directory containing this README) and run the database build command as follows:

clojure -M:build-db build-all [--dev-data]

This will begin by creating a new database and role called ceo and then add the postgis and pgcrypto extensions to it. Next, the script will populate the database with the schemas, tables, and functions that are necessary for storing and processing ceo’s data. Finally, it will load some default data into these tables that is necessary for the website to function properly. You can optionally load dev data with --dev-data. This will initialize the DB with 3 users, an imagery source, and a project.

If you wish to use a live copy of the CEO database instead of the dev data, please see the documentation of the build-db command in the Triangulum README.

If you want to improve the performance of your Postgresql server, one way is to visit the pgtune website and input your system settings. This will generate a set of configuration options that you can add to the postgresql.conf file in your system-specific Postgresql data directory.

If you would like to keep these settings separate for your own reference, you can add them to a file called “performance.conf” that you should place in your Postgresql data directory. You can then import these settings into postgresql.conf by adding this line to the end of that file:

include = 'performance.conf'

Note that the Postgresql server will need to be restarted for these changes to take effect.

In order for your command line shell to find the programs installed above, you will need to update your environment variables. Under GNU/Linux, *BSD, or MacOS X, you can set these through your shell’s .*rc or .*profile configuration files in your $HOME directory. In a Windows environment, you will need to set either the USER (for the current user only) or SYSTEM (for any user) environment variables under your system settings menu.

On a Unix machine using bash (e.g., GNU/Linux or MacOS X):

export JAVA_HOME=<jdk-installation-directory>

On a Unix machine using csh or tcsh (e.g., *BSD):

setenv JAVA_HOME <jdk-installation-directory>

On a Windows machine:

JAVA_HOME = C:\Program Files\Java\jdk-X.X.X

Replace X.X.X with the JDK version installed on your computer.

On a Unix machine using bash (e.g., GNU/Linux or MacOS X):

export PATH=$PATH:$JAVA_HOME/bin

On a Unix machine using csh or tcsh (e.g., *BSD):

setenv PATH $PATH\:$JAVA_HOME/bin

On a Windows machine, add these entries to the PATH environment variable:

%JAVA_HOME%\bin
C:\Program Files\PostgreSQL\X\lib
C:\Program Files\PostgreSQL\X\bin
C:\Program Files\7-Zip

Replace X with the PostgreSQL version installed on your computer.

Copy config.default.edn to config.edn and update any settings that you wish. The base set of defaults should be sufficient to run CEO in a development setting.

Use pip to install dependencies. The python / clj interop does not support virtual environments.

pip install -r requirements.txt
pip install earthengine-api --upgrade

You will need a Google Earth Engine account to use the GeoDash. To apply for a GEE account, visit https://signup.earthengine.google.com/#!/. Once you receive an account, you can store local authentication information with:

earthengine authenticate

This command will open a window in your browser containing a key that you should copy and paste back into your terminal. You do not need to update the :gee section of config.edn if you use this method.

To be able to crate accounts without having to send email for confirmation, set :auto-validate? of :mail to true in config.edn.

There can only be one superuser account with user_uid of 1 adminstrator flag set to true. Activating this flag for other users can cause their queries to sometimes not return the full result, as some of those queries assume only one root superuser to be existing.

To compile and run the web application, navigate to the top level project directory and run:

npm install
npm run webpack-dev

Now, in a separate terminal window (because webpack needs to be running in the background), run the following commands:

npm run server-dev

The website will then be available at http://localhost:8080 unless a port is specified. These can also be configured using the :server section in your config.edn file. When using the clojure -M:server start command (which is what npm run server-dev is an alias for) an http port can be specified with -p and an https port can be specified with -P. In dev mode, server-side exceptions will be displayed in the browser and JavaScript source files will be reloaded whenever you refresh the page.

Going forward you will usually only need to run npm run webpack-dev and npm run server-dev (still in separate terminals) to get your dev environment set up. Watch for updates to SQL files or package.json and run npm run build-db-functions or npm install respectively.

From the top level project directory run:

clojure -M:check-reflection

This will emit warnings when reflection is needed to resolve Java method calls or field accesses. To resolve any that appear, add type hints to your code. Resolving all reflection warnings can improve system performance.

It is very important to change the default :session-key in config.edn. This key is used to encrypt user session data and should be unique to each deployment. The key must be exactly 16 characters long.

To set up the email server for system emails, open the “config.edn” file in the root directory of the application. Edit the default EDN object containing server details to the file, replacing the values with your own.

To enable HTTPS from within the server, view the Triangulum HTTPS page for further instructions on enabling HTTPS.

For production it is recommended that you use a service account with a key file. You can obtain your key file by logging into your service account, navigating to the account menu, and clicking “Create key > JSON”. Then, download that JSON key file and place it in the root directory of CEO. Set the email for your service account and key path in the :gee section of config.edn.

:gee {:ee-account  "[email protected]"
      :ee-key-path "ceo-gee-key.json"}

To compile and run the web application, navigate to the top level project directory and run:

npm install
npm run webpack-prod
clojure -M:build-db functions -d ceo
clojure -M:server start -m [dev|prod] [-p 8080] [-P 8443] [-r]

The website will then be available at http://localhost:8080 unless a port is specified. These can also be configured using the :server section in your config.edn file. An http port can be specified with -p and an https port can be specified with -P. In dev mode, server-side exceptions will be displayed in the browser and Clojure source files will be reloaded whenever you refresh the page. These features are disabled in prod mode. If -m is unspecified, it will default to prod mode.

View the Triangulum Systemd page for further instructions on enabling the app as a system service.

By default the server will log to standard out. If you would like to have the system log to YYYY-DD-MM.log, use the “-o path” option to specify an output path. You can either specify a path relative to the top level directory of this repository or an absolute path on your filesystem. The logger will keep the 10 most recent logs.

On each page load clojure will read the value of announcement.txt. If text is found, the value will be inserted into a HTML element that displays as a red banner at the top of the page. To add a new announcement, edit announcement.txt and add a new message. To remove the announcement, edit announcement.txt and remove all text.

Authors:

• David S. Saah (SIG)
• Matt Spencer (SIG)
• Gary W. Johnson (SIG)
• Billy Ashmall (NASA)
• Roberto Fontanarosa (FAO)
• Oliver Baldwin Edwards (SIG)
• Githika Tondapu (NASA)
• Stefano Ricci (FAO)
• Alfonso SanchezPausDiaz (FAO)
• Biplov Bhandari (SIG)

Copyright © 2016-2022 FAO.

Collect Earth Online is distributed by FAO under the terms of the MIT License. See LICENSE in this directory for more information.