TravisCI build:

Name	Version	Dockerfile	Build
nginx	1.13.x	Dockerfile
mssql	2017	Dockerfile
php54	5.4	Dockerfile
php54-debug	5.4 + xdebug 2.4.1	Dockerfile
php55	5.5	Dockerfile
php55-debug	5.5 + xdebug 2.5.5	Dockerfile
php56	5.6	Dockerfile
php56-debug	5.6 + xdebug 2.5.5	Dockerfile
php70	7.0	Dockerfile
php70-debug	7.0 + xdebug 2.6.1	Dockerfile
php71	7.1	Dockerfile
php71-debug	7.1 + xdebug 2.7.1	Dockerfile
php72	7.2	Dockerfile
php72-debug	7.2 + xdebug 2.7.1	Dockerfile
php73	7.3	Dockerfile
php73-debug	7.3 + xdebug 2.7.1	Dockerfile

This project aims to provide an easy way to start developing for Totara by providing a Docker setup.

This setup was created and tested intensively on a MAC and Linux. It works on Windows as well but wasn't tested that much yet.

Although this project started as a development environment for Totara Learn it can be used for any other PHP project.

• NGINX as a webserver
• PHP 5.4, 5.5, 5.6, 7.0, 7.1, 7.2, 7.3 to test for different versions
• PostgreSQL (9.3, 9.6, 10, 11), MariaDB (10.2.x) and MySQL (5.7.x), and Microsoft SQL Server 2017 support
• A PHPUnit and Behat setup to run tests (including Selenium)
• A mailcatcher instance to inspect mails
• Redis for caching and/or session handling
• XHProf for profiling
• XDebug installed, ready for debugging with your favorite IDE

• Totara source code: https://help.totaralearning.com/display/DEV/Getting+the+code
• Docker: https://www.docker.com (for Docker on Mac please read warning below)
• Docker-compose: https://docs.docker.com/compose/install (included in Docker for Mac/Windows)
• Mutagen: http://mutagen.io/ (optional, for more speed on Mac, not needed for Linux)
• At least 3.25GB of RAM for MSSQL

1. Clone the Totara source code (see requirements)
2. Clone this project
3. Install mutagen (optionally, recommended for MAC)
4. Copy the file .env.dist to .env and change at least the path to your local Totara source folder (LOCAL_SRC)
5. Make sure you have all the hosts in your /etc/hosts file to be able to access them via the browser

Example:

127.0.0.1   localhost totara54 totara54.debug totara54.behat totara55 totara55.debug totara55.behat totara55 totara55.debug totara56.behat totara70 totara70.debug totara70.behat totara71 totara71.debug totara71.behat totara72 totara72.debug totara72.behat totara73 totara73.debug totara73.behat

If you are already using the docker setup but you want to make sure you get the latest changes and features:

1. make sure you pull the latest code from this repository
2. and use the tpull script in the bin/ folder to pull the latest images
3. tup any alreay running containers to apply changes

tpull [all]   # updates all images already present locally by pulling the latest changes from docker hub
tpull nginx   # to update a specific image use the last part of the repository name, for example nginx resolves to docker-dev-nginx
tpull php73

Alternatively to pulling the pre-built images you can also rebuild themselve by using tbuild [container], for example tbuild php-7.3. Please note that rebuilding the images can take a while.

To speed up performance you can use a sync tool called mutagen.

This is especially relevant for Mac OS and Windows as the performance of mounted volumes on those platforms is really bad. If you are using Linux you can skip this as performance there is pretty good, almost native.

See chapter mutagen for instructions on how to set it up.

It is recommended to specify the containers you really need. The minimum you probably need is the db and the php container of your choice, the nginx container is started automatically alongside the php container.

The scripts for the following commands are located in the bin/ folder of this project. Either run the commands directly, like bin/tup, or add the bin folder to your PATH to not bother about your current folder.

tup pgsql php-7.2

If you need additional containers at a later point just run tup with the container you need:

tup php-5.6
tup mariadb
tup selenium-hub

This starts a lot of containers so consider to run only those you need.

tup

# this just stops the containers, equivalent to docker-compose stop
tstop
# this shuts all containers (and pauses an existing mutagen session) down, equivalent to docker-compose down
tdown

This project comes with a few bash scripts to simplify usage across platforms. The scripts are located in the bin/ folder. Ideally you add the bin folder to your PATH environment variable so you can run the commands from anywhere.

tdocker                           # shortcut to general docker-compose ... command
tup [containers]                  # start (all) container(s)            
tbash [container]                 # log into a container, i.e. php-7.2
tstop [container]                 # stop (all) container(s)
trestart [container]              # restart (all) container(s)
tdown                             # shutdown all containers
tstats                            # show docker stats including container names
tbuild [container]                # build (all) container(s)
tgrunt [subfolder]                # run grunt scripts in container, if you use subfolders for version pass it as 2nd argument
tscale [container]=6              # scale up the number of containers, i.e. selenium-chrome
tunit [container] [folder] [init] # run or init unit tests in given container for given version

I recommend to check out each Totara Learn version in a different subfolder below the folder LOCAL_SRC defined in .env. This enables you to access different versions without having to switch branches all the time.

This is just a suggestion which worked fine for me. There are different ways to handle this and at the end you need to decide yourself how to do it.

Make sure you have configured Totara and created the databases you need. You can connect to the databases from your host using any tools you prefer (host = localhost, use default ports).

To use the command line clients provided by the containers you can use the following commands:

# PostgreSQL
tdocker exec pgsql psql -U postgres

# MySQL / MariaDB
tdocker exec mysql mysql -u root -p"root"
tdocker exec mariadb mysql -u root -p"root"

# Microsoft SQL Server
tdocker exec php-7.1 /opt/mssql-tools/bin/sqlcmd -S mssql -U SA -P "Totara.Mssql1"

Create a database for each Totara version you would like to develop on.

Example commands:

# PostgreSQL
CREATE DATABASE totara_13;

# MariaDB/MySQL
CREATE DATABASE totara_13 DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;

# MSSQL
CREATE DATABASE totara_13 COLLATE Latin1_General_CS_AS
ALTER DATABASE totara_13 SET ANSI_NULLS ON
ALTER DATABASE totara_13 SET QUOTED_IDENTIFIER ON
ALTER DATABASE totara_13 SET READ_COMMITTED_SNAPSHOT ON;

The nginx container automatically creates a bunch of data folders ready to be used.

All data directories have to be created within the /var/www/totara/data directory to be part of the data volume and be persistent even after shutting down docker.

/var/www/totara/data/ver[versionnumber].[database]
/var/www/totara/data/ver[versionnumber].[database].phpunit
/var/www/totara/data/ver[versionnumber].[database].behat
# example
/var/www/totara/data/ver13.pgsql
/var/www/totara/data/ver13.pgsql.phpunit
/var/www/totara/data/ver13.pgsql.behat

versionnumber = 22, 24, 25, 26, 27, 29, 9, 10, 11, 12, 13 database = pgsql, mysql, mssql

To create a custom data directory just log into the nginx container (tbash nginx) and then create your custom folder inside /var/www/totara/data. You may need to create custom directories for multiple installations. If you do this, you will also need to set the correct permissions on custom directories that you create (e.g.; chown www-data:www-data /var/www/totara/data/ver13_custom.pgsql).

This is an example for the t13 branch with the 3 different databases and the correct data directories. Please note: You will need additional configuration parameters for PHPUnit and Behat. Please refer to Totara docs and have a look at config-dist.php for examples.

//=========================================================================
// 1. DATABASE SETUP
//=========================================================================
// First, you need to configure the database where all Moodle data       //
// will be stored.  This database must already have been created         //
// and a username/password created to access it.                         //


//$CFG->dbtype    = 'mysqli';
//$CFG->dbhost    = 'mysql';  // eg 'localhost' or 'db.isp.com' or IP
//$CFG->dbuser    = 'root';   // your database username
//$CFG->dbpass    = 'root';   // your database password
//$CFG->dataroot  = '/var/www/totara/data/ver13.mysql';
//$CFG->behat_dataroot = '/var/www/totara/data/ver13.mysql.behat';
//$CFG->phpunit_dataroot = '/var/www/totara/data/ver13.mysql.phpunit';

//$CFG->dbtype    = 'sqlsrv';
//$CFG->dbhost    = 'mssql';  // eg 'localhost' or 'db.isp.com' or IP
//$CFG->dbuser    = 'SA';   // your database username
//$CFG->dbpass    = 'Totara.Mssql1';   // your database password
//$CFG->dataroot  = '/var/www/totara/data/ver13.mssql';
//$CFG->behat_dataroot = '/var/www/totara/data/ver13.mssql.behat';
//$CFG->phpunit_dataroot = '/var/www/totara/data/ver13.mssql.phpunit';

$CFG->dbtype    = 'pgsql';      // 'pgsql', 'mariadb', 'mysqli', 'mssql', 'sqlsrv'
$CFG->dbhost    = 'pgsql';  // eg 'localhost' or 'db.isp.com' or IP
$CFG->dbuser    = 'postgres';   // your database username
$CFG->dbpass    = '';   // your database password
$CFG->dataroot  = '/var/www/totara/data/ver13.pgsql';
$CFG->behat_dataroot = '/var/www/totara/data/ver13.pgsql.behat';
$CFG->phpunit_dataroot = '/var/www/totara/data/ver13.pgsql.phpunit';


$CFG->dblibrary = 'native';     // 'native' only at the moment
$CFG->dbname    = 'totara_13';     // database name, eg moodle
$CFG->prefix    = 'mdl_';       // prefix to use for all table names
$CFG->dboptions = array(
    'dbpersist' => false,       // should persistent database connections be
                                //  used? set to 'false' for the most stable
                                //  setting, 'true' can improve performance
                                //  sometimes
    'dbsocket'  => false,       // should connection via UNIX socket be used?
                                //  if you set it to 'true' or custom path
                                //  here set dbhost to 'localhost',
                                //  (please note mysql is always using socket
                                //  if dbhost is 'localhost' - if you need
                                //  local port connection use '127.0.0.1')
    'dbport'    => '',          // the TCP port number to use when connecting
                                //  to the server. keep empty string for the
                                //  default port
);

Make sure your config file contains the PHPUnit configuration needed and the database is ready.

Log into one of the PHP containers:

tbash php-5.6
tbash php-7.2
# or if you need xdebug support
tbash php-5.6-debug
tbash php-7.2-debug

If your project is in a subfolder:

cd subfolder

If needed initiate the PHPUnit setup:

php admin/tool/phpunit/cli/init.php

Start the testsuite:

vendor/bin/phpunit

Make sure your config file contains the Behat configuration needed and the database is ready.

Log into one of the PHP containers:

tbash php-5.6
tbash php-7.2
# or if you need xdebug support
tbash php-5.6-debug
tbash php-7.2-debug

If your project is in a subfolder:

cd subfolder

If needed initiate the behat tests

# use --parallel=x if needed
php admin/tool/behat/cli/init.php

Run behat with:

# for t11
vendor/bin/behat
# for others use the command prompted after init, for example:
vendor/bin/behat --config /var/www/totara/data/ver9.pgsql.behat/behatrun/behat/behat.yml

By default prebuilt images from Docker Hub will be used. If you want to modify any of the containers to your needs then you can rebuild them locally with the following command:

tbuild
# or for individual images
tbuild php-7.2

You can run the cron manually by logging into a php container and run php admin/cli/cron.php from your source root.

You can also use the cron containers to run the cron automatically using crontab. Just create your own crontab files within the cron.d folder and start a cron container like:

# in the foreground
tdocker up php-7.2-cron

# in the background
tup php-7.2-cron

# access the logs anytime with
tdocker logs -f php-7.2-cron

# stop a daemonized cron container
tstop php-7.2-cron

The setup comes with mailcatcher support. Just add the following to your config and all mails will be sent to it:

$CFG->smtphosts = 'mailcatcher:25';

Open http://localhost:8080 to open the mailcatcher GUI.

If needed, modify the local port in the docker-compose.yml file.

If you want to use grunt or npm you can log into the nodejs container and issue the commands there:

tdocker run nodejs bash
# go to your source directory and
npm install
npm install grunt-cli
./node_modules/.bin/grunt

Or you use the shortcut bash script:

tgrunt
# if your project lives in the subfolder 13 then run
tgrunt 13
# if you want to run a specific grunt task
tgrunt 13 gherkinlint

This should work on all platforms but is especially relevant for Mac OS and Windows as the performance of mounted volumes on those platforms is really bad. If you are using Linux you can skip this as performance there is pretty good, almost native.

Mutagen is a two-way-sync tool with focus on performance. Read more about it here: https://mutagen.io.

It runs in the background and keeps syncing your files onto a mounted volume inside your docker containers. It's pretty performant and the delay is minimal even if you change a lot of files at once.

To use mutagen first install it. On Mac OS you can use homebrew for that or alternatively download the appropriate release file from https://github.com/havoc-io/mutagen/releases

brew install havoc-io/mutagen/mutagen

To have mutagen automatically start up with your machine

mutagen daemon register

Then start the daemon. This is a background process without the sync does not work. If you have registered the daemon with the command above you won't need to do this every time.

mutagen daemon start

To activate the use of mutagen copy the file .use-mutagen.dist to .use-mutagen.

cp .use-mutagen.dist .use-mutagen

If you then use the commands tup and tdown as described in the following chapters the correct sync session is automatically created for you.

To find out if your sync is working you can use the following command:

mutagen list

which shows something like:

⇒  mutagen list
--------------------------------------------------------------------------------
Session: 58438dbf-d8c1-43f5-ae43-61257b307574
Alpha:
        URL: /your/local/path/to/totara/src
        Connection state: Connected
Beta:
        URL: docker://totara_sync/var/www/totara/src
                DOCKER_HOST=
                DOCKER_TLS_VERIFY=
                DOCKER_CERT_PATH=
        Connection state: Connected
Status: Watching for changes
--------------------------------------------------------------------------------

You can use the session id or any part of the paths to monitor the session, for example:

mutagen monitor totara

You can customise the docker compose configurations simply adding your own .yml or .yaml compose files into the custom folder. Any containers or other options you have will automatically override any existing default container options.

The bash folder lets you add custom aliases and functions to your php containers. Any file with the .bash extension will be sourced into your php container whenever you bash into it. This is useful for when you need to run complex commands often during development, such as initialising tests. To get started, simply copy aliases.bash.dist to aliases.bash and define your aliases.