There is a linker error with print_my_diff in the dcgp/wrapped_functions.hpp file being declared multiple times when separating the Expression bindings to the expression.cpp file. This probably happens because both expression.cpp and kernel_set.cpp include dcgp/kernel_set.hpp which intern includes dcgp/wrapped_functions.hpp and therefor print_my_diff is declared in two object files and the linker doesn't know which one to pick.

Currently initialising dcgp immediately returns the classes and downloads, compiles and initialises the wasm file in the background. This makes it unsafe to call one of the returned classes because the wasm file might not be fully initialised yet which results in errors.

Initialising dcgp should return a promise that resolves with the initialised classes when the WebAssembly instance is downloaded, compiled and initialised so the user knows it's safe to use the dcgp classes.

The JavaScript bindings can be tested using jest.

At least the following points need to be tested both in node and in the browser:

• createInstance
• getTypedMemory
• setInHeap
• encodeStringArray
• Expression methodes
• Kernel methodes
• KernelSet methodes

To test for the browser use puppeteer.

The generated glue code by emscripten is currently ~5000 lines but contains a lot of unnecessary functions. Smaller and more customisable glue code would be a big improvement. The customisability also ties in with issue #2.

Use a form of line search to make the learning rate adaptive.

Currently dcgp.js assumes that the dcgp.wasm file is in the current directory or in the root folder of the server. I would like to let the user of this library decide where the dcgp.wasm file should be stored. This could come in handy when using this library together with webpack to create a web app.

The limitation factor in this issue is the by emscripten created glue code dcgp.js. A start has been made in the index.js file, see the commented code. This however didn't work because the generated dcgp.wasm file needs some of the generated dcgp.js code.

WebAssembly by default only supports single threaded applications and piranha works multi-threaded. To resolve errors and remove the need for the user to set the experimental multi-threading flag in the browser piranha should be configured to only use the main thread.

Main issue is #10.

1. Create C++ functions for all the methodes in Expression.js. The C++ functions should be placed in kernel_set.cpp.
2. Bind all the commented out methodes in Expression.js to the corresponding C++ function.

For API implementation details see the python docs.

Expression.getEquation returns a string with a null byte "\0" at the end which might cause errors or inconsistencies in the JS code that uses the equation string.

A solution could be to add the following snipped before returning the equation

equation = equation.substring(0, equation.length - 1)

Setup CI so that when the master branch updates a new version of dcgp.js is published to NPM. This also implies that direct development on the master branch will be forbidden.

It would be nice if the development setup of this project could all be done with npm install. For that to work some platform specific commands need to be executed.

Node OS.platform()

I found a chromosome returned from Expression.getChromosome that contained pointers. Investigate how that could happen and how to prevent it.

Add a C++ implementation of the ES-(1+λ) algorithm as implemented in this python example.

The algorithm function header is:

// returns the loss
unsigned es_algorithm(dcgp::expression<double> *self, unsigned offsprings, unsigned max_gen, double *x, double *yt)

At the end of the algorithm the best chromosome is set in the expression, this makes it possible to get the best chromosome from JS afterwards.

Because of the progress of issue #10 it should now be possible to compile the current bindings with the unedited AuDi library and the dCGP library with flags for multi-treading. Currently a customised AuDi and dCGP version is used that disabled the functionalities that could not be compiled.

This also means that the other dependencies need to be added to the build/compile pipeline.

Change some unix specific commands to also work on windows. Places to change commands include: package.json scripts, the development install script in README.md.

Add differentiable support by compiling with AuDi.

Dependencies

• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]

Requirements

• To compile gmp a 32 bit system is required, I use i386/ubuntu:latest.
• The m4, build-essentials, gcc-multilib, git, cmake, curl, node.js and python. I use the following command to install the dependencies:

apt-get -y update
apt-get -y install build-essential m4 gcc-multilib git cmake nodejs default-jre python3.6 python3-distutils vim curl
ln /usr/bin/python3.6 /usr/bin/python

Note: the command ln /usr/bin/python3.6 /usr/bin/python is needed to access python on the comand line instead of using Python with python3.6.

Compilation steps

Boost

curl -L -o boost_1_68_0.tar.bz2 https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.bz2
tar --bzip2 -xf ./boost_1_68_0.tar.bz2
cd ./boost_1_68_0
./bootstrap.sh
# see Using boost with emscripten
sed -i 's/using gcc ;/using gcc : : "\/root\/emsdk\/emscripten\/tag-1.38.27\/em++" ;/g' project-config.jam 
./b2 variant=release threading=single --with-timer --with-chrono --with-serialization --with-system --with-test --with-iostreams --with-regex install

Eigen

# curl -L -o eigen_3_3_7.tar.bz2 http://bitbucket.org/eigen/eigen/get/3.3.7.tar.bz2
# tar --bzip2 -xf ./eigen_3_3_7.tar.bz2
# mkdir eigen-eigen-323c052e1731/build
# cd eigen-eigen-323c052e1731/build
# emconfigure cmake ..
# emmake make install
apt-get install libeigen3-dev

gmp

curl -L -o gmp_6_1_2.tar.bz2 https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
tar --bzip2 -xf ./gmp_6_1_2.tar.bz2
cd gmp-6.1.2
CC_FOR_BUILD=/usr/bin/gcc emconfigure ./configure --build i686-pc-linux-gnu --host none --disable-assembly --enable-cxx
# apply patches found in: https://github.com/marcosscriven/cgaljs/tree/master/components/gmp/patches
emmake make -j4
emmake make install

mpfr

curl -L -o mpfr-4.0.1.tar.bz2 https://www.mpfr.org/mpfr-4.0.1/mpfr-4.0.1.tar.bz2
tar --bzip2 -xf ./mpfr-4.0.1.tar.bz2
curl -L -o mpfr-4.0.1-allpatches https://www.mpfr.org/mpfr-4.0.1/allpatches
cd mpfr-4.0.1
patch -N -Z -p1 < ../mpfr-4.0.1-allpatches
emconfigure ./configure --build i686-pc-linux-gnu --host none --with-gmp-include=/usr/local/include/gmp --with-gmp-lib=/usr/local/lib
emmake make -j 4
emmake make install

mp++

git clone https://github.com/bluescarni/mppp.git
cd mppp
git checkout tags/v0.9
mkdir build && cd build
emconfigure cmake .. -DMPPP_WITH_MPFR=ON -DGMP_INCLUDE_DIR=/usr/local/include/gmp -DGMP_LIBRARY=/usr/local/lib -DMPFR_INCLUDE_DIR=/usr/local/include/mpfr -DMPFR_LIBRARY=/usr/local/lib
emmake make install

piranha

git clone https://github.com/bluescarni/piranha.git
cd piranha
git checkout tags/v0.11
mkdir build && cd build
# disable all try_compile statements in ../CMakeLists.txt and in ../cmake_modules/PiranhaCompilerLinkerSettings.cmake
emconfigure cmake .. -DGMP_INCLUDE_DIR=/usr/local/include -DMPFR_INCLUDE_DIR=/usr/local/include -DMPFR_LIBRARIES=/usr/local/lib -DGMP_LIBRARIES=/usr/local/lib -DBoost_INCLUDE_DIR=/usr/local/include -DBoost_LIBRARY_DIR_RELEASE=/usr/local/lib
emmake make install

audi

git clone https://github.com/darioizzo/audi.git
cd audi
git checkout tags/1.6.1
mkdir build && cd build
# disable try_compile statements in ../CMakeLists.txt
emconfigure cmake .. -DAUDI_WITH_MPPP=ON -DBoost_INCLUDE_DIR=/usr/local/include -DBoost_LIBRARY_DIR=/usr/local/lib -DEIGEN3_INCLUDE_DIR=/usr/local/include/eigen3 -DPiranha_INCLUDE_DIR=/usr/local/include -DGMP_INCLUDE_DIR=/usr/local/include -DGMP_LIBRARY=/usr/local/lib -DMPFR_INCLUDE_DIR=/usr/local/include -DMPFR_LIBRARY=/usr/local/lib -DAUDI_BUILD_TESTS=OFF
emmake make install

dcgp

git clone https://github.com/darioizzo/dcgp.git
mkdir dcgp/build && cd dcgp/build
# disable try_compile statements and statements with TBB in ../CMakeLists.txt 
emconfigure cmake .. -DBoost_INCLUDE_DIR=/usr/local/include -DBoost_LIBRARY_DIR_RELEASE=/usr/local/lib -DEIGEN3_INCLUDE_DIR=/usr/include/eigen3/ -DPiranha_INCLUDE_DIR=/usr/local/include/ -DGMP_INCLUDE_DIR=/usr/local/include/ -DGMP_LIBRARY=/usr/local/lib/ -DMPFR_INCLUDE_DIR=/usr/local/include/ -DMPFR_LIBRARY=/usr/local/lib/ -DAUDI_INCLUDE_DIRS=/usr/local/include/ -DDCGP_BUILD_TESTS=OFF
emmake make install

Resources

The following links might be helpful for figuring out how to compile all the necessary libraries correctly.

1. dCGP Github
2. Boost Getting Started on Unix Variants
3. Using boost with emscripten
4. Building Boost with emscripten
5. Boost Building the Unit Test Framework
6. Status on using webidl binder or embind for C++ bindings
7. Compiling GMP to WASM instructions
8. Compiling GMP/MPFR with Emscripten
9. Error when configuring gmp, missing m4
10. GMP and MPFR comilation script
11. Setting Up a 32-bit chroot Environment in Ubuntu
12. 32-bit Ubuntu docker image
13. Compiling GMP/MPFR practical example
14. GMP/MPFR/BOOST python build script with emscripten