This the is the project directory of my (Anton Fuxjaegers, s1455952) MScR Project. Roughly speaking the presented program consists of 2 subprograms namely:

• Phase one: the lowlevel feature learning part, done by autoencoders (implemented in PyTorch)
• Phase two: the highlevel feature learning part as well as evaluation, done by PSDDs (implemented in scala/python)

• Python >= 3.5 (https://www.python.org/) -- numpy(http://www.numpy.org/) -- scipy (https://www.scipy.org/) -- pytorch (https://pytorch.org/) -- tqdm (https://pypi.org/project/tqdm/) -- matplotlib (https://matplotlib.org/)
• scala (https://scala-lang.org/) -- corresponding sbt compiler
• cuda (https://developer.nvidia.com/cuda-downloads)
• (optional) Graphviz (https://graphviz.org/)

When using the code, it is recommended to use miniconda as a package manager. All experiments and test where run using miniconda with the packages listed above

The datasets used for the experiments are contained in the ./dataset directory and where downloaded from the following websites:

• MNIST: http://yann.lecun.com/exdb/mnist/
• FASHION: https://github.com/zalandoresearch/fashion-mnist/tree/master/data/fashion using the ./datasets/download_make_fashion_dataset.py file which also splits the files into train, valid and test set
• EMNIST: https://www.nist.gov/node/1298471/emnist-dataset

In order to execute a given experiments two programs have to be executed separately preferably on separate clusters corresponding to a GPU cluster for AE learning and a multi-CPU cluster for the PSDD learning. (in this order)

graph LR
A{learn AE} --> B[convert dataset]
B --> C{learn VTREE}
C --> D{learn PSDD}
D --> E(measure accuracy)
E --> F(evaluate model by sampling)
F --> G(analyse FL)

The code base used for learning the AEs from data can be found in the directory ./src/lowlevel/ where the main execution file is located at: ./src/lowlevel/main.py. Information about arguments can be obtained by navigating to the the directory ./src/lowlevel/ and running the command:

python main.py -h

Further information is found in the option parser files located at: ./src/lowlevel/options/.

When it comes to the experiments we conducted on the mentioned datasets, please navigate to the directory ./tasks/lowlevel_gpu/ for example execution commands. The files in this directory are slurm scripts allocating 2 GPUs with 32GB memory before learning a convolutional-variational-autoencoder with 32 latent boolean (categorical dimention 2) variables (FL). Such a file is present for each of the 3 datasets used (MNIST, FASHION, EMNIST). As such the scipts are executed using the sbatch command on any cluster running slurm.

example (can be run on cpu): navigate to ./tasks/lowlevel_gpu/ and run chmod 755 ./trainAE_mnist_example.sh then ./trainAE_mnist_example.sh full version navigate to ./tasks/lowlevel_gpu/ and run sbatch trainAE_mnist_slurm.sh

The code base for this part of the experiments can be found in two seperate locations: ./src/learnPSDD/ for learning and ./src/Scala-LearnPSDD/ for evalutation. Each of those programs has to be compiled if the target does not exists yet, which is done by navigating to the corresponding directory and executing the command:

sbt assembly

Once the source is build a target directory is created containing executable .jar files. Each of the programs is then run by executing the cmd below from the corresponding directory:

java -jar ./target/scala-2.11/psdd.jar -h

with the optional -h for inforation on arguments in the corresponding directory. However in order to mitigate this process we created a well documented python wrapper that handles building and execution of the scala programs located at ./src/learn_psdd_wrapper.py. Within the first lines of this file the user should specify if the program graphviz is installed on the machine for (optional) visualization perposed.

Now when it comes to handle experimetns within this phase yet another file wraps the privously mentioned wrapper such that the experiment flow is easier to manage. This files can be found at: ./src/experiment.py. Within this files methonds and classed are defined that handle hyperparameters as well as the learning and evaluation of a given experiment, such that this file wrapps the learn_psdd_wrapper as well as the ./src/lowlevel/main.py. This makes it possible to seamlessly generated samples from the PSDD before decoding them using the NN with one command. To illustrate this the ./src/experiment.py is used by the scripts we used for running experiments located in the folder: ./tasks/highlevel_cpu/. Within these scipts a experiment is created where the experiment_parent_name has to correspond to the a previously run lowlevel AE experiment such that the pretrained NNs can be used to encode and ecode the data. This is illustrated with the examples present in the directory. Furthermore we see that in the scripts we can specify what parts of the experiment we want to run, such as training, evaluation, analysis or all of them. Finally it shoud be mentined at this point, that we can reference the same AE in multiple differnt Phase two tasks such as a classification task and a blxor task. This again is illustrated with the files present in the directory. These scipt files are run using the python interpreter such that one shoudl navigate to the directory before running a command like:

example: navigate to ./tasks/highlevel_cpu/ and run python train_eval_mnist_example.py (about 10 min) example: navigate to ./tasks/highlevel_cpu/ and run python train_eval_mnist.py (about 2 days)