This is a monorepo for my class project and inlab, outlab deliverables for the NE591-008 course, titled Mathematical and Computational Methods in Nuclear Engineering. The course aims to provide a theoretical foundation of mathematical methods applied broadly in nuclear engineering and to construct algorithms to implement the resulting formalisms on digital computers. The labs involve designing computer programs in low-level languages (exclusively Fortran or C++) and their implementation, verification, and testing. I have chosen to organize this repo in a manner that builds on the cumulative code changes from each deliverable, and reuses as many helper functions as possible. A secondary objective is to benchmark my implementations against the C++17, CUDA, and Boost library functions.

1. Locating Source Files
2. Building
   • Build Environments
     • EOS
     • Hazel
   • Class Project
   • InLabs and OutLabs
   • Optional CMake Arguments
3. Docker Usage Guide
   • Why Docker?
   • Building the Docker Image
   • Running the Docker Container
   • Building and Running Tests
4. Development - JetBrains Clion Support
   • Prerequisites
   • Clion Docker Toolchain
     • Instructions
     • Troubleshooting Windows Issues
   • Clion Remote Debugging via SSH
5. License
6. FAQs

Sources and file paths for each deliverable follow the following structure:

├── README.md   <= You are now here
├── src
│   ├── project     <= Class project deliverables
│   │   ├── project1
│   │   │   ├── main.cpp         <= Stub entrypoint main() method
│   │   │   ├── project1.cpp     <= run() method
│   │   │   ├── Compute.h        <= Core compute logic
│   │   │   ├── Parser.h         <= Argument parsing and error checking 
│   │   │   ├── InputOutputs.h   <= Struct definitions for file I/O
│   │   │   ├── README.md        <= Usage and examples
│   │   │   ├── examples         <= Directory with example inputs and outputs
│   │   │   ├── tests            <= CTests implemented using Google Test Suite
│   │   │   ├── ...
│   │   ├── project2
│   │   ├── ...
│   ├── labs        <= InLab, OutLab deliverables
│   │   ├── inlab1
│   │   │   ├── main.cpp         <= Stub entrypoint main() method
│   │   │   ├── inlab1.cpp       <= run() method
│   │   │   ├── Compute.h        <= Core compute logic
│   │   │   ├── Parser.h         <= Argument parsing and error checking 
│   │   │   ├── InputOutputs.h   <= Struct definitions for file I/O
│   │   │   ├── README.md        <= Usage and examples
│   │   │   ├── examples         <= Directory with example inputs and outputs
│   │   │   ├── tests            <= CTests implemented using Google Test Suite
│   │   │   ├── ...
│   │   ├── outlab1
│   │   ├── inlab2
│   │   ├── outlab2
│   │   ├── ...
│   ├── utils                        <= Helper methods
│   │   ├── math                     
│   │   │   ├── blas                 <= Self implemented Math, BLAS libraries
│   │   ├── CheckBounds.h
│   │   ├── CommandLine.h            <= Other helper functions
│   │   ├── ...
├── ...
├── external                     <= External dependencies

The utils directory contains helper methods used across the project. These include a helper math library, methods for checking bounds, interacting with the command line, I/O operations on CSV and JSON files, general helper functions, and a high-precision profiler for performance analysis.

The code has been built and tested on the NCSU Hazellogin.hpc.ncsu.edu and NCSU EOS remote.eos.ncsu.edu servers. It requires no additional configuration except choosing the build target, and output file. Here is a repeatable script to perform the build and run the project2 target executable:

There are two ways to do run jobs on Hazel, currently, we support the interactive mode option.

This mode requires a few additional steps, but for the most part, it follows the same process as the EOS server.

#!/bin/bash

# Start by logging in
ssh login.hpc.ncsu.edu

# Then, start an interactive session
bsub -Is -n 4 -R "span[hosts=1]" -W 20 bash

# Create a working directory, in this case we call it earthperson_project2 to avoid naming conflicts
mkdir -p /share/$GROUP/$USER/earthperson_project2

# Go into that directory.
cd /share/$GROUP/$USER/earthperson_project2

# Load the build modules
module load openmpi-gcc/openmpi4.1.0-gcc10.2.0 cmake/3.24.1

# Begin by copying the files over using rsync, scp, sftp, etc...
# Then, assuming the repo root is the current directory:

## Specify the build target
export BUILD_TARGET=project2

## Create the build directory, configure and compile the $BUILD_TARGET
mkdir -p build && cd build && \
cmake .. -DCMAKE_BUILD_TYPE=Release && \
make -j$(nproc) $BUILD_TARGET && cd ../

## Specify the input and output files.
## NOTE: This path is relative to the repo root directory
export INPUT_PARAMETERS=./src/project/project2/examples/project2_example_input_parameters.json
export INPUT_SOURCETERMS=./src/project/project2/examples/project2_example_source_terms.csv
export OUTPUT_RESULTS=./src/project/project2/examples/project2_example_output_results.json
export OUTPUT_COMPUTED_FLUX=./src/project/project2/examples/project2_example_computed_flux

## Execute
./build/bin/$BUILD_TARGET -i $INPUT_PARAMETERS -s $INPUT_SOURCETERMS -o $OUTPUT_RESULTS -f $OUTPUT_COMPUTED_FLUX

Coming Soon

#!/bin/bash

# Begin by copying the files over using rsync, scp, sftp, etc...
# Then, assuming the repo root is the current directory:

## Specify the build target
export BUILD_TARGET=project2

## Create the build directory, configure and compile the $BUILD_TARGET
mkdir -p build && cd build && \
cmake .. -DCMAKE_BUILD_TYPE=Release && \
make -j$(nproc) $BUILD_TARGET && cd ../

## Specify the input and output files.
## NOTE: This path is relative to the repo root directory
export INPUT_PARAMETERS=./src/project/project2/examples/project2_example_input_parameters.json
export INPUT_SOURCETERMS=./src/project/project2/examples/project2_example_source_terms.csv

export OUTPUT_RESULTS=./src/project/project2/examples/project2_example_output_results.json
export OUTPUT_COMPUTED_FLUX=./src/project/project2/examples/project2_example_computed_flux

## Execute
./build/bin/$BUILD_TARGET -i $INPUT_PARAMETERS -s $INPUT_SOURCETERMS -o $OUTPUT_RESULTS -f $OUTPUT_COMPUTED_FLUX

The entire codebase has been built and tested on the NCSU Hazel login.hpc.ncsu.edu and EOS remote.eos.ncsu.edu servers, which, as of Fall 2023, provide support for the following dependencies:

• Boost : 1.83.0
• C/C++ compiler : GNU 8.5.0

• Boost : 1.83.0
• C/C++ compiler : GNU 10.2.0
• CUDA: 12.0.0
• MPI: OpenMPI 4.1.0

To facilitate local development, equivalent Dockerfiles have been created. Checkout eos.Dockerfile and hazel.Dockerfile and the Docker instructions below.

Currently supported project build targets include:

Currently supported inlab, outlab build targets include:

Available options are:

To use these options, you can add them to the cmake command in the build script. For example, to build with tests, you would modify the cmake command as follows:

cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTS=ON

This repository contains Dockerfiles that are designed to mimic the OS and packages available on the login.hpc.ncsu.edu and remote.eos.ncsu.edu servers. These Dockerfiles also set up SSH for remote debugging access with tools such as JetBrains Clion.

Docker is a platform that allows us to package our application and its dependencies into a container which can run on any Linux/macOS/Windows machine. This helps to eliminate the "it works on my machine" problem. In this case, the Dockerfiles were needed to ensure that the code can be built, run, and debugged in an environment that closely matches the login.hpc.ncsu.edu and remote.eos.ncsu.edu servers.

The current default Dockerfile is a symlink to eos.Dockerfile. You can change it to build for hazel.Dockerfile instead. Alternatively, you can always specify the Dockerfile to use using the -f <your.Dockerfile> option.

To simply run the executables, build an image from the Dockerfile using the following command:

docker build -t ne591:binary .

To run the Docker container, use the following command:

docker run --rm -it ne591:binary

This command tells Docker to run the container interactively (i.e., it attaches a terminal session to the running container) and starts a bash shell.

The Dockerfile includes arguments for building tests and performance benchmarks. To build and run the tests, you can modify the docker build command as follows:

docker build --build-arg CMAKE_BUILD_TESTS=ON -t ne591:binary .

Then, you can run the tests in the Docker container as you would normally do.

You can use JetBrains Clion as the debug environment. Use Docker as the debug toolchain.

• Docker
• CLion

1. In Clion, first set your terminal to Bash.

   1. Go to Settings / Preferences | Tools | Terminal
   2. Under Application Settings, look for Shell Path and set it to Bash.

2. Then, build the docker debugger target, and save the built image with the tag eos:debugger.

       docker build --target=debugger -t eos:debugger .

Developing in Clion using the Docker Toolchain provides a consistent experience. The debugging environment is an ephemeral Docker container based on the eos:debugger image you just built.

1. Create Docker Toolchain

   1. Go to Settings / Preferences | Build, Execution, Deployment | Toolchains.
   2. Click Add toolchain and select Docker.
   3. Click the screw nut icon next to the Docker field to select a Docker image.
      1. You can also configure a Docker server in Settings / Preferences | Build, Execution, Deployment | Docker and then select it in the toolchain settings.
   4. Select the Docker Image eos:debugger and wait until the tool detection finishes.
   5. Set the C Compiler to Let CMake Detect. It should detect gcc.
   6. Set the C++ Compiler to Let CMake Detect. It should detect g++.
   7. Set the Debugger to Docker GDB.
   8. Then save the settings.

2. Build, Run, Debug with a Docker Toolchain

   1. After configuring a Docker toolchain, you can select it in CMake profiles or in Makefile settings. Alternatively, move the toolchain to the top of the list to make it default.
   2. The project folder will mount to the Docker container and building, running, and debugging will be performed in it. CLion will start the container and shut it down after the command is executed.

• Improve Docker Toolchain performance on Windows

  • To get better performance on Windows, we recommend using Docker with the WSL 2 backend.
    1. Set up Docker Desktop with the WSL 2 backend.
    2. In the Docker desktop application, navigate to Settings | Resources | WSL Integration and enable integration with your WSL distribution.
    3. Place the project sources into the WSL filesystem, then open it in CLion and configure a Docker toolchain.

The Dockerfile sets up SSH for root access in the ssh-debugger stage. This allows you to connect to the running container using SSH and debug the code. To do this, you need to build the SSH target and run the container in the background with port 22 exposed:

docker build --target=ssh-debugger -t ne591:ssh-debugger .
docker run --privileged --name=eos-ssh-debugger -d -p 2222:22 -it ne591:ssh-debugger

Then, you can connect to the running container using SSH. Clion provides native support for debugging remotely. Follow these instructions to set it up.

ssh root@localhost

The password for the root user is "debugger".

This project is licensed under the Unlicense - a license with no conditions whatsoever which dedicates works to the public domain. Unlicensed works, modifications, and larger works may be distributed under different terms and without source code.

However, note that this code uses libraries that are bound by their own licenses.

• This is a course project. Why all the extra effort?
  • My hope is that a stripped version of this code can serve as boilerplate so future students can limit all the hair-pulling to their algorithms, instead of worrying about I/O and input validation.