Giter Club home page Giter Club logo

benchmark-rngs's Introduction

Random Number Engine Benchmark

benchmark.cpp benchmarks Philox.h, PhiloxSIMD.h xoshiro256starstar.h, PCG.h and std::mt19937

Build and run with the following instructions:

mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make VERBOSE=1
./bench

Usage:

Random Number Engine Benchmark
Usage: ./bench [OPTIONS]

Options:
  -h,--help                   Print this help message and exit
  -r,--min-num-randoms UINT   Minimum number of randoms to produce
  -t,--min-num-threads INT    Minimum number of threads to use
  -e,--engine UINT ...        Selects specific tests to run by supplying an int. Runs with all tests if none provided.
                              Available tests are:
                              {0: philox (global)}
                              {1: philox (thread local)}
                              {2: philox_simd (global)}
                              {3: philox_simd (thread local)}
                              {4: xoshiro256**}
                              {5: pcg64}
                              {6: at::mt19937}
                              {7: std::mt19937}
                              {8: at::mt19937 (chunking)}
                              {9: pcg64 (chunking)}
                              {10: std::mt19937 (chunking)}
                              {11: xoshiro256** (chunking)}
                              {12: philox (global) (chunking)}
                              {13: philox_simd (global) (chunking)}
  -x,--num-x-data-points INT  Bins of x data points to produce, where x is either threads or number of randoms
[Option Group: benchmark_type]
  Decides if the independent variable is number of threads or number of randoms 
  [At most 1 of the following options are allowed]
  Options:
    -a,--benchmark-threads      Run full benchmark with increasing number of threads
    -b,--benchmark-num-randoms  Run full benchmark with increasing number of randoms

Running the benchmark:

Step 1: Adjust cpu settings

Turn off turbo-boost (if available) using the following script (https://askubuntu.com/questions/619875/disabling-intel-turbo-boost-in-ubuntu):

#!/bin/bash

if [[ -z $(which rdmsr) ]]; then
    echo "msr-tools is not installed. Run 'sudo apt-get install msr-tools' to install it." >&2
    exit 1
fi

if [[ ! -z $1 && $1 != "enable" && $1 != "disable" ]]; then
    echo "Invalid argument: $1" >&2
    echo ""
    echo "Usage: $(basename $0) [disable|enable]"
    exit 1
fi

cores=$(cat /proc/cpuinfo | grep processor | awk '{print $3}')
for core in $cores; do
    if [[ $1 == "disable" ]]; then
        sudo wrmsr -p${core} 0x1a0 0x4000850089
    fi
    if [[ $1 == "enable" ]]; then
        sudo wrmsr -p${core} 0x1a0 0x850089
    fi
    state=$(sudo rdmsr -p${core} 0x1a0 -f 38:38)
    if [[ $state -eq 1 ]]; then
        echo "core ${core}: disabled"
    else
        echo "core ${core}: enabled"
    fi
done

Adjust NUMA if needed (not required if your CPUs are in one socket)

https://developer.download.nvidia.com/video/gputechconf/gtc/2019/presentation/s9956-best-practices-when-benchmarking-cuda-applications_V2.pdf

Set CPU frequency to performance mode:

sudo cpupower frequency-set -g performance

Use i7z to monitor benchmark run

sudo i7z

Step 2: Run benchmark

# benchmarks with varying number of threads, and 10 data points 
./bench -a -x 11
# benchmarks with varying number of randoms
./bench -b -x 11
# benchmark a subset of engines
./bench -e 0 1 -a -x 11

Step 3: Copy results to result folder

# from build folder
mv result-increasing-* ../results/
cd ../results/
jupyter notebook # run plot-results.ipynb

benchmark-rngs's People

Contributors

colesbury avatar syed-ahmed avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.