dionhaefner / pyhpc-benchmarks Goto Github PK

View Code? Open in Web Editor NEW

301.0 10.0 26.0 1.22 MB

A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

License: The Unlicense

Python 96.61% Jupyter Notebook 2.71% Shell 0.68%

benchmarks python tensorflow pytorch jax cupy parallel-computing high-performance-computing gpu

pyhpc-benchmarks's Introduction

Hi there 👋

Now that you're here, why don't you check out some of my projects?

Main projects

^{(bigger projects where I am a core developer)}

team-ocean / veros — ⭐ 305

The versatile ocean simulator, in pure Python, powered by JAX.

DHI / terracotta — ⭐ 638

A light-weight, versatile XYZ tile server, built with Flask and Rasterio 🌍

mpi4jax / mpi4jax — ⭐ 371

Zero-copy MPI communication of JAX arrays, for turbo-charged HPC applications in Python ⚡

Side projects

dionhaefner / pyhpc-benchmarks — ⭐ 301

A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python 🚀

dionhaefner / yahtzotron — ⭐ 18

The friendly robot that beats you in Yahtzee 🤖 🎲

dionhaefner / dionhaefner.github.io

Repository for my personal homepage, hosted on GitHub Pages, created with Pelican.

Science applications

dionhaefner / rogue-wave-discovery — ⭐ 10

Code for the paper "Machine-Guided Discovery of a Real-World Rogue Wave Model" (Häfner et al., 2023)

dionhaefner / FOWD — ⭐ 10

Processing framework for FOWD, a free ocean wave dataset, ready for your ML application 🌊

Misc

^{(one-off projects and everything else)}

dionhaefner / cv — ⭐ 3

My personal CV and résumé, auto-generated via GitHub Actions 📄

dionhaefner / bayesian-histograms — ⭐ 27

Bayesian histograms for estimation of binary rare event rates, with fully automated bin pruning 📊

dionhaefner / dionsthesis — ⭐ 8

Custom LaTeX2e documentclass for typesetting beautiful, modern theses.

dionhaefner / pgfcache — ⭐ 11

LaTeX package for caching of PGF figures created with Matplotlib, just like tikz-externalize

dionhaefner / shallow-water — ⭐ 10

Powerful shallow-water implementations in pure Python

dionhaefner / attractive — ⭐ 3

Compute and plot beautiful Clifford Attractors 💫

dionhaefner / fly — ⭐ 9

An interactive geophysical flow visualizer in Python.

_{Last updated 2024-05-03}

pyhpc-benchmarks's People

Contributors

Stargazers

Watchers

pyhpc-benchmarks's Issues

fastmath

Hi @dionhaefner, great comparisons, thanks for that! Out of interest. Did you ever try to run numba with fastmath=True; does it make any difference, and if, how much?

Compare with TACO Python binding

The Tensor Algebra Compiler (https://github.com/tensor-compiler/taco) seems to be good at sparse/dense linear algebra and has Python frontend: http://tensor-compiler.org/docs/pycomputations/index.html

Add TPU support

Jax now supports using the TPU on Google Colab. It would be nice to try how it performs out of the box.

Error while installing GPU environment

Using:
conda env create -f environment-gpu.yml

leads to an error saying:

Exception:

  =========================================================
  The "tensorflow-gpu" package has been removed!

  Please install "tensorflow" instead.

  Other than the name, the two packages have been identical
  since TensorFlow 2.1, or roughly since Sep 2019. For more
  information, see: pypi.org/project/tensorflow-gpu
  =========================================================


  [end of output]

turbulent_kinetic_energy returns inconsistent results

I am working on #14.
The command has inconsistent result output:

$ python run.py -r 2 -s 1048576 --device cpu -b pytorch benchmarks/turbulent_kinetic_energy/

Using pytorch version 1.13.0.dev20220617+cu113
Running 3 benchmarks...  [------------------------------------]    0%Error: inconsistent results for size 1048576
Error: inconsistent results for size 1048576
Error: inconsistent results for size 1048576
Running 3 benchmarks...  [####################################]  100%

benchmarks.turbulent_kinetic_energy
===================================
Running on CPU

size          backend     calls     mean      stdev     min       25%       median    75%       max       Δ
------------------------------------------------------------------------------------------------------------------
   1,048,576  pytorch            2     0.573     0.028     0.544     0.559     0.573     0.587     0.601     1.000

(time in wall seconds, less is better)

Looks like two consecutive runs will generate inconsistent results for turbulent_kinetic_energy. I guess the root cause is this line: https://github.com/dionhaefner/pyhpc-benchmarks/blob/master/benchmarks/turbulent_kinetic_energy/tke_pytorch.py#L264

There could be non-deterministic numeric results when running mask = tke[2:-2, 2:-2, -1, taup1] < 0.0

Adding Transonic to comparison

https://github.com/fluiddyn/transonic can accelerate Python/Numpy code and run on accelerators

deprecated jax ops: index_update

the call
for backend in jax; do python run.py benchmarks/isoneutral_mixing/ --device gpu -b $backend -b numpy; done
yields

dTdz = jax.ops.index_update(
AttributeError: module 'jax.ops' has no attribute 'index_update'

and indeed index_update is no longer a thing: https://jax.readthedocs.io/en/latest/jax.ops.html

DaCe (https://github.com/spcl/dace) is a parallel computing framework that also support Numpy frontend, similar to JAX and Numba. It runs on CPU/GPU/FPGA. Would be interesting to add it for comparison!

Compare with an MLIR-based stencil DSL

This project https://github.com/spcl/open-earth-compiler/ provides a DSL frontend for stencil/PDE programs, and rely on MLIR & LLVM to run on NVIDIA and AMD GPUs. It is not a Python frontend, but can be called from Python I think (see https://arxiv.org/abs/2005.13014)

Adding Weld to comparison

https://github.com/weld-project/weld can accelerate Python/numpy code by optimizing data movement.