Giter Club home page Giter Club logo

profile_numpy_numba's Introduction

Profiling Numpy and Numba code

The module in this repository compares execution time and maximum memory usage for different methods of converting a color image to black-and-white.

The conversion methods are:

  • using Numpy
  • using Numba to jit the Numpy method
  • using a for loop in Numba

This project is derived from and replicates the results from Itamar Turner-Trauring's blog post The wrong way to speed up your code with Numba. For a more thorough and detailed explanation, please see his post.

Results

The execution times and maximum memory usage of the conversion methods are below. The results from a first run of all three methods are printed on the first three lines; the second set of three lines shows results for the second run with a different color image as input. The second run shows the optimized execution times. Please note that the exact execution times will stochastically vary from run to run, though the maximum memory usage should be consistent for a given image.

Image

The results from the second run above replicate the primary theme of Mr. Turner-Trauring's blog post: trying to optimize the Numpy code by merely jitting it with Numba is far inferior to rewriting the code with a for loop to take advantage of how Numba optimizes code. The for loop Numba code is much faster than the jitted written-for-Numpy code, and it requires much less memory.

Note that if one wants a more careful quantification of the code speed-up and reduced memory usage, one should statistically summarize results from multiple profiling runs on images that are representative of the ones that will be encountered in production.

Run with Anaconda

From the top-level project directory, you can create and activate the Anaconda environment by running these commands:

conda env create -f environment.yml

conda activate numba01

Then you can run the code and save the results with this command:

python src/convert_bw.py > output/profiling_results.txt

Run with Poetry

The commands to run with Poetry in the top-level directory are:

poetry install

poetry run python src/convert_bw.py

profile_numpy_numba's People

Contributors

afairless avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.