pythonspeed / profila Goto Github PK
View Code? Open in Web Editor NEWA profiler for Numba
License: Apache License 2.0
A profiler for Numba
License: Apache License 2.0
Right now, it seems profiling is done at a practically infinite depth, so my calls usually end up going way deeper than is important to me (i.e. I don't need to see the deepest numpy source code). Is it possible to limit the depth to which the profiling is done, either by simply doing x
levels away from the executed file or (possibly even better) only profiling calls in files contained in a certain directory/namespace?
File "/home/itamarst/devel/low-level-performance-book/venv/lib/python3.11/site-packages/profila/__main__.py", line 54, in get_stats
async for sample in read_samples(process):
File "/home/itamarst/devel/low-level-performance-book/venv/lib/python3.11/site-packages/profila/_gdb.py", line 100, in read_samples
async for sample in _sample(process):
File "/home/itamarst/devel/low-level-performance-book/venv/lib/python3.11/site-packages/profila/_gdb.py", line 53, in _sample
if "stack" not in message["payload"]: # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable
On Linux prctl(PR_SET_PTRACER, pid_of_subprocess)
/prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY)
might allow gdb to access current process without needing root. prctl(PR_SET_PTRACER, 0)
to reset.
Would probably need controller to be an additional Python process to prevent deadlocks.
The controlling Python subprocess would need to communicate with the parent process:
Profiling-wise, this is probably straightforward, should be able to do the equivalent of apply thread all bt
in GDB MI and get samples across multiple threads.
Whether the line annotation visualization actually makes sense for multiple threads is something I have to think about.
lldb
currently works.lldb-mi
is third-party tool supposedly compatible with GDB MI, but not readily available on macOS. E.g. VS Code plugins will ship it themselves.debuginfo
in the way debuggers do. Initial experiments have failed, this is why there hasn't been profiling support for Numba until now! But will keep trying.lldb-mi
in the wheel.lldb-mi
for Homebrew.On macOS, use Docker, Podman, or a VM to get a Linux environment.
I installed this onto a venv ran into this error:
{'type': 'result', 'message': 'error', 'payload': {'msg': '"/home/jje/.pyenv/shims/python": not in executable format: file format not recognized'}, 'token': None}
But /home/jje/.pyenv/shims/python
is executable, so I'm not sure what the issue is.
ls /home/jje/.pyenv/shims/python -l
# -rwxr-xr-x 1 jje jje 176 Sep 8 22:12 /home/jje/.pyenv/shims/python*
Does it need to be a proper binary?
cat /home/jje/.pyenv/shims/python
# #!/usr/bin/env bash
# set -e
# [ -n "$PYENV_DEBUG" ] && set -x
#
# program="${0##*/}"
#
# export PYENV_ROOT="/home/jje/.pyenv"
# exec "/usr/share/pyenv/libexec/pyenv" exec "$program" "$@"
But it's not the executable I wanted anyway. I wanted the one from the venv I was calling python -m profilia
in. Changing this line got it working the way I needed in my particular case:
- process.stdin.write(b"-file-exec-file python\n")
+ process.stdin.write(bytes(f"-file-exec-file {sys.executable}\n", encoding="utf-8"))
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.