Comments (12)
@der-flo You're absolutely right! Based on a read of the test-prof sources, it seems like the file speedscope is failing to import is Marshal encoded. Aside from that though, I suspect it will import just fine.
To convert from the Marshal generated code to JSON for import, this might work:
ruby -e "require 'json'; puts JSON.generate(Marshal.load(File.binread('/path/to/your/stackprof.dump')))" > profile-for-speedscope.json
Then drop the resulting profile-for-speedscope.json
into speedscope.
That's obviously not a great workflow. To support this better, I could either contribute a change to stackprof to make this integration tighter, or automatically convert Marshal data to JSON in speedscope.
from stackprof.
@NickLaMuro Thanks for great amount of information. I'll have a look into it asap.
from stackprof.
@der-flo Sorry to hear that it wasn't much more help... 😞
I might take a look and see what would be involved with method elimination in post for flamegraphs, as I think that does seem like the preferred way of handling it on this project (adding more processing while taking the samples seems like it is against the original goals of the project). No promises though.
from stackprof.
(Shameless self-plug) You should be able to use speedscope to examine stackprof profiles in an interactive way that should be more pleasant for massive profiles (https://github.com/jlfwong/speedscope#stackprof-ruby-profiler)
from stackprof.
@jlfwong I did a quick test. Selecting my 18 MB file gives me "Failed to load format. See console for details". The JS console gives me a quite interesting undefined
in speedscope.86b357fe.js:144
😉.
@ALL: More solutions/ideas are very welcome.
from stackprof.
@der-flo Interesting! Is it a file you'd be willing to share, or have a method of reproducing something that fails similarly? it should be able to handle 18MB files without too much issue.
Here's an example ruby file that I used to generate a test case that can import successfully into speedscope: https://github.com/jlfwong/speedscope/blob/master/sample/programs/ruby/simple.rb
from stackprof.
@jlfwong Um, my file contains quite sensitive information. I can not hand it out easily.
To your Ruby example: To be honest, that's a hello world script. I'm profiling a Rails app - an enourmous difference.
Your test obviously exports JSON, test-prof exports a binary dump file for Stackprof profiles.
from stackprof.
@jlfwong I converted two of the dump files and tried them:
Importing as stackprof profile
undefined speedscope.57cb68ca.js:150
from stackprof.
Ah, sorry. I'm not sure what's going on now.
Just to make sure, when running test-prof
, are you using TEST_STACK_PROF=raw
? (from https://test-prof.evilmartians.io/#/stack_prof)
I'm not sure what's wrong, and I'm not sure how to proceed at this point without having a file that fails to import. I'd really love to get this working since the whole point of speedscope is to be able to interactively examine really large flamegraphs. Thanks for sticking with me this far -- if you have other ideas of things to try to reproduce what you're running into, I'd love to fix this.
from stackprof.
@jlfwong My failure! I tried to analyse non-raw dumps. It now works. The UI is indeed quite nice!
My core problem unfortunately remains. I have enormous amounts of stack frames which I'm not interested in. They are dominating the view, preventing me to identify hot spots/problems.
@ALL RubyProf offers "method exclusion" while profiling and "method elimination" after profiling. With this tools I can get rid of all the ActiveSupport, Bundler, Rack, ActionView, ... stuff a usual Rails app pollutes the graph with.
Does anybody see a possibility to do such a reduction with Stackprof, speedscope, or an intermediate tool?
from stackprof.
RubyProf offers "method exclusion" while profiling and "method elimination" after profiling.
@der-flo Not part of the core team on this, but it doesn't currently support this when doing flame graph generation. That said, there are options in place for '"method elimination" after profiling' when viewing via plain text:
Lines 26 to 29 in 1af2220
Again, not part of the flamegraph generation, and what is done for the plain text output does not even translate to the flamegraph generation since it uses the :raw
bit from the profile data, and not the frame data (which is what gets filtered out by those options).
Those differ a quite a bit, so off the top of my head I couldn't tell you if filtering the raw data would even make sense. I would have to look into how the :raw
is implemented.
@der-flo Alternatively, you might get better use out of the perl version of the "flamegraph", versus the web UI one. You can running it by doing:
$ stackprof --stackcollapse [DATA.stackprof] > data.stackcollapse
$ stackprof-flamegraph.pl data.stackcollapse > flamegraph.svg
You can also just use the original script from the repo, since the one packaged here is very old. Just used the flamegraph.pl
file from that repo (just a single perl file).
I don't want make too much of a 📖 to explain the difference, but the perl version that is packaged with stackprof
doesn't display information chronologically, but grouped by stack location, and might be more useful when the hot spots are hit non-sequentially. This is "technically" a flamegraph, versus a "flamechart" (as I think Brendan Gregg describes it), which is mostly what speedscope and the current --flamegraph-viewer
are.
Brendan has a good presentation on the subject over on his website, which does a better job of explaining it than I can if you are interested.
Hope this helps.
from stackprof.
@NickLaMuro I tested your suggestion using the original flamegraph script. The output is a bit more readable. But the core issue remains. Most frames are not interesting and pollute the UI.
from stackprof.
Related Issues (20)
- Using wall-clock profiling on macOS makes Ruby thread scheduling starve background threads HOT 4
- TruffleRuby support HOT 4
- Stackprof raises NameError when wrapped in Rails environment conditional HOT 3
- `test_gc` fails on (32-bit, hardfloat, QEMU-emulated) arm HOT 1
- Incompatible with YJIT 3.2.0 HOT 1
- Problem with Ruby 2.7.6 and version 0.2.20 HOT 6
- Error installing with Ruby 3.1 HOT 3
- Segmentation fault on Ruby 2.7.6p219 HOT 4
- Possibility to filter certain files or classes?
- flaky test failure with ruby3.1
- Test failures on 32bit arch
- Flamegraph way too short in `wall` mode HOT 5
- Support Gecko format? HOT 2
- Weird timestamp deltas with puma HOT 2
- bug: "missed_samples" integer overflow possible HOT 1
- bug: possible state leak of `buffer_count` state between stackprof invocations HOT 2
- C99 error while compiling v0.2.25 HOT 1
- Add marker metadata feature HOT 1
- stackprof fails to build native extensions on ubuntu 22.04 and ruby 3.2.2 HOT 1
- "lines" attribute format?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stackprof.