Giter Club home page Giter Club logo

flamescope's Introduction

FlameScope

FlameScope

Gitter TravisCI NetflixOSS Lifecycle License

FlameScope is a visualization tool for exploring different time ranges as Flame Graphs, allowing quick analysis of performance issues such as perturbations, variance, single-threaded execution, and more.

FlameScope begins by displaying the input data as an interactive subsecond-offset heat map. This shows patterns in the data. You can then select a time range to highlight on different patterns, and a flame graph will be generated just for that time range.

Disclaimer

FlameScope is in early stages of development and under constant change, so bugs and issues are expected. We count on your support to find and report them!

Installation / Instructions

The quickest way to get started is to run the pre-built client bundle:

$ git clone https://github.com/Netflix/flamescope
$ cd flamescope
$ pip install -r requirements.txt
$ python run.py

(Note python3 is assumed, python2 may work)

Then browse to http://127.0.0.1:5000/, and you can begin exploring profiles from the examples directory. You can add new profiles to that directory, collected using Linux perf. Here are instructions for a generic CPU profile at 49 Hertz for 120 seconds:

$ sudo perf record -F 49 -a -g -- sleep 120
$ sudo perf script --header > stacks.myproductionapp.2018-03-30_01
$ gzip stacks.myproductionapp.2018-03-30_01	# optional

If you are profiling C++ code, you may want to pipe stacks through c++filt to get readable frames.

There are extra steps to fetch stacks correctly for some runtimes, depending on the runtime. For example, we've previously published Java steps in Java in Flames: java needs to be running with the -XX:+PreserveFramePointer option, and perf-map-agent must be run immediately after the perf record to dump a JIT symbol table in /tmp.

FlameScope can visualize any Linux perf script output that includes stack traces, including page faults, context switches, and other events. See the References section below for documentation.

FlameScope is composed of two main components, the Python backend, and a React client interface. A pre-built client bundle is distributed with the backend, so the quickest way to get started is to install the Python requirements and start the application, as described earlier.

Although not necessary, we strongly suggest using virtualenv to isolate your Python environment.

By default, FlameScope will load a list of files from the examples directory, which includes a two profile examples.

Configuration Options

FlameScope configuration file can be found in app/config.py.

DEBUG = True # run the web server in debug mode
PROFILE_DIR = 'examples' # path where flamescope will look for profiles
HOST = '127.0.0.1' # web server host
PORT = 5000 # web server port
JSONIFY_PRETTYPRINT_REGULAR = False # pretty print api json responses

Building Client from Source

In order to build the client application from source, the following command line tools must be installed:

Once those tools are available, you will be able to install the project dependencies and generate a build.

$ npm install
$ npm run webpack

The npm run webpack command will generate a new build under app/public. This directory is exposed by the Python web server.

Webpack can also watch and recompile files whenever they change. To build and start the watch task, run the following command:

$ npm run webpack-watch

Building a Docker Image

FlameScope provides a Dockerfile to build a Docker image:

$ cd flamescope
$ docker build -t flamescope .

The container expects the profiles to be bind-mounted into /profiles and listens on port 5000. To view profiles from /tmp/profiles, start the container with the following command:

$ docker run --rm -it -v /tmp/profiles:/profiles:ro -p 5000:5000 flamescope

Then access FlameScope on http://127.0.0.1:5000

References

flamescope's People

Contributors

ampresent avatar bahtiyarkaba avatar bobrik avatar brendangregg avatar dependabot[bot] avatar ekmartin avatar iori-yja avatar ispequalnp avatar jacoblearned avatar jasonk000 avatar kroemeke avatar minusome avatar mmarchini avatar nhick avatar nobeh avatar sghill avatar sonicdoe avatar spiermar avatar steven-aerts avatar takac avatar vmdz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flamescope's Issues

pixelate count legend

screen shot 2018-04-13 at 8 58 40 am

The current count scale implies many colors you won't actually see, due to the gradient fill. This example heat map (with "enhance") only has 8 colors. Does it make sense to turn the gradient fill into rectangles for each possible color? I'd prototype it and paste an example screenshot here for comment.

make the duration of a column configurable

Right now, columns are hardcoded to be 1 second long, it'd be nice if this was configurable (I tried changing YRATIO in heatmap.py but that doesn't seem to be enough).

multiple profiler support

As has been mentioned elsewhere, a generic data file format would allow for converters to be written to support formats other than perf. Has there been any work towards this yet?

Another approach seems to be what it being implemented in the source with "native" support of various profilers, with each format having its own parsing routine. I prefer this second approach, but it seems the interface should be formalized a bit more.

Has a decision been made on which approach to take? I would like to start sending some PRs to provide support for other profilers.

Could not find a version that satisfies the requirement click==6.7 (from -r requirements.txt (line 1))

When running pip install -r requirements.txt then I get the below error message:

Could not find a version that satisfies the requirement click==6.7 (from -r requirements.txt (line 1)) (from versions: ) No matching distribution found for click==6.7 (from -r requirements.txt (line 1))
I am using Python 3.6.5. Also, when I then try to run python run.py then I get a error saying:
ImportError: No module named flask.

Thanks

Compatibility with stackprof (ruby)?

My attempts at using flamescope with stack profiles obtained through stackprof have failed.

Initially, I had assumed the general steps for flamegraph.pl would work, but flamescope chokes on the collapsed stack file.

Is there any plans to support stack profiles from stackprof, or document how one would do this?

cache / preprocess heatmap response data

Every time people access the flamescope's heatmap, the server processes the perf output data which makes the user awaited. So I am considering to cache preprocessed or first-returned data.

Structure of the response is quite simple;

{
    "columns": [0, 1, .....],
    "rows": [980, 960....],
    "values": [ [2, 2, ...], [...], [...], ....],
    "maxvalue": 16,
}

The most quick hack to cache it is storing in sessionStorage in local browser. Better idea is to save the data as a metadata beside actual data (In that case, we need to decide naming convention for metadata). Other idea is to store data in DB.

I am going to implement a cache anyway, but I want to ask this before starting, which is, what kind of cache is easier to accept for you?.

Ability to view Golang graphs

This tool looks really awesome and I'd like to try it out on some profiles taken from our production services, but the tool seems to be having trouble with profiles generated using the go tool pprof command.

Has anyone tried or gotten this to work with Golang profiles yet?

Here's the stacktrace I'm getting:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/richardartoul/Documents/flamescope/app/views/heatmap.py", line 34, in get_heatmap
    return jsonify(generate_heatmap(filename, rows))
  File "/Users/richardartoul/Documents/flamescope/app/util/heatmap.py", line 120, in generate_heatmap
    cols = int(ceil(end) - floor(start))
OverflowError: cannot convert float infinity to integer

EDIT: Ah ok it looks like this tool is designed to handle the linux perf output whereas golang generates pprof-formatted files.

Various issues making it work

Hi

To get this nice tool working I had to make a few tweaks (after setting up a virtualenv)

% python -m pip list
Package       Version
------------- -------
cachetools    3.0.0  
click         6.7    
Flask         0.12.2 
itsdangerous  0.24   
Jinja2        2.10   
MarkupSafe    1.0    
pip           19.0.3 
pkg-resources 0.0.0  
protobuf      3.6.1  
python-magic  0.4.15 
setuptools    40.8.0 
simplejson    3.16.0 
six           1.12.0 
Werkzeug      0.14.1 
wheel         0.33.1 

First issue -

ImportError: No module named cpuprofile.flame_graph

and similarly for trace_event.flame_graph. I was able to fix this with:

$ touch app/cpuprofile/__init__.py
$ touch app/trace_event/__init__.py

Second issue - the version of the 'json' module did not have the JSONDecodeError method.

Traceback (most recent call last):
  File "run.py", line 6, in <module>
    from app import APP
  File "/data/git/clones/flamescope/app/__init__.py", line 34, in <module>
    from app.views.flame_graph import MOD_FLAME_GRAPH
  File "/data//git/clones/flamescope/app/views/flame_graph.py", line 21, in <module>
    from app.controllers.flame_graph import generate_flame_graph
  File "/data/git/clones/flamescope/app/controllers/flame_graph.py", line 25, in <module>
    from app.trace_event.flame_graph import trace_event_generate_flame_graph
  File "/data/git/clones/flamescope/app/trace_event/flame_graph.py", line 26, in <module>
    from json.decoder import JSONDecodeError
ImportError: cannot import name JSONDecodeError

Perhaps you should specify the one you want in requirements.txt or just use simplejson as below.
For me the fix was

$ git diff
diff --git a/app/trace_event/flame_graph.py b/app/trace_event/flame_graph.py
index 6e8df08..8c56745 100644
--- a/app/trace_event/flame_graph.py
+++ b/app/trace_event/flame_graph.py
@@ -23,7 +23,7 @@
 import json
 import math
 from flask import abort
-from json.decoder import JSONDecodeError
+from simplejson.decoder import JSONDecodeError
 from app.common.fileutil import get_file
 from app.trace_event.common import get_time_range
 
diff --git a/app/trace_event/heatmap.py b/app/trace_event/heatmap.py
index 261eca3..c12e5f6 100644
--- a/app/trace_event/heatmap.py
+++ b/app/trace_event/heatmap.py
@@ -24,7 +24,7 @@ import collections
 import math
 import json
 from flask import abort
-from json.decoder import JSONDecodeError
+from simplejson.decoder import JSONDecodeError
 
 from app.common.fileutil import get_file
 from app.trace_event.common import get_time_range

sort FileList

I expect the FileList page isn't meant for very long list of files but to make it easier to find a new result in medium to long file lists sorting is a quick thing to put in.
Added it to our automated benchmarking on Sunday and the list is growing. future enhancement could be pagination, reverse sort (so latest result comes up first), etc

ERROR: Bad range, 622353.26 -> 622351.26.

Wondering if this could be less fatal. I'm assuming out of order or missed perf record elements.

Got no output in browser with this error log:

9.192.254.114 - - [04/Apr/2018 20:52:53] "GET /stack?filename=stacks.mysql-prep.2018-04-04%2020:49:33-05:00&start=18.26&end=16.26 HTTP/1.1" 301 -
ERROR: Bad range, 622353.26 -> 622351.26.

Decent file size:

31235407 Apr  4 20:49 stacks.mysql-prep.2018-04-04 20:49:33-05:00

Trying to look at data at 622353.26, 622351.26:

 grep  62235[13].26 stacks.mysql-prep.2018-04-04\ 20\:49\:33-05\:00
mysqld 26608 [176] 622351.261861:  114686965 cycles:ppp:
mysqld 26652 [116] 622351.262978:  114685776 cycles:ppp:
mysqld 26060 [092] 622351.265249:  114685758 cycles:ppp:
mysqld 26502 [176] 622353.262046:  114692803 cycles:ppp:
mysqld 25987 [116] 622353.263136:  114683275 cycles:ppp:
mysqld 26751 [092] 622353.265415:  114686145 cycles:ppp:
swapper     0 [088] 622353.265751:  260814858 cycles:ppp:
 sudo perf script --debug-mode -i perf-prep.data --header > "stacks.mysql-prep.`date --rfc-3339=seconds`"
Misordered timestamps: 0

This generated a header only:
# ========
# captured on: Wed Apr  4 20:48:53 2018
# hostname : bobby
# os release : 4.13.0-37-generic
# perf version : 4.15.0
# arch : ppc64le
# nrcpus online : 46
# nrcpus avail : 184
# cpudesc : POWER9, altivec supported
# cpuid : 78,4609
# total memory : 536032448 kB
# cmdline : /usr/local/bin/perf record -F 20 --proc-map-timeout=50000 -o perf-prep.data -a -g -- sleep 30
# event : name = cycles:ppp, , size = 112, { sample_period, sample_freq } = 20, sample_type = IP|TID|TIME|CALLCHAIN|CPU|PERIOD, disabled = 1, inherit = 1, mmap = 1, comm = 1, fre
q = 1, task = 1, precise_ip = 3, sample_id_all = 1, exclude_guest = 1, mmap2 = 1, comm_exec = 1
# CPU_TOPOLOGY info available, use -I to display
# pmu mappings: nest_mba1_imc = 9, nest_mba5_imc = 13, nest_phb1_imc = 22, nest_phb5_imc = 26, nest_nvlink0_imc = 18, breakpoint = 5, nest_powerbus0_imc = 27, nest_mba2_imc = 10,
 nest_nx_imc = 20, nest_mba6_imc = 14, nest_phb2_imc = 23, cpu = 4, nest_nvlink1_imc = 19, nest_mcs01_imc = 16, software = 1, thread_imc = 28, nest_mba3_imc = 11, nest_mba7_imc =
 15, nest_phb3_imc = 24, nest_capp0_imc = 6, nest_xlink1_imc = 29, nest_mcs23_imc = 17, nest_mba0_imc = 8, nest_mba4_imc = 12, nest_phb0_imc = 21, tracepoint = 2, nest_phb4_imc =
 25, nest_xlink2_imc = 30, core_imc = 7
# missing features: TRACING_DATA NUMA_TOPOLOGY BRANCH_STACK GROUP_DESC AUXTRACE STAT CACHE
# ========
#

General issue with (perf) for flamescope using [Hardware cache event] counters

Hi,
Im trying to use flamescope on L1-dcache-load-misses, I use
sudo node --perf-basic-prof dist/cli.js &
sudo perf record -e L1-dcache-load-misses -g -c 100000 -p pgrep -n node
sudo perf script --header > /examples/node.example.benchmark

Perf script is only producing the header output, I have not checked for which counter event types this happen.

I can do perf report and I can see that stack traces information is present, but obviously not in the format that I can use currently for flamescope -- anyone got any pointers here on how to fix this.

NOTE: if I do something like below for a [Hardware event] as reported by perf list
sudo perf record -e cache-misses -g 100000 -p pgrep -n node
then the perf script command produces stack traces.

perf version 4.13.13
kernel (uname -r) 4.13.0-36-generic

Apologies in advance if I've done something dumb ....

Thanks,

Andy

metric other than sample count

Thanks, great tool.

It would be interesting to have the heat map present other metrics than cpu sample count.

For example throughput and especially some latency statistic (eg median, 99th etc).

The data would come from elsewhere so would need to follow some predefined format, at the right interval.

Expect It likely not be possible to collect this kind of data from production systems. Test/ Benchmark systems may be able to though.

It might be a problem to marry up the perf kernel timestamps to userspace timestamps. it might fall on the perf record end to get the right clock.

$ perf record -h 2>&1|grep -i clock
    -k, --clockid <clockid>
                          clockid to use for events, see clock_gettime()```

drag and drop to select range

I think drag and drop to select range would feel more natural than clicking first start, then end.

At least that was the first thing I tried, even after looking at the image in the readme.

option to toggle flamegraph to threaded view

We often need to review individual threads for load balancing, utilisation, etc.
Being able to toggle between the 2 views would be very useful.
It would need source data in the perf script output according to stackcollapse-perf.pl.

USAGE: ./stackcollapse-perf.pl [options] infile > outfile

	--tid		# include TID and PID with process names [1]
...

[1] perf script must emit both PID and TIDs for these to work; eg, Linux < 4.1:
	perf script -f comm,pid,tid,cpu,time,event,ip,sym,dso,trace
    for Linux >= 4.1:
	perf script -F comm,pid,tid,cpu,time,event,ip,sym,dso,trace

master `Dockerfile` is broken

Recent refactorings specifically b496f77 seems to have bumped the required Python from 2.7 to 3.6. In addition, libmagic is assumed to be installed for Python binding. These are not yet updated in Dockerfile; the following patch seems to fix the problem until a fix is merged:

diff --git a/Dockerfile b/Dockerfile
index e4c1346..23cba5c 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,11 +1,12 @@
-FROM python:2-alpine3.7
+FROM python:3-alpine3.8
 
 ADD app /app/app
 ADD run.py /app
 ADD requirements.txt /app
 
-RUN cd /app && \
-  pip install -r requirements.txt && \
+RUN apk add libmagic && \
+  cd /app && \ \
+  pip3 install -r requirements.txt && \
   mkdir /profiles && \
   sed -i -e s/127.0.0.1/0.0.0.0/g -e s~examples~/profiles~g app/config.py

Side Note It would be great if there would be tags on the repo so that community could rely on a working version.

reproduce profile with animated flame graphs

I read the blog post and found it very relatable:

It would be awesome to have animated mode where one could select a range and then replay either step by step or at some reasonable rate (say 20ms per frame, 10 fps).

Flamegraph "bricks" would then shrink and expand with each "frame". This is effectively watching individual flamegraphs one after another, but with smooth transformations.

Hopefully this makes sense.

Idle regexp eats up softirq frames

We borrowed idle_regexp for our internal system and it looks like the regexp hides some legitimate stacks in softirq context.

Current filter produces the following flamegraph:

image

If I make idle_regexp match nothing, I can see much more interesting stuff:

image

The trace comes from on Intel Skylake.

event annotation on the heatmap visualization

We are considering to add an annotation feature on the heatmap to represent simultaneous events on the heatmap so that users can visually grasp relation between "hot-spot" on the heatmap and co-occurring events or can seek to an event on the heatmap.
I'd like to discuss with you about this feature before we start to implement, because this feature may require several decision to contribute to the mainline.

The hardest thing to decide is representation of the metadata, which will contain the absolute time information to relate start time of the sampling to the date time, and the information where to retrieve the event collection.
The other one to be decided is event format which may contain {name, start_at, in_or_out, length...} for example. in_or_out is presenting whether the event is shown as "in" event on top of the heatmap or as "out" event below (for more concisely, it will be good representation to rotate the map 90° clock wise so the map extends below as time goes by; and the event is plotted left and right corresponding to the attribute).

                (start)

(in)           ++++++++++           (out)
               ++++++++++
/login_________±±±$++++++
  :            ++++++++++
  :            ++++++#±±±______ Redis [session]
  :            ++++#±±±±±______ MySQL [account] 
  \____________±$++++++++

                 (end)
[a blueprint of annotating the heatmap with events]

A possible solution is just leave not to decide the precise detailed specification of the event collection and metadata but prepare a standardized way for add-ons to overlay any information. In this case, the work on flamescope won't be so large and most part of it is laying on d3 specific (actually I'm not front guy but I think letting add-ons to invoke d3.compose can solve most part of it; yet questions "what is 'add-ons' and how to place it" still remain).

The other solution is define the structure of those data and support them. In this case, I will be happy if I could discuss all the people interested in the feature and seek well-defined data format for that.

icicle layout

There could be a button on the flame graph view to switch it to icicle layout. This could be sticky, and/or configured via a settings page, so it always did it this way. I'm not sure what to call the button: I guess the two layout orderings are "flame" and "icicle". (Note that there's also a different layout type, sunburst, which we could also switch to, but that's changing the type of visualization rather than just the ordering; I'll file a separate ticket).

Adding this feature probably means adding code to https://github.com/spiermar/d3-flame-graph .

option to allow users to split heatmap by thread

Currently there seem to be the only mode with seconds on X axis and 20ms buckets on the Y axis.

I'd like to suggest a mode with threads on the Y axis and smaller buckets on the X axis.

My real world usage is ClickHouse. We have loads of data in our instances and they take quite a bit of time to both start up and tear down. During that time we can observe only some low number of cores being saturated.

With the proposed mode we'd be able to see which threads are doing what and when. This mode should work nicely with specialized thread pools.

Enable Settings view for UI controls

With more requests coming through for enhanced controls (#39 , #2 , #30 , ...), we should look at enabling the settings view or some other UI component and push some/all of the UI controls there that modify the view.

flamescope failing to render flamegraph

GIT TAG=91bc199
OS: Mac
Browser: Chrome

The heatmap page works, but on the flamegraph page, the flamegraph itself fails to load. I see the following error in the javascript console:

main.js:33 Uncaught TypeError: Cannot read property 'bind' of undefined
    at main.js:50
    at Array.forEach (<anonymous>)
    at new e (main.js:50)
    at zr (main.js:33)
    at ui (main.js:33)
    at fi (main.js:33)
    at Ki (main.js:33)
    at qi (main.js:33)
    at Da (main.js:33)
    at _a (main.js:33)

The flamegraph does load on git tag df67004

The examples provided on the current commit, 91bc199, fail with the same error.

flame graphs sometimes do not render correctly; uncaught exception

Sometimes when I select ranges of time the flamegraph comes out broken (no text and not clickable). Looking at my console I see the following exception:

Uncaught (in promise) TypeError: Cannot read property 'match' of undefined
    at main.js:17
    at F (main.js:17)
    at SVGRectElement.<anonymous> (main.js:17)
    at SVGRectElement.<anonymous> (main.js:1)
    at G.each (main.js:1)
    at G.attr (main.js:1)
    at HTMLDivElement.<anonymous> (main.js:17)
    at G.each (main.js:1)
    at T (main.js:17)
    at O (main.js:17)

It appears to happen when I pick bigger time windows. Attached a stack file that you can drop into examples. Selecting the bottom left to top right pixels leads to a broken flame graph. Smaller selections can also trigger the issue.

mesos-agent.stacks.1.zip

Here's an example of what it looks like:

image

filter flame graph stacks

I'd really like filtering out some stacks in the GUI.. (probably it's possible in the perf cli somehow though I'm kinda newish to perf ).

Example usage:

Did whole system perf record, found out process node_exporter does its periodic work for 30s. Ok, I'm interested what the rest of the system is doing, so I could filter out matching regex (( e.g. if this stack or its parent stack match the regex ignore/include them ))

gitignore app/ files

$ git status
...
#	modified:   app/public/main.js

should this be in .gitignore?

master currently not running

Hi there,

currently when fetching the origin/master HEAD and trying to use python run.py on ubuntu it will not run due to some import issues:

sergej@L530:~/GitHub/flamescope$ python run.py
Traceback (most recent call last):
File "run.py", line 6, in
from app import APP
File "/home/sergej/GitHub/flamescope/app/init.py", line 34, in
from app.views.flame_graph import MOD_FLAME_GRAPH
File "/home/sergej/GitHub/flamescope/app/views/flame_graph.py", line 21, in
from app.controllers.flame_graph import generate_flame_graph
File "/home/sergej/GitHub/flamescope/app/controllers/flame_graph.py", line 21, in
from app.common.fileutil import get_profile_type
File "/home/sergej/GitHub/flamescope/app/common/fileutil.py", line 25, in
from json import JSONDecodeError
ImportError: cannot import name JSONDecodeError

It looks like this issue is caused by recent restructuring. When I revert to some later commit everything workds just fine.

UnboundLocalError when the file is invalid

Stack trace when trying to open an invalid file:

ERROR: Can't read stack file examples/perf.data.
127.0.0.1 - - [05/Apr/2018 15:21:29] "GET /heatmap/?filename=perf.data&rows=50 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/google/home/strandmark/source/flamescope/app/views/heatmap.py", line 34, in get_heatmap
    return jsonify(generate_heatmap(filename, rows))
  File "/usr/local/google/home/strandmark/source/flamescope/app/util/heatmap.py", line 107, in generate_heatmap
    o = read_offsets(filename)
  File "/usr/local/google/home/strandmark/source/flamescope/app/util/heatmap.py", line 66, in read_offsets
    f.close()
UnboundLocalError: local variable 'f' referenced before assignment

sunburst layout toggle

Similar to #8, this would be an option to draw the flamegraphs in the sunburst layout (radial coordinates).

First use of it I think came from here by @paulirish: https://bugs.chromium.org/p/chromium/issues/detail?id=452624

I've also written about the layout here: http://www.brendangregg.com/blog/2017-02-06/flamegraphs-vs-treemaps-vs-sunburst.html

We can have it as a mode. This is why I wanted flamegraphs to go d3 in the first place, so we can make it more interactive and manipulatable.

Flamescope dashboard not available

I had followed all steps as stated in README. Still not able to view dashboard.
Please find the screenshots below:
screen shot 2018-04-14 at 11 32 46 am
Here of dashboard:
imageedit_2_7523124955
Port Number: 5000 is open & whitelisted for any ip address.
I am running the flamescope tool on my ec2 instance.

OverflowError: cannot convert float infinity to integer

Ended up being caused by an empty file:

-rw-rw-r-- 1 dan dan 0 Apr 4 20:30 stacks.mysql-64-oltp_read_only_1.2018-04-04 20:30:08-05:00

Traceback (most recent call last):
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/dan/pyenv/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/dan/flamescope/app/views/heatmap.py", line 34, in get_heatmap
    return jsonify(generate_heatmap(filename, rows))
  File "/home/dan/flamescope/app/util/heatmap.py", line 120, in generate_heatmap
    cols = int(ceil(end) - floor(start))
OverflowError: cannot convert float infinity to integer

Remove invalid chars from the stack file in medium.com blog (was - README example).

Hello and thanks for flamescope,
In https://medium.com/netflix-techblog/netflix-flamescope-a57ca19d47bb and https://github.com/Netflix/flamescope/blob/master/README.md#installation--instructions the example filename used is stacks.myproductionapp.2018-03-30_01 , (UPDATE - the first dash on the medium site is a non ASCII char), as of cbd13e1 for #7 this is now invalid and the heatmap generation fails with a 500 from app/util/heatmap.py::read_offsets when fileutil.validpath returns false.
I was thinking that in addition to a change to the filename used in the example (UPDATE - on medium.com), that an error could be printed to the terminal, something like:

     if not fileutil.validpath(path):
+        print("ERROR: File %s path %s is not valid" % (filename, path) )
         return abort(500)

Thanks.

Reproducer:

> cp examples/perf.test01 examples/stacks.myproductionapp.2018-03-30_01 (copy-and-paste from blog on medium.com)
> python run.py
Click on link for the heatmap.

Error in terminal is:

[16/Apr/2018 07:38:09] "GET /heatmap/?filename=stacks.myproductionapp.2018%E2%80%9303%E2%80%9330_01&rows=50 HTTP/1.1" 500 -

Error in the web app is a missing heatmap.

time labels below plot

First of all: thank you for this very powerful visualization!

One thing that was counter-intuitive to me was that each seconds progresses bottom->top. Since the time labels are above the plot, my brain parses the upper left corner as the origin.

I think that moving the time labels to below the plot might provide the right visual cue to move the origin to the bottom left corner.

The application lacks protection against CSRF

The application lacks any sort of protection against CSRF (let alone DNS rebinding).

As some, uhhh, creative development time saving techniques are used, an attacker can execute arbitrary shell commands on the computer running this software by redirecting the users browser to something like:

<a href="http://127.0.0.1:5000/stack/?filename=%3Bxcalc%3Bfoo.gz">Link</a>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.