Giter Club home page Giter Club logo

ghc-events-analyze's Introduction

ghc-events-analyze

See Performance profiling with ghc-events-analyze for an introduction to this tool. Below we describe some features that were introduced since the blog post.

Controlling layout

As of version 0.2.3, there are a number of options for controlling the layout. To slice time coarsely, as we did in the blog post, you can run

ghc-events-analyze -b 50 ...

This results in something like

By default (in 0.2.0 and again from 0.2.3) time is split into 100 buckets. To slice time more finely (this was the default for versions 0.2.1 and 0.2.2), you can run

ghc-events-analyze -b 500 --tick-every 10 --bucket-width 1 --border-width 0

The results in something like

Windowing

Windowing can be used to split all events into a bunch of reports, one per window. You can use it like this:

  traceEventIO "START WINDOW"
  ...
  traceEventIO "STOP WINDOW"

  traceEventIO "START WINDOW"
  ...
  traceEventIO "STOP WINDOW"

  traceEventIO "START WINDOW"
  ...
  traceEventIO "STOP WINDOW"

If you then run ghc-events-analyze using --window "WINDOW" it will create one report per window; for instance, in the above example it would create

example.0.timed.svg
example.1.timed.svg
example.2.timed.svg

Event subscripts

Suppose you have multiple events that should all show up as request in the generated reports, but should nonetheless be distinguished from each other. There are two ways to do this. One is to call the events request0, request1, etc.

traceEventIO "START request0"
...
traceEventIO "STOP request0"

traceEventIO "START request1"
...
traceEventIO "STOP request1"

and then use the ghc-events-analyze DSL to add some renaming instructions. However, that might get tedious if there are a lof of these. Alternatively, you can use event subscripts, like this:

traceEventIO "START 0 request"
...
traceEventIO "STOP 0 request"

traceEventIO "START 1 request"
...
traceEventIO "STOP 1 request"

These subscripts are used to distinguish events, but do not show up in the report.

ghc-events-analyze's People

Contributors

basvandijk avatar bgamari avatar cartazio avatar dcoutts avatar dpwiz avatar edsko avatar fuuzetsu avatar ganeshrapolu avatar idontgetoutmuch avatar maoe avatar osa1 avatar pepeiborra avatar qnikst avatar sergv avatar sjakobi avatar willsewell avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ghc-events-analyze's Issues

User event questions

I hope this isn't spam but I have a few questions after reading the blog post and the README, and playing with this a little more:

  1. You write "[user START/STOP event] periods are completely independent of threads". Am I right that this means we can't just e.g. wrap a handler function HttpRequest -> IO JSON in traceEventIO "START/STOP mainHandlerFunc" and expect it to give sensible results (since STARTs and STOPs will be interleaved across threads, and ghc-events-analyze will pair a START from one thread with a STOP from a different one)?
  2. If so is this a usecase for "Event subscripts", where I could do above but hash the threadID and use that as a subscript, allowing us to correlate stops and starts?
  3. How should I understand how the tool behaves when there is nesting (START foo >> START bar >> STOP bar >> STOP foo)? It looks like that is supported but it's the users job to make note of when that's happening and what part of the time of an event should be attributed to a child event. I'm just interested in being able to use the same instrumentation to be able to get a more traditional profiling report as well, and I'm curious what you think about that (see also #29)

Build failed.

Hey, only starting to learn Haskell here.
Cloned the repo, did stack init and stack install which failed for me with:

.../ghc-events-analyze/src/GHC/RTS/Events/Analyze/Reports/Timed/SVG.hs:25:11: error:
    • Couldn't match type ‘(,) (F.FontData Double)’ with ‘IO’
      Expected type: IO (F.OutlineMap Double)
        Actual type: F.PreparedFont Double
    • In a stmt of a 'do' block: font <- F.bit
      In the expression:
        do font <- F.bit
           uncurry (renderSVG path)
             $ renderReport options quantized report font
      In an equation for ‘writeReport’:
          writeReport options quantized report path
            = do font <- F.bit
                 uncurry (renderSVG path)
                   $ renderReport options quantized report font
   |
25 |   font <- F.bit
   |           ^^^^^

.../ghc-events-analyze/src/GHC/RTS/Events/Analyze/Reports/Timed/SVG.hs:26:68: error:
    • Couldn't match type ‘Data.Map.Internal.Map
                             String (D.Path V2 Double)’
                     with ‘(F.FontData Double, F.OutlineMap Double)’
      Expected type: F.PreparedFont Double
        Actual type: F.OutlineMap Double
    • In the fourth argument of ‘renderReport’, namely ‘font’
      In the second argument of ‘($)’, namely
        ‘renderReport options quantized report font’
      In a stmt of a 'do' block:
        uncurry (renderSVG path)
          $ renderReport options quantized report font
   |
26 |   uncurry (renderSVG path) $ renderReport options quantized report font
   |                                                                    ^^^^

Opened a text editor, changed this function to

writeReport :: Options -> Quantized -> Report -> FilePath -> IO ()
writeReport options quantized report path = do
  uncurry (renderSVG path) $ renderReport options quantized report F.bit

After that it successfully built and ate my eventlog, producing nice pictures.
Only learning here, have no Idea what I'm doing. Just close this if I did something stupid.

Make it easy to filter finaliser threads out

Finalizer threads clobber the view when present, since their number is proportional to the number of collections and can be quite large.

Unfortunately there is no easy way to filter them out. labelThreadId can be used on a finaliser body, but finaliser threads get reused sometimes and as it turns out labelThreadId is not idempotent: labels get appended.

I don't see any mention of finalisers in the AST defined by ghc-events so perhaps this ticket should be raised in the ghc-events issue tracker or even in the GHC issue tracker, but I thought I'd ask here first.

Support GHC >= 9.0.

It seems the latest version of ghc that is still supported is 8.10 which is the next version that will be deprecated.

Fails to open SVG font file

When executing ghc-events-analyze it says:

ghc-events-analyze: /home/ricardo/.cabal/share/x86_64-linux-ghc-7.6.3/SVGFonts-1.4.0.3/fonts/LinLibertine.svg: hGetContents: invalid argument (invalid byte sequence)

Not sure if this error comes from ghc-events-analyze or SVGFonts. Tested with Haskell platform using GHC (7.6.3).

Rendering unlike blogpost..

This is SVG produced on a Mac and on Ubuntu.
How can I get the beautiful and readable square-rendering for each bucket ?
On my machine(s) each bucket is just a very slim rectangle and does not fill an entire time-line section like in your blog-post.

screenshot from 2016-06-12 21-56-30

Fails to install

stack --version
Version 2.9.3 aarch64

stack install ghc-events-analyze

Error: [S-4804]
       Stack failed to construct a build plan.
       
       While constructing the build plan, Stack encountered the following errors:
       
       In the dependencies for ghc-events-analyze-0.2.8:
           SVGFonts-1.8.0.1 from Stack configuration does not match >=1.7 && <1.8 (latest matching version is 1.7.0.1)
           base-4.16.4.0 from Stack configuration does not match >=4.9 && <4.15 (latest matching version is 4.14.3.0)
           ghc-events-0.18.0 from Stack configuration does not match >=0.13 && <0.14 (latest matching version is 0.13.0)
           hashable-1.4.1.0 from Stack configuration does not match >=1.2 && <1.4 (latest matching version is 1.3.5.0)
           lens-5.1.1 from Stack configuration does not match >=3.10 && <4.20 (latest matching version is 4.19.2)
           optparse-applicative-0.17.0.0 from Stack configuration does not match >=0.11 && <0.17 (latest matching
                                         version is 0.16.1.0)
       needed since ghc-events-analyze is a build target.

SortIndex in the wrong place?

We have this constructor of EventId:

    -- | User events
    --
    -- To use user events, do
    --
    -- > traceEventIO "START <label>"
    -- > ...
    -- > traceEventIO "STOP <label>"
  | EventUser String SortIndex

This SortIndex was introduced in e6bc836 (where it was called GroupId). However, looking at it now, it seems that the constructor should instead be

  | EventUser SortIndex String

so that the SortIndex takes priority over the name when sorting events (we sort using the derived Ord; see compareEventIds, case SortByName). @xich , is there a specific reason you added the SortIndex (you called it GroupId) as the second field?

improve space complexity of ghc-events analyze

even modestly complex programs that are the slightest bit parallel tend to product pretty substantial event logs in just 5-6 seconds (on the order of 100mb)

currently ghc-events-analyze requires several gigabytes to compute the resulting visual

Chrome Trace Event support?

Hi,

First off, very nice project! I've only tried it on a toy example, but enjoying it so far and I want to dig deeper in my real projects using the tool.

I was watching a talk about the Xi text editor implementation, where they exported tracing data in the Chrome Trace Event format and used chrome://tracing as a frontend.

These blog posts describe the general idea:

Would this be a feature you'd be interested to have in ghc-events-analyze, i.e. also exporting to the Chrome Trace Event format, in addition to SVG and text files?

Cheers!

Synthesize start and stop "events" corresponding to absence of user events

I'm using this tool to profile some non-trivial pure code. It's very helpful, thank you.

This issue is a feature request. The feature is for ghc-events-analyze to synthesize events corresponding to START no-genuine-user-events-are-open and STOP no-genuine-user-events-are-open. Informally, my goal is for the synthesized no-genuine-user-events-are-open event to be the complement of all other user events.

This feature would help me determine if I've placed my traceEvent calls carefully and thoroughly enough for my profiling data to be useful.

(In my local branch, I've hacked up something similar by post-processing the quantized user events, but it's severely inaccurate due to double-counting when user events overlap in time within a bucket. However, it already does make it obvious on the graph when none of my genuine user events are active in a given bucket.)

Why are threads recorded as having terminated at the end of a window?

@WillSewell I've merged #16 tentatively, but I'm not 100% sure it's correct. So if I understand your code correctly, if a thread start outside the scope of any window, then we basically don't have a EventAnalysis to add it to, and so you add it to pendingIds and record them as having started at the start of the next window. That makes a certain amount of sense to me.

However, then when a window finishes you record all threads as having terminated. Why?

How to debug `_startup not set` ?

How should I interpret the error ghc-events-analyze: _startup not set? Is this a problem with my eventlog? Is there something I can do to work around it?

Does not compile on macOS

Building all executables for `ghc-events-analyze' once. After a successful build of all of them, only specified executables will be rebuilt.
ghc-events-analyze> build (exe)
Preprocessing executable 'ghc-events-analyze' for ghc-events-analyze-0.2.7..
Building executable 'ghc-events-analyze' for ghc-events-analyze-0.2.7..
[ 7 of 11] Compiling GHC.RTS.Events.Analyze.Analysis

/Users/arlyon/Downloads/ghc-events-analyze-0.2.7/src/GHC/RTS/Events/Analyze/Analysis.hs:85:55: error:
    • Couldn't match type ‘text-1.2.4.0:Data.Text.Internal.Text’
                     with ‘[Char]’
      Expected type: String
        Actual type: text-1.2.4.0:Data.Text.Internal.Text
    • In the second argument of ‘labelThread’, namely ‘l’
      In the expression: labelThread tid l
      In a case alternative: ThreadLabel tid l -> labelThread tid l
   |
85 |         ThreadLabel tid l          -> labelThread tid l
   |                                                       ^

/Users/arlyon/Downloads/ghc-events-analyze-0.2.7/src/GHC/RTS/Events/Analyze/Analysis.hs:95:27: error:
    • Couldn't match type ‘text-1.2.4.0:Data.Text.Internal.Text’
                     with ‘[Char]’
      Expected type: text-1.2.4.0:Data.Text.Internal.Text
        Actual type: String
    • In the pattern: prefix optionsUserStart -> Just e
      In the pattern: UserMessage (prefix optionsUserStart -> Just e)
      In an equation for ‘startId’:
          startId (UserMessage (prefix optionsUserStart -> Just e))
            = Just $ parseUserEvent e
   |
95 |     startId (UserMessage (prefix optionsUserStart -> Just e)) = Just $ parseUserEvent e
   |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/Users/arlyon/Downloads/ghc-events-analyze-0.2.7/src/GHC/RTS/Events/Analyze/Analysis.hs:101:26: error:
    • Couldn't match type ‘text-1.2.4.0:Data.Text.Internal.Text’
                     with ‘[Char]’
      Expected type: text-1.2.4.0:Data.Text.Internal.Text
        Actual type: String
    • In the pattern: prefix optionsUserStop -> Just e
      In the pattern: UserMessage (prefix optionsUserStop -> Just e)
      In an equation for ‘stopId’:
          stopId (UserMessage (prefix optionsUserStop -> Just e))
            = Just $ parseUserEvent e
    |
101 |     stopId (UserMessage (prefix optionsUserStop -> Just e)) = Just $ parseUserEvent e
    |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


--  While building package ghc-events-analyze-0.2.7 using:
      /Users/arlyon/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_3.0.1.0_ghc-8.8.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-3.0.1.0 build exe:ghc-events-analyze --ghc-options " -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.