Giter Club home page Giter Club logo

illimani-memory-profiler's Introduction

Illimani: a Memory Profiler

Pharo version Pharo version

It works on Pharo 11 and Pharo 12. But, keep in mind that the profiler should be faster in Pharo 12. This is because there were some optimizations done in Pharo 12 to make the instrumentation faster.

Illimani is a library of memory profilers. It provides a memory allocation profiler and a finalization profiler. The allocation profiler gives you information about the allocation sites and where the objects where produced in your application. The finalization profiler gives you information about how much time did the objects lived, and about how many GC cycles (both scavenges and full GC) they survived.

How to install it

EpMonitor disableDuring: [
	Metacello new
		baseline: 'IllimaniProfiler';
		repository: 'github://jordanmontt/illimani-memory-profiler:main';
		load ].

Quick Getting Started

Profiling a given code snippet

profiler
	profileOn: [ 15 timesRepeat: [ StPlaygroundPresenter open close ] ] ;
	open;
	yourself

Profiling the Pharo IDE activity for a given amount of time

profiler
	profileFor: 6 seconds;
	open;
	yourself

Example 1, allocation profiler for profiling the Pharo IDE activity

IllAllocationSiteProfiler new
	copyExecutionStack;
	profileFor: 6 seconds;
	open;
	yourself

Example 2, finalization profiler on a code snippet:

IllFinalizationProfiler new.
	profileOn: [ 15 timesRepeat: [ StPlaygroundPresenter open close ] ] ;
	open;
	yourself

How to use

Profile a code snippet or the Pharo IDE

You can decide both to profile a given method block or just watching the activity of the image for some time.

"With this the profiler will block the ui and you will only capture the objects created by your code snippet"
profiler profileOn: [ anObject performSomeAction ].

"With this the profiler with not block the UI nor the image. So, you will capture all the allocations of the image"
profiler profileFor: 2 seconds.

Profiler manual API

For starting the stoping the profiling manually. This can be useful if you don't know how long your program will run and you need to interact with the Pharo's IDE.

profiler startProfiling.
profiler stopProfiling.

Open the GUI

You can open the ui at any time with the message open (even if the profiler is still profiling)

profiler open.

Sample the allocations

By default, the profiler captures all the object allocations. You can configure it to sample the samples. This can be useful for reducing the overhead when your application makes lots of allocations.

"Capture only 10% of the allocations"
profiler samplingRate: 10.

Export the profiled data to files

You can export the data to csv and json files by doing:

profiler exportData

This will create a csv file with all the information about the allocated objects, and some other auxiliary files in json and csv. This auxiliary files can be the meta data about the total profiled time, the gc activity, etc.

Monitor the GC activity

You can monitor the GC activity while the profiler is profiling with the message monitorGCActivity. This will fork a process that will take GC statistics once per second. Then, when exporting the profiler data, two csv files will be exported containing both the scavenges and full GCs.

profiler monitorGCActivity

Implement your own memory profiler

Illimani is also a profiling framework. A user can implement his own profiler by subclassing the IllAbstractProfiler class and defining the few missing methods. Especially, the internalRegisterAllocation: method. The internalRegisterAllocation: method will be called each time that an allocation is produced (or when sampling, each time that the sampling rates matches) with the newly allocated object as a parameter. You can the IllAllocationRateProfiler class as an example of a very simple memory profiler.

Allocation Site profiler

GIF1

It is also possible to copy the execution stack of each of the allocated objects with the message. This is very useful when you to make analysis, for example indentify in which context the allocations were prodoced, etc. Keep in mind that with a lot of allocations, copying the stack can cause the image to grow in size rapidly and making it slow to use.

profiler copyExecutionStack.

Without the UI, because the profiler is independent from the UI, you can access to some statistics. See the protocol accessing - statistics in the profiler to see the methods. Also, the profiler has a statistics model that groups and sorts the allocation by class and by methods. For example check 'profiler stats allocationsByClass.'

Finalization profiler

GIF2

Implementation details

  • Illimani uses method proxies library to capture the allocations. It inserts a proxy in Behavior>>basicNew:, Behavior>>basicNew, Array>>#new:, and Object >> #shallowCopy.
  • It uses Ephemerons to know when an object is about to be finalized.
  • It has an statistics model that helps with the calculations of allocations grouping them by classes and methods and sorting them by number of allocations.
  • The UI is independent of the profiler. It can be used without it. You will have access to all allocations and to the same statistics.

illimani-memory-profiler's People

Contributors

akevalion avatar guillep avatar jordanmontt avatar pdebruic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

illimani-memory-profiler's Issues

Doesn't load in Pharo 12

EpMonitor disableDuring: [
	Metacello new
		baseline: 'IllimaniProfiler';
		repository: 'github://jordanmontt/illimani-memory-profiler:main';
		load ].

Result :(

Warning: The following definitions had errors while loading.  Press Proceed to try to load them again (they may work on a second pass):
  IllFinalizationProfilerUI
  IllFrequencyHistogramPresenter

Loading error in P11 (v0.8)

Hello,

I read in main branch "The release version v0.8 works on Pharo 11 and Pharo 10. To use it on Pharo 12, please use the version that is on the main branch." and I wanted to load it in P11.

So, I changed to the tag (https://github.com/jordanmontt/illimani-memory-profiler/tree/v0.8) and modified the snippet to load that version:

EpMonitor disableDuring: [
	Metacello new
		baseline: 'IllimaniAllocationProfiler';
		repository: 'github://jordanmontt/illimani-memory-profiler:v0.8';
		load ].

I got:

Warning: Package *MethodProxies depends on the following classes:
  InstrumentationEnsurer
You must resolve these dependencies before you will be able to load these definitions: 
  MpInstrumentationUnwinder
  MpInstrumentationUnwinder>>#newWithHandler:receiver:arguments:
  MpInstrumentationUnwinder>>#arguments
  MpInstrumentationUnwinder>>#arguments:
  MpInstrumentationUnwinder>>#handler
  MpInstrumentationUnwinder>>#handler:
  MpInstrumentationUnwinder>>#receiver
  MpInstrumentationUnwinder>>#receiver:
  MpInstrumentationUnwinder>>#value

DNU: Instance of WriteStream did not understand #inspectionFullString

Hi,
Clicking on the GUI, I got:

Screenshot 2023-09-28 at 14 42 30

And DNU seems to be fixed when I inserted the send "contents" like this:

inspectorExtensionContextFingerprint: aBuilder
	<inspectorPresentationOrder: 2 title: 'Context Fingerprint'>

	^ contextFingerprint contents inspectionFullString

Please tell me if my bug description is not precise enough.

Use ephemerons

In the new vm the ephemerons are implemented. I need to use them in the profiler to know when an Object is being garbage collected. It can be also used to calculate the hash of the objects. Because we can assume that at that time the object that configurated enough to extract the hash from it. Because when I tried to extract the hash at the allocation time sometimes I was getting erros because the object was failing when calculating the hash

Pharo 12 image unresponsive

Hello. I loaded the project with:

EpMonitor disableDuring: [
	Metacello new
		baseline: 'IllimaniProfiler';
		repository: 'github://jordanmontt/illimani-memory-profiler:main';
		load ].

This is a snippet that reproduces it:

IllAllocationProfiler new
	profileFor: 1 seconds;
	open;
	yourself

But also copy-pasted scripts from readme, and a script that I used in the past for Bloc.

My Pharo has 1 or 2 weeks:

Pharo 12.0.0
Build information: Pharo-12.0.0+build.1058.sha.d7fe9c39709fd9c1855730743d80c4c7d49a116d (64 Bit)

Is this information enough?

Context fingerprint

The context fingerprint was removed because it was causing the profiler to break in the latest Pharo 12. One new implementation is needed

Created separeted classes for the presenters in the Inspector

Currently, for example for the statistics nodes, there are inspector extensions (presenters). Those presenters are created in methods. I need to put them into separated classes like that I can override the transmission port and have custom transmissions in the inspector.

Add a spinner when taking time to load

When there is a lot of data, to open the UI takes a lot of time. Especially the heat-map. It is needed to add a spinner to show to the user that the tool is calculating something.

After it is worth optimizing the speed up things

Improve sampling

Currently, the sampling only accepts numbers between 1 and 100 (1% to 100%). The mechanism needs to be improved to accept smaller rates. Maybe a user wants to sample 1 of every 1000 objects (0.1%)

Add human readable size text

image

Please in total size in byte column use humanReadableSIByteSize or a similar method to express to users more meaningful information.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.