samsaffron / memory_profiler Goto Github PK

View Code? Open in Web Editor NEW

1.7K 21.0 88.0 246 KB

memory_profiler for ruby

License: MIT License

Ruby 100.00%

memory_profiler's Introduction

MemoryProfiler

A memory profiler for Ruby

Requirements

Ruby(MRI) Version 3.1.0 and above.

Installation

Add this line to your application's Gemfile:

gem 'memory_profiler'

And then execute:

$ bundle

Or install it yourself as:

$ gem install memory_profiler

Usage

There are two ways to use memory_profiler:

command line
convenience API

Command Line

The easiest way to use memory_profiler is via the command line, which requires no modifications to your program. The basic usage is:

$ ruby-memory-profiler [options] run [--] command [command-options]

Example:

$ ruby-memory-profiler --pretty run -- rubocop --cache false

$ ruby-memory-profiler --max=10 --pretty run -- ruby notify_users.rb 1 2 3 --quiet

For a full list of options, execute the following command:

$ ruby-memory-profiler -h

Convenience API

require 'memory_profiler'
report = MemoryProfiler.report do
  # run your code here
end

report.pretty_print

Or, you can use the .start/.stop API as well:

require 'memory_profiler'

MemoryProfiler.start

# run your code

report = MemoryProfiler.stop
report.pretty_print

NOTE: .start/.stop can only be run once per report, and .stop will be the only time you can retrieve the report using this API.

Options

`report`

The report method can take a few options:

top: maximum number of entries to display in a report (default is 50)
allow_files: include only certain files from tracing - can be given as a String, Regexp, or array of Strings
ignore_files: exclude certain files from tracing - can be given as a String or Regexp
trace: an array of classes for which you explicitly want to trace object allocations

Check out Reporter#new for more details.

pry> require 'memory_profiler'
pry> MemoryProfiler.report(allow_files: 'rubygems'){ require 'mime-types' }.pretty_print
Total allocated 82375
Total retained 22618

allocated memory by gem
-----------------------------------
rubygems x 305879

allocated memory by file
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb x 285433
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/basic_specification.rb x 18597
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems.rb x 2218
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/specification.rb x 1169
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/defaults.rb x 520
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_gem.rb x 80
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/version.rb x 80

. . .

`pretty_print`

The pretty_print method can take a few options:

to_file: a path to your log file - can be given a String
color_output: a flag for whether to colorize output - can be given a Boolean
retained_strings: how many retained strings to print - can be given an Integer
allocated_strings: how many allocated strings to print - can be given a Integer
detailed_report: should report include detailed information - can be given a Boolean
scale_bytes: flag to convert byte units (e.g. 183200000 is reported as 183.2 MB, rounds with a precision of 2 decimal digits) - can be given a Boolean
normalize_paths: flag to remove a gem's directory path from printed locations - can be given a Boolean Note: normalized path of a "location" from Ruby's stdlib will be prefixed with ruby/lib/. e.g.: ruby/lib/set.rb, ruby/lib/pathname.rb, etc.

Check out Results#pretty_print for more details.

For example to report to file, use pretty_print method with to_file option and path_to_your_log_file string:

$ pry
pry> require 'memory_profiler'
pry> MemoryProfiler.report(allow_files: 'rubygems'){ require 'mime-types' }.pretty_print(to_file: 'path_to_your_log_file')

$ less my_report.txt
Total allocated 82375
Total retained 22618

allocated memory by gem
-----------------------------------
rubygems x 305879

. . .

Example Session

You can easily use memory_profiler to profile require impact of a gem, for example:

pry> require 'memory_profiler'
pry> MemoryProfiler.report{ require 'mime-types'  }.pretty_print
Total allocated 82375
Total retained 22618

allocated memory by gem
-----------------------------------
mime-types-2.0 x 3668277
2.1.0-github/lib x 2035704
rubygems x 305879
other x 40

allocated memory by file
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb x 3391763
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb x 2021853
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb x 285433
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader.rb x 227033
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/cache.rb x 48663
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/basic_specification.rb x 18597
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/parser.so x 8200
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/generator.so x 2463
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems.rb x 2218
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/specification.rb x 1169
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/defaults.rb x 520
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader_path.rb x 417
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/version.rb x 409
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types.rb x 361
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/generic_object.rb x 241
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/ext.rb x 200
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json.rb x 120
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_gem.rb x 80
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/version.rb x 80
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime-types.rb x 40
(pry) x 40

allocated memory by location
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 1985709
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:522 x 927503
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 827676
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:302 x 499525
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:625 x 281047
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:521 x 265920
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 265920
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:523 x 265920
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 222705
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader.rb:215 x 218105
[REDACTED]

allocated]objects by gem
-----------------------------------
mime-types-2.0 x 56564
2.1.0-github/lib x 22210
rubygems x 3600
other x 1

allocated objects by file
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb x 56237
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb x 21978
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb x 3388
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/cache.rb x 291
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/basic_specification.rb x 169
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/parser.so x 124
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems.rb x 53
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/generator.so x 37
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/specification.rb x 26
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader.rb x 24
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/defaults.rb x 13
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/version.rb x 7
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types.rb x 6
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader_path.rb x 5
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/ext.rb x 5
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json.rb x 3
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/generic_object.rb x 3
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_gem.rb x 2
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/version.rb x 2
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime-types.rb x 1
(pry) x 1

allocated objects by location
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:522 x 21337
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 21095
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 9972
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:523 x 6648
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:521 x 6648
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 6648
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 3307
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:302 x 2955
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:625 x 1663
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:60 x 837
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:224 x 312
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/cache.rb:62 x 287
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/parser.so:0 x 124
[REDACTED]

retained memory by gem
-----------------------------------
mime-types-2.0 x 1496813
2.1.0-github/lib x 519954
rubygems x 76195

retained memory by file
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb x 1447316
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb x 516679
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb x 75946
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/cache.rb x 48583
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/generator.so x 2263
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/parser.so x 892
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader.rb x 577
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader_path.rb x 297
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/basic_specification.rb x 169
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/specification.rb x 80
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types.rb x 40
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/generic_object.rb x 40
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/ext.rb x 40
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/version.rb x 40

retained memory by location
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 516181
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:302 x 499525
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 414007
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:625 x 280878
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 132960
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:523 x 66480
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135 x 58151
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:224 x 52728
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/cache.rb:62 x 48503
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 17795
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/generator.so:0 x 2263
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/parser.so:0 x 892
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader.rb:239 x 577
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader_path.rb:15 x 257
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/basic_specification.rb:124 x 169
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:122 x 89
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:46 x 89
[REDUCTED]

retained objects by gem
-----------------------------------
mime-types-2.0 x 15211
2.1.0-github/lib x 7089
rubygems x 318

retained objects by file
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb x 14918
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb x 7045
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb x 315
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/cache.rb x 289
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/generator.so x 32
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/parser.so x 9
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/specification.rb x 2
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader_path.rb x 2
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/loader.rb x 1
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types.rb x 1
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/version.rb x 1
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/generic_object.rb x 1
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/ext.rb x 1
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/basic_specification.rb x 1

retained objects by location
-----------------------------------
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 7035
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 4987
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 3324
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:302 x 2955
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:625 x 1662
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:523 x 1662
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:224 x 312
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 300
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/types/cache.rb:62 x 287
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/x86_64-linux/json/ext/generator.so:0 x 32
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135 x 15
[REDUCTED]

Allocated String Report
-----------------------------------
"application" x 12050
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:522 x 4820
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:521 x 2410
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 2410
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 2410
"" x 6669
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:522 x 6648
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 11
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:266 x 4
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:71 x 1
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:72 x 1
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:69 x 1
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:0 x 1
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_gem.rb:39 x 1
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:70 x 1
"/" x 3342
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:523 x 3324
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 13
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:60 x 4
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135 x 1
"encoding" x 3336
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 3324
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 10
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135 x 1
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:60 x 1
"registered" x 3328
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 3324
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 4
"content-type" x 3327
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 3324
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 3
"references" x 2944
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 2940
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 4
"IANA" x 2824
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 1412
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:302 x 1412
[REDUCTED]

Retained String Report
-----------------------------------
"IANA" x 2824
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 1412
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:302 x 1412
"application" x 2410
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 1205
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 1205
"base64" x 1527
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb:155 x 1525
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:70 x 1
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 1
"audio" x 300
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 150
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 150
"video" x 188
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 94
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 94
"text" x 155
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:629 x 77
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb:632 x 77
    /home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55 x 1
[REDUCTED]

The data is also available in the MemoryProfiler::Results object returned.

Retained vs Allocated

The report breaks down 2 key concepts.

Retained: long lived memory use and object count retained due to the execution of the code block.

Allocated: All object allocation and memory allocation during code block.

As a general rule "retained" will always be smaller than or equal to allocated.

Memory profiler will tell you aggregate costs of the above, for example requiring the mime-types gem above results in approx 2MB of retained memory in 22K or so objects. The actual RSS cost will always be slightly higher as MRI heaps are not squashed to size and memory fragments. In future we may be able to calculate a rough long term GC cost of retained objects (for major GCs).

Memory profiler also performs some String analysis to help you find strings that would heavily benefit from #freeze. In the example above the string IANA is retained in memory 2824 times, this costs you a minimum of RVALUE_SIZE (40 on x64) * 2824.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

memory_profiler's People

Contributors

Stargazers

Watchers

memory_profiler's Issues

How to optimize module names as allocated strings

I'm profiling COSMOS which is a qtbindings application. My highest allocated string report is 'Qt' because it's how everything is namespaced. Is there anyway to optimize this? Thanks for an awesome gem!

can't convert ItsIt::It to String

I'm trying to profile my gems in a Rails 2.2.5 app.

I've created script/memory-profiler.rb as follows:

require 'memory_profiler'

report = MemoryProfiler.report do
  require_relative '../config/environment'
  # Add specific code here if you want to profile deeper
end

report.pretty_print

ruby script/memory-profiler.rb gives me the following:

ernsputer:sign2web elsurudo$ ruby script/memory-profiler.rb 
/Users/elsurudo/.rbenv/versions/2.2.5/lib/ruby/gems/2.2.0/gems/memory_profiler-0.9.6/lib/memory_profiler/reporter.rb:102:in `block in object_list': can't convert ItsIt::It to String (ItsIt::It#to_str gives ItsIt::It) (TypeError)
    from /Users/elsurudo/.rbenv/versions/2.2.5/lib/ruby/gems/2.2.0/gems/memory_profiler-0.9.6/lib/memory_profiler/reporter.rb:92:in `each'
    from /Users/elsurudo/.rbenv/versions/2.2.5/lib/ruby/gems/2.2.0/gems/memory_profiler-0.9.6/lib/memory_profiler/reporter.rb:92:in `object_list'
    from /Users/elsurudo/.rbenv/versions/2.2.5/lib/ruby/gems/2.2.0/gems/memory_profiler-0.9.6/lib/memory_profiler/reporter.rb:40:in `run'
    from /Users/elsurudo/.rbenv/versions/2.2.5/lib/ruby/gems/2.2.0/gems/memory_profiler-0.9.6/lib/memory_profiler/reporter.rb:27:in `report'
    from /Users/elsurudo/.rbenv/versions/2.2.5/lib/ruby/gems/2.2.0/gems/memory_profiler-0.9.6/lib/memory_profiler.rb:14:in `report'
    from script/memory-profiler.rb:3:in `<main>'

Not sure how to proceed...

I am using memory_profiler (0.9.6).

Let me know if you need any more details about my environment.

Print report during profiling

Is it possible to print the report while memory profiling is being done?

P.S. The program I wish to profile would take about an hour to run. I'd like to see the memory it is using while executing.

Usage with Ractors causes a segmentation fault

When using memory_profiler with Ractors, it causes a segfault in (I believe) gc_event_hook_body. I'm unsure if that's a VM issue or something in the library code, but I have a minimal reproduction that came from michaelherold/benchmark-memory#27:

# frozen_string_literal: true

require "memory_profiler"

def factorial(n)
  n == 0 ? 1 : n * factorial(n - 1)
end

MemoryProfiler.start

ractor = Ractor.new do
  1000.times { factorial(1000) }
end

# take response from ractor, so it will actually execute
ractor.take

MemoryProfiler.stop.pretty_print

Running on 3.2.2 causes a segfault. When you don't call Ractor#take to execute it, there is no segfault.

I thought you'd want to know!

Error on Ruby 2.2.1 & Rails 4.2

Running the profiler using:


require 'memory_profiler'

report = MemoryProfiler.report do
  require_relative '../config/environment'
  # Add specific code here if you want to profile deeper
end

report.pretty_print


/Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/top_n.rb:14:in `sort_by': comparison of Array with Array failed (ArgumentError)
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/top_n.rb:14:in `sort_by!'
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/top_n.rb:14:in `top_n'
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/results.rb:26:in `block in register_results'
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/results.rb:24:in `each'
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/results.rb:24:in `register_results'
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/reporter.rb:63:in `run'
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler/reporter.rb:27:in `report'
    from /Users/rishav/.rvm/gems/ruby-2.2.1@edcast/gems/memory_profiler-0.9.6/lib/memory_profiler.rb:14:in `report'

Feature request: ability to profile commands

Currently, CLI is able to profile only single ruby files via memory-profiler ruby_file.rb (we pass a file name and the gem loads it and profiles the memory).
It will be very convenient to be able to profile whole commands/processes, like rspec, rails etc via something like
memory-profiler run ...gem options... -- rails test ...command options....

The interface will like the one added for stackprof (tmm1/stackprof#187).

If this is OK, I can open a PR.

MemoryProfiler falsely report retained string

MemoryProfiler.report{ 200.times{|i| "SOME RANDOM TEXT#{i}: SOME Value".dup } }.pretty_print

The above code result into

Total allocated: 41600 bytes (800 objects)
Total retained:  17600 bytes (200 objects)

I've tried running same code without memory_profiler and in infite loop and it doesn't increase memory.

The result changes for short string:

MemoryProfiler.report{ 200.times{|i| "TEXT#{i}: Value".dup } }.pretty_print

Total allocated: 24000 bytes (600 objects)
Total retained:  0 bytes (0 objects)

retained memory (maybe) reports released objects

I am debugging an application for high memory usage and came across to something weird 😕 I am not 100% sure if it is a bug but our expectations and tests point to that direction.

report = MemoryProfiler.report(top: 10) do
  # some method
end
report.pretty_print

reports

Total allocated: 491959315 bytes (337088 objects)
Total retained:  150970508 bytes (7106 objects)

allocated memory by gem
-----------------------------------
 147231872  active_hash-ed1faced2e4a
  84880339  activesupport-5.2.3
  84054872  rest-client-2.0.2
  68179352  json-2.2.0
  59272721  optimizer-api/lib     
  26251503  socket
  21429171  net
    303143  uri
    280025  other
     58593  http-cookie-1.0.3
...
retained memory by gem
-----------------------------------
  71562722  rest-client-2.0.2       (* we can't explain this since the variables are long gone)
  59166800  optimizer-api/lib
  20165388  net                     (* neither this)
     42321  uri
     15085  activesupport-5.2.3
      9893  other
      7179  http-cookie-1.0.3
       424  redis-3.3.5
       312  active_hash-ed1faced2e4a
       264  multi_json-1.13.1

while

HeapProfiler.report('./') do
  # the same method
end

reports (which agrees with what we expected)

Total allocated: 552.43 MB (347935 objects)
Total retained: 59.29 MB (4713 objects)

allocated memory by gem
-----------------------------------
 147.23 MB  active_hash-ed1faced2e4a
 132.07 MB  activesupport-5.2.3
  96.41 MB  rest-client-2.0.2
  68.18 MB  json-2.2.0
  59.27 MB  optimizer-api/lib
  27.46 MB  socket
  20.96 MB  net
 330.77 kB  other
...
retained memory by gem
-----------------------------------
  59.17 MB  optimizer-api/lib
  41.19 kB  rest-client-2.0.2
  31.74 kB  http-cookie-1.0.3
  26.26 kB  activesupport-5.2.3
  19.17 kB  other

plus if we call GC inside the MemoryProfiler report block -- of course the allocated only reports the retained objects as expected but also -- the unexpected entries of the retained list disappears.

report = MemoryProfiler.report(top: 10) do
  # the same method
  GC.enable
  4.times { GC.start }
end
report.pretty_print

reports

Total allocated: 59323368 bytes (6062 objects) 
Total retained:  59226928 bytes (4093 objects)

allocated memory by gem
-----------------------------------
  59166800  optimizer-api/lib
    102138  other
     34785  rest-client-2.0.2
     12664  activesupport-5.2.3
      6669  http-cookie-1.0.3
       312  active_hash-ed1faced2e4a
...
retained memory by gem
-----------------------------------
  59166800  optimizer-api/lib
     34313  rest-client-2.0.2
     12664  activesupport-5.2.3
      6669  http-cookie-1.0.3
      6170  other
       312  active_hash-ed1faced2e4a

I can't understand how calling the GC inside the block can change the retained memory since MemoryProfiler.stop method calls GC as well. I check the beginning of the MemoryProfiler.stop method with no luck to see if there is something that would lead to objects being retained ¯_(ツ)_/¯

Would appreciate your input about the observations above and if this is a memory-profiler bug or not.

Happy holiday season and a happy new year!

MemoryProfiler.stop or .end causes 100% CPU usage

When trying to use MemoryProfiler in the rails console I get 100% CPU usage and the process RSS use explodes into the GB's of data. It keeps growing and eventually sinks my entire machine...

If I interrupt the profiler the 100% CPU goes away but the memory usage does not decline (even GC'ing does nothing).

Example:

2.4.4 :021 > MemoryProfiler.start
 => nil 
2.4.4 :022 > 100.times { t= "hello!" }
 => 100 
2.4.4 :024 > r = MemoryProfiler.stop

Using "MemoryProfiler.report do; ... end" makes no difference.

I use Rails 5.1.6.1, ruby 2.4.4, memory_profiler 0.9.12 on a Linux Mint 18.3 system.

What can I do to track down the cause of this behavior?

Best regards, Thomas.

pretty_print results in NO DATA

I'm investigating memory usage in a Ruby on Rails 6 app. I wrote the following code around an initializer:

    if !ENV["MEMPROFILE"].blank?
      require 'memory_profiler'
      MemoryProfiler.start
    end

    # My initializer
    
    if !ENV["MEMPROFILE"].blank?
      puts "MEMPROFILE"
      report = MemoryProfiler.stop
      report.pretty_print
    end

Normally, the process uses about 1GB and with the profiler, it uses about 6GB, so it's clearly doing something, but the results printed show NO DATA. The # My initializer section above is definitely executing and doing a lot (I can see the print statements). Any thoughts?

MEMPROFILE
Total allocated: 0 bytes (0 objects)
Total retained:  0 bytes (0 objects)

allocated memory by gem
-----------------------------------
NO DATA

allocated memory by file
-----------------------------------
NO DATA

allocated memory by location
-----------------------------------
NO DATA

allocated memory by class
-----------------------------------
NO DATA

allocated objects by gem
-----------------------------------
NO DATA

allocated objects by file
-----------------------------------
NO DATA

allocated objects by location
-----------------------------------
NO DATA

allocated objects by class
-----------------------------------
NO DATA

retained memory by gem
-----------------------------------
NO DATA

retained memory by file
-----------------------------------
NO DATA

retained memory by location
-----------------------------------
NO DATA

retained memory by class
-----------------------------------
NO DATA

retained objects by gem
-----------------------------------
NO DATA

retained objects by file
NO DATA

retained objects by location
-----------------------------------
NO DATA

retained objects by class
-----------------------------------
NO DATA

Ruby version:

$ ruby -v
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]

Running on Linux Fedora 37 x64

Release of new version with scale_bytes ?

The Readme mentions a scale_bytes option. However, it is not available in the latest stable 0.9.12 version. Got me confused as I though the version was not working.

Is there any ETA for the release of a new version with this option ?

Hash reported as retained

MemoryProfiler will report that an object of type Hash is being retained by the following code:

require "memory_profiler"

def run
  {{:a => 1} => 2}
end

MemoryProfiler.report { run }.pretty_print

Output:

Total allocated: 696 bytes (3 objects)
Total retained:  232 bytes (1 objects)

allocated memory by gem
-----------------------------------
       696  other

allocated memory by file
-----------------------------------
       696  report.rb

allocated memory by location
-----------------------------------
       464  report.rb:5
       232  report.rb:4

allocated memory by class
-----------------------------------
       696  Hash

allocated objects by gem
-----------------------------------
         3  other

allocated objects by file
-----------------------------------
         3  report.rb

allocated objects by location
-----------------------------------
         2  report.rb:5
         1  report.rb:4

allocated objects by class
-----------------------------------
         3  Hash

retained memory by gem
-----------------------------------
       232  other

retained memory by file
-----------------------------------
       232  report.rb

retained memory by location
-----------------------------------
       232  report.rb:5

retained memory by class
-----------------------------------
       232  Hash

retained objects by gem
-----------------------------------
         1  other

retained objects by file
-----------------------------------
         1  report.rb

retained objects by location
-----------------------------------
         1  report.rb:5

retained objects by class
-----------------------------------
         1  Hash

I've actually observed similar behaviour when using an array or a custom class as hash key.

What does it mean the Hash is being retained in such cases? What do you think is preventing the object being unallocated?

method `hash' called on hidden T_ARRAY object

Well, I've never seen something like this before. Don't even know if this would be considered a bug or not?

ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-darwin14]
memory_profiler revision 8a421fb

method `hash' called on hidden T_ARRAY object (0x007fd6d0879d40 flags=0x20827)
/Users/ssilver/.rvm/gems/ruby-2.2.2/bundler/gems/memory_profiler-8a421fb8bb2c/lib/memory_profiler/results.rb:79:in `each'
/Users/ssilver/.rvm/gems/ruby-2.2.2/bundler/gems/memory_profiler-8a421fb8bb2c/lib/memory_profiler/results.rb:79:in `group_by'
/Users/ssilver/.rvm/gems/ruby-2.2.2/bundler/gems/memory_profiler-8a421fb8bb2c/lib/memory_profiler/results.rb:79:in `string_report'
/Users/ssilver/.rvm/gems/ruby-2.2.2/bundler/gems/memory_profiler-8a421fb8bb2c/lib/memory_profiler/results.rb:63:in `register_results'
/Users/ssilver/.rvm/gems/ruby-2.2.2/bundler/gems/memory_profiler-8a421fb8bb2c/lib/memory_profiler/reporter.rb:64:in `run'
/Users/ssilver/.rvm/gems/ruby-2.2.2/bundler/gems/memory_profiler-8a421fb8bb2c/lib/memory_profiler/reporter.rb:22:in `report'
/Users/ssilver/.rvm/gems/ruby-2.2.2/bundler/gems/memory_profiler-8a421fb8bb2c/lib/memory_profiler.rb:14:in `report'

Comparison of Array with Array failed

Using this gem via the derailed tool (bundle exec derailed bundle:objects), I receive the following output:

bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/top_n.rb:12:in `sort_by': comparison of Array with Array failed (ArgumentError)
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/top_n.rb:12:in `sort_by!'
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/top_n.rb:12:in `top_n' 
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/results.rb:26:in `block in register_results'
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/results.rb:24:in `each'
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/results.rb:24:in `register_results'
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/reporter.rb:63:in `run'
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler/reporter.rb:27:in `report'
bundler/gems/memory_profiler-c8255596d985/lib/memory_profiler.rb:14:in `report'
gems/derailed_benchmarks-1.3.1/bin/derailed:51:in `block in <class:DerailedBenchmarkCLI>'

Unfortunately I have no idea what that might mean. Do you think it's a problem of memory_profiler or rather with derailed? Anything I can do to help debug?

[BUG] Segmentation fault at 0x0000000109e63000 ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]

When I run memory_profiler I get an error when it's generating the report:

/Users/xxxxx/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/memory_profiler-0.9.14/lib/memory_profiler/helpers.rb:40: [BUG] Segmentation fault at 0x0000000109e63000
ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]

Memory Profiler causes extreme memory usage during profiling.

Using MemoryProfiler.start on one of our jobs, and then waiting to dump MemoryProfile.stop&.pretty_print took a job that uses 2gb of memory to 76gb of memory. Is there a way we can prevent this kind of memory usage? Even running memory profiler for a few short seconds causes it to jump to exponential levels.

release cli?

Time for a 0.9.15?

Memory allocations reported on wrong line for blocks in Ruby 2.6.x

The following test case reports the allocation of method arguments on different lines for Ruby MRI 2.5/2.7 vs. Ruby MRI 2.6:

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'memory_profiler', '0.9.14'
end

puts "Ruby: #{RUBY_VERSION}"

def do_something
  yield
end

def do_something_else(options)
  yield
end

report = MemoryProfiler.report do
  100_000.times do
    do_something do
      do_something_else({foo: 1, bar: 2, baz: 3}) do
        # no-op
      end
    end
  end
end

report.pretty_print(retained_strings: 0, allocated_strings: 0)

Running this in Ruby 2.5 and 2.7 reports 100,000 allocations on the do_something_else({foo: 1, bar: 2, baz: 3}) line as expected. Ruby 2.6 incorrectly reports the allocations on the do_something line.

undefined method `name' in 0.9.5

When trying to profile some stuff, I get:

NoMethodError: undefined method `name' for nil:NilClass
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler/helpers.rb:28:in `lookup_class_name'
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler/reporter.rb:100:in `block in object_list'
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler/reporter.rb:92:in `each'
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler/reporter.rb:92:in `object_list'
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler/reporter.rb:40:in `run'
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler/reporter.rb:27:in `report'
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler.rb:14:in `report'
/Users/vwoo/coderpad.io/lib/tasks/stripe.rake:7:in `block in <top (required)>'
/Users/vwoo/.gem/ruby/2.2.3/gems/bugsnag-2.8.12/lib/bugsnag/rake.rb:12:in `execute_with_bugsnag'
Tasks: TOP => stripe
(See full trace by running task with --trace)

probably related to the new performance improvements. Checking for nil should probably do the trick.

https://github.com/SamSaffron/memory_profiler/blob/master/lib/memory_profiler/helpers.rb#L28

What is the proper way to use memory_profiler with a rake task?

This seems to work but not sure if it creates incorrect results:

require 'memory_profiler'

report = MemoryProfiler.report do
  `rake theTask`
end

report.pretty_print

Question about MemoryProfiler results

Hi,

I'm not having an actual issue with this gem, is just that i have a couple of questions to help me understand better the results that the gem gives me.

I'm using the gem to profile the memory in some parts of my code base, after profiling i get some results like

Total allocated: 6980019 bytes (75878 objects)
Total retained:  184943 bytes (943 objects)

My first question is: being 943 the amount of total retained objects this actually means that i have a memory leak? Or it means something else? I'm not sure i completely understand this results.
I get some results like

retained memory by location
-----------------------------------
     22288  /Users/msouto/Desktop/Test-App/app/models/person.rb:1

The line 1 of person.rb is the declaration of the class, specifically

class Person < ActiveRecord::Base
end

how do i intepret this? The class Person is leaking? Should i profile more specific methods of this class?

Thanks for your help

Feature request: CLI for quick and dirty memory profiling

Would you be open for such a cli?

Something like
$ memory-profiler script.rb ..options..

instead of code modifications and writing required wrapping code.

Avoid the color white in pretty_print

As some people are using a white console, the white ANSI color is probably best left unused.
Suggest replacing it with one of the other colors.

Profiling live rails apps

Hi! Im just curious if there is a way to report memory usage on a running rails app (with puma).

I'm thinking of deploying one app instance with memory profiler and route some small amount of arbitrary client calls on it (like 1-5%) to see where the memory goes.

I'm trying to do something like this in my config.ru:

require ::File.expand_path('../config/environment', __FILE__)
require 'memory_profiler'

report = MemoryProfiler.report do
  run Rails.application
end

REPDIR = "#{Rails.root}/tmp/memreport".freeze

Dir.mkdir(REPDIR) unless File.exists?(REPDIR)

report_th = Thread.new do
  loop do
    sleep(6.hours.seconds)
    report.pretty_print(scale_bytes:true,to_file:"#{REPDIR}/#{Time.now.to_i}.txt")
  end
end

Signal.trap('INT') do # just for development
  report_th.exit
end

But all I've got is some poor info about rack and newrelic middleware and that's all.

Is there a way to do it properly?

private method `puts' called in 0.9.5

NoMethodError: private method `puts' called for {:to_file=>"1444691948.log"}:Hash
/Users/vwoo/.gem/ruby/2.2.3/gems/memory_profiler-0.9.5/lib/memory_profiler/results.rb:72:in `pretty_print'
/Users/vwoo/coderpad.io/lib/tasks/stripe.rake:22:in `block in <top (required)>'
/Users/vwoo/.gem/ruby/2.2.3/gems/bugsnag-2.8.12/lib/bugsnag/rake.rb:12:in `execute_with_bugsnag'
Tasks: TOP => stripe
(See full trace by running task with --trace)

https://github.com/SamSaffron/memory_profiler/blob/master/lib/memory_profiler/results.rb#L66

recent change from

def pretty_print(io = STDOUT, **options)
  io = File.open(options[:to_file], "w") if options[:to_file]

def pretty_print(io = STDOUT, options = {})
  io = File.open(options[:to_file], "w") if options[:to_file]

**args behaves different, in the case where no io is passed.

Showing line number of the end of the calling function rather than the location of the allocation

I found an issue where in memory profiler returns the location of an allocation on a different line.

Here's the script. You would expect that memory profiler would tell me the allocation happens on line 2 but instead it gets reported as line 3:

def my_concat(a, b)
  return a + b # <====================
end

require 'fileutils'
FileUtils.mkdir_p("tmp")

require 'memory_profiler'

def run
  report = MemoryProfiler.report do
    10.times do
      my_concat("hello".freeze, "world".freeze)
    end

    10.times do
      my_concat("hello".freeze, "world".freeze)
    end
  end

  report.pretty_print
end

run

Output

allocated memory by location
-----------------------------------
       800  script.rb:3

If you add space to the method body it keeps incrementing the line number of the output. For example, changing the method body to this, should not change the allocation location:

def my_concat(a, b)
  return a + b # <====================





end

However, it is reported as a different line:

allocated memory by location
-----------------------------------
       800  script.rb:8

Is this expected? Is this an issue with memory profiler or the underlying ObjectSpace allocation tracing?

How to intrepret the results and get the memory size

Excuse my ignorance here, but as I can tell, the numbers here represent the objects created. I understand this is helpful specially when tracing memory leaks by checking the increased number of objects. However, is there a way where I can convert this number to real memory size consumed by the profiled code?
Right now I'm doing this to get it in Mbs, but for the whole running process:

# (number of heaps * size of a heap * size of an object in bytes) / bytes per mb
(GC.stat[:heap_length] * GC::INTERNAL_CONSTANTS[:HEAP_OBJ_LIMIT] * GC::INTERNAL_CONSTANTS[:RVALUE_SIZE]) / 1024 ** 2

THANK YOU

this is just a quick thank you note to say how awesome this tool is an how much it saved me. I optimized a bunch of string allocations in our code and saved a huge amount of memory using data from your gem!

Large memory usage in Timeout::timeout

I'm profiling my application with memory_profiler to try and reduce our memory usage. We don't have a leak, but the usage is still a bit higher than I'd like. One thing that stands out to me in the memory_profiler dump is this:

Total allocated: 2.12 GB (6186720 objects)
Total retained:  37.94 MB (19885 objects)

allocated memory by gem
-----------------------------------
   1.25 GB  timeout
...
allocated memory by file
-----------------------------------
   1.25 GB  /home/lfm/.rbenv/versions/2.3.3/lib/ruby/2.3.0/timeout.rb
...
allocated memory by location
-----------------------------------
   1.25 GB  /home/lfm/.rbenv/versions/2.3.3/lib/ruby/2.3.0/timeout.rb:81

That line corresponds to the Timeout::timeout method, excerpt:

 73   def timeout(sec, klass = nil)   #:yield: +sec+
     74     return yield(sec) if sec == nil or sec.zero?
     75     message = "execution expired".freeze
     76     from = "from #{caller_locations(1, 1)[0]}" if $DEBUG
     77     e = Error
     78     bl = proc do |exception|
     79       begin
     80         x = Thread.current
     81         y = Thread.start {
     82           Thread.current.name = from
     83           begin
     84             sleep sec
     85           rescue => e
     86             x.raise e
     87           else
     88             x.raise exception, message
     89           end
     90         }

This says one of two things to me -- either Timeout::timeout is really expensive and/or we're calling it way too much, or this is a limitation of the memory_profiler gem / red herring.

Can you confirm one way or the other?
I found this issue while Googling -- stompgem/stomp#121 -- which seems to point to this being a real thing. But I'm not entirely convinced by it

Problems if GC runs during profiling? RangeError (0x003fdc710144c8 is recycled object)

I sometimes get the following error while generating the report for a very large action I am profiling.

RangeError (0x003fdc710144c8 is recycled object)

It triggers on this line after the block has finished:

report = MemoryProfiler.report do

Full gem backtrace:

~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/results.rb:71:in `_id2ref'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/results.rb:71:in `block in string_report'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/results.rb:71:in `each'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/results.rb:71:in `map'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/results.rb:71:in `string_report'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/results.rb:58:in `register_results'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/reporter.rb:45:in `run'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler/reporter.rb:7:in `report'
~/.gem/ruby/2.2.0/gems/memory_profiler-0.0.4/lib/memory_profiler.rb:12:in `report'

I am running Ruby 2.2.0. Whether it errors or not appears to be random, but it may have to do with GC since it appears to be an access to something that doesn't exist.

Thanks for the gem. It's great (albeit slow) when it works!

Problems with tests on ruby 3.2

Hello,

we are currently updating our Arch Linux ruby setup to version 3.2.4 and we ran into issues when testing memory_profiler.

First of all the tests currently need to run with MT_COMPAT= otherwise Minitest is reporting multiple issues. This is only a temporary solution as Minitest announced already to remove this compatibility layer in a future major release.

Secondly when running the tests (with rake test) we get the following errors:

Run options: --seed 18359

# Running:

....F.............FF......................F.............FF....................F.........F....F.F..

Finished in 1.072546s, 91.3714 runs/s, 245.2110 assertions/s.

  1) Failure:
TestReporter#test_counts [test/test_reporter.rb:80]:
Expected: 16
  Actual: 15

  2) Failure:
TestReporter#test_ignore_file_with_regex [test/test_reporter.rb:99]:
Expected: 3
  Actual: 2

  3) Failure:
TestReporter#test_ignore_file_with_string [test/test_reporter.rb:105]:
Expected: 3
  Actual: 2

  4) Failure:
TestReporterPrivateStartStop#test_counts [test/test_reporter.rb:80]:
Expected: 16
  Actual: 15

  5) Failure:
TestReporterPrivateStartStop#test_ignore_file_with_regex [test/test_reporter.rb:99]:
Expected: 3
  Actual: 2

  6) Failure:
TestReporterPrivateStartStop#test_ignore_file_with_string [test/test_reporter.rb:105]:
Expected: 3
  Actual: 2

  7) Failure:
TestReporterPublicStartStop#test_counts [test/test_reporter.rb:80]:
Expected: 16
  Actual: 15

  8) Failure:
TestReporterPublicStartStop#test_module_double_start [test/test_reporter_public_start_stop.rb:40]:
Expected: 17
  Actual: 16

  9) Failure:
TestReporterPublicStartStop#test_ignore_file_with_regex [test/test_reporter.rb:99]:
Expected: 3
  Actual: 2

 10) Failure:
TestReporterPublicStartStop#test_ignore_file_with_string [test/test_reporter.rb:105]:
Expected: 3
  Actual: 2

98 runs, 263 assertions, 10 failures, 0 errors, 0 skips
rake aborted!
Command failed with status (1)

Tasks: TOP => test
(See full trace by running task with --trace)

I would appreciate any help you can provide.

Best regards
Segaja

undefined method `name' in 0.9.6 using Sequel

Related to #27 I guess - though I have the object this time:
/Users/me/.rvm/gems/ruby-2.3.1/gems/memory_profiler-0.9.6/lib/memory_profiler/helpers.rb:28:in 'lookup_class_name': undefined method 'name' for #<Sequel::SQL::Identifier @value=>:class> (NoMethodError)

This appears when I am using derailed_benchmarks to list the objects created at require time. My app is a Padrino app using Sequel ORM.

It seems to be caused by Sequel::SQL::VirtualRow objects. These subclass BasicObject & return another object subclassing Object (Sequel::SQL::Identifier) when :class is called on them.

So the question: is memory_profiler expected to deal with objects subclassing BasicObject & if so how (Jeremy Evans, the maintainer of Sequel asked me this question)? If not, would you accept a PR to fix the issue - I'd propose rather than calling :class on an unknown object using this clever trick instead - which works on objects sub-classed from BasicObject.

Many thanks

Bruce

request for a bit of documentation

does this still only work on head or are these features in 2.1?

also, could you link to a guide somewhere for how to use, or show a minimal example?

thanks!

Showing only application specific results

Currently, the reports are mostly populated by gems that are used in the code. Having an option to suppress them would be awesome. Then one can focus on how his/her code is doing.

Memory profile report missing substantial amounts of the memory allocated

I will update with more information after the weekend.

memory_profiler reporting itself

I noticed today (I also updated my Gems today , so it might be related).
That memory_profiler now reports it's own footprint as a part of the analysis report.
I believe this wasn't the case before and it creates a false top level analysis metrics.

See the example here:
https://gist.github.com/ohaddahan/cdaef533abf963f16f7bccec141c2185

Most of the memory report is actually caused by memory_profiler.


Total allocated: 28693134 bytes (206218 objects)
Total retained:  28671354 bytes (205866 objects)

allocated memory by gem
-----------------------------------
  28664857  memory_profiler-0.9.8
     26437  chili-hunter-2/app
      1840  activesupport-5.0.0

Option to only report retained would be useful

Allocated is usually not all that important to optimize, and the output is really verbose, so option to cut it in half would be much appreciated (especially since there's no simple grep can get the more relevant part).

Test failures on ruby 3.0.4

When running rake test on version 1.0.0 I get this output:

Run options: --seed 15293

# Running:

..F.F........................F...F...........F.F...................F.F.....................F......

Finished in 1.031771s, 94.9823 runs/s, 240.3635 assertions/s.

  1) Failure:
TestReporterPublicStartStop#test_symbols_report [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_reporter.rb:247]:
Expected: 3
  Actual: 2

  2) Failure:
TestReporterPublicStartStop#test_no_strings_retained_report [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_reporter.rb:235]:
45 strings should be allocated.
Expected: 45
  Actual: 55

  3) Failure:
TestCLI#test_normalize_paths [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_cli.rb:84]:
Expected /ruby\/lib\/set.rb/ to match "Total allocated: 13690 bytes (78 objects)\nTotal retained:  0 bytes (0 objects)\n\nallocated memory by gem\n-----------------------------------\n      9263  memory_profiler-1.0.0/lib\n      2144  usr/lib\n      1920  longhorn-0.1.0\n       363  other\n\nallocated memory by file\n-----------------------------------\n      9263  memory_profiler-1.0.0/lib/memory_profiler/cli.rb\n      2144  usr/lib/ruby/3.0.0/set.rb\n      1920  longhorn-0.1.0/lib/longhorn.rb\n       363  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb\n\nallocated memory by location\n-----------------------------------\n      9263  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n      1216  usr/lib/ruby/3.0.0/set.rb:355\n       928  usr/lib/ruby/3.0.0/set.rb:89\n       888  longhorn-0.1.0/lib/longhorn.rb:10\n       640  longhorn-0.1.0/lib/longhorn.rb:12\n       363  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n       312  longhorn-0.1.0/lib/longhorn.rb:9\n        40  longhorn-0.1.0/lib/longhorn.rb:7\n        40  longhorn-0.1.0/lib/longhorn.rb:8\n\nallocated memory by class\n-----------------------------------\n      8432  File\n      3050  String\n      1240  Array\n       928  Hash\n        40  Set\n\nallocated objects by gem\n-----------------------------------\n        44  longhorn-0.1.0\n        17  usr/lib\n        13  memory_profiler-1.0.0/lib\n         4  other\n\nallocated objects by file\n-----------------------------------\n        44  longhorn-0.1.0/lib/longhorn.rb\n        17  usr/lib/ruby/3.0.0/set.rb\n        13  memory_profiler-1.0.0/lib/memory_profiler/cli.rb\n         4  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb\n\nallocated objects by location\n-----------------------------------\n        19  longhorn-0.1.0/lib/longhorn.rb:10\n        16  longhorn-0.1.0/lib/longhorn.rb:12\n        16  usr/lib/ruby/3.0.0/set.rb:355\n        13  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n         7  longhorn-0.1.0/lib/longhorn.rb:9\n         4  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n         1  longhorn-0.1.0/lib/longhorn.rb:7\n         1  longhorn-0.1.0/lib/longhorn.rb:8\n         1  usr/lib/ruby/3.0.0/set.rb:89\n\nallocated objects by class\n-----------------------------------\n        48  String\n        27  Array\n         1  File\n         1  Hash\n         1  Set\n\nretained memory by gem\n-----------------------------------\nNO DATA\n\nretained memory by file\n-----------------------------------\nNO DATA\n\nretained memory by location\n-----------------------------------\nNO DATA\n\nretained memory by class\n-----------------------------------\nNO DATA\n\nretained objects by gem\n-----------------------------------\nNO DATA\n\nretained objects by file\n-----------------------------------\nNO DATA\n\nretained objects by location\n-----------------------------------\nNO DATA\n\nretained objects by class\n-----------------------------------\nNO DATA\n\n\nAllocated String Report\n-----------------------------------\n         3  \"/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb\"\n         3  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         2  \"/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures\"\n         2  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n\n         2  \"/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/gems/longhorn-0.1.0/lib/longhorn\"\n         2  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n\n         2  \"allocated memory by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"gems/longhorn-0.1.0/lib/longhorn\"\n         2  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         2  \"retained memory by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         1  \"\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"# frozen_string_literal: true\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"-\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"Longhorn.run\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"frozen_string_literal\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"require_relative \\\"gems/longhorn-0.1.0/lib/longhorn\\\"\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"true\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n".

  4) Failure:
TestCLI#test_pretty [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_cli.rb:94]:
Expected /ruby\/lib\/set.rb/ to match "Total allocated: 13.69 kB (78 objects)\nTotal retained:  0 B (0 objects)\n\nallocated memory by gem\n-----------------------------------\n   9.26 kB  memory_profiler-1.0.0/lib\n   2.14 kB  usr/lib\n   1.92 kB  longhorn-0.1.0\n  363.00 B  other\n\nallocated memory by file\n-----------------------------------\n   9.26 kB  memory_profiler-1.0.0/lib/memory_profiler/cli.rb\n   2.14 kB  usr/lib/ruby/3.0.0/set.rb\n   1.92 kB  longhorn-0.1.0/lib/longhorn.rb\n  363.00 B  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb\n\nallocated memory by location\n-----------------------------------\n   9.26 kB  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n   1.22 kB  usr/lib/ruby/3.0.0/set.rb:355\n  928.00 B  usr/lib/ruby/3.0.0/set.rb:89\n  888.00 B  longhorn-0.1.0/lib/longhorn.rb:10\n  640.00 B  longhorn-0.1.0/lib/longhorn.rb:12\n  363.00 B  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n  312.00 B  longhorn-0.1.0/lib/longhorn.rb:9\n   40.00 B  longhorn-0.1.0/lib/longhorn.rb:7\n   40.00 B  longhorn-0.1.0/lib/longhorn.rb:8\n\nallocated memory by class\n-----------------------------------\n   8.43 kB  File\n   3.05 kB  String\n   1.24 kB  Array\n  928.00 B  Hash\n   40.00 B  Set\n\nallocated objects by gem\n-----------------------------------\n        44  longhorn-0.1.0\n        17  usr/lib\n        13  memory_profiler-1.0.0/lib\n         4  other\n\nallocated objects by file\n-----------------------------------\n        44  longhorn-0.1.0/lib/longhorn.rb\n        17  usr/lib/ruby/3.0.0/set.rb\n        13  memory_profiler-1.0.0/lib/memory_profiler/cli.rb\n         4  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb\n\nallocated objects by location\n-----------------------------------\n        19  longhorn-0.1.0/lib/longhorn.rb:10\n        16  longhorn-0.1.0/lib/longhorn.rb:12\n        16  usr/lib/ruby/3.0.0/set.rb:355\n        13  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n         7  longhorn-0.1.0/lib/longhorn.rb:9\n         4  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n         1  longhorn-0.1.0/lib/longhorn.rb:7\n         1  longhorn-0.1.0/lib/longhorn.rb:8\n         1  usr/lib/ruby/3.0.0/set.rb:89\n\nallocated objects by class\n-----------------------------------\n        48  String\n        27  Array\n         1  File\n         1  Hash\n         1  Set\n\nretained memory by gem\n-----------------------------------\nNO DATA\n\nretained memory by file\n-----------------------------------\nNO DATA\n\nretained memory by location\n-----------------------------------\nNO DATA\n\nretained memory by class\n-----------------------------------\nNO DATA\n\nretained objects by gem\n-----------------------------------\nNO DATA\n\nretained objects by file\n-----------------------------------\nNO DATA\n\nretained objects by location\n-----------------------------------\nNO DATA\n\nretained objects by class\n-----------------------------------\nNO DATA\n\n\nAllocated String Report\n-----------------------------------\n         3  \"/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb\"\n         3  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         2  \"/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures\"\n         2  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n\n         2  \"/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/gems/longhorn-0.1.0/lib/longhorn\"\n         2  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/script.rb:3\n\n         2  \"allocated memory by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"gems/longhorn-0.1.0/lib/longhorn\"\n         2  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         2  \"retained memory by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         1  \"\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"# frozen_string_literal: true\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"-\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"Longhorn.run\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"frozen_string_literal\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"require_relative \\\"gems/longhorn-0.1.0/lib/longhorn\\\"\\n\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n         1  \"true\"\n         1  memory_profiler-1.0.0/lib/memory_profiler/cli.rb:47\n\n".

  5) Failure:
TestReporter#test_no_strings_retained_report [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_reporter.rb:235]:
45 strings should be allocated.
Expected: 45
  Actual: 55

  6) Failure:
TestReporter#test_symbols_report [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_reporter.rb:247]:
Expected: 3
  Actual: 2

  7) Failure:
TestReporterPrivateStartStop#test_no_strings_retained_report [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_reporter.rb:235]:
45 strings should be allocated.
Expected: 45
  Actual: 55

  8) Failure:
TestReporterPrivateStartStop#test_symbols_report [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_reporter.rb:247]:
Expected: 3
  Actual: 2

  9) Failure:
TestResults#test_normalize_paths_true [/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_results.rb:83]:
Expected /ruby\/lib\/set.rb/ to match "Total allocated: 4309 bytes (64 objects)\nTotal retained:  0 bytes (0 objects)\n\nallocated memory by gem\n-----------------------------------\n      2144  usr/lib\n      1920  longhorn-0.1.0\n       245  other\n\nallocated memory by file\n-----------------------------------\n      2144  usr/lib/ruby/3.0.0/set.rb\n      1920  longhorn-0.1.0/lib/longhorn.rb\n       245  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_helper.rb\n\nallocated memory by location\n-----------------------------------\n      1216  usr/lib/ruby/3.0.0/set.rb:355\n       928  usr/lib/ruby/3.0.0/set.rb:89\n       888  longhorn-0.1.0/lib/longhorn.rb:10\n       640  longhorn-0.1.0/lib/longhorn.rb:12\n       312  longhorn-0.1.0/lib/longhorn.rb:9\n       245  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_helper.rb:11\n        40  longhorn-0.1.0/lib/longhorn.rb:7\n        40  longhorn-0.1.0/lib/longhorn.rb:8\n\nallocated memory by class\n-----------------------------------\n      2029  String\n      1312  Array\n       928  Hash\n        40  Set\n\nallocated objects by gem\n-----------------------------------\n        44  longhorn-0.1.0\n        17  usr/lib\n         3  other\n\nallocated objects by file\n-----------------------------------\n        44  longhorn-0.1.0/lib/longhorn.rb\n        17  usr/lib/ruby/3.0.0/set.rb\n         3  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_helper.rb\n\nallocated objects by location\n-----------------------------------\n        19  longhorn-0.1.0/lib/longhorn.rb:10\n        16  longhorn-0.1.0/lib/longhorn.rb:12\n        16  usr/lib/ruby/3.0.0/set.rb:355\n         7  longhorn-0.1.0/lib/longhorn.rb:9\n         3  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_helper.rb:11\n         1  longhorn-0.1.0/lib/longhorn.rb:7\n         1  longhorn-0.1.0/lib/longhorn.rb:8\n         1  usr/lib/ruby/3.0.0/set.rb:89\n\nallocated objects by class\n-----------------------------------\n        34  String\n        28  Array\n         1  Hash\n         1  Set\n\nretained memory by gem\n-----------------------------------\nNO DATA\n\nretained memory by file\n-----------------------------------\nNO DATA\n\nretained memory by location\n-----------------------------------\nNO DATA\n\nretained memory by class\n-----------------------------------\nNO DATA\n\nretained objects by gem\n-----------------------------------\nNO DATA\n\nretained objects by file\n-----------------------------------\nNO DATA\n\nretained objects by location\n-----------------------------------\nNO DATA\n\nretained objects by class\n-----------------------------------\nNO DATA\n\n\nAllocated String Report\n-----------------------------------\n         2  \"allocated memory by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated memory by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"allocated objects by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained memory by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by class\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by file\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by gem\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         2  \"retained objects by location\"\n         1  longhorn-0.1.0/lib/longhorn.rb:12\n         1  usr/lib/ruby/3.0.0/set.rb:355\n\n         1  \"/build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/fixtures/gems/longhorn-0.1.0/lib\"\n         1  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_helper.rb:11\n\n         1  \"longhorn-0.1.0\"\n         1  /build/ruby-memory_profiler/src/memory_profiler-1.0.0/test/test_helper.rb:11\n\n".

98 runs, 248 assertions, 9 failures, 0 errors, 0 skips
rake aborted!
Command failed with status (1)

Tasks: TOP => test
(See full trace by running task with --trace)

I'm not sure what I'm doing wrong.

Include breakdown of shared strings in objects allocated

I've been using memory_profiler to profile a few optimizations around string allocations and usage of byteslice vs string mutation, and although the number of strings allocated has increased, I suspect that the majority is shared. Unfortunately, there's no way to tell just based on the breakdown charts.

Are the current breakdowns already taking that into account, and if not, would it be an interesting addition?

[BUG] segfault at call to memsize_of for 0.9.14 over 2.4.10

/Users/sitano/.asdf/installs/ruby/2.4.10/lib/ruby/gems/2.4.0/gems/memory_profiler-0.9.14/lib/memory_profiler/reporter.rb:117: [BUG] Segmentation fault at 0x00000000000000e0
ruby 2.4.10p364 (2020-03-31 revision 67879) [x86_64-darwin20]

-- Crash Report log information --------------------------------------------
   See Crash Report log file under the one of following:
     * ~/Library/Logs/DiagnosticReports
     * /Library/Logs/DiagnosticReports
   for more details.
Don't forget to include the above Crash Report log file in bug reports.

-- Control frame information -----------------------------------------------
c:0194 p:---- s:1392 e:001391 CFUNC  :memsize_of
c:0193 p:0240 s:1387 e:001386 BLOCK  /Users/sitano/.asdf/installs/ruby/2.4.10/lib/ruby/gems/2.4.0/gems/memory_profiler-0.9.14/lib/memory_profiler/reporter.rb:117 [FINISH]
c:0192 p:---- s:1375 e:001374 CFUNC  :each_object
c:0191 p:0064 s:1371 e:001370 METHOD /Users/sitano/.asdf/installs/ruby/2.4.10/lib/ruby/gems/2.4.0/gems/memory_profiler-0.9.14/lib/memory_profiler/reporter.rb:95
c:0190 p:0025 s:1363 e:001362 METHOD /Users/sitano/.asdf/installs/ruby/2.4.10/lib/ruby/gems/2.4.0/gems/memory_profiler-0.9.14/lib/memory_profiler/reporter.rb:48
...

   def object_list(generation)
   ...
      ObjectSpace.each_object do |obj|
        ...

        begin
          ...
          memsize = ObjectSpace.memsize_of(obj) # <---------
          string = klass == String ? helper.lookup_string(obj) : nil

crashing obj:

grpc-1.1.2-universal-darwin - GRPC::Core::MetadataArray - /Users/sitano/.asdf/installs/ruby/2.4.10/lib/ruby/gems/2.4.0/gems/grpc-1.1.2-universal-darwin/src/ruby/lib/grpc/generic/active_call.rb:337 - 337

It does not crash if I skip the objects from GRPC module.
I have hundreds of other modules.

Two tests are failing with Ruby 3

Hi,

there are two tests of the 0.9.14 release failing when run with Ruby 3.0.

test/test_results.rb: The report for Ruby 3 does not contain the last string as expected here:

  def test_normalize_paths_true
    report = normalized_paths_report
    io = StringIO.new
    report.pretty_print(io, normalize_paths: true)
    assert_match(%r!\d+\s{2}longhorn-0.1.0/lib/longhorn.rb:\d+!, io.string)
    assert_match(%r!ruby/lib/set.rb!, io.string)
  end

I think this test (without adjustments) should only been run via:

--- a/test/test_results.rb
+++ b/test/test_results.rb
@@ -80,6 +80,6 @@ class TestResults < Minitest::Test
     io = StringIO.new
     report.pretty_print(io, normalize_paths: true)
     assert_match(%r!\d+\s{2}longhorn-0.1.0/lib/longhorn.rb:\d+!, io.string)
-    assert_match(%r!ruby/lib/set.rb!, io.string)
+    assert_match(%r!ruby/lib/set.rb!, io.string) if RUBY_VERSION < '3'
   end
 end

test/test_reporter.rb: There is a difference in the number of total allocated strings for test_no_strings_retained_report. I added some debugging output to see why. This basically happens for each of the test strings:

[.. Ruby 2.7 ...]
         2  "LONG TEXT #0 12345678901234567890123456789012345678901234567890"
         1  /build/ruby-memory-profiler-5G9Cml/ruby-memory-profiler-0.9.14/test/test_reporter.rb:40
         1  /build/ruby-memory-profiler-5G9Cml/ruby-memory-profiler-0.9.14/test/test_reporter.rb:41
[.. Ruby 3 ...]
         3  "LONG TEXT #0 12345678901234567890123456789012345678901234567890"
         2  /build/ruby-memory-profiler-5G9Cml/ruby-memory-profiler-0.9.14/test/test_reporter.rb:41
         1  /build/ruby-memory-profiler-5G9Cml/ruby-memory-profiler-0.9.14/test/test_reporter.rb:40
[..]

If this is the correct result for Ruby 3 then I suggest changing the assertion to be:

--- a/test/test_reporter.rb
+++ b/test/test_reporter.rb
@@ -232,7 +232,7 @@ class TestReporter < Minitest::Test
       end
     end
 
-    assert_equal(45, results.total_allocated, "45 strings should be allocated")
+    assert_includes([45, 55], results.total_allocated, "45 or 55 strings should be allocated")
     assert_equal(20, results.strings_allocated.size, "20 unique strings should be observed")
     assert_equal(0, results.strings_retained.size, "0 unique strings should be retained")
   end

Option to normalize filepaths

First off, this gem rules! Is there any way for 'normalizing' a path to the file? I'm hoping to find a way to make these reports more shareable between developers with an end goal of maybe even checking these memory reports into version control to check for performance regressions. With the report containing machine dependent filepaths like /home/sam/.rbenv/versions/2.1.0-github/ or /Users/will/.rvm/gems/ruby-2.5.1/gems/ would generate a lot of noise in diffs.

For example,

/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/gems/2.1.0/gems/mime-types-2.0/lib/mime/type.rb x 3391763
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/json/common.rb x 2021853
/home/sam/.rbenv/versions/2.1.0-github/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb x 285433

Could be normalized to something like the following (Warning: not entirely sure what part from the filepaths above are environment dependent vs not):

mime-types-2.0/lib/mime/type.rb x 3391763
json/common.rb x 2021853
core_ext/kernel_require.rb x 285433

Frozen Strings / Line report

This is run on Ruby 2.2 and 2.3 (same result) using memory_profiler 0.9.4.

I have the following case – see questions at the end.

require 'memory_profiler'

GC.start

MemoryProfiler.report do
  h1 = { "key": "value" } # line 6
  h2 = { "key": "value" } # line 7

  key = "key" # line 9
  h3 = {}
  h3[key] = "value" # line 11
  key = nil

  GC.start
end.pretty_print

The "Allocated part ":

Allocated String Report
-----------------------------------
         3  "value"
         1  performance/memory/experiments.rb:11
         1  performance/memory/experiments.rb:7
         1  performance/memory/experiments.rb:6

         2  "key"
         1  performance/memory/experiments.rb:11
         1  performance/memory/experiments.rb:9

We have three "value" Strings corresponding to the values of the hashes – as expected.
The key part is interesting - Ruby may immediately freeze the Strings here on lines 6 and 7.

Retained String Report
-----------------------------------
         1  "key"
         1  performance/memory/experiments.rb:11

With the retained String report, I would have expected a single frozen "key" String to show up, originating from line 6, and three "value" Strings, from lines 6, 7, and 11.

Frozen Strings do not show up – should they?
Why would the three "value" Strings not be reported?
And why is the String reported to originate from line 11?

Thanks for any help! :)

Unclear usage options

Could you please tell me (and add to the README.md if possible) whether it's safe to simultaneously have multiple Reporter instances and collect data using them?

In case it's allowed by design, is the code threadsafe? If I were to run it concurrently with puma - will it break?

Test failures on arm64

Hi there,

Thanks for your work on this gem so far.
However, while building this against Ruby2.7 on arm64, I got the following errors:

Finished in 2.134689s, 39.3500 runs/s, 103.9964 assertions/s.

  1) Failure:
TestReporter#test_symbols_report [/tmp/autopkgtest-lxc.stcsf78a/downtmp/build.vf5/src/test/test_reporter.rb:248]:
Expected: 0
  Actual: 1

  2) Failure:
TestReporterPrivateStartStop#test_symbols_report [/tmp/autopkgtest-lxc.stcsf78a/downtmp/build.vf5/src/test/test_reporter.rb:248]:
Expected: 0
  Actual: 1

84 runs, 222 assertions, 2 failures, 0 errors, 0 skips
rake aborted!
Command failed with status (1): [ruby -w  "/usr/lib/ruby/vendor_ruby/rake/rake_test_loader.rb" "test/test_helper.rb" "test/test_helpers.rb" "test/test_reporter.rb" "test/test_reporter_private_start_stop.rb" "test/test_reporter_public_start_stop.rb" "test/test_results.rb" "test/test_stat_hash.rb" "test/test_top_n.rb" -v]

Could you help us figure this out, please? :)

Integration with rack-mini-profiler

Can this profile memory of requests and theirs parts (SQL queries, etc) and output the result in the rack-mini-profiler HTML?

Top profiled memory by gem is "other"

What does "other" mean? We're trying to track down some high memory usage in a ruby cucumber test project.

Also, seeing memory by file say "internal:prelude" for our top memory by file. Any clues as to what these mean would be super helpful. This isn't necessarily an issue, but more just questions about what these mean.

Thanks!

ObjectSpace.dump_all diff based profiling

👋 @SamSaffron, it's super early so I don't really have much code to show, but I've been playing with an idea since this morning.

Context

In MRI there was many objects that are hidden to ObjectSpace.each_object, that include many internal objects such as IMEMO, instruction sequences, etc, as well as regular objects hidden with rb_obj_hide because they are literal constants etc.

On our app, I'm running memory_profiler around the entire boot, and it reports a total of ~600MB. However in parallel (another CI job) I also call ObjectSpace.dump_all for analysis with harb, and a simple:

$ jq 'if .memsize == null then 0 else .memsize end' dumps/shopify-production-boot-heap-2020-07-09.dump | awk '{ sum += $1 } END { print sum }'
885314785 # 844MB

So in our case, a good third of the ruby heap is invisible to memory profiler.

Idea

So I think memory profiler could do the following:

GC.start && GC.disable
dump_the_heap '/tmp/pre-heap.json'
run_the_snippet
dump_the_heap '/tmp/post-run-heap.json'
GC.enable && GC.start
dump_the_heap '/tmp/post-gc-heap.json'

allocated = "post-run-heap" - "pre-heap"
retained =  "post-gc-heap" - "pre-heap"

One big challenge though is that we'd need to parse the heap files several times, and it's not that fast. The jq above takes 52s on my machine to parse that 1.6GB file. However using simdjson, I'm able to get the same result in 13s:

require 'fast_jsonparser'
memsize = 0
FastJsonparser.load_many(ARGV.first, batch_size: 33_554_432) do |object|
  memsize += object[:memsize].to_i
end
p memsize

The same thing using stdlib's json is 42s. So it would be possible to use fast_jsonparser if available, and otherwise fallback to the stdlib.

Required data

The heap dump contains:

memsize, same value as ObjectSpace.memsize_of.
string values (unless they're very big), so the String Report could be implemented.
the class, even though it's an address pointer, so we'd need to build a map of class names.
file and line for the allocation source.

So unless I'm missing something, there's all the data needed.

It might also even help with #78

so?

So I wonder if you'd be interested in such change. It would almost be rewrite, so I'd rather check with you first if you are open to it of if I should instead create a new gem. Let me know.

report print to file

It would be great if there was a way to print the results directly to a file; csv would be ideal for starters. This would make it easier to compare results over time to show what improvements have been made.

I could give a go at implementing this if you are open to the idea?

buffer problem

hi guys
i have a problem on youtube-dl.rb gem when i want download a big video ( 1/5gb) from youtube my ram is crashed ( i have 4 gb ram on server) because after download video i see ram graph and up to 2gb after download again ram up to 3 and again 4 and as a result server crashed

now i use memory_profiler gem this awesome but after download video ram up to 1.100 mb and again 1.200 mb ....... and again up to 4 and again crashed
also after complete download ram buffer not start to flush buffer