dimitrisandreou / memory-measurer Goto Github PK

It seems that ObjectGraphMeasurer does not take into account the memory used by 
an array.  It handles visiting the elements in the array, but the array itself 
is only incrementing the object count by 1.  This causes severe underestimation 
of memory used since the best we can do with Footprint is to multiply object 
count by 16 (overhead in bytes for an object) plus a little randomness for 
padding.

Java collections use a lot of arrays internally, so even if you don't use 
arrays directly in your code, there can still be lots of arrays in use 
indirectly.  These arrays can take up a lot a memory based on length * size of 
objects in array (anywhere from 1, 2, 4 or 8). 

Perhaps Footprint could return arrays the same way it returns primitives which 
would allow caller to more accurately estimate size by doing the multiplication 
after graph is visited.

I was able to use it as a javaagent if I launch the application from 
eclipse/CMD by providing "-javaagent:path/to/object-explorer.jar", but I am not 
able to use it in my Jboss app server. I am using Jboss 7.2. I am getting error 
as : "Instrumentation is not set properly.....". I added the path to the agent 
as JAVA_OPTS, but of no avail. Please tell me where to add the path.

What steps will reproduce the problem?
1. Running a psvm w/ -javaagent:object-explorer.jar
2. OR Running a tomcat server with the same setting.

What is the expected output? What do you see instead?
Instrumentation should work.
Instead the following error is received when using memory-measurer: 
"No instrumentation available (has the VM started with the appropriate agent?)".
All the while, JAMM (https://github.com/jbellis/jamm) works.

What version of the product are you using? On what operating system?
Tried various JDKs: 1.6 - 1.8.

Please provide any additional information below.

Hi

I am doing the following

long totalMemory = Runtime.getRuntime().totalMemory();
MyObject = createMyObject(args);
memoryGuessed = Runtime.getRuntime().totalMemory() - totalMemory;

Footprint footprint = ObjectGraphMeasurer.measure(appSession);
log.info("footprint.toString()
      + " memory:" + MemoryMeasurer.measureBytes(appSession) 
      + " memoryGuessed:" + memoryGuessed
      );

And the result is 
footprint:Footprint{Objects=70992, References=194939, Primitives=[int x 63043, 
long x 452, char x 437906, byte x 78594, boolean x 12100, float x 2331]} 
memory:2821976 memoryGuessed:327680

I am using:
Ubuntu 12.04.2 LTS, Precise Pangolin
Linux ubuntu 3.2.0-43-generic-pae #68-Ubuntu SMP Wed May 15 03:55:10 UTC 2013 
i686 i686 i386 GNU/Linux

Why is memory x8 greater than memoryGuessed? Could it be that memory is bits 
and not bytes? Am I missing something? Thank you for your help!

scala> case class Person(name: String, surname: String)
defined class Person

scala> objectexplorer.MemoryMeasurer.measureBytes(Person("", ""))
res1: Long = 64

scala> objectexplorer.MemoryMeasurer.measureBytes(Person("h", ""))
res2: Long = 112

scala> objectexplorer.MemoryMeasurer.measureBytes(Person("h", "h"))
res3: Long = 72

It would be nice to have this project deployed to maven central repository. I 
think it will be not that hard to distill a pom.xml for it.

I will try it later and attach the patch.

java.lang.NoClassDefFoundError: com/google/common/base/Preconditions
    at objectexplorer.InstrumentationGrabber.checkSetup (InstrumentationGrabber.java:20)
    at objectexplorer.InstrumentationGrabber.instrumentation (InstrumentationGrabber.java:25)
    at objectexplorer.MemoryMeasurer.<clinit> (MemoryMeasurer.java:24)
    at com.example.App.main (App.java:19)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:834)
Caused by: java.lang.ClassNotFoundException: com.google.common.base.Preconditions
    at jdk.internal.loader.BuiltinClassLoader.loadClass (BuiltinClassLoader.java:581)
    at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass (ClassLoaders.java:178)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:521)
    at objectexplorer.InstrumentationGrabber.checkSetup (InstrumentationGrabber.java:20)
    at objectexplorer.InstrumentationGrabber.instrumentation (InstrumentationGrabber.java:25)
    at objectexplorer.MemoryMeasurer.<clinit> (MemoryMeasurer.java:24)
    at com.example.App.main (App.java:19)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:834)

export MAVEN_OPTS=-javaagent:memory-measurer-1.0-SNAPSHOT.jar
mvn -e exec:java -Dexec.mainClass=com.example.App

---------------------------------------------------
constituent[0]: file:/Library/Downloads/apache-maven-3.8.5/conf/logging/
constituent[1]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-resolver-util-1.6.3.jar
constituent[2]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-builder-support-3.8.5.jar
constituent[3]: file:/Library/Downloads/apache-maven-3.8.5/lib/plexus-component-annotations-2.1.0.jar
constituent[4]: file:/Library/Downloads/apache-maven-3.8.5/lib/plexus-sec-dispatcher-2.0.jar
constituent[5]: file:/Library/Downloads/apache-maven-3.8.5/lib/slf4j-api-1.7.32.jar
constituent[6]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-core-3.8.5.jar
constituent[7]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-resolver-spi-1.6.3.jar
constituent[8]: file:/Library/Downloads/apache-maven-3.8.5/lib/plexus-utils-3.3.0.jar
constituent[9]: file:/Library/Downloads/apache-maven-3.8.5/lib/plexus-interpolation-1.26.jar
constituent[10]: file:/Library/Downloads/apache-maven-3.8.5/lib/plexus-cipher-2.0.jar
constituent[11]: file:/Library/Downloads/apache-maven-3.8.5/lib/jcl-over-slf4j-1.7.32.jar
constituent[12]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-model-builder-3.8.5.jar
constituent[13]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-embedder-3.8.5.jar
constituent[14]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-settings-3.8.5.jar
constituent[15]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-model-3.8.5.jar
constituent[16]: file:/Library/Downloads/apache-maven-3.8.5/lib/javax.annotation-api-1.2.jar
constituent[17]: file:/Library/Downloads/apache-maven-3.8.5/lib/commons-cli-1.4.jar
constituent[18]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-resolver-transport-wagon-1.6.3.jar
constituent[19]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-artifact-3.8.5.jar
#!/bin/bash
constituent[20]: file:/Library/Downloads/apache-maven-3.8.5/lib/wagon-file-3.5.1.jar
constituent[21]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-resolver-provider-3.8.5.jar
constituent[22]: file:/Library/Downloads/apache-maven-3.8.5/lib/org.eclipse.sisu.inject-0.3.5.jar
constituent[23]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-slf4j-provider-3.8.5.jar
constituent[24]: file:/Library/Downloads/apache-maven-3.8.5/lib/commons-io-2.6.jar
constituent[25]: file:/Library/Downloads/apache-maven-3.8.5/lib/wagon-http-3.5.1-shaded.jar
constituent[26]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-resolver-connector-basic-1.6.3.jar
constituent[27]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-resolver-api-1.6.3.jar
constituent[28]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-repository-metadata-3.8.5.jar
constituent[29]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-shared-utils-3.3.4.jar
constituent[30]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-plugin-api-3.8.5.jar
constituent[31]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-compat-3.8.5.jar
constituent[32]: file:/Library/Downloads/apache-maven-3.8.5/lib/org.eclipse.sisu.plexus-0.3.5.jar
constituent[33]: file:/Library/Downloads/apache-maven-3.8.5/lib/wagon-provider-api-3.5.1.jar
constituent[34]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-settings-builder-3.8.5.jar
constituent[35]: file:/Library/Downloads/apache-maven-3.8.5/lib/guice-4.2.2-no_aop.jar
constituent[36]: file:/Library/Downloads/apache-maven-3.8.5/lib/maven-resolver-impl-1.6.3.jar
constituent[37]: file:/Library/Downloads/apache-maven-3.8.5/lib/javax.inject-1.jar
constituent[38]: file:/Library/Downloads/apache-maven-3.8.5/lib/commons-lang3-3.8.1.jar
constituent[39]: file:/Library/Downloads/apache-maven-3.8.5/lib/jansi-2.4.0.jar
constituent[40]: file:/Library/Downloads/apache-maven-3.8.5/lib/guava-25.1-android.jar
---------------------------------------------------
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
	at com.google.inject.Key.ensureRetainedAtRuntime(Key.java:341)
	at com.google.inject.Key.strategyFor(Key.java:335)
	at com.google.inject.Key.get(Key.java:219)
	at org.eclipse.sisu.wire.ParameterKeys.<clinit>(ParameterKeys.java:28)
	at org.eclipse.sisu.wire.DependencyAnalyzer.<init>(DependencyAnalyzer.java:93)
	at org.eclipse.sisu.wire.ElementAnalyzer.<init>(ElementAnalyzer.java:104)
	at org.eclipse.sisu.wire.WireModule.configure(WireModule.java:74)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:344)
	at com.google.inject.spi.Elements.getElements(Elements.java:103)
	at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:137)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
	at com.google.inject.Guice.createInjector(Guice.java:87)
	at com.google.inject.Guice.createInjector(Guice.java:69)
	at com.google.inject.Guice.createInjector(Guice.java:59)
	at org.codehaus.plexus.DefaultPlexusContainer.addPlexusInjector(DefaultPlexusContainer.java:481)
	at org.codehaus.plexus.DefaultPlexusContainer.<init>(DefaultPlexusContainer.java:206)
	at org.apache.maven.cli.MavenCli.container(MavenCli.java:651)
#!/bin/bash
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)

First of all: thanks for creating this project. It looks great, and I can't 
wait to play a with it.

I was very interested by the 
http://code.google.com/p/memory-measurer/wiki/ElementCostInDataStructures wiki 
page. It lets the visitor immediately get interesting insights into data 
structures, without having to run the measurements himself.

It would be nice if IdentityHashMap were represented on this page: it is 
supposedly more memory-efficient than HashMap, and I was interested in that. Of 
course, I'll run the test myself when I go home, but I think this would be a 
nice addition.

I would also be interested in IdentityHashSet (from Guava r08's 
Sets.newIdentityHashSet()), but it's really a wrapped IdentityHashmap anyway.

Thanks again