adoptium / bumblebench Goto Github PK

A microbenchmarking test framework for Eclipse Adoptium

License: Apache License 2.0

Java 96.83% Shell 1.97% C++ 1.20%

bumblebench's Introduction

bumblebench

This microbenchmarking test framework for Eclipse Adoptium, along with jmh will be used to create and run different types of benchmarks on binaries produced at Adoptium (to verify that large perf regressions are caught). While we may have limited access to isolated performance machines, many microbenchmarks can still be useful to catch regressions early. You can find a Quick Start Guide and other useful documentation at the BumbleBench and HumbleBench wiki page.

BumbleBench is a microbenchmark tool intended to make it as easy as possible to avoid common pitfalls when microbenchmarking Java. It is intended to make sure that test runs spend most of their time running the desired piece of code compiled at the highest possible level of quality. This is surprisingly tricky in an environment with dynamic compilation employing aggressive speculative optimizations.

The name "BumbleBench" derives from the manner in which the tool varies the iteration count of the benchmark's main loop in order to determine the highest iteration count that can be completed within a given target duration. The target score vacillates around the estimated maximum achievable score, alternating between low and high target scores in an attempt to converge on the actual achievable score, while remaining sensitive to variations in performance that can occur due to effects like jit compilation occurring during the run.

bumblebench's People

Contributors

Stargazers

Watchers

bumblebench's Issues

DigestBench and HMACBench taking too long to run

Problem Description

DigestBench and HMACBench seem to take forever to run. Benchmarks in BumbleBench usually take less than 2 mins. I had to abort DigestBench run after it kept running for 1 hr 45 min, generating an extremely large output (> 3 GB) as shown in the screenshots below. HMACBench behaves in a similar way.

I'm trying to add these BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

Console Output

13:55:45  ===============================================
13:55:45  Running test bumbleBench-DigestBench_0 ...
13:55:45  ===============================================
13:55:45  bumbleBench-DigestBench_0 Start Time: Fri Nov  1 13:55:43 2019 Epoch Time (ms): 1572630943888
13:55:45  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
13:55:45  JVMSHRC005I No shared class caches available
13:55:45  JVMSHRC005I No shared class caches available
13:55:45  cache cleanup done
13:55:45  variation: NoOptions
13:55:45  JVM_OPTIONS: -Xcompressedrefs 
13:55:45  { itercnt=1; \
13:55:45  mkdir -p "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726308899881/bumbleBench-DigestBench_0"; \
13:55:45  cd "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726308899881/bumbleBench-DigestBench_0"; \
13:55:45  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar DigestBench; \
13:55:45  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-DigestBench_0""_PASSED"; echo ""; cd /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/..;  else echo ""; echo "bumbleBench-DigestBench_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726308899881/TestTargetResult";
13:55:45  Using Provider SUN
13:55:45  Payload size: 4096 bytes
13:55:45  
13:55:45  -= BumbleBench series 7 version 5.10 running DigestBench  Fri Nov 01 13:55:44 EDT 2019 =-
13:55:45  
13:55:45                Target	Est	Uncert%	MaxPeak	Peak	Peak%	%paused
13:55:45      0.0s:  <  110	100	 12.0	-inf	-inf	--
13:55:45      0.0s:  <! 94	88	 14.4	-inf	-inf	--
13:55:45      0.0s:  <! 81.66	75.33	 17.3	-inf	-inf	--
13:55:45      0.0s:  <! 68.82	62.31	 20.7	-inf	-inf	--
13:55:45      0.0s:  <! 55.85	49.39	 24.9	-inf	-inf	--
13:55:45      0.0s:  <! 43.25	37.10	 29.9	-inf	-inf	--
13:55:45      0.0s:  <! 31.56	26.02	 35.8	-inf	-inf	--
13:55:45      0.0s:  <! 21.36	16.70	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 13.36	10.02	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 8.015	6.011	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 4.809	3.607	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 2.885	2.164	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 1.731	1.298	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 1.039	0.7791	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.6233	0.4674	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.3740	0.2805	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.2244	0.1683	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.1346	0.1010	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.08077	0.06058	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.04846	0.03635	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.02908	0.02181	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.01745	0.01309	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.01047	0.007851	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.006281	0.004711	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.003769	0.002826	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.002261	0.001696	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.001357	0.001018	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0008140	0.0006105	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0004884	0.0003663	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0002930	0.0002198	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0001758	0.0001319	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0001055	0.00007912	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00006330	0.00004747	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00003798	0.00002848	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00002279	0.00001709	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00001367	0.00001025	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000008203	0.000006153	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000004922	0.000003692	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000002953	0.000002215	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000001772	0.000001329	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000001063	797.4n	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 637.9n	478.4n	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 382.7n	287.1n	 40.0	-inf	-inf	--
...
...

To Reproduce

Add (or enable if it already exists) the target(s) in the BumbleBench playlist (https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml)
Use the Adopt Grinder: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/build?delay=0sec
Set the TARGET to the relevant benchmark

NullPointerException while Running DispatchBench.InnerClasses with JDK 11 Hotspot

Problem Description

Running bumbleBench-DispatchBench-InnerClasses with JDK 11 Hotspot gives NullPointerException as shown below. This benchmark works with JDK 8 Hotpot and JDK 11 and 8 OpenJ9.

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

Error

18:12:19  ===============================================
18:12:19  Running test bumbleBench-DispatchBench-InnerClasses_0 ...
18:12:19  ===============================================
18:12:19  bumbleBench-DispatchBench-InnerClasses_0 Start Time: Fri Oct 11 22:12:19 2019 Epoch Time (ms): 1570831939782
18:12:19  Nothing to be done for setup.
18:12:19  variation: NoOptions
18:12:19  JVM_OPTIONS:  
18:12:19  { itercnt=1; \
18:12:19  mkdir -p "/home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_157083193966/bumbleBench-DispatchBench-InnerClasses_0"; \
18:12:19  cd "/home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_157083193966/bumbleBench-DispatchBench-InnerClasses_0"; \
18:12:19  "/home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar DispatchBench.InnerClasses; \
18:12:19  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-DispatchBench-InnerClasses_0""_PASSED"; echo ""; cd /home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../..;  else echo ""; echo "bumbleBench-DispatchBench-InnerClasses_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_157083193966/TestTargetResult";
18:12:20  Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:187)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfPossiblyFreshlyLoadedClass(Util.java:223)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench$InnerClasses.getCallee(DispatchBench.java:126)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench.computeCallees(DispatchBench.java:39)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench.<init>(DispatchBench.java:34)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench$InnerClasses.<init>(DispatchBench.java:124)
18:12:20  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
18:12:20  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
18:12:20  	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
18:12:20  	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
18:12:20  	at java.base/java.lang.Class.newInstance(Class.java:584)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
18:12:20  Caused by: java.lang.NullPointerException
18:12:20  	at java.base/java.lang.reflect.Method.invoke(Method.java:559)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:196)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:184)
18:12:20  	... 11 more
18:12:20  
18:12:20  bumbleBench-DispatchBench-InnerClasses_0_FAILED
18:12:20  
18:12:20  Nothing to be done for teardown.
18:12:20  bumbleBench-DispatchBench-InnerClasses_0 Finish Time: Fri Oct 11 22:12:20 2019 Epoch Time (ms): 1570831940345
18:12:20  
18:12:20  ===============================================

To Reproduce

Remove the implementation constraint for this test: bumbleBench-DispatchBench-InnerClasses (https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml)
Use the bumbleBench-DispatchBench-InnerClasses target on TestKitGen with the relevant SDK

NullPointerException while Running Humble Benchmarks with JDK 11 Hotspot

Problem Description

Running Humble benchmarks with JDK 11 Hotspot gives NullPointerException as shown below. This benchmark works with JDK 8 Hotpot and JDK 11 and 8 OpenJ9.

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

Error

15:46:53  Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:185)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfPossiblyFreshlyLoadedClass(Util.java:221)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfFreshlyLoadedClass(Util.java:208)
15:46:53  	at net.adoptopenjdk.bumblebench.core.HumbleBench$DelayThread.<init>(HumbleBench.java:134)
15:46:53  	at net.adoptopenjdk.bumblebench.core.HumbleBench.<init>(HumbleBench.java:57)
15:46:53  	at net.adoptopenjdk.bumblebench.humble.DistinctStringsEqualsBench.<init>(DistinctStringsEqualsBench.java:35)
15:46:53  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
15:46:53  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
15:46:53  	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
15:46:53  	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
15:46:53  	at java.base/java.lang.Class.newInstance(Class.java:584)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
15:46:53  Caused by: java.lang.NullPointerException
15:46:53  	at java.base/java.lang.reflect.Method.invoke(Method.java:559)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:194)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:182)
15:46:53  	... 11 more
15:46:53  
15:46:53  bumbleBench-DistinctStringsEqualsBench_0_FAILED

To Reproduce

Remove the implementation constraint for the tests such as DistinctStringsEqualsBench (https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml)
Use the bumbleBench-DistinctStringsEqualsBench target on TestKitGen with the relevant SDK

NoSuchMethodException: sun.misc.IOUtils.readAllBytes on java 8

Works with jdk 10 and jdk 14, but not java 8

java -jar BumbleBench.jar StringEqualsBench
Exception in thread "main" java.lang.ExceptionInInitializerError
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: sun.misc.IOUtils.readAllBytes(java.io.InputStream)
	at net.adoptopenjdk.bumblebench.core.Util.<clinit>(Util.java:44)
Caused by: java.lang.NoSuchMethodException: sun.misc.IOUtils.readAllBytes(java.io.InputStream)
	at java.lang.Class.getDeclaredMethod(Class.java:2130)
	at net.adoptopenjdk.bumblebench.core.Util.<clinit>(Util.java:36)

java version

openjdk version "1.8.0_222"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode)

InstantiationException while Running KmeansBench GPULambda

Problem Description

Running KmeansBench GPULambda gives java.lang.InstantiationException as shown below. I've saw this issue while using Java 8 OpenJ9 and Java 11 HotSpot but it should happen on all JDKs.

I'm adding this test: bumbleBench-KmeansBench-GPULambda to Adopt Test repo in this playlist for BumbleBench: https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml.

Relevant Console Output with Error

18:07:39  ===============================================
18:07:39  Running test bumbleBench-KmeansBench-GPULambda_0 ...
18:07:39  ===============================================
18:07:39  bumbleBench-KmeansBench-GPULambda_0 Start Time: Mon Oct 21 22:07:39 2019 Epoch Time (ms): 1571695659641
18:07:39  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
18:07:39  JVMSHRC005I No shared class caches available
18:07:39  JVMSHRC005I No shared class caches available
18:07:39  cache cleanup done
18:07:39  variation: NoOptions
18:07:39  JVM_OPTIONS: -Xcompressedrefs 
18:07:39  { itercnt=1; \
18:07:39  mkdir -p "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15716952597279/bumbleBench-KmeansBench-GPULambda_0"; \
18:07:39  cd "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15716952597279/bumbleBench-KmeansBench-GPULambda_0"; \
18:07:39  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar KmeansBench.GPULambda; \
18:07:39  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-KmeansBench-GPULambda_0""_PASSED"; echo ""; cd /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../..;  else echo ""; echo "bumbleBench-KmeansBench-GPULambda_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15716952597279/TestTargetResult";
18:07:40  Exception in thread "main" java.lang.InstantiationException: net.adoptopenjdk.bumblebench.gpu.KmeansBench$GPULambda
18:07:40  	at java.lang.J9VMInternals.newInstanceImpl(Native Method)
18:07:40  	at java.lang.Class.newInstance(Class.java:1848)
18:07:40  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
18:07:40  
18:07:40  bumbleBench-KmeansBench-GPULambda_0_FAILED
18:07:40  
18:07:40  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
18:07:40  
18:07:40  Attempting to destroy all caches in cacheDir /home/jbench/javasharedresources/
18:07:40  
18:07:40  JVMSHRC806I Compressed references persistent shared cache "sharedcc_jbench" has been destroyed. Use option -Xnocompressedrefs if you want to destroy a non-compressed references cache.
18:07:40  JVMSHRC005I No shared class caches available
18:07:40  cache cleanup done
18:07:40  bumbleBench-KmeansBench-GPULambda_0 Finish Time: Mon Oct 21 22:07:39 2019 Epoch Time (ms): 1571695659956
18:07:40  
18:07:40  ===============================================

To Reproduce

Use the bumbleBench-KmeansBench-GPULambda target on TestKitGen

Missing package statement in net.adoptopenjdk.bumblebench.string.StringTestData

The net.adoptopenjdk.bumblebench.string.StringTestData class is missing a package statement:

package net.adoptopenjdk.bumblebench.string;

Which results in build errors when building the string test classes e.g:

[javac] symbol: variable StringTestData [javac] location: class StringConversionBench [javac] /bumblebench/net/adoptopenjdk/bumblebench/string/StringConversionBench.java:73: error: package StringTestData does not exist [javac] sb.append(StringTestData.POSSIBLE_CHARS[rand.nextInt(StringTestData.POSSIBLE_CHARS.length)]); [javac] ^

sun.misc.IOUtils.readFully throws exception due to negative length argument

When trying to run a HumbleBench using the latest adoptopenjdk openjdk8 nightly builds, I got the following exception:

Exception in thread "main" java.lang.AssertionError: java.lang.reflect.InvocationTargetException
	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:201)
	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:184)
	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfPossiblyFreshlyLoadedClass(Util.java:223)
	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfFreshlyLoadedClass(Util.java:210)
	at net.adoptopenjdk.bumblebench.core.HumbleBench$DelayThread.<init>(HumbleBench.java:134)
	at net.adoptopenjdk.bumblebench.core.HumbleBench.<init>(HumbleBench.java:57)
	at CxfMessageHumbleBench.<init>(CxfMessageHumbleBench.java:7)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:194)
	... 12 more
Caused by: java.io.IOException: length cannot be negative: -1
	at sun.misc.IOUtils.readFully(IOUtils.java:305)
	... 17 more

I think the problem is here: https://github.com/AdoptOpenJDK/bumblebench/blob/master/net/adoptopenjdk/bumblebench/core/Util.java#L194

On that line, method readFully is called with a length of -1.

I also found this PR in openj9 where a similar issue was dealt with: eclipse-openj9/openj9#8312

It looks like the fix would either be to pass Integer.MAX_VALUE to readFully or use the new IOUtils.readAllBytes method.

Credit to the original author?

Might I humbly suggest it would be appropriate to give some kind of credit/recognition to the author of Bumblebench (namely me) somewhere in the code or docs?

Update build.xml file to include newer bumblebench additions to jar file

arraycopy (https://github.com/AdoptOpenJDK/bumblebench/tree/master/net/adoptopenjdk/bumblebench/arraycopy) needs to be added to the build.xml file for inclusion into the BumbleBench.jar file

SSLSocketBench gives java.io.FileNotFoundException

Problem Description

Running SSLSocketBench gives java.io.FileNotFoundException as shown below. I believe Vlad worked on this benchmark since he has his username in the file path.

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

Error

13:47:11  java.io.FileNotFoundException: /home/vpapro/mySrvKeystore (No such file or directory)

Console Output

13:47:08  ===============================================
13:47:08  Running test bumbleBench-SSLSocketBench_0 ...
13:47:08  ===============================================
13:47:08  bumbleBench-SSLSocketBench_0 Start Time: Fri Nov  1 13:47:08 2019 Epoch Time (ms): 1572630428469
13:47:08  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
13:47:08  JVMSHRC005I No shared class caches available
13:47:08  JVMSHRC005I No shared class caches available
13:47:08  cache cleanup done
13:47:08  variation: NoOptions
13:47:08  JVM_OPTIONS: -Xcompressedrefs 
13:47:08  { itercnt=1; \
13:47:08  mkdir -p "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726303782557/bumbleBench-SSLSocketBench_0"; \
13:47:08  cd "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726303782557/bumbleBench-SSLSocketBench_0"; \
13:47:08  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar SSLSocketBench; \
13:47:08  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-SSLSocketBench_0""_PASSED"; echo ""; cd /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/..;  else echo ""; echo "bumbleBench-SSLSocketBench_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726303782557/TestTargetResult";
13:47:11  java.io.FileNotFoundException: /home/vpapro/mySrvKeystore (No such file or directory)
13:47:11  	at java.io.FileInputStream.open(FileInputStream.java:195)
13:47:11  	at java.io.FileInputStream.<init>(FileInputStream.java:138)
13:47:11  	at java.io.FileInputStream.<init>(FileInputStream.java:93)
13:47:11  	at net.adoptopenjdk.bumblebench.crypto.SSLSocketBench.<clinit>(SSLSocketBench.java:51)
13:47:11  	at java.lang.Class.forNameImpl(Native Method)
13:47:11  	at java.lang.Class.forName(Class.java:333)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.loadTestClass(Launcher.java:104)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.loadTestClass(Launcher.java:85)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:48)
13:47:11  
13:47:11  -= BumbleBench series 7 version 5.10 running SSLSocketBench  Fri Nov 01 13:47:08 EDT 2019 =-
13:47:11  
13:47:11                Target	Est	Uncert%	MaxPeak	Peak	Peak%	%paused
13:47:11  java.net.ConnectException: Connection refused (Connection refused)
13:47:11  	at java.net.PlainSocketImpl.socketConnect(Native Method)
13:47:11  	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
13:47:11  	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
13:47:11  	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
13:47:11  	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
13:47:11  	at java.net.Socket.connect(Socket.java:607)
13:47:11  	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666)
13:47:11  	at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
13:47:11  	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
13:47:11  	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
13:47:11  	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
13:47:11  	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
13:47:11  	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
13:47:11  	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
13:47:11  	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
13:47:11  	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
13:47:11  	at net.adoptopenjdk.bumblebench.crypto.SSLSocketBench.doBatch(SSLSocketBench.java:88)
13:47:11  	at net.adoptopenjdk.bumblebench.core.MicroBench.attempt(MicroBench.java:66)
13:47:11  	at net.adoptopenjdk.bumblebench.core.BumbleBench.runAttempt(BumbleBench.java:139)
13:47:11  	at net.adoptopenjdk.bumblebench.core.BumbleBench.run(BumbleBench.java:245)
13:47:11  	at net.adoptopenjdk.bumblebench.core.MicroBench.run(MicroBench.java:107)
13:47:11  	at net.adoptopenjdk.bumblebench.core.BumbleBench.bumbleMain(BumbleBench.java:287)
13:47:11  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
13:47:11  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
13:47:11  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
13:47:11  	at java.lang.reflect.Method.invoke(Method.java:498)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.runBumbleMainOn(Launcher.java:61)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
13:47:11      0.2s:  >! 110	695.5	 40.0	110	110	470.0
...
...

To Reproduce

Add (or enable if it already exists) the target in the BumbleBench playlist (https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml)
Use the Adopt Grinder: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/build?delay=0sec
Set the TARGET to the relevant benchmark

BumbleBench doesn't build with JDK 12

Problem Description

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379. BumbleBench compiles and runs successfully with JDK 8 and JDK 11 for both OpenJ9 and HotSpot. It doesn't compile with JDK 12, and hence the targets are just limited to 8 and 11 in the BumbleBench playlist.xml file.

I believe we should see a similar issue with JDK 13 as well even though I haven't tested with it.

Error

16:42:31  compile-all:
16:42:31      [javac] Compiling 8 source files to /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/perf/bumbleBench/bumblebench
16:42:31      [javac] warning: [options] bootstrap class path not set in conjunction with -source 6
16:42:31      [javac] error: Source option 6 is no longer supported. Use 7 or later.
16:42:31      [javac] error: Target option 6 is no longer supported. Use 7 or later.
16:42:31  
16:42:31  BUILD FAILED

Possible Solution

I believe, we'll need to update source and target for javac commands in https://github.com/AdoptOpenJDK/bumblebench/blob/master/build.xml.

To Reproduce

Remove the subsets constraint for any of the BumbleBench tests (https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml)
Use the Adopt Grinder: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/build?delay=0sec
Set the JDK_VERSION to 12.

adoptium / bumblebench Goto Github PK

bumblebench's Introduction

bumblebench

bumblebench's People

Contributors

Stargazers

Watchers

Forkers

bumblebench's Issues

Problem Description

Console Output

To Reproduce

Problem Description

Error

To Reproduce

Problem Description

Error

To Reproduce

Problem Description

Relevant Console Output with Error

To Reproduce

Problem Description

Error

Console Output

To Reproduce

Problem Description

Error

Possible Solution

To Reproduce

Recommend Projects

Recommend Topics

Recommend Org