aurbroszniowski / rainfall-core Goto Github PK

In order to indicate the overall progression, (nb of tx, TPS, and maybe ETA ?) we should log cumulative stats

seems useful for some people to load properties for some parameters of the DSL

alternatively, might want to parse text file for a scenario in plain text

when too many data, x axis doesn't display anymore the graph

to display the SLA on the final graph

Proposition : report by result (to compare each operation individually)

add gc activity + net activity + cpu + memory

Graph format example:

1. GET:

Stress test

TPS max:
TOTAL | cache1 | cache2
Response time
TOTAL | cache1 | cache2
Hardware
GC | Net | CPU | Memory
Response time percentiles distribution
TOTAL | cache1 | cache2

Normal load

TPS : 80% or defined in SLA
TOTAL | cache1 | cache2
Response time
TOTAL | cache1 | cache2
Hardware
GC | Net | CPU | Memory
Response time percentiles distribution
TOTAL | cache1 | cache2

2. PUT

3. MISS

etc.

In reporting, final reporting should be displayed
for instance, when Times Execution is used for 100,000 elements, the final line of the report should indicate that there were 100,000 Tx
Adjust Statistics thread not to stop too early

see io.rainfall.statistics.Metrics.increaseCounter

for instance,
compare CHM vs CHMv8 vs other?
or ehcache 2 vs ehcache 3?

verify key generationto see if distribution is really gaussian

As a reference, to show how we do it in Rainfall
and finalize y lines where they cut the graph, and add a legend to display the value

currently they are calculated during stat collection

Assertion could be added to be used for Performance Driven Development
e.g.
assert that test passes in less than 1 sec
so perf tests can be written first then code can be fixed to have performance test passing

Enum is limited because all results may not be listed
Using an Object/String instead so unexpected results can be gathered

add in the DSL something like
scenario exec(...).andCompareTo(...).andCompareTo(...) -> to compare scenarios in final report

in the html report, each graph has one line per scenario when scenarios are compared

Currently two files are generated - report everything on one page

Text/Html Reporter is called every second, we should be able to parameterize it

Generics shouldn't be needed for Ehcache 2 tests
Some common classes between ehcache 2 and 3 may produce unchecked calls warnings
Asses what would give this modification

Filling a cache with a single loop that is the size of the cache will fill it completely
Using a Times Execution will fill it almost but not completely

stats are measured like this:

statisticsHolder.measure(cache.getName(), new Task() {

it requires the instantiation of a Task class just to have better readability, maybe there is a way not to have to instantiate the object on every call and still have something readable

A concrete tutorial to explain step by step how to use Rainfall
(maybe comparing AS with a typical webapp?)

JMeter XML reporting could be used by
https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin

1. X axis of dates is going from 01:00:00 to ...
   fix it to start at 00:00:00

2. and when it is longer than 1h, it makes the chart break
   e.g. starts at
   01:00:00
   ...
   02:00:00
   breaks it

3. add option to add current hour as offset instead of 0

try to use
exec(operation1, operation2, operation3...).exec(operation4...)
instead of
exec(operation1).exec(operation2).exec(operation3).exec(operation4)

see TODO in project

physical disk, memory, process, CPU, and network

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-analyzeperf-cli-tools.html

• disk: http://www.cyberciti.biz/tips/linux-disk-performance-monitoring-howto.html
• cpu: http://www.cyberciti.biz/tips/how-do-i-find-out-linux-cpu-utilization.html
• gc activity
• process: ps ???
• memory : free -m
• network: sar : http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html