[Question] Stats about scarab HOT 5 OPEN

Hey Heiner,

Printing stats:
Scarab can print stats in one of two ways: 1) Scarab will dump all stats at the end of simulation. This is the *.stat.out files that appear in your run directory. 2) Scarab can be configured to periodically dump a stat (e.g, dump cycle_count every x number of instructions.)

We are working on a stats parser for batch runs; i.e., to rollup results across multiple SimPoints or a given stat for each benchmark in a suite. Is this the functionality you are looking for? We do not have a script currently available, but this is actively being worked on and should be ready relatively soon (a month, give or take).

In terms of printing the stats in a machine-readable format (I assume you mean something like JSON), we have discussed this, but there are no current plans to do this.

Duplicate rows (columns?)
Are you referring to the multiple columns in the stats file? If so, there is a subtle difference between columns 2/3 vs 4/5.

Columns 2/3 are affected when the stats are reset. For example, you may choose to simulate for a warmup period and reset the stats before you begin full simulation. Once full simulation ends, columns 2/3 will reflect the stat values during simulation only, whereas 4/5 will represent the complete stats, including warmup. Resetting the stats does not reset columns 4/5.

Does that make sense?
Stephen

from scarab.

Hi Stephen,
thank you!
With machine-readable, I mean that the format can be easily plotted using e.g. matplotlib. If the stats were in CSV format, it would be trivial to read it into a python script for plotting. In the current form, it requires a more advanced parser to extract the right values from the output files.

duplicate rows: yes I meant columns. Thanks!

from scarab.

Yeah, I see. The script we are working on will solve your problem. It is a python script that will parse the stats files, then can either be included as a library for matplotlib or can be run as a statndalone script. The only trouble is it has not been finished yet. The "scarab_batch_update" branch of this repository is currently developing that and a few other script updates.

That being said, there is a current (not working) version of the script in bin/scarab_globals/scarab_stats.py, which you can look at if you like. I will leave this question open for now, and update it when the working script has been pushed.

from scarab.

@spruett Do you happen to know whether this script(bin/scarab_globals/scarab_stats.py) is functional, or if the aforementioned issues still persist?

from scarab.

Yes these scripts are functional. I’ve been meaning to write some documentation on them for a while but have not gotten around to it.

In short there is a scarab_batch script which can be used to launch jobs either locally or on a SLURM system. Then scarab stats can be used to collect stats across multiple benchmarks or a suite.

The stats script can be used to collect any stats in the stat file or user defined stats. Either way the stats are rolled up and weighted properly in the case of checkpoints.

from scarab.