gwinel / likwid Goto Github PK

[deleted issue]

Extend cpuid with the Pentium M variants

Groups can only be set statically for all architectures. This
is inflexible because some architectures have more counters or other events.

Solution:

Implement a dynamic group approach.
Add a routine to ask for the supported groups.
The API can be implemented with function pointers.
These are initialized according to the architecture.

Extend the module for Pentium M.
Probably a good opportunity for a partial rewrite.

The marker API could be simpler in usage.

Analyse the requirements and specify and implement new revised API.

When using markers it should be possible to accumulate a measurement
with multiple calls to the library.

The solution at the moment with Intel MPI is very specific and
does not scale.

Design a method based on a shell wrapper to implmement this independent from 
the MPI library used.

For many events in order to count cycles it is necessary apart from eventID and 
umask also to initialize the CMASK and INVERT field.

Plan:

Analyze the implications on the code. 
Implement for Core 2 and Nehalem.

Group ticket for release 2.0.

Open points:
* Formal review
* Testing
* Final benchmark set for likwid-bench

Convert programming style to WIKI

[deleted issue]

K8 is still a very common architecture.

Implement support for K8 in

* cpuid
* perfCtr

The clock measurement at the beginning of every start takes some time.
On automated runs it should be possible to turn it off.

1. Evaluate how far the duration can be reduced for exact results.
2. Add an option to turn it off.

What steps will reproduce the problem?
1. likwid-pin -c 0,1 ls

What is the expected output? What do you see instead?

Expected output: Message from likwid-pin about pinning, then output from ls.

Actual output:

[likwid-pin] Main PID -> core 0 - OK
ERROR: ld.so: object '/home.local/jan/bin/lib/liblikwidpin.so' from
LD_PRELOAD cannot be preloaded: ignored.
... and then the output from ls.

Please use labels and text to provide additional information.

Intel specifies besides the kernel provided logical numbering
a topological numbering. 

Provide a switch that likwid-pin can be also used with such an topological
numbering.

Compilation on woody fails with
./GCC/magma_dot.s:32: Error: no such instruction: `dpps xmm6,xmm5,0xFF'

[deleted issue]

The asm directives in cpuid module and probably else where do not compile
with gcc 4.4.1.

If counters are zero it apprears that NAN occur because of divide through
zero. 

Solution:

Test for zeros before division.

There is a socket lock for turning on and off the Nehalem Uncore events. 
Still with the Marker API on multiple runs different cores might akkumulate 
the results.

Solution:

Either force one core to turn counters on or off. Or later in the result 
presentation sum up all core results.

Add a make install target.
Put all configurable make settings in a separate file.

A special switch should re-interpret the specified core numbers as logical
numbers.

$ export OMP_NUM_THREADS=4  # dunnington1
$ echo $KMP_AFFINITY 
disabled
$ likwid-pin -t intel_omp -c 0,3,6,9  ./stream_omp_NT.exe
[likwid-pin] Main PID -> core 0 - OK
[...]
[pthread wrapper] [pthread wrapper] PIN_MASK: 0->9  
[pthread wrapper] SKIP MASK: 0x0
[pthread wrapper 0] Notice: Using libpthread.so.0 
        threadid 1073809728 -> core 9 - OK
[pthread wrapper 1] Notice: Using libpthread.so.0 
        threadid 1078008128 -> core 0 - OK
[pthread wrapper 2] Notice: Using libpthread.so.0 
        threadid 1082206528 -> core 0 - OK
[pthread wrapper 3] Notice: Using libpthread.so.0 
        threadid 1086404928 -> core 0 - OK
---------------------

In fact, all 4 application threads are running on core 0.
The only one on core 9 is the shepherd thread.

Up to now the sys fs file cpumap was used. I did not found any Documentation 
about these files. This will be changed to use the cpulist file which is plain 
integer list.

At the moment there is much redundant processor specific code.

Solution:

Reduce the processor specific parts to data configurations which are
processed by generic routines.

The detection of the processor clock does not work reliable with speedstep
active.

Solution:

Add a check if speedstep is enabled and ouput a warning.

Label for enabled SMT is too wide for box.

Solution:

1. Make the label width variable
or Increase the box width

The performance groups should be easily extensible.

A solution is to specify the groups in terms of simple text files which
are parsed by a Perl script which generated the according code.

Implement support for Sandy Bridge: Core and Uncore.

At the moment the skip mask only supports up to 64 threads due to the usage of 
64 bit integers.

Solution:

* Provide a solution where everything above 64 threads is truncated.
* Use a bitset of configurable size

Determine and print also NUMA information in likwid-topology

For threaded applications it is tedious to specify the core list
two times. A better solution is to integrate the pinning functionality in
likwid-perfctr.

[deleted issue]

The performance groups in perfCtr are not validated.
This is especially true for Intel Nehalem.

Solution:

Validate the existing groups with the implemented events.
Think about new metrics for performance groups.

The Intel Nehalem EX Uncore is different from Nehalem.

Task:

Implement the Nehalem EX Uncore environments 

It is more convenient if this necessary environment variable is set by
likwid-pin. This can be done in the -t switch clause.

[deleted issue]

likwid-perfCtr should support a mode were the counters run during 
execution of the application and in configured time steps read out and 
printed. This allows to generate graphs of events or derived metrics over an 
applications execution time.

Tasks:
1. Adopt the multiplex module to allow continous measurements.
2. Extend the architectures code by a routine to just read out results.
3. Provide output and computations of raw events and derived metrics.

Just found likwid (i.e. minutes ago) and I am very happy to see it.  I need to 
measure main memory traffic on a dual socket Nehalem system (i.e. across both 
sockets).  After investigating all the performance counters, it appears that 
some combination of the following is what I want:
UNC_QHL_REQUESTS.LOCAL_(READS|WRITES)
UNC_IMC_NORMAL_READS.ANY
UNC_IMS_WRITES_FULL.ANY

On to my question then -- in reading your wiki pages for Nehalem, I see that 
the performance group "MEM" uses UNC_L3_LINES_IN_ANY as part of its bandwidth 
measurement.  I believe that this counter will count the allocation of a cache 
line in any state, i.e., including modified or exclusive.  This means that if I 
assign a value without first reading it, you would incorrectly count this as 
part of the memory traffic.

This is relevant, in (e.g.), my particular case which is sparse matrix vector 
multiply (SpMV).  The inner loop of SpMV accumulates the dot product of one row 
in the matrix with the vector operand.  The accumulation is held in a processor 
register (ideally).  Once finished, the value is simply written into the 
destination vector, i.e., so as to avoid a write-miss.  Thus, L3 cache should 
allocate a line in the modified state without performing any DRAM access, thus 
defeating the use of UNC_L3_LINES_IN_ANY as a useful counter for memory traffic.

Finally, I must admit that I am at the moment simply searching for a good 
solution.  If the above is wrong, then please simply let me know :)

As a related question:  can likwid access all of the uncore performance 
counters (if yes, then you can add this to your brag sheet as perf cannot do 
this AFAIK).

Thank you,
Pete Stevenson

[deleted issue]

This is the ticket keeping track about the TODOs for the V1 release.
There is already a named branch for this release.

Current open issues:
1. Port P6 and Pentium M to new perfmon module
2. Write test procedure document for release
3. Update and Review Documentation (man pages and WIKI)
4. Provide examples in WIKI for marker use case
5. Review, extend and validate the groups for Core 2 and Nehalem

Problem:

At the moment the core ids can only be given as comma sparated list.
For high core counts it is convenient to anter ranges e.g. 0-7

Fix:

Implement ragens in the command line treatment of PErfCtr main routine.

At the moment if using the marker API only one instance of likwid
can run on a node.

It is useful to be able to start several instances of likwid on a node.

Solution:

Include the pid of either the wrapper process or the application in
the filename of the marker results.

To enhance the security of likwid a proper
handling of strings is crucial. bstring provides 
a secure, fast and feature rich interface.

What steps will reproduce the problem?
1. Instrument the code as described in:
http://code.google.com/p/likwid/wiki/Introduction

2. Run the command
./likwid-perfCtr -m  -c 0  -g L2  ./likwid-pin -c 0 <executable>


3.

What is the expected output? What do you see instead?

Expected output: the hardware counter information
Actual output: The message: "WARNING: Number of threads in marker file unequal 
to number of threads in likwid-perfCtr!"

and No output is seen.

What version of the product are you using? On what operating system?
Using 1.0.
On linux

Please provide any additional information below.

Due to multiple applications and many supported architectures in likwid it 
is necessary to follow a systematic work instruction to test likwid before 
releases.

Task:
Write document with work instructions and checklists to test and review 
likwid before releases.

The Uncore events on the Intel Nehalem processors are implemented but
do not yet work reliably.

Solution:

Test and fix the Uncore events.
Add appropriate events to the performance groups.

What steps will reproduce the problem?
1. Set install path with a slash at the end
2. make && make install
3. likwid-pin will result in an error

What is the expected output? What do you see instead?
Usual likwid-pin behavior.
Instead, no pinning and error:
~/helper/likwid-2.2> likwid-pin -c0-1 uname
[likwid-pin] Main PID -> core 0 - OK
ERROR: ld.so: object '~/helper/likwid-2.2-inst' from LD_PRELOAD cannot be 
preloaded: ignored.   

What version of the product are you using? On what operating system?
2.2, Linux

Please provide any additional information below.

Optionally support perf_event interface 
available since Linux version 2.6.31.

A major problem in using the tools is if the msr module is not loaded
or if the rights on the device files are not sufficient.

Solution:

Add a check on startup and issue a informative warning if
problems are detected.