pmodels / openpa Goto Github PK

Reported by apenya on 27 Jun 2014 21:53 UTC
Jeff Hammond developed a C11 implementation of OpenPA.

It's sitting on https://github.com/jeffhammond/OpenPA

Mailing list thread: http://lists.mcs.anl.gov/pipermail/opa-discuss/2014-March/000045.html

This ticket is for the code review and integration.

Reported by goodell on 27 Apr 2009 16:43 UTC
Right now the src/primitives/opa_gcc_ppc.h file is somewhat BG/P-specific (PPC440-special). We ought to do one of two things:

1. Make it more general. This means supporting PPC G5 Macs (PPC970 family) and probably the POWER![5-7] series of machines.
2. Rename the file and add comments to make it clear that this is really only for BG/P or at least PPC4XX family machines.

Note that if we pick (2) then we'll probably still have to do (1) in some form at some point, particularly for the POWER machines.

Reported by buntinas on 19 May 2009 20:04 UTC
I've addressed some of the issues getting PGI to compile OpenPA. Here is the remaining (known) problem:

When an integer is passed as an asm parameter with a "m" constraint it uses the address of that parameter. However if the parameter happens to be a void*, then it dereferences the pointer. Gcc uses the address in both cases. The code below illustrates the problem.

This problem also revealed a gap in our testing code. I'll file a separate ticket for that.

#include <stdio.h>

volatile void *d = (void *)10;
void *prev, *newv=(void *)20, *oldv=(void *)10;

volatile int d_i = 10;
int prev_i, newv_i=20, oldv_i=10;


int main ()
{
    /* Integer version */
    printf("oldv_i = %d newv_i = %d d_i = %d\n", oldv_i, newv_i, d_i);
    asm ("lock ; cmpxchg %3,%4"
         : "=a" (prev_i), "=m" (d_i)
         : "0" (oldv_i), "q" (newv_i), "m" (d_i));
    printf("prev_i = %d d_i = %d\n", prev_i, d_i);
    
    printf("\n");

    /* void * version */
    printf("oldv = %d newv = %d d = %d\n", oldv, newv, d);
    asm ("lock ; cmpxchg %3,%4"
         : "=a" (prev), "=m" (d)
         : "0" (oldv), "q" (newv), "m" (d));
    printf("prev = %d d = %d\n", prev, d);

    return 0;
}

Reported by goodell on 22 May 2009 19:06 UTC
We might be able to write a primitives implementation that uses OpenMP atomic directives to provide atomicity. This might let us expand to a few more platforms where there is a working omp compiler but we don't yet support that particular platform's/compiler's inline assembly.

We might be able to have one of the summer students here at ANL work on this.

Reported by goodell on 27 Apr 2009 16:46 UTC
We have nightly tests running on a number of platforms but we don't run them on any BG/P machines right now. We should setup a nightly or bi-nightly cron job to run them on the ALCF's surveyor machine.

C++11 clang++ programs cannot compile because of missing spaces between literals and identifiers. g++ emits similar compilation warnings.

Short reproducer:

#include <iostream>
#include "src/opa_primitives.h"

int main()
{
	OPA_int_t a;
	OPA_store_int(&a, 4);
	OPA_decr_int(&a);
	std::cout << "a = " << OPA_load_int(&a) << std::endl;
	return 0;
}

The following compilation errors are emitted when compiling with clang++:

opa_gcc_intel_32_64_ops.h:93:39: error: invalid suffix on literal; C++11 requires a space between literal and identifier [-Wreserved-user-defined-literal]
   93 |     __asm__ __volatile__ ("lock ; add"OPA_SS" %1,%0"
      |                                       ^
      |                                        
opa_gcc_intel_32_64_ops.h:101:39: error: invalid suffix on literal; C++11 requires a space between literal and identifier [-Wreserved-user-defined-literal]
  101 |     __asm__ __volatile__ ("lock ; inc"OPA_SS" %0"
      |                                       ^
      |                                        
opa_gcc_intel_32_64_ops.h:109:39: error: invalid suffix on literal; C++11 requires a space between literal and identifier [-Wreserved-user-defined-literal]
  109 |     __asm__ __volatile__ ("lock ; dec"OPA_SS" %0"
      |                                       ^
      |                                        
opa_gcc_intel_32_64_ops.h:119:39: error: invalid suffix on literal; C++11 requires a space between literal and identifier [-Wreserved-user-defined-literal]
  119 |     __asm__ __volatile__ ("lock ; dec"OPA_SS" %0; setz %1"

Reported by jonathan morton on 22 May 2009 09:40 UTC
The PowerPC implementation uses the LL/SC paradigm extensively, and thus relies on the LL and SC primitives being implemented correctly. Unfortunately, the SC primitive has an elementary bug which could be extremely difficult to debug in-the-wild.

The 'ret' variable is not initialised, and is only set (to zero) if the operation fails. This could result in the operation always appearing to fail, even when it actually succeeds.

The fix is simply to initialise 'ret' to a non-zero value at definition.

Reported by goodell on 27 Apr 2009 16:47 UTC
We need to setup automated nightly testing to run on the SiCortex machine here at ANL.

Reported by buntinas on 23 Mar 2009 15:17 UTC
It looks like autoconf or automake or something created a COPYING file in my OPA directory. This file contains a GPL license (which conflicts with the license in our COPYRIGHT file).

-d

Reported by buntinas on 19 May 2009 20:07 UTC
The test suite doesn't catch an incorrect implementation of OPA_cas_ptr()

This is a placeholder to remind us to add the correct test.

Reported by ramon j moral on 18 Dec 2013 14:43 UTC
WHen running make check on MPI 3.0.4 I got the following error:

ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Unexpected return: 6535520 is not less than -2715 /bin/sh: line 5: 12680 Aborted (core dumped) ${dir}$tst
FAIL: test_primitives

Here is my configure script:FC=pgf90 F77=pgf77 CC=pgcc CXX=pgCC CPP="pgcc -E" FCFLAGS="-O2 -tp istanbul" F77FLAGS="-O2 -tp istanbul" CFLAGS="-O2 -tp istanbul" CXXFLAGS="-O2 -tp istanbul" ./configure --prefix=/u/rmoral/CTHV10.0/mpich --with-device=ch3:sock

This was on a CRAY XE 6 in cluster compatibility mode.

Reported by jhammond on 9 Jan 2014 19:47 UTC
High-level:

make  check-TESTS
make[Entering directory `/home/jhammond/MPICH/kayla/build/src/openpa/test'
make[5](4]:): Entering directory `/home/jhammond/MPICH/kayla/build/src/openpa/test'
PASS: sanity
FAIL: test_primitives
PASS: test_barriers
PASS: test_queue
make[Entering directory `/home/jhammond/MPICH/kayla/build/src/openpa/test'
make[6](6]:): Nothing to be done for `all'.
make[6]: Leaving directory `/home/jhammond/MPICH/kayla/build/src/openpa/test'
============================================================================
Testsuite summary for OpenPA 1.0.3
============================================================================
# TOTAL: 4
# PASS:  3
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See test/test-suite.log
Please report to https://trac.mcs.anl.gov/projects/openpa/newticket
============================================================================

Details:

Testing integer LL/SC ABA with 2 threads                                  Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
    Unexpected success of OPA_SC_int
*FAILED*
        at ../../../../../git/src/openpa/test/test_primitives.c:3949 in test_threaded_llsc_int_aba()...
    False positives: 25 / 1000456
Testing pointer LL/SC ABA with 2 threads                                  Unexpected success of OPA_SC_ptr
    Unexpected success of OPA_SC_ptr
*FAILED*
        at ../../../../../git/src/openpa/test/test_primitives.c:4178 in test_threaded_llsc_ptr_aba()...
    False positives: 26 / 999790

jhammond@kayla:~/MPICH/kayla/build/src/openpa/test$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-linux-gnueabihf/4.6/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.6.4-1ubuntu1~12.04' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --enable-multilib --disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16 --with-float=hard --with-mode=thumb --disable-werror --enable-checking=release --build=arm-linux-gnueabihf --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf
Thread model: posix
gcc version 4.6.4 (Ubuntu/Linaro 4.6.4-1ubuntu1~12.04) 

I am running on https://developer.nvidia.com/content/kayla-platform.

jhammond@kayla:~/MPICH/kayla/build/src/openpa/test$ cat /proc/cpuinfo 
Processor	: ARMv7 Processor rev 9 (v7l)
processor	: 0
BogoMIPS	: 1987.37

processor	: 1
BogoMIPS	: 1987.37

processor	: 2
BogoMIPS	: 1987.37

processor	: 3
BogoMIPS	: 1987.37

Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x2
CPU part	: 0xc09
CPU revision	: 9

Hardware	: cardhu
Revision	: 0000
Serial		: 3232423443424142

Reported by jhammond on 2 Mar 2014 21:30 UTC
http://git.mcs.anl.gov/radix/openpa.git/blob/HEAD:/COPYRIGHT includes contact information that is no longer useful.

D. Goodell: (630) 252-6082; FAX: (630) 252-5986; e-mail: [email protected]
D. Buntinas: (630) 252-7928; FAX: (630) 252-5986; e-mail: [email protected]

I recommend that either [email protected] be included here or the contact info stricken altogether.

Reported by goodell on 30 Mar 2009 23:41 UTC
We just got a bug report in MPICH2 that 1.0.8p1 doesn't support the PGI compilers. I don't have any real details at this point, but I'm filing this to make sure that we test and support PGI in OpenPA.

I think they have gcc-style inline assembly, so it should just be a matter of getting the keywords (asm vs asm) and the configure tests right.

-Dave

Reported by goodell on 10 Jul 2009 13:59 UTC
Unfortunately, it seems that the x86 lfence and mfence instructions aren't supported on many older processors and will require some sort of run-time test to correctly detect. Specifically, I believe that these instructions were introduced as part of the SSE2 instruction set. When one of these instructions is encountered on a machine that doesn't support it, an invalid opcode processor exception is raised (!#UD/INT6).

Wikipedia notes a few CPUs that you might have expected to support SSE2 but do not: http://en.wikipedia.org/wiki/SSE2#Notable_IA-32_CPUs_not_supporting_SSE2

This is causing a problem in MPICH2 on older systems right now, which is being tracked in https://trac.mcs.anl.gov/projects/mpich2/ticket/694.

The primary solution being considered right now is to convert some of our AC_TRY_LINK tests to AC_TRY_RUN tests with the current AC_TRY_LINK behavior as a fallback if we are cross-compiling. There are some other run-time (not configure run-time) workarounds we could use.

-Dave

Reported by tg on 5 Sep 2013 20:41 UTC
Hi,

as can be seen in http://buildd.debian-ports.org/status/fetch.php?pkg=mpich&arch=m68k&ver=3.0.4-1&stamp=1378413251 mpich 3.0.4-1 fails to build from source on Debian/m68k (on an Atari VM using ARAnyM).

If theres anything we can do to help tracking this down, contact [email protected] (sorry, no porterboxen available right now, but https://wiki.debian.org/Aranym/Quick contains quickstart VM images).

Reported by jayesh on 7 Oct 2009 16:10 UTC
Currently the OPA test suite does not work on windows. This is a reminder to port it to windows.

-jayesh

I use OPA to atomically handle pointer put/get/swap operations. The data pointed at was written before putting it, and read after getting it, of course.

I investigated memory barriers, and am confused about whether OPA handles these, or not. There are barriers in the include files, but for pointers it is not clear whether they apply to the pointer alone, or to the data referenced.

It almost always makes sense to handle memory order issues for all pointer operations. The one exception is when code handles the pointer alone (which I've encountered in the example I point at).

1. Is it right that OPA does not handle memory ordering of swaps that are being pointed-at?
2. Is it an idea to add primitives that clearly help users to portable memory barriers for pointers used via OPA?

Reported by michael forstner on 19 Sep 2013 11:20 UTC
Release tarball for V1.0.4 is missing the GCC ARM header for primitives.
I've downloaded the header from the current trunk and successfully compiled and tested the release on a nVidia Tegra2 system.