test t/io/sem fails on Mac PPC64 about perl5 HOT 14 OPEN

Description It was easy enough, with minor tweaks to CFLAGS, to build Perl in 64-bit mode on my Power Mac G5 – but something is not quite right in the semaphore handling. Running the test suite produces failures with no obvious cause at steps 6, 7, and 9 of t/io/sem.t. The exact output is:

# Failed test 6 - Checking value of semaphore 3 at io/sem.t line 70
#      got "0"
# expected "192"
# Failed test 7 - Check value via GETVAL at io/sem.t line 73
#      got "0 but true"
# expected "192"
# Failed test 9 - Checking value of semaphore 3 after fetch into originally UTF-8 buffer at io/sem.t line 82
#      got "0"
# expected "192"
FAILED at test 6

Steps to Reproduce From the top-level directory,

cd t
./perl -I../lib harness io/sem.t

Expected behavior All three sub-tests are apparently supposed to return the value 192.

Perl configuration

Nosferalto:perl-5.40.0 gsteemso$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 40 subversion 0) configuration:
   
  Platform:
    osname=darwin
    osvers=9.8.0
    archname=darwin-thread-multi-2level
    uname='darwin nosferalto.local 9.8.0 darwin kernel version 9.8.0: wed jul 15 16:57:01 pdt 2009; root:xnu-1228.15.4~1release_ppc power macintosh powerpc '
    config_args='-des -Dprefix=/Users/Shared/Brewery/Cellar/perl/5.40.0 -Uvendorprefix= -Dprivlib=/Users/Shared/Brewery/Cellar/perl/5.40.0/lib -Darchlib=/Users/Shared/Brewery/Cellar/perl/5.40.0/lib -Dman1dir=/Users/Shared/Brewery/Cellar/perl/5.40.0/share/man/man1 -Dman3dir=/Users/Shared/Brewery/Cellar/perl/5.40.0/share/man/man3 -Dman3ext=3pl -Dsitelib=/Users/Shared/Brewery/Cellar/perl/5.40.0/lib/site_perl -Dsitearch=/Users/Shared/Brewery/Cellar/perl/5.40.0/lib/site_perl -Dsiteman1dir=/Users/Shared/Brewery/Cellar/perl/5.40.0/share/man/man1 -Dsiteman3dir=/Users/Shared/Brewery/Cellar/perl/5.40.0/share/man/man3 -Dperladmin=none -Dstartperl='#!/usr/local/opt/perl/bin/perl' -Duseshrplib -Duselargefiles -Dusenm -Dusethreads -Acppflags=-m64 -Duse64bitall'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-std=gnu99 -fno-common -DPERL_DARWIN -mmacosx-version-min=10.5 -DNO_THREAD_SAFE_QUERYLOCALE -DNO_POSIX_2008_LOCALE -arch ppc64 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_FORTIFY_SOURCE=2'
    optimize='-O3'
    cppflags='-arch ppc64 -m64 -std=gnu99 -fno-common -DPERL_DARWIN -mmacosx-version-min=10.5 -DNO_THREAD_SAFE_QUERYLOCALE -DNO_POSIX_2008_LOCALE -arch ppc64 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 (Apple Inc. build 5666) (dot 3)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=87654321
    doublekind=4
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=6
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc -arch ppc64'
    ldflags =' -mmacosx-version-min=10.5 -arch ppc64 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib
    libs=-lpthread -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=/usr/lib/libc.dylib
    so=dylib
    useshrplib=true
    libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.5 -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_STRTOLD
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under darwin
  Compiled at Jul 18 2024 08:47:34
  @INC:
    /Users/Shared/Brewery/Cellar/perl/5.40.0/lib/site_perl
    /Users/Shared/Brewery/Cellar/perl/5.40.0/lib

Can you determine at what commit or release of Perl this test failure first appeared on your machine?

from perl5.

Do you see these failures in older perls?

There hasn't been a substantive change here since November 2020.

from perl5.

I notice that the OP's system is big-endian, and wonder whether the problem is somehow related to the "64 bit big-endian semctl SETVAL bug" mentioned a few lines earlier (line 53) in t/io/sem.t.
Annoyingly my PPC64 box died a few years ago.

from perl5.

I had noticed that same remark, but have not yet had time to dig into this any further. Given how long it takes to build stuff on a 2GHz machine from 20 years ago, I'd like to strategically build as few distinct versions of Perl as I can to get the answer to jkeenan's and tonycoz' questions above. Does anyone know offhand where I'd look to find out what version of Perl was being worked on when those last substantive changes were made? And do you want to hear about odd-numbered development releases or just the even-numbered production releases?

from perl5.

Does anyone know offhand where I'd look to find out what version of Perl was being worked on when those last substantive changes were made?

The change I was thinking of was d43c116 which was first in 5.34.

And do you want to hear about odd-numbered development releases or just the even-numbered production releases?

5.even.0 seems reasonable to me. If you're working from a git checkout

I notice that the OP's system is big-endian, and wonder whether the problem is somehow related to the "64 bit big-endian semctl SETVAL bug" mentioned a few lines earlier (line 53) in t/io/sem.t.

I don't think so, with the change that fixed that (64d7628 first in 5.20.0) we always go through the val member of union semun for SETVAL, and we use either the system defined union semun or one that matches the man semctl pages for Linux, current OS X and FreeBSD.

from perl5.

I have now verified that 5.34.0 produces the same result. Currently waiting for free time in which to test 5.32.1.

from perl5.

(A comment on my earlier concern about build times... I found that the actual builds go quite quickly indeed; but the tests, on the other hand, take a very long time simply because there are so many of them.)

from perl5.

Differing somewhat from 5.34.0, 5.32.1 produces similar results to every subsequent version, but the specifics of the failure are different:

# Failed test 6 - Checking value of semaphore 3 at io/sem.t line 66
# got "0"
# expected "17"
# Failed test 7 - Checking value via GETVAL at io/sem.t line 69
# got "0 but true"
# expected "17"
FAILED at test 6

Further investigation reveals that the difference in expected values was merely a change in the constants in the test file, and test 9 didn't fail because it didn't exist yet. Ultimately, there was in fact no change.

I will continue testing my way backwards in time to see if this ever does change at any point. My immediately-next tests will be of 5.20.0 and 5.18.0, bracketing when that big-endian-64-bit bug was stated to have been addressed... assuming that the test existed at that time. If it didn't, I will copy the version from 5.34.0 (which is identical to the current one, save for using slightly older syntax at the Config import).

At some version, I will not be able to go further backwards, as 64-bit Mac OS was not a thing before a certain point and I can't see how Perl could have been ready to compile for it in advance. “No time machines were harmed in the making of this software”, etc. ...It will be interesting to see just how far it can be pushed back, around or even past that point in history.

from perl5.

Well, 5.18.0 suffers a wide array of failures in this test, as well as raising numerous compiler warnings that were improved away in later editions of Perl. (I did indeed need to copy the t/io/sem.t from Perl 5.34, as well as editing it to set @INC directly rather than through a convenience function.) The failures were in tests 9-11, 13, 17-20, and 22. It also complained of malformed UTF-8 at line 81. This shows that, as annoying as the current test failure is, it used to be so much worse! I also find it interesting that tests 6 and 7 passed on that elderly version of Perl, but fail on more modern ones.

from perl5.

You can run just the io/sem.t test:

make test TEST_FILES=io/sem.t

If you've already built perl you can just do:

./perl t/io/sem.t

which is safe for this test (but not all tests)

You might be running into the limits of testing older versions.

With blead or 5.40, please add the two lines as follows:

diff --git a/t/io/sem.t b/t/io/sem.t
index 30beafabfd..7d3637970a 100644
--- a/t/io/sem.t
+++ b/t/io/sem.t
@@ -63,6 +63,8 @@ $SIG{__WARN__} = sub { push @warnings, "@_"; print STDERR @_; };
     ok(semctl($id, $ignored, GETALL, $semvals),
        'Get current semaphore values');
 
+    use Devel::Peek;
+    Dump($semvals);
     my @semvals = unpack("s!*", $semvals);
     is(scalar(@semvals), $nsem, 
        "Make sure we get back statuses for all $nsem semaphores");

and run:

./perl t/io/sem.t

The result should include a dump of $semvals, something like:

...
ok 4 - Get current semaphore values
SV = PV(0x55b8fdc2e1f0) at 0x55b8fdeab450
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x55b8fdebc2d0 "\x00\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"\0
  CUR = 20
  LEN = 22
ok 5 - Make sure we get back statuses for all 10 semaphores
...

which we can use to see if any semaphore values are coming through at all.

If this isn't returning something some sort of sensible we'll need to:

• make a simple C test case to see if it works sensibly there, if it does
• make a simpler perl test case and debug that with gdb

from perl5.

Tonycoz, your suspicions were correct -- I tried your variable-dumping recommendation, using the test file from 5.40.0 with the minimum changes needed to make it parse in each case, on numerous versions of perl going back to 5.12.5; and everything since 5.20.0 seems to produce a vector composed entirely of nulls for the semaphore values. I’m pretty sure that’s not what it’s supposed to look like.
(This probably doesn’t help much, but 5.12.5 and 5.18.4 both instead produced this vector: \0\0\0\0\0\0\0\300\0\0\0\0\0\0\0\0\0\0 )
I don’t know enough about the guts of Perl to help with the C test case you mentioned, and as you may have inferred from the pace of my replies here, I don’t really have any time to read up on it. If there are any other variables I can usefully dump (or other points in time at which to dump that one) in tests such as I have been doing, please let me know.

from perl5.

Unsurprisingly, 5.41.2 gave exactly the same results as well.

from perl5.

I have also verified that 32-bit PPC builds behave correctly in this test.

from perl5.

Are you building perl from tarballs or from a git clone?

from perl5.