The CM-(\d+) names in the branches point to our old internal stash issues numbers. CM stands for "Code Monkeys"
They are on the way to be imported here into this public github issues, and the branches need to be renamed then to point to the new issue numbers.

I would like to add cperl to my smoker similar to Schmorp's stableperl. Is there a set of tarballs I can feed to perlbrew?

Thanks!

The rationale to remove the for qw() syntax was bogus and annoying.
re-instate it for for loops, but not for the rest.
do not insist on the backwards syntax to require (qw( ... )) around the for list.

Keep qw() as list for for and foreach but not as lists for
if, given, when, while, until, elsif and function calls.

Use of qw(...) as parentheses is disallowed for:

    if qw(a) {}
    unless qw(a) {}
    if (0) {} elsif qw(a) {}
    given qw(a) {}
    when qw(a) {}
    while qw(a) {}
    until qw(a) {}
    $obj->meth qw(a b c)
    do foo qw(a b c)
    do $subref qw(a b c)
    &foo qw(a b c)
    $a[0] qw(a b c)

but allowed for (foreach also):

    for qw(a b c) {}
    for $x qw(a b c) {}
    for my $x qw(a b c) {}
    for my \$x qw(a b c) {}

commit 417a992
Author: Zefram [email protected]
Date: Fri May 25 22:25:21 2012 +0100

remove deprecated qw-as-parens behaviour

commit ea25a9b
Author: Zefram [email protected]
Date: Wed Sep 8 09:51:29 2010 +0200

make qw(...) first-class syntax

This makes a qw(...) list literal a distinct token type for the
parser, where previously it was munged into a "(",THING,")" sequence.
The change means that qw(...) can't accidentally supply parens to parts
of the grammar that want real parens.  Due to many bits of code taking
advantage of that by "foreach my $x qw(...) {}", this patch also includes
a hack to coerce qw(...) to the old-style parenthesised THING, emitting
a deprecation warning along the way.

use Exporter; package A; sub a { // };
for (keys %::) {delete $::{$_} unless Internals::SvREADONLY(%{$_})}

e.g. https://travis-ci.org/perl11/cperl/jobs/80072283#L1391

It crashes when %Internals:: is gone, and replaced by __ANON__::SvREADONLY

cperl-5.22.1 installed on OSX 10.9.5 with perlbrew

$ cpanm LCS
--> Working on LCS
Fetching http://www.cpan.org/authors/id/W/WO/WOLLMERS/LCS-0.10.tar.gz ... OK
==> Found dependencies: Module::Build::Tiny
--> Working on Module::Build::Tiny
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/Module-Build-Tiny-0.039.tar.gz ... OK
==> Found dependencies: ExtUtils::Helpers, ExtUtils::InstallPaths, DynaLoader, ExtUtils::Config
--> Working on ExtUtils::Helpers
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Helpers-0.022.tar.gz ... OK
Configuring ExtUtils-Helpers-0.022 ... OK
Building and testing ExtUtils-Helpers-0.022 ... OK
Successfully installed ExtUtils-Helpers-0.022
--> Working on ExtUtils::InstallPaths
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-InstallPaths-0.011.tar.gz ... OK
Configuring ExtUtils-InstallPaths-0.011 ... OK
==> Found dependencies: ExtUtils::Config
--> Working on ExtUtils::Config
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Config-0.008.tar.gz ... OK
Configuring ExtUtils-Config-0.008 ... OK
Building and testing ExtUtils-Config-0.008 ... OK
Successfully installed ExtUtils-Config-0.008
Building and testing ExtUtils-InstallPaths-0.011 ... OK
Successfully installed ExtUtils-InstallPaths-0.011
skipping N/NW/NWCLARK/perl-5.8.6.tar.gz
! Installing the dependencies failed: Module 'DynaLoader' is not installed
! Bailing out the installation for Module-Build-Tiny-0.039.
! Installing the dependencies failed: Module 'Module::Build::Tiny' is not installed
! Bailing out the installation for LCS-0.10.
3 distributions installed

Generally see my distroprefs for the known patches and fixes:
https://github.com/rurban/distroprefs/commits/master

cd ~/.cpan
git clone https://github.com/rurban/distroprefs
mv distroprefs/* .

Check if your .cpan/CPAN/MyCPAN.pm contains a proper YAML module (YAML::XS or YAML::Syck are good. With lots of .yml in build, YAML alone is too slow)

cperl5.22.1 -S cpan Module...

Failing modules

• Variable-Magic-0.58 broken
• Coro broken by upstream

Patched

• version-0.9912 patched, but still broken upstream

Not cperl related failures

• Internals::DumpArenas broken upstream. See mgruberman/internals-dumparenas#1 for the fixes

Here are 2 patches I propose for cherry picking into cperl.

http://perl5.git.perl.org/perl.git/commit/c7a622b32220ef59969124212598b6848bcaa247

This is required for the patch below to apply.

http://perl5.git.perl.org/perl.git/commit/9b669ea1e2997fbb78558e1fc0a7ecae3aa23af0

This patch prevents doing a ton of gv lookups while registering XSUBs with NewXS and etc, and a dozen less (or a dozen saved) useless GVs are in main:: after this patch.

stricter than strict. Fatalize stricter and misc warnings.
like javascript strong mode, but not so strict.
See https://github.com/rurban/stricter/

my (@a, $x) = (0, 1);

=> ERROR: Wrong slurpy assignment with @A in LIST, leaving $x uninitialized

FATAL misc:

my (%h) = 0; 

=> ERROR: Odd number of elements in hash assignment

use stricter adds stricter compile-time checks than strict, enables the default warnings, enables no multidimensional and fatalizes the compile-time warnings from this stricter and the misc category.

use stricter;

might be a better replacement for the typical idiom:

use strict;
use warnings;

plus

no multidimensional;
use warnings 'FATAL' => qw(stricter misc);

common::sense or strictures are similar but have misleading or bad names, and do not catch wrong slurpy assignments.

stricter adds a new warnings category stricter, and throws a warning on the "Possibly" cases. In the non "Possibly" cases the warnings are FATAL.

Convert perlbug to cperlbug, which files issues on github.
We don't need to send email, just post a web request with some simple json.
But we need the github username, or some administrative name with its token.
Already started with that.

perl-5.003_02 added an incomplete op_const_sv to check an optree for a single const or pad, and to mark it as CvCONST then. 760ac83
Nothing much changed since then.

This is

1. wrong. only const pad values may be used like this. sideeffects changing the lexical are not safe. 5.22 started warning about this condition.
2. not enough. proper constant folding should be done to check if the body may be inlinable.

Splice a OP* into the caller if inlinable, not just a CONSTSUB via the const_sv logic.
With some heuristic/cost calculation when it makes sense.

we can also use the new :const and :pure attributes to mark functions as inlinable.
:const to bypass the constant folding check, :pure to mark the closed over lexicals as not being changed by sideeffects.

Note: inlined subs and methods are a huge performance win in most dynamic languages. esp. with the slow perl function calls, it might be a big win.

Analyze the body and check if a enter/leave is needed, and if the args can be hardcoded into the inlined body (optim_args). this can be done by ref (with consts and with $_[]) or by value via copies of the sv/pad.
otherwise use a simple push @_ , args oplist instead of the pushmark args gv entersub sequence.

The inlined body needs to be copied, as the nextstate and args needs to be changed. We can do this on the fly, or 2nd or more generally add a new op_clone_oplist() API, which just clones the ops, not the data.

See the branch feature/CM-707-cperl-inline-subs, now feature/gh311-opclone

osx -DDEBUGGING clang
https://travis-ci.org/perl11/cperl/jobs/80126329#L3521

../dist/ExtUtils-CBuilder/t/00-have-compiler.t                  (Wstat: 6 Tests: 3 Failed: 0)
  Non-zero wait status: 6
  Parse errors: Bad plan.  You planned 7 tests but ran 3.
../dist/ExtUtils-CBuilder/t/03-cplusplus.t                      (Wstat: 6 Tests: 0 Failed: 0)
  Non-zero wait status: 6
  Parse errors: No plan found in TAP output

osx -Duselongdouble -DDEBUGGING gcc
https://travis-ci.org/perl11/cperl/jobs/80126331#L3536

../dist/ExtUtils-CBuilder/t/00-have-compiler.t                  (Wstat: 6 Tests: 3 Failed: 0)
  Non-zero wait status: 6
  Parse errors: Bad plan.  You planned 7 tests but ran 3.
../dist/ExtUtils-CBuilder/t/03-cplusplus.t                      (Wstat: 6 Tests: 0 Failed: 0)
  Non-zero wait status: 6
  Parse errors: No plan found in TAP output
../ext/DynaLoader/t/XSLoader.t                                  (Wstat: 256 Tests: 24 Failed: 1)
  Failed test:  11
  Non-zero exit status: 1

linux -Duse64bitall -Dusethreads -DDEBUGGING clang
https://travis-ci.org/perl11/cperl/jobs/80127917#L3607

op/aelem_u.t                                                    (Wstat: 139 Tests: 0 Failed: 0)
  Non-zero wait status: 139
  Parse errors: No plan found in TAP output
op/delete.t                                                     (Wstat: 139 Tests: 0 Failed: 0)
  Non-zero wait status: 139
  Parse errors: No plan found in TAP output
op/signatures.t                                                 (Wstat: 0 Tests: 1164 Failed: 0)
  TODO passed:   1041
porting/utils.t                                                 (Wstat: 0 Tests: 81 Failed: 1)
  Failed test:  58
../ext/B/t/concise.t                                            (Wstat: 139 Tests: 0 Failed: 0)
  Non-zero wait status: 139
  Parse errors: No plan found in TAP output
Files=2388, Tests=730581, 418 wallclock secs (86.27 usr  7.47 sys + 512.49 cusr 33.42 csys = 639.65 CPU)
Result: FAIL

linux -Duse64bitall -Dusethreads gcc
https://travis-ci.org/perl11/cperl/jobs/80127916#L3747

op/signatures.t                                                 (Wstat: 0 Tests: 1164 Failed: 0)
  TODO passed:   1041
../dist/ExtUtils-CBuilder/t/00-have-compiler.t                  (Wstat: 139 Tests: 3 Failed: 0)
  Non-zero wait status: 139
  Parse errors: Bad plan.  You planned 7 tests but ran 3.
../dist/ExtUtils-CBuilder/t/03-cplusplus.t                      (Wstat: 134 Tests: 0 Failed: 0)
  Non-zero wait status: 134
  Parse errors: No plan found in TAP output
Files=2388, Tests=727988, 286 wallclock secs (57.21 usr  5.56 sys + 333.47 cusr 29.20 csys = 425.44 CPU)
Result: FAIL

TODO:

• fix goto to hassig, treat it like a XSUB (MARK->SP): pp2sig, sig2sig, sig2pp. (mostly done)
• check defgv access in the bodies, error or warn if @_ is used. This is for convenience, but needed for automatic FAKE_SIGNATURE conversion.
• use it in most core libs (in work, see #97)

LATER:

• add invocant support, :self => $self as 1st argument with methods. fix the arity checker with sig->proto (multi branch)
• convert call-by-value to ref for constants when it's made faster (inline branch)
• multi (#16) and method combinations (#98)

STATUS:

• 80% faster than 5.22 sigs, 40% faster than without sigs.
• based on PERL_FAKE_SIGNATURE by davem, can automatically convert most
  old-style sigs to the new fast ones. this version uses the stack values directly, as with XSUBs. No @_ copying.
• adds compile-time coretype and arity checks, references, better error messages.

DONE:

• add typechecking for cv args and return value (done with 9b77750b4b62d38dad21fbf20159b36f48c10250)
• enable (\@array) and access it as $array->[] arrayref, ditto for hashes. (done)
• improve error reporting with sigs (name the missing arg, too many arguments: wanted 0-2, got 3, ...)
• accept proto or sig without any feature or pragma.
• call-by-value or ref for scalar, array and hash. i.e. copy or ref the arg.
  bind scalarrefs transparently as in perl6 or $_[n] as in perl5.
• auto-convert signature to prototype. but do not store the proto. only for compile-time arg checking.
• accept alternative ($a?) syntax for optional arg, as in perl6

See https://github.com/perl11/cperl/commits/feature/gh7-signatures

Overview

p5p came with the worst of all signature implementions which already do exist. The slowest and with the least features, actually blocking critical progress. The newer one in a branch for 5.22 actually looks pretty good (OP_SIGNATURE), but needs a complete rewrite so I wait on this until this lands and stabilizes.
smoke-me/davem/op_signature3

Update: It didn't land, so we use the temp. version and improve it by ourselves. See feature/gh7-signatures https://github.com/perl11/cperl/commits/feature/gh7-signatures

missing support for optional types

as provided for lexical variable declarations, in leading position as
with my int $a; and as attribute, as with ($i :int :const)
we need to seperate core (int, str, num) and user-defined types (class names),
and :const. Maybe more later.
store the typestash similar as in pads. do it again with the new OP_SIGNATURE.

s = scan_word(s, PL_tokenbuf, sizeof PL_tokenbuf, TRUE, &len);
typestash = find_in_my_stash(PL_tokenbuf, len);
var = parse_opt_lexvar();
PADNAME *pn = padnamelist_fetch(PL_comppad_name, var->op_targ);
SvPAD_TYPED_on(pn);
PadnameTYPE_set(pn,
  MUTABLE_HV(SvREFCNT_inc_simple_NN(MUTABLE_SV(typestash))));

missing syntax for return types (as attr)

for easier implementation we support subattributes, : only.
Note that we can add :method and :const also here.

sub add (int $a, int $b=1): int { $a + $b }

=> use i_add op

missing syntax and support for references, call-by-ref

with perl5 all arguments are copied only, as with my $arg1 = shift; but
syntax for fast $_[0] access is not provided.
cperl uses \$name to denote references to scalar lvalues.
with constant arguments use call-by-ref also.

e.g.

sub myfunc(int \$i) : int { $i++ }

modifies the calling argument.

for now scalar lvalue references only, \@a or \%h would be nice with type checks for arrayref
or hashref. maybe \[$] also.

wrong overly slow treatment of @_

Use the stack ABI as with OPs and XS, and do not copy the args into @_ for each call.
@_ is empty. Use your own slurpy parameter.
This makes cperl signatures about 2x faster.

The following applies only to the old slow zefram purple signatures, which are not use anymore:

The elements of @_ are accessed via aelem, not aelemfast,
the arity check is overly big and needs two checks and big strings with fixed arity subs.
replace @_ with direct stackaccess ST(n) with OP_SIGNATURE.

./perl -Ilib -MO=Deparse -e'sub x($a){$a++}'
sub x {
  die sprintf("Too many arguments for subroutine at %s line %d.\n", 
    (caller)[1, 2]) unless @_ <= 1;
  die sprintf("Too few arguments for subroutine at %s line %d.\n", 
    (caller)[1, 2]) unless @_ >= 1;
  my $a = $_[0];
  ();
  $a++;
}

=>

die sprintf("Wrong number arguments for subroutine at %s line %d.\n", 
  (caller)[1, 2]) if @_ != 1;

improve error reporting

In compile-time errors do not only print the position, also the declaration which is violated.
e.g.

@a=(); sub x(\@b) {$b->[0]++} print x(\$a)

Type of arg 1 to main::x must be arrayref (not a scalar ref) at -e line 1, near "\$a)"
=>
Type of arg 1 \@b to x must be arrayref (not a scalar ref) at -e line 1, near "\$a)"

Harmonize error message with the rest:

• Same compile-time and run-time error messages for the same error.
• Skip main:: when printing the function name
• Not enough arguments for %s %s as with the ops and not Too few arguments for %s %s

Add subroutine type and name to error message on arity errors:

Too many arguments for subroutine myfunc
Too many arguments for subroutine entry myfunc
Not enough arguments for multi subroutine myfunc
Too many arguments for method myfunc

Disallow nameless parameters

When they clash with the prototype syntax.

Proper signature types are not only a great help for catching errors
early. they are performance critical, see coffescript, dart, microsoft
typescript, facebook flow and hack. the type inferencer will not be
able to infer many types without explicit types. but with typed
signatures, besides the obvious solution of private methods or closed
classes we can inline most small methods, and improve most loops and
array accesses. it is also critical to implement multi methods, i.e. proper multi-dispatch,
which would enable a proper oo system within perl5.
Dios is close but a hack (PPI macros!), and only with Inside-Out objects.

Installed with perlbrew on OSX 10.9.5

$ wget https://github.com/perl11/cperl/archive/cperl-5.22.1.tar.gz \
    -O cperl-cperl-5.22.1.tar.gz
$ perlbrew install --as=cperl-5.22.1 $(pwd)/cperl-cperl-5.22.1.tar.gz

helmut@helmut:~/github/LCS-XS$ cpanm .
--> Working on .
Configuring /Users/helmut/github/LCS-XS ... OK
==> Found dependencies: Test::Deep, LCS
--> Working on Test::Deep
Fetching http://www.cpan.org/authors/id/R/RJ/RJBS/Test-Deep-0.117.tar.gz ... OK
Configuring Test-Deep-0.117 ... OK
Invalid version format (non-numeric data) at /Users/helmut/perl5/perlbrew/perls/cperl-5.22.1/lib/5.22.1/Safe.pm line 51.
BEGIN failed--compilation aborted at /Users/helmut/perl5/perlbrew/perls/cperl-5.22.1/lib/5.22.1/Safe.pm line 51.
Compilation failed in require at /Users/helmut/perl5/perlbrew/bin/cpanm line 382.
BEGIN failed--compilation aborted at /Users/helmut/perl5/perlbrew/bin/cpanm line 382.
Compilation failed in require at /Users/helmut/perl5/perlbrew/bin/cpanm line 36.

signatures containing a newline cannot be parsed yet. see feature/gh9-signatures3-gh25-nl
and feature/gh9-signatures3-gh25-nl1 for a not yet working attempt to parse it properly with the parser, not inside the lexer.

the lex restore code, when validate_proto fails in toke.c:8204, is wrong.

sub x ($a=undef, $b=0, $c=1, $d=2, $e="foo",
                     $f=$lex, $g=$pkg) { }

in feature/gh9-signatures3-gh25-nl I removed all newlines to pass the tests. A temp. workaround to continue with classes

The old hash table uses:

• linked lists for its collisions, with slow out-of-cache pointer chasing and data overhead.
• unsorted flags at the end, while some flags are needed for compare.
• has questionable security measures to slow down all cases. seed ok, randomize iter maybe, but randomize the collisions and slow hash funcs is stupid. the security should be fixed with proper implementations, not by pseudo-security theatre.

either use:

• rurban/hash-sortbuckets: a sorted static list of collisions (up to 8, maybe start with 3, then 8) as in knuth sorted hash table,
• khash: use open addressing as everyone else. faster, less space.
• ✓ PERTURB_KEYS_TOP move-to-front with a linked list is the only sane strategy for simple chained bucket lists.
• ✓ HV_FILL_RATE: try lower fill rates than 100%. 100% is pretty insane, esp. with our bad hash funcs. make it fast with builtin_ctz.
• ✓ use __builtin_ctz for faster division in DO_HSPLIT. Allow -DHV_FILL_RATE=90 definition. (Tested to be the best with FNV1A)
• HE_ARRAY: According to http://goanna.cs.rmit.edu.au/~jz/fulltext/spire05.pdf
  the best for chained hashing is currently a cache-friendly array:
  cache-friendly continuous buffer of HE's w/ inlined HEK (char_) + SV_ val, but no hash, no next ptr. Also for shared he's: PL_strtab.
• ✓ array_he: inline parts of the HE into the array. array_he vs ll_he. (linked list, see also the he-array branch). array_he (HvARRAY = AHE[]) can contain { hent_he, hent_hash }. this way the hash catches 99% of all comparisons already, and we don't have to chase the external hek ptr, when the hash check fails. every HE entry will then be 2 words (128), instead of one word (64), called AHE. The linked list still contains the old HE*, with { hent_next, hent_hek, hent_val }.
• one-word-AHE: As possible improvement on that on 64bit use 48bits for the HE ptr, and 16bits of the hash to be compared first. See https://www.semanticscholar.org/paper/Fast-Dynamically-Sized-Concurrent-Hash-Table-Barnat-Rockai/ab7dd007587f411cf99bfe056639e055eff22e0c/pdf
• use robin-hood as this is currently the best worse-case strategy (being super defensive, but not so stupid to use SipHash, which adds no benefit). with better threading support (shared hashes) eventually use leapfrog.
• compact ordered hash. use an array of short indices into a compacted array of hash/key/val entries as in PyPy and now python: "compact ordered dict". This saves a lot of space and only add's one indirect lookup into cache-friendly array. See methane/cpython#1 https://mail.python.org/pipermail/python-dev/2016-September/146327.html
  This also iterates over the hash in insertion order, which effectively hides any attempt to get the seed over the iterators. For attacks you need to get collision and robin-hood reordering timings.

maybe use the glibc htable (hsearch, hcreate), just add random seeds and flood detection (collision counts). (or khash, which is basically the same). coucal looks good also. https://github.com/xroche/coucal
but best is probably preshing's http://preshing.com/20160314/leapfrog-probing/, which even can be made concurrent easily.

open addressing is cache friendly and much faster, but table growth is slow.
potion uses khash, first double open, then quadratic open.

either way, first I need to abstract away the collision search in MACROS
and reduce the needed comparisons from 4 to 1. (edit: after my critic, they improved that part to one ptr cmp)
maybe we can normalize all keys to utf8 for faster checks. (probably not)
utf8 is a HEK flag. See http://cpansearch.perl.org/src/RURBAN/illguts-0.49/index.html#hek
The current implementation is just insanity and it only got worse over the years.

The last attempt was here: https://github.com/rurban/perl/commits/rurban/hash-sortbuckets
See the featurex/gh24-*hash* branches.
See also GH #102

plan

• ✓ -Dhash_func, add and test many more funcs
• ✓ PERTURB_KEYS_TOP
• ✓ HV_FILL_RATE
• ✓ d_builtin_ctz
• ✓ seperate hv_common_magic
• ✓ security: warn and protect against hash flood
• ✓ fix NODEFAULT_SHAREKEYS (#201)
• ✓ hash_loop: abstract loops and array_he
• ✓ array_he: seperate HvARRAY from collisions entries, add hent_hash
• 80% new_hash_table: merge HE fields for faster loop, usehekbitfield, d_little_endian.
• one-word-AHE (hash+he on 64bit)
• 40% open_hash, compact_ordered: various strategies with the new abstractions

gperf puts our tables into the lookup function, which makes them not exported on ELF, great trick.
But it forgets to add const to it, which is horrible. They are RW, ie COW on most platforms, but some platforms cannot handle that well. Looking at you, Windows.
Thanks for bulk88 for detecting that.

we have a simple gperf postprocessor already, trivial patch.

See pjcj/Devel--Cover#125 and FB#187957

Felipe found the _GEN sym problem which is not yet fixed, and our current workaround is not yet good enough.
One idea is that is caused by an \undef (coming from \shift in the args code),
the other an empty stash caused by Symbol, an autodie IO gensym.

The current workaround is to use:

foreach my $sym ( sort keys %$symref ) {
+ next if $sym =~ m<\A_GEN_[0-9]+\z>;

(was CM-821)

Starting with early September t/op/inc.t is reporting fails in all the use warnings 'imprecision' tests on inc and dec. First only for 32bit, but with the 32bit fixes (with 60d0c28) now also for 64bit.

I'm not sure how severe that is. I doubt if our faster path should be made slower to warn on overflows, and also if there's no imprecision at all in this range of small doubles. double's have a big range of precise arithmetic and we don't want to check real mantissa overflow.

Empty is success. A message means failure.

C:\sources\cperl>perl -e"require Win32;"
Can't locate DynaLoader.pm in @INC (you may need to install the DynaLoader modul
e) (@INC contains: C:/sources/cperl/lib .) at C:/sources/cperl/lib/Win32.pm line
 8.
Compilation failed in require at -e line 1.

C:\sources\cperl>perl -e"glob('*.h')"
Can't locate XSLoader.pm in @INC (you may need to install the XSLoader module) (
@INC contains: C:/sources/cperl/lib .) at C:/sources/cperl/lib/File/Glob.pm line
 6.
Compilation failed in require at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

C:\sources\cperl>perl -e"DynaLoader::boot_DynaLoader(); require Win32;"

C:\sources\cperl>perl -e"DynaLoader::boot_DynaLoader(); glob('*.h');"
Can't locate XSLoader.pm in @INC (you may need to install the XSLoader module) (
@INC contains: C:/sources/cperl/lib .) at C:/sources/cperl/lib/File/Glob.pm line
 6.
Compilation failed in require at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

C:\sources\cperl>perl -e"BEGIN{ DynaLoader::boot_DynaLoader();} glob('*.h');"

C:\sources\cperl>

Notice perl phase is important for the glob() example.

The problem is, there is nothing to call boot_DynaLoader sub except by hand.
The only reference to XS_DynaLoader_bootstrap is from xs_init which is defined at https://github.com/perl11/cperl/blob/master/win32/perllib.c#L39 and that is a newXS call, not execution of XS_DynaLoader_bootstrap XSUB.

    perl522.dll!xs_init(interpreter * my_perl=0x00383e54)  Line 39 + 0x10   C
    perl522.dll!S_parse_body(interpreter * my_perl=0x00000056, char * * env=0x00384e00, void (interpreter *)* xsinit=0x280c55c7)  Line 2229 + 0x4   C
    perl522.dll!perl_parse(interpreter * my_perl=0x00383e54, void (interpreter *)* xsinit=0x280c55c7, int argc=4, char * * argv=0x00383db8, char * * env=0x00384e00)  Line 1627 C
    perl522.dll!RunPerl(int argc=4, char * * argv=0x01383db8, char * * env=0x00382f10)  Line 251 + 0x10 C
    perl.exe!mainCRTStartup()  Line 398 + 0xe   C
    kernel32.dll!_BaseProcessStart@4()  + 0x23  

other cperl perma-XS-.pms are loaded at https://github.com/perl11/cperl/blob/master/perl.c#L2223 in S_parse_body through boot_core_xsutils but there is alot of stuff disabled in https://github.com/perl11/cperl/blob/master/xsutils.c#L239 since day 1 of the file in d328f3e "strict 1.10c as builtin, re-add xsutils.c ". So how is DynaLoader::boot_DynaLoader XSUB called on your build?

Win32 perl has PERL_IS_MINIPERL on every one of miniperl's .o, but on unix perl I think PERL_IS_MINIPERL is much more restricted in use since most of the .o files are used unchanged for linking full perl but I know something on unix perl does use PERL_IS_MINIPERL.

Example of calling DynaLoader::boot_DynaLoader by hand from BEGIN block on my build.

    perl522.dll!dl_boot(interpreter * my_perl=0x280f9038)  Line 151 C
    perl522.dll!dl_generic_private_init(interpreter * my_perl=0x00000001)  Line 161 + 0x5   C
    perl522.dll!boot_DynaLoader(interpreter * my_perl=0x00383e74, cv * cv=0x008fcec4)  Line 515 C
    perl522.dll!Perl_pp_enterxssub(interpreter * my_perl=0x01383e74)  Line 3515 C
    perl522.dll!Perl_pp_entersub(interpreter * my_perl=0x0090b53c)  Line 3346 + 0x7 C
    perl522.dll!Perl_runops_standard(interpreter * my_perl=0x00383e74)  Line 41 + 0x4   C
    perl522.dll!Perl_call_sv(interpreter * my_perl=0x009088d4, sv * sv=0x00908904, volatile long flags=9472244)  Line 2782 + 0xc    C
    perl522.dll!Perl_call_list(interpreter * my_perl=0x00383e74, long oldscope=2, av * paramList=0x009085a4)  Line 4906 C
    perl522.dll!S_process_special_blocks(interpreter * my_perl=0x00070023, long floor=39, const char * const fullname=0x0090a35c, gv * const gv=0x009085b4, cv * const cv=0x009085c4)  Line 8957    C
    perl522.dll!Perl_newATTRSUB_x(interpreter * my_perl=0x01000002, long floor=39, op * o=0x0090b1c0, op * proto=0x00000000, void * attrs=0x28031203, op * block=0x0091215c, char o_is_gv=0)  Line 8885 + 0x11  C
    perl522.dll!Perl_yyparse(interpreter * my_perl=0x00383e74, int gramtype=9483344)  Line 295 + 0x26   C
    perl522.dll!S_parse_body(interpreter * my_perl=0x00000001, char * * env=0x00384e20, void (interpreter *)* xsinit=0x280c55c7)  Line 2323 + 0xd   C
    perl522.dll!perl_parse(interpreter * my_perl=0x00383e74, void (interpreter *)* xsinit=0x280c55c7, int argc=3, char * * argv=0x00383db8, char * * env=0x00384e20)  Line 1627 C
    perl522.dll!RunPerl(int argc=3, char * * argv=0x01383db8, char * * env=0x00382f10)  Line 251 + 0x10 C
    perl.exe!mainCRTStartup()  Line 398 + 0xe   C
    kernel32.dll!_BaseProcessStart@4()  + 0x23  

nextstate resets the stack at every ; which is a waste and adds enormous run-time costs.
Track the stack depth for each op result if statically possible, add the oplines field to each op, thus get rid of most nextstate COPs.

nextstate is one of the most costly ops (see dtrace Porting/op.d), and is called every single line.

stack tracking+oplines. see the jit #220

Rewrite warnings.pm which is generated as pure XS. esp. to save memory

A regular XS extension, with a stable set of .xs and .h files, and a warnings_xs.PL regen and definition
script (prev. regen/warnings.pl), which updates the keys in place.
required gperf so far, as with Config as XS.

Seperate into a pure read-only perfect hash, and a user-extendable warnings::register hash.
Check against Carp in XS and PP Carp.

See branches feature/CM-367-cperl-warnings-xs-carp and feature/CM-370-cperl-warnings-xs

See https://github.com/imneme/pcg-c-basic for the 32bit
and https://github.com/imneme/pcg-c for the 64bit variant. And my fork and fixes.

This is 20x faster and much better statistically.
Est. 3d

See also perl11/potion#99 for the results of the pcg branch in potion.

Or use the new xoroshift++, which should be even better. See https://www.reddit.com/r/programming/comments/4gtlfz/xoroshiro128_the_fastest_fullperiod_pseudorandom/
Nope: xoroshift128++ is officially broken.

The only remaining task to remove miniperl is one function in ext/DynaLoader/dlboot_c.PL

resolve xx/../path, symlinks, duplicates
e.g.
("/usr/lib", "/lib/x86_64-linux-gnu", "/lib/../lib", "/usr/lib/x86_64-linux-gnu", "/usr/lib/../lib")
=> ("/usr/lib", "/lib/x86_64-linux-gnu", "/lib", "/usr/lib/x86_64-linux-gnu")

Rename lib/buildcustomize.pl to lib/bootlib.pm which is included in RUN_PERL.

resolve_duplicate_paths in Configure

first resolve any/../w => w
then remove duplicates, but keep order intact
then resolve symlinks
easily done via sh or sed or awk. (Done)

add this to config.h:

/* LIBPTH
*     This variable contains a space seperated (or list) set of paths for the linker
*      and DynaLoader to search for additional libraries.
*/
#define LIBPTH "$libpth"

remove all references to miniperl, improve and streamline the build system. (In work)

See branch feature/CM-916-cperl-no-miniperl https://github.com/perl11/cperl/commits/feature/CM-916-cperl-no-miniperl, was CM-916

Optimizer (compile-time)

We typed now all ops, and can optimize (up and downgrade) on them.

But we want to pass native typed data (e.g. INT_PADSV) to subs also, when they allow
native types.

We want to keep optimized sub args when the sub allows it, and don't want to downgrade all typed ops within pushmark.

Dispatch (run-time)

Our compile-time type double-dispatch is slow but needs not much memory, we just cycle all type variants until we find it.

The user-based dispatch needs to be fast, so we need dispatch tables, or even a dynamically adjusted tree. it is also multi-dispatch, not just a simple double dispatch

Damian went in Dios with a too trivial arity check first, while he really should have started from left.

probe for libffi and link to it if found. useffi, ffi_targets for the valid :nativconv() ABI args.
parse syntax:

use ffi; # NativeCall already taken and incompatible
extern sub NAME SIG ATTRIB?;
#or
sub NAME SIG :native() ATTRIB?;

The alternative for extern is the perl6-style :native attribute, with optional
arg :native("libname") and optional :symbol("c_name").
Str arg or return types can have optional :encoded('utf8') attributes.
The default return type is :void (not :Int).
See https://docs.perl6.org/language/nativecall
Resolve the attribute arguments, safely. See #291. (Not as in Attribute::Handlers)

Use a new op dlcall/enterffi for it, instead of entersub. leaveffi not needed, as no blocks are stored for extern subs, just the call and the sigop. the sigop holds the info for argument and returntype translations, the callop does the translation, call, translation.

Without a :native(libname) the symbol searched in all loaded shared libraries.
So you can preload shared libs via the DynaLoader.

Add and separate a function to DynaLoad a pure shared library, without xs bootstrap, so extern does not need any library name. Other languages do it eg. with use lib ("mysqlclient.so");

Translate args in the optimizer to such a sub, upgrade to native.

Enforce native types for the sub parameters, resp. provide casts (unbox).
ditto for the return value.

A CvEXTERN(cv) needs a temp. libptr for the loaded shared lib (shared with the ffi cif), and an optional PV for the symbolname.
(mangled or unmangled, :symbol() override, _ prefix or not).
It doesn't need a CvSTART, nor a CvROOT, so overload those two.
At runtime libffi needs the XSUB address, and the cif, i.e. the ffi signature.
The symbolname is resolved to the XSUB address via DynaLoader, the sig is converted to a cif at compile-time.

We need a few more Configure probes and keys: useffi (to disable it), d_libffi (if the definition probed ok), i_ffi (if to include <ffi.h>), i_alloca (if to include alloca.h), d_alloca or d__alloca (for fast arg preperation).

Note that Larry had this in his TODO, but p5p removed it later.
Est. 2w

Architecture

parser: extern sub name (sig) :native() :symbol() :nativconv() :encoded();

• sig is stored as sigop attached to the CvEXTERN sub, not in the block (CvROOT). needed for prep_cif.
• attrs store the lib and symname in xsutils,
• NY: add the runtime attributes->import call to the stmt list there. compile-time works fine.

attributes::import (either compile-time or run-time):

• resolve the lib and name via DynaLoader (i.e. not in miniperl). See #291 for run-time attrs support :native($libname).
• set libhandle (temp. CvFFILIB) via dl_load_file
• resolve symbol ptr via dl_find_symbol
• store ptr at CvXFFI(). CvFFILIB is now unneeded, and re-used for the cif.
• ffi_prep_cif: prepare sig arity and type translations.
• set CvFFILIB from lib handle to cif.
• store the optional :symbol() override as proto (shared pvn). protos are not needed, a sig is mandatory.

compiler:

• ck_entersub_args_proto_or_list checks for CvEXTERN
• sets op from ENTERSUB to ENTERFFI (as with ENTERXSSUB)
• type check the call args

runtime: pp_enterffi

• prep_sig: copy and type-check args from perl stack to arg_values[] via sigop aux info.
• ffi_call(cif, CvXFFI(cv), &rvalue, argvalues);
• prep_ret: copy rvalue to perl stack.

rewrite Exporter.pm as XS, analog to attributes.
That's probably a bigger one

Est 1w

• panic: free from wrong pool, 0!=100803200 during global destruction:

t/io/pipe ........................................
t/io/socket ......................................
t/op/getppid .....................................
t/op/magic .......................................
t/op/write .......................................
cpan/File-Temp/t/fork ............................
cpan/libnet/t/nntp_ipv6 ..........................
cpan/libnet/t/pop3_ipv6 ..........................
cpan/libnet/t/smtp_ipv6 ..........................
cpan/Socket/t/socketpair .........................
cpan/Test-Simple/t/Builder/fork_with_new_stdout ..
cpan/Test-Simple/t/fork ..........................
cpan/Test-Simple/t/subtest/fork ..................
dist/IO/t/cachepropagate-tcp .....................
dist/IO/t/cachepropagate-unix ....................
dist/IO/t/io_multihomed ..........................
dist/IO/t/io_sock ................................
dist/IO/t/io_unix ................................
ext/B/t/b ........................................

leading to no test error.

and

• Failed 14 tests out of 2299, 99.39% okay.

    ../cpan/Memoize/t/errors.t
    ../cpan/Pod-Usage/t/pod/pod2usage2.t
    ../cpan/Test-Simple/t/subtest/fork.t
    ../ext/B/t/concise.t
    ../ext/B/t/optree_check.t
    ../ext/B/t/optree_concise.t
    ../ext/Devel-Peek/t/Peek.t
    ../lib/warnings.t
    io/pipe.t
    op/aelem_u.t
    op/delete.t
    op/fork.t
    op/magic.t
    porting/utils.t

Currently I only have one patch in my distroprefs for it, but the big fix is still missing.
https://github.com/rurban/distroprefs/blob/master/prefs/Variable-Magic.yml

See also #36 to make it easier to use it.

add a build matrix for that, and fix the errors.
You shouldn't use that though. It's only to ease taking patches upstream.

Supporting this would have made it easier to take patches to upstream.
But p5p closed the door and refuse even to take bug reports, so it's very low priority.

Need to get the branch new-5.22.2 passing, before I can test the other fixes: win32 #39 and dbg-globs #38

    porting/corelist.t
    porting/manifest.t
    porting/pod_rules.t
    porting/podcheck.t
    porting/regen.t
    porting/utils.t

no documentation in utils/cperldoc
/home/travis/perl-blead/man/man1/perldoc.1
Replacing link() with File::Copy::copy(): Couldn't link /home/travis/perl-blead/man/man1/cperldoc.1 to /home/travis/perl-blead/man/man1/perldoc.1: No such file or directory
/home/travis/perl-blead/man/man1/perldoc.1
Couldn't copy /home/travis/perl-blead/man/man1/cperldoc.1 to /home/travis/perl-blead/man/man1/perldoc.1: No such file or directory

e.g. https://travis-ci.org/perl11/cperl/jobs/80042040

not even Config_boot is ready for dist/PathTools

Can't locate Config.pm in @INC (you may need to install the Config module) (@INC contains: /home/travis/build/perl11/cperl/cpan/AutoLoader/lib /home/travis/build/perl11/cperl/dist/Carp/lib /home/travis/build/perl11/cperl/dist/PathTools /home/travis/build/perl11/cperl/dist/PathTools/lib /home/travis/build/perl11/cperl/cpan/ExtUtils-Command/lib /home/travis/build/perl11/cperl/cpan/ExtUtils-Install/lib /home/travis/build/perl11/cperl/cpan/ExtUtils-MakeMaker/lib /home/travis/build/perl11/cperl/cpan/ExtUtils-Manifest/lib /home/travis/build/perl11/cperl/cpan/File-Path/lib /home/travis/build/perl11/cperl/ext/re /home/travis/build/perl11/cperl/dist/Term-ReadLine/lib /home/travis/build/perl11/cperl/dist/Exporter/lib /home/travis/build/perl11/cperl/ext/File-Find/lib /home/travis/build/perl11/cperl/cpan/Text-Tabs/lib /home/travis/build/perl11/cperl/dist/constant/lib /home/travis/build/perl11/cperl/lib .) at /home/travis/build/perl11/cperl/cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker/Config.pm line 7.
BEGIN failed--compilation aborted at /home/travis/build/perl11/cperl/cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker/Config.pm line 7.
Compilation failed in require at /home/travis/build/perl11/cperl/cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm line 9.
BEGIN failed--compilation aborted at /home/travis/build/perl11/cperl/cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm line 9.
Compilation failed in require at Makefile.PL line 5.
BEGIN failed--compilation aborted at Makefile.PL line 5.
Error 512 from dist/PathTools's Makefile.PL at make_ext.pl line 524.
Warning: No Makefile!
make[1]: *** No rule to make target `all'.  Stop.
make[1]: *** No rule to make target `all'.  Stop.
Unsuccessful make(dist/PathTools): code=512 at make_ext.pl line 575.
make: *** [lib/auto/Cwd/Cwd.so] Error 2
make: *** Waiting for unfinished jobs....

use better bison error and debugging reporting facilities.
do not throw away the useful perlytmp.output and tmp files when regenerating.

• add -v -r states. And -t eventually on DEBUGGING
• search for a proper bison-2.7.1 or bison-2.5.1 binary, when we see an unsupported bison. newer bisons are currently not supported, so this usecase is now very common
• do not duplicate the tokens table. bison-2.4 started with them as enum, so we have them now twice,
  as #defines and as enum's.
  Update: win32 undefines the enum, because of a clash with their typedef WORD ! We don't want to rename the WORD token, do we?
• define YYPRINT with DEBUGGING
• use a better printer macro

    Reducing stack by rule 19 (line 218), stmtseq fullstmt -> stmtseq
    Entering state 103

=> Entering state 103 stmtseq

        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_EXTERNAL
_GLOB -DPERL_IS_MINIPERL -Fo.\mini\utf8.obj ..\utf8.c
utf8.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_EXTERNAL
_GLOB -DPERL_IS_MINIPERL -Fo.\mini\util.obj ..\util.c
util.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_EXTERNAL
_GLOB -DPERL_IS_MINIPERL -Fo.\mini\xsutils.obj ..\xsutils.c
xsutils.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_EXTERNAL
_GLOB -DPERL_IS_MINIPERL -Fo.\mini\miniperlmain.obj ..\miniperlmain.c
miniperlmain.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_EXTERNAL
_GLOB -DPERL_IS_MINIPERL -Fo.\mini\perlio.obj ..\perlio.c
perlio.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_IS_MINIP
ERL -Fo.\mini\win32.obj win32.c
win32.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_IS_MINIP
ERL -Fo.\mini\win32io.obj win32io.c
win32io.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_IS_MINIP
ERL -Fo.\mini\win32sck.obj win32sck.c
win32sck.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_IS_MINIP
ERL -Fo.\mini\win32thread.obj win32thread.c
win32thread.c
        cl -c -nologo -GF -W3 -I..\lib\CORE -I.\include -I. -I.. -DWIN32 -D_CONS
OLE -DNO_STRICT -DPERLDLL -DPERL_CORE   -O1 -MD -Zi -DNDEBUG -GL -DPERL_IS_MINIP
ERL -Fo.\mini\fcrypt.obj fcrypt.c
fcrypt.c
        link -out:..\miniperl.exe @C:\WINDOWS\TEMP\nm1EC0.tmp
Generating code
Finished generating code
LINK : warning LNK4089: all references to 'WS2_32.dll' discarded by /OPT:REF
        if exist ..\miniperl.exe.manifest mt -nologo -manifest ..\miniperl.exe.m
anifest -outputresource:..\miniperl.exe;1 &&  if exist ..\miniperl.exe.manifest
del ..\miniperl.exe.manifest
        ..\miniperl.exe -I..\lib -f ..\write_buildcustomize.pl ..
        copy config.vc config.w32
        1 file(s) copied.
        ..\miniperl.exe -I..\lib config_sh.PL "INST_TOP=c:\perl"
         "INST_VER="                     "INST_ARCH="            "archname=MSWin
32-x86-multi-thread"                     "cc=cl"
 "ld=link"                               "ccflags=-nologo -GF -W3 -O1 -MD -Zi -D
NDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT  -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLIC
IT_CONTEXT -DPERL_IMPLICIT_SYS -D_USE_32BIT_TIME_T"      "usecplusplus="
         "cf_email="                     "d_mymalloc=undef"              "libs=o
ldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi3
2.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.l
ib winmm.lib  version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib"
                 "incpath=C:\Program Files\Microsoft Visual Studio .NET 2003\VC7
\include"                "libperl=perl523.lib"           "libpth=C:\Program File
s\Microsoft Visual Studio .NET 2003\VC7\lib;"    "libc=msvcrt.lib"
                 "make=nmake"                            "static_ext=Win32CORE"
         "usethreads=define"             "useithreads=define"            "usemul
tiplicity=define"                "use64bitint=undef"             "uselongdouble=
undef"                   "uselargefiles=define"  "usesitecustomize=undef"
 "LINK_FLAGS=-nologo -nodefaultlib -debug -opt:ref,icf -ltcg  -libpath:\"c:\perl
\lib\CORE\"  -machine:x86"               "optimize=-O1 -MD -Zi -DNDEBUG -GL"
         "WIN64=undef" config.w32 > ..\config.sh
Can't locate Config.pm in @INC (you may need to install the Config module) (@INC
 contains: C:\sources\cperl\lib C:\sources\cperl\cpan\AutoLoader\lib C:\sources\
cperl\dist\Carp\lib C:\sources\cperl\dist\PathTools C:\sources\cperl\dist\PathTo
ols\lib C:\sources\cperl\cpan\ExtUtils-Command\lib C:\sources\cperl\cpan\ExtUtil
s-Install\lib C:\sources\cperl\cpan\ExtUtils-MakeMaker\lib C:\sources\cperl\cpan
\ExtUtils-Manifest\lib C:\sources\cperl\cpan\File-Path\lib C:\sources\cperl\ext\
re C:\sources\cperl\dist\Term-ReadLine\lib C:\sources\cperl\dist\Exporter\lib C:
\sources\cperl\ext\File-Find\lib C:\sources\cperl\cpan\Text-Tabs\lib C:\sources\
cperl\dist\constant\lib C:\sources\cperl\cpan\Text-ParseWords\lib .) at FindExt.
pm line 7.
BEGIN failed--compilation aborted at FindExt.pm line 7.
Compilation failed in require at config_sh.PL line 3.
BEGIN failed--compilation aborted at config_sh.PL line 3.
NMAKE : fatal error U1077: '..\miniperl.exe' : return code '0x2'
Stop.

C:\sources\cperl\win32>

caused by 7c3330a

when 'config_sh.PL' runs during the Win32 build process, 'Config.pm' doesn't exist yet. static extensions list is passed on command line to 'config_sh.PL' from the d/n/gmake process. Static extension list can't be looked up from 'config.sh' since 'static_ext' from 'config.sh' would be the canned 'static_ext' from the canned 'config.sh' for miniperl which is always empty string when 'config_sh.PL' runs. I reverted 7c3330a in my repo so I can continue the Win32 build but some other solution will be needed.

New lexical pragma for exact or fast arith, promote to double or bigint? We probably need a PL_hints bit for this, more then for no magic or strict names.

Add a core pragma to select arithmetic overflow semantics.
Overflow to fast NV (double) as with perl5, or to exact bigint/num/rat as with perl6.

This disallowed using static strings in B::C since 5.18.
We really need static strings for the shared keys and filenames.
When done, we could eventually be even better than 5.14, storing more COW (now "real cow")
strings than before.

CowREFCNT: seperate from string, share with sv_refcnt

For read-only strings (i.e. embedded and with the compiler) and with threads
it makes no sense to share the volatile cowrefcnt with the read-only buffer of the
string. We cannot use strings as read-only, and cannot compile them into a
constant segment, we even need to them to be initialized dynamically. This is one
of the reasons why the compiler is not performant since 5.18 anymore

Maybe: Add PV* and STRUCT_PV as alternative view of an SV*, add PvREFCNT() to make
it clear. PVs are all structs starting with xpv, just not AV, HV, IO as they have no cowable strings.

Add stats for highest SvREFCNT and CowREFCNT with -DC at the end.

Result with the core testsuite:
Typically 820 refcnt (the "_>file" GV), 5 cow

Highest outliers:
t/re/uniprops.t
== max refcnt: 118480 max cowrefcnt: 255
t/porting/podcheck.t
== max refcnt: 6121 max cowrefcnt: 255

See https://github.com/perl11/cperl/commits/feature/CM-600-cperl-cowrefcnt

a major memory and performance win.

as native types in Perl6
new box_{int,num,str} and unbox_{int,num,str} ops, and all the arithmetic ops for int and num, and for const and padsv (padav for native arrays later).

Allow native int, double and ASCIIZ pointers on the stack, and provide optimized ops for those.
Maybe uni strings later.

Check in the compiler (op.c) for safe ranges to use unboxed sequences of ops, and box the stack values when escaping: sub calls, exceptions, jumps outside.

I needed to extend opcodes and opcode.pl to mark the ops with new arg types, change msgrcv and sockpair, because there's no room for 5 op args anymore (no, undone.)

type num (double) can only be used when IVSIZE==NVSIZE, i.e. 32bit with NV=float and 64bit with double. The compiler needs to emits different code when unboxed num is avail. or not.

Optimizer:
native types strength reduction:
Change ops with typed or const args into typed.
e.g. padsv[$a:int] const(iv) add => i_add

With more than 2 ops with unboxable args, maybe unbox it.
e.g. padsv[$a:int] const(iv) add padsv[$b:int] multiply
=>

 padsv[$a:int] const(iv) unbox[2] int_add
 padsv[$b:int] unbox int_multiply[b]

[b] meaning box result as in box_int

native ops can have OPpBOXRET and UNBOX_* can have the number of stack elements in op_private. Extend B::Concise to display those, and also display the types of pads, like
padsv[$a:int 63,64]

I had a major thinko btw. Those ops can only refer to other ops, never to data directly. The data ops are const and padsv. Either the native arith ops replace in the optimizer the padsv/const ops by the unboxed variants, where op_first/op_last points to an unboxed data (ptr or literal const?, ptr tagging cannot be used as we need all bits for the literal, which might end with 0b00)
so we need new types const_int, pad_int, ... which hold the unboxed data.

int_padsv holds the data in special curpad[] SVs with SVf_NATIVE. Or keep it there unboxed, but then we need to assure that all the padops referring to this targ are changed to be properly typed.

See feature/CM-713-cperl-native-types

In the fastarith codepath which is taken on the overflow doing the add or multiply as NV, the slow fallback branch misbehaves with 32bit IVs or -m32 IVSIZE==4, with overlarge negative IV's (>IV_MAX, < UV_MAX).

Add some uok checks and take the proper IV or UV value for the SETn() calculation

appveyor only supports Visual Studio 2014 and 2015, but perl5 only supports the older msvc versions.
mingw smoking failed so far on appveyor.

• backport the latest v5.23.2 fixes to support Visual Studio 2014 and 2015, but this is still buggy.
• find another windows smoker: win32 + win64, msvc + mingw

and make it easy to use.
i.e. offer to cp it to ~/.cpan/{prefs,sources/authors}

this would make it easier to fix affected modules on the fly. currently version and Variable::Magic.

cloning a restricted stash (e.g. a coretype) leads to "Attempt to access disallowed key 'CLONE' in the restricted hash '%main::UInt::'" error

gv_fetchmethod_autoload which is used to find a CLONE or CLONE_SKIP method is not usable for protected stashes. This is an upstream bug and not really related to threads. Just that only threads call a method in a restricted stash and fail currently.

For the time being use gv_fetchsv(name, GV_NOADD_NOINIT, SVt_PVCV) there instead to avoid
those errors. but try to fix it in gv_fetchmethod_autoload.

BTW: The new gv api is together with the lexicals warnings non-API a highlight of confusion and API horrors.

gv_fetchmethod_autoload is really a gv_fetchmethod_flags because it does not autoload, only with a true 3rd arg. gv_fetchmethod_flags and gv_fetchmethod_pv_flags are not generally usable because of the non-support of binary names, but ok internally for core.

gv_fetchmethod_* cannot deal with protected stashes, because it tries a destructive hash lookup in a restricted hash just to find a method name which causes a run-time error. so we have now to protect each gv_fetchmethod_autoload from protected stashes.

gv_fetchmethod_pvs_flags is my new helper for compile-time static strings.

TODO: With protected stashes and an existing AUTOLOAD method do not error and perform the AUTOLOAD. Without autoload just search for the method, but not GV_ADD, thus do not error.

Nicolas found a bug in base v2.20 added with 892089c

utf8::encode on the filename removed the utf8 flag from the cow'ed package names as bad sideeffect.
we only found it out via double encoding in the compiler, with utf8 pkgnames, which was in the end only a B::C bug in precomp setting a wrong SVf_UTF8 flag.

But an underlying bug is that Perl_sv_utf8_encode forgets to uncow a string. Need to perlbug this fix and come up with a testcase.

Perl_sv_utf8_upgrade_flags_grow uncows it in most cases, just not in our, when we pass in a normal UTF8 string, then it goes to

    if (SvUTF8(sv)) {
    return SvCUR(sv);
    }

and does not uncow it for us.

We saw something like this:

$ p -e'$fn="ƂƂƂƂ.pm";utf8::encode($fn);require $fn;'
Can't locate ����.pm in @INC (you may need to install the ���� module)
$ p -e'use utf8; $fn="ƂƂƂƂ.pm";utf8::encode($fn);require $fn;'
Can't locate ƂƂƂƂ.pm in @INC (you may need to install the ƂƂƂƂ module)

Replace CvFILE by GvFILE, where possible.
http://www.nntp.perl.org/group/perl.perl5.porters/2001/05/msg36781.html

Benefit: memory savings. lots of unneeded static strings, some of them even dynamic.

Cost: We need the files in B::C to decide where to split modules. GvFILE should be enough though.
The only problems are: lexical subs, imported functions and anon subs.

For imports: they will have the wrong filename then, the one from the original definition, not the one where the import was done. This is not concern I think
bulk88 came up with a really good patch, I will adopt, I think.

Parse them into the existing structures.
See https://design.perl6.org/S12.html#multi_Declarations

perl6 creates a longname of the sig types and aliases them to the combined shortname, the proto (aka generic). Their multiple dispatch is the global shortest distance over all argument types, as in Damian's Class::MultiMethods http://cpansearch.perl.org/src/DCONWAY/Class-Multimethods-1.70/tutorial.html with an error if there are equally ranked candidates.
perl6 allows multi subs and methods, plus proto and only prefix keywords. (similar to extern/native). sub is even optional, so they favor multi subs over methods.

For the beginning with cperl only allow it within classes, and favor methods over subs:
allow method only within class {} blocks, to allow stricter and better semantics.
allow sub class{} and class() sub decl and calls, which e.g. is used in B
allow multi (aka generics) outside of class {} blocks.
allow $class: invocant syntax
pass $self onto the signature of methods automatically, but not with :method attributes, only method declarations.
add has fields and methods accessor methods
re-implement pseudo-hashes for the internal object layout for the default BUILD method for class->new objects.
method-dispatch via global shortest distance, as in Class::MultiMethods, seems to be too much overhead and counterintuitive. much better would be a trivial left-to-right shortest distance (via c3 of course) without backtracking. This can be optimized to simple dynamic decision trees embedded into the method op, as in most other efficient method dispatch implementations. (see e.g "Efficient Method Dispatch in PCL", Kiczales, Rodriguez 1990", or the Cecil, profile guided Self-93 or Squeak dispatcher)
classes can have the :native attribute, to layout the hash fields as in C, to be used as native structs for the ffi.

fail at 'Invalid method call on class subroutine' and vice versa 'Invalid subroutine call on class method'.
warn on redefining packages as classes, as classes are more restrictive.
when redefining classes as packages it will fail at the readonly stash.

See the branch feature/gh16-multi

backport 3 commits from upstream starting at 3bdc51a
d89ea36
c2c7bda

This fixes many platform and build issue quirks, and also enables someone to eventually upload it to CPAN.
bulk88 did this, thanks!

Out of my head only pp_sort relies on that obscure schema.
We have now a nice S_op_next_nn() which returns the next non-null op.

Don't punish run-time for an inefficient compiler.

See rurban/perl-compiler#214 where we still use it in B::C, but only for DESTROY methods which are known at compile-time.
And run-time added DESTROY method will be skipped.

Rationale: The PL_sv_objcount++ in each bless is much cheaper than running global destruction for simple scripts without any blessed objects (disregarding internal IO Filehandle objects). Doing the DESTROY callback on every SV object in sv_clean_objs in global destruction is very expensive, one of the most expensive tasks even. Being able to skip it is 10x faster.

Relevant commits:

commit 8dab3ba5fe05d49fe039bc05012fdab8bc572a5c
Author: Karl Williamson <[email protected]>
Date:   Tue Apr 8 18:46:25 2014 -0600

    Remove deprecated 'PL_sv_objcount'

    This was scheduled to be removed in 5.20, but was forgotten.

commit d7a08570f1b66c1947fbca76061dfa2f85d0ade0
Author: Steffen Mueller <[email protected]>
Date:   Thu Feb 28 18:25:19 2013 +0100

    PL_sv_objcount deprecation notice

commit ddf23d4a1ae98daa8608179f83aad311d03898ef
Author: Steffen Mueller <[email protected]>
Date:   Thu Feb 28 18:23:32 2013 +0100

    Prepare PL_sv_objcount removal

    This used to keep track of all objects. At least by now, that is
    for no particularly good reason. Just because it could avoid a
    bit of work during global destruction if no objects remained.
    Let's do less work at run-time instead.

    The interpreter global will remain for one deprecation cycle.

GV is the only major type which cannot be declared static for the compiler (besides op_ppaddr and AV,HV,CV arrays), thus leads to startup-time delays, 90% of the startup-time is spent in initializing GVs and aliases, ie. imports. improving AvARRAY init led to a substantial (>5%) win.

The body has seperate GvFLAGS so's there room for GvSTATIC.
Fix the destruction code to allow static names.

Fix one SEGV in bugfix/CM-891-glob_assign_glob-crash

See branch feature/gh11-static-gv
Est. 2w

... in sigs for the stack pointer as in other languages, and not as in perl5+6 for empty bodies only.
this is better than passing @_ through, which needs to copy the args. ... just passes the mark stack pointer through

... as empty function body already has a special meaning as yadayada
operator, just croaking, but interestingly not the usual meaning of varargs.

cperl uses ... in the natural way to denote unnamed and uncopied
rest args, and passes the varargs through to the next call.

... denotes a slurpy unnamed signature, and ... in a otherwise
non-empy function body denotes passing those arguments efficiently to
the next function. Internally ... does not need to copy the values
into a temporary array, we just need to pass the argument stack
position down to the next calls using .... By using ... instead
of @_ we can avoid copying the values to @_, we only need the stack
index, not all the values.

sub foometh($self, ...) { func(...) }