exodist / child Goto Github PK

I'm trying to test perl5i with AppVeyor, a Windows CI service. Unfortunately, Child is failing somewhere in its tests.

https://ci.appveyor.com/project/schwern/perl5i/build/1#L105

--> Working on Child
Fetching http://www.cpan.org/authors/id/E/EX/EXODIST/Child-0.012.tar.gz ... OK
Configuring Child-0.012 ... OK
Building and testing Child-0.012 ... ! Installing Child failed. See C:\Users\appveyor.cpanm\work\1465754302.2520\build.log for details. Retry with --force to force install it.
FAIL

This is going to be hard to debug from the perl5i side. If Child were using AppVeyor it would be easier to work out, and it would be CI tested on Windows.

A sample Perl module config for AppVeyor can be found here. http://blogs.perl.org/users/eserte/2016/04/testing-with-appveyor.html

I just started using Child, so please bear with me if i'm wrong...

I'm planning to use Child on a long-running process (months) who's job it is to spawn dozens to hundreds of children per second, each of which will only be short-lived (maybe 10s). I intended to call $proc->wait every few seconds on all processed returned by Child->all_procs() which have is_complete set - so that all zombies are killed. However it turns out that these processes will show up again in the next round, because entries in @Child::PROCS is always pushed to, but never removed from.

For me, the expected behavior would be that once a child has been wait()ed for, it no longer shows up hen the all_procs list...

Do you agree, or did I misunderstand the intended use of Child?

With a max files ulimit of 256, after roughly 110 children are executed, pipes start failing to be created:

Starting job 129
Got data back from child: my pid is 26033
print() on unopened filehandle $aout at /usr/share/perl5/Child/Link/IPC.pm line 61.
Got data back from child: my pid is 26034
Starting job 130
readline() on unopened filehandle $ain at /usr/share/perl5/Child/Link/IPC.pm line 50.
Use of uninitialized value $data in concatenation (.) or string at runner.pl line 27.
Got data back from child: 

And an strace will start showing output like:

pipe(0x3b56c69dc30)                     = -1 EMFILE (Too many open files)
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x2d1ac5d79d0) = 25675

To use the below proof of concept, you'll need to launch runner.sh on any Linux system. POC likely also works on OS X, but not tested there.

runner.sh:

#!/bin/bash
ulimit -n 256
exec perl runner.pl

runner.pl:

#!/usr/bin/env perl
use strict;
use warnings;
use Parallel::Runner;
use IO::Handle;

STDOUT->autoflush(1);
STDERR->autoflush(1);

my $Runner = Parallel::Runner->new(
    50,
    data_callback => \&runner_data_callback,
);
for my $i (1..1000) {
    print "Starting job $i\n";
    $Runner->run(sub { return "my pid is $$" });
}
$Runner->finish;

sub runner_data_callback {
    my ($data) = @_;
    print "Got data back from child: $data\n";
}

I can write the .travis.yml, but I think only you can flip the switch on https://travis-ci.org/

No idea what's going on, this is with Strawberry 5.10.1 using 0.007 after I applied the Carp patch.

t\Child.t ... 1/? wait returned -1500: No such process -1500 at t\Child.t line 29
# Tests were run but no plan was declared and done_testing() was not seen.
t\Child.t ... Dubious, test returned 9 (wstat 2304, 0x900)
All 5 subtests passed
t\Import.t .. ok
t\Manage.t .. 1/? wait returned -1492: No such process -1492 at C:\cygwin\home\Administrator\src\Child\blib\lib/Child.pm line 32
# Tests were run but no plan was declared and done_testing() was not seen.
t\Manage.t .. Dubious, test returned 9 (wstat 2304, 0x900)
All 9 subtests passed

20:54 mst something I would really like in Child.pm
20:54 mst or as an extension
20:54 mst "kill this child -and- anything it spawned"
20:54 mst ala the shell's response to a hangup
20:56 Exodist oooh.. thats a good idea
20:56 Exodist can you add a github ticket for it? I do not have time in the
immediate couple days

typo: Parrallel::Runner instead of Parallel::Runner in the History section

This fairly consistently shows up while running a full prove suite:

Use of uninitialized value $_ in numeric ne (!=) at Child.pm line 30 during global destruction.

use Child;
say "start: $$";
eval { $c = Child->new(sub { die "Foo\n" }); $proc = $c->start; $proc->wait; };
say "end: $$"'
__END__
start: 58747
end: 58748
end: 58747

Child->start does this...

$code->( $parent );
exit;

Normally if $code dies then the child dies. But if $child is called in an eval block, control will jump back to the eval bock and the exit will never be executed.