exodist / child Goto Github PK
View Code? Open in Web Editor NEW(perl) Object oriented simple interface to fork()
(perl) Object oriented simple interface to fork()
I'm trying to test perl5i with AppVeyor, a Windows CI service. Unfortunately, Child is failing somewhere in its tests.
https://ci.appveyor.com/project/schwern/perl5i/build/1#L105
--> Working on Child
Fetching http://www.cpan.org/authors/id/E/EX/EXODIST/Child-0.012.tar.gz ... OK
Configuring Child-0.012 ... OK
Building and testing Child-0.012 ... ! Installing Child failed. See C:\Users\appveyor.cpanm\work\1465754302.2520\build.log for details. Retry with --force to force install it.
FAIL
This is going to be hard to debug from the perl5i side. If Child were using AppVeyor it would be easier to work out, and it would be CI tested on Windows.
A sample Perl module config for AppVeyor can be found here. http://blogs.perl.org/users/eserte/2016/04/testing-with-appveyor.html
I just started using Child, so please bear with me if i'm wrong...
I'm planning to use Child on a long-running process (months) who's job it is to spawn dozens to hundreds of children per second, each of which will only be short-lived (maybe 10s). I intended to call $proc->wait
every few seconds on all processed returned by Child->all_procs()
which have is_complete
set - so that all zombies are killed. However it turns out that these processes will show up again in the next round, because entries in @Child::PROCS
is always pushed to, but never removed from.
For me, the expected behavior would be that once a child has been wait()
ed for, it no longer shows up hen the all_procs
list...
Do you agree, or did I misunderstand the intended use of Child?
With a max files ulimit of 256, after roughly 110 children are executed, pipes start failing to be created:
Starting job 129
Got data back from child: my pid is 26033
print() on unopened filehandle $aout at /usr/share/perl5/Child/Link/IPC.pm line 61.
Got data back from child: my pid is 26034
Starting job 130
readline() on unopened filehandle $ain at /usr/share/perl5/Child/Link/IPC.pm line 50.
Use of uninitialized value $data in concatenation (.) or string at runner.pl line 27.
Got data back from child:
And an strace will start showing output like:
pipe(0x3b56c69dc30) = -1 EMFILE (Too many open files)
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x2d1ac5d79d0) = 25675
To use the below proof of concept, you'll need to launch runner.sh on any Linux system. POC likely also works on OS X, but not tested there.
runner.sh:
#!/bin/bash
ulimit -n 256
exec perl runner.pl
runner.pl:
#!/usr/bin/env perl
use strict;
use warnings;
use Parallel::Runner;
use IO::Handle;
STDOUT->autoflush(1);
STDERR->autoflush(1);
my $Runner = Parallel::Runner->new(
50,
data_callback => \&runner_data_callback,
);
for my $i (1..1000) {
print "Starting job $i\n";
$Runner->run(sub { return "my pid is $$" });
}
$Runner->finish;
sub runner_data_callback {
my ($data) = @_;
print "Got data back from child: $data\n";
}
I can write the .travis.yml
, but I think only you can flip the switch on https://travis-ci.org/
No idea what's going on, this is with Strawberry 5.10.1 using 0.007 after I applied the Carp patch.
t\Child.t ... 1/? wait returned -1500: No such process -1500 at t\Child.t line 29
# Tests were run but no plan was declared and done_testing() was not seen.
t\Child.t ... Dubious, test returned 9 (wstat 2304, 0x900)
All 5 subtests passed
t\Import.t .. ok
t\Manage.t .. 1/? wait returned -1492: No such process -1492 at C:\cygwin\home\Administrator\src\Child\blib\lib/Child.pm line 32
# Tests were run but no plan was declared and done_testing() was not seen.
t\Manage.t .. Dubious, test returned 9 (wstat 2304, 0x900)
All 9 subtests passed
20:54 mst something I would really like in Child.pm
20:54 mst or as an extension
20:54 mst "kill this child -and- anything it spawned"
20:54 mst ala the shell's response to a hangup
20:56 Exodist oooh.. thats a good idea
20:56 Exodist can you add a github ticket for it? I do not have time in the
immediate couple days
typo: Parrallel::Runner instead of Parallel::Runner in the History section
This fairly consistently shows up while running a full prove suite:
Use of uninitialized value $_ in numeric ne (!=) at Child.pm line 30 during global destruction.
use Child;
say "start: $$";
eval { $c = Child->new(sub { die "Foo\n" }); $proc = $c->start; $proc->wait; };
say "end: $$"'
__END__
start: 58747
end: 58748
end: 58747
Child->start
does this...
$code->( $parent );
exit;
Normally if $code
dies then the child dies. But if $child
is called in an eval block, control will jump back to the eval bock and the exit
will never be executed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.