Giter Club home page Giter Club logo

parallel's Introduction

parallel

Linux AddressSanitizer Windows Coverage Status

A succinct parallel concurrency API for PHP 8

GoFundMe

Documentation

Documentation can be found in the PHP manual: https://php.net/parallel

Requirements and Installation

See INSTALL.md

Hello World

<?php
$runtime = new \parallel\Runtime();

$future = $runtime->run(function(){
    for ($i = 0; $i < 500; $i++)
        echo "*";

    return "easy";
});

for ($i = 0; $i < 500; $i++) {
    echo ".";
}

printf("\nUsing \\parallel\\Runtime is %s\n", $future->value());

This may output something like (output abbreviated):

.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*
Using \parallel\Runtime is easy

Development

See CONTRIBUTING.md for guidelines on contribution and development (and debugging).

parallel's People

Contributors

cmb69 avatar dixyes avatar dktapps avatar dotfry avatar flavioheleno avatar jellynoone avatar krakjoe avatar marsonge avatar realflowcontrol avatar remicollet avatar sanmai avatar tsoftware-org avatar z3niths avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

parallel's Issues

PECL

Hi @remicollet, can you start the ball rolling on PECL things please ?

I need to rest now, but I'll do a bunch more testing tomorrow, coverage is at 97%~ which is good enough for me, I can't possibly test all error conditions when it comes to threads because you cannot force them to occur.

Ta, and thanks for help earlier :)

Objects

I've started to work on this, in a branch ... I've come up with a way to copy objects that works safely for pure user objects.

I was going to try to introduce support for internal, or internal derived objects, however, for this to work properly, we need an additional internal handler for objects; Should parallel ever be merged, we can talk about adding that and implementing the necessary handlers for all internal objects.

We could, theoretically abuse the clone object handler, however, some objects do not implement this properly now, and none of them are really prepared for it to be executed as we need to execute it, so it would probably just introduce more crashes than anything.

Any thoughts are welcome ...

Usefulness of channel naming

It seems to me that channel naming is no longer useful, thanks to the ability to pass channels directly between threads with the newly implemented object support. I'm now finding myself needing ways to uniquely identify channels, which is starting to involve things like randomly generated UUIDs.

It seems to me that channel names could be dropped, along with Channel::make() and Channel::open().

Killing

We need a way to kill a thread immediately, ignoring the remainder of the stack.

Co-Op

I've implemented the beginnings of coop ...

Notes:

Co-Op in parallel allows you to effectively achieve 1:N threading, by yielding to the scheduler and allowing it to run any waiting tasks on the stack, optionally re-scheduling the current task.

I've chosen the method name yield, but it may not be a good name: yield in PHP means a very specific thing today, and yielding to the scheduler is somewhat different ...

This should probably be marked experimental in the first release that includes it ...

TODO

Important:

  • native frames
  • yield at instruction (closer to native generators)
  • yielding values
  • Cleanup code

Less important:

  • Discover how well supported pthread_yield is, and how practical it is to use it

Incompatibility with timecop extension

When timecop is loaded:

at the end of execution... for each thread:

Warning: Unknown: timecop couldn't find method datetime::timecop_orig___construct. in Unknown on line 0

Warning: Unknown: timecop couldn't find method datetime::timecop_orig_createfromformat. in Unknown on line 0

Warning: Unknown: timecop couldn't find method datetimeimmutable::timecop_orig___construct. in Unknown on line 0

Warning: Unknown: timecop couldn't find method datetimeimmutable::timecop_orig_createfromformat. in Unknown on line 0

And the, the process hangs

Channels send/recv with timeout ?

In #21 I said that I wanted recv/send to have a timeout parameter ... I changed my mind, I think.

In go, timeouts are only handled by select {}, and in parallel they can be handled by Events in the same sort of way ... it seems superfluous to have API's that do the same thing ...

Can anyone think of a use case that could not be implemented with Events ?

/cc @everyone #26

Constant values of Events\Event\Type

So I'm working on PHPStorm stubs for this extension. Now neither can I find nor get the values from my docker image with ext-parallel in it for those constants. (Will look into that later.) I did find the following header file: https://github.com/krakjoe/parallel/blob/cee25ecef92904bbaeebbfff3fb098fb3d755473/src/event.h

So I'm assuming that the values for the constants are even though the enum only specifies the first:

final class Type
{
    public const Read = 1;
    public const Write = 2;
    public const Close = 3;
    public const Cancel = 4;
    public const Kill = 5;
    public const Error = 6;
}

Is that correct?

Getting value of a finished Future

Tryingt to write some example, implementing a -j# like option

Is it possible to run a $f->value, from the $done array returned by a select ?

See example:

<?php
if (!extension_loaded("parallel")) {
	die("parallel extension required\n");
}

$runtime = new \parallel\Runtime();
$tasks = [];
for ($i=1 ; $i < 10 ; ) {

	if (count($tasks) < 4) { // Never launch more than 4 workers
		$workers[$i] = new \parallel\Runtime();
		$tasks[$i] = $workers[$i]->run(function($i){
			$t = rand(1,4);
			sleep($t);
			return "$i done after $t\"\n";
		}, [$i]);
		echo "$i launched\n";
		var_dump($tasks[$i]);
		$i++;

	} else { 				// Wait 1 worker to end
		echo "select: ";
		$done = $err = [];
		if (\parallel\Future::select($tasks, $done, $err)) {
			printf("%d done, %d errored, %d still running\n", count($done), count($err), count($tasks));

			foreach($done as $task) {
				var_dump($task);
				//echo $task->value();
			}

		}
	}
}

// wait all remaining worker
while (\parallel\Future::select($tasks, $done, $err)) {
	printf("%d done, %d errored, %d still running\n", count($done), count($err), count($tasks));
	foreach($done as $task) {
		var_dump($task);
		//echo $task->value();
	}
}

Is seems the call to value() hangs...

Documentation

I've started documentation in the manual, it needs more work and examples added. We can clean up the readme now also.

@remicollet do you want to do the readme changes with install info as normal please ?

I'll close this when I'm happy that there's enough examples in the manual ...

select clarification

See #10 (comment) for reproducer

Working on some tests / examples, I notice select always wait for the first task in the array to finish, while we may expect to wait for any task which finish first.

Using linked example

Output:

+ 0 started
+ 1 started
+ 2 started
+ 0 ended   <===== why ?
+ 3 started
+ 1 ended
+ 4 started
Done

+ 2 ended
+ 3 ended
+ 4 ended

Expected:

+ 0 started
+ 1 started
+ 2 started
+ 2 ended
+ 3 started
+ 1 ended
+ 4 started
Done

+ 3 ended
+ 4 ended
+ 0 ended <=== Task 0 is the longer, so should terminate last.

Can you please confirm if this is expected, or an issue ?

Release 0.9.0

Upcoming release contains:

  • Channels: bi-directional communication between tasks and runtimes
  • Improvements to copying making it vastly more efficient
  • Improvements to runtime stack, making it more efficient
  • Tasks may use lexical scope
  • Events - an experimental, quite primitive event loop for reading/writing sets of channels and futures
  • Support for rethrowing exceptions uncaught in tasks
  • Simplified Runtime constructor
  • Remove Future::select (in favour of the superior events interface)
  • Task cancellation
  • Closure support

Will assign this to you @remicollet when I think ready ... ta ...

Anyone got any last minute thoughts before this next release happens ?

Release v1.0.2

  • Fix #45 segfault on new Channel
  • Fix #46 scheduler is leaking function allocated in frame

Behavioural inconsistency with normal classes on wrong argument types

Normal typehinted methods will throw \TypeError when wrong types are passed to them. parallel is currently throwing its own baked exception, which seems strange to me. It seems like it would be less work to use ZEND_PARSE_PARAMS_THROW instead? Am I missing some obvious reason why this is different?

Not using future blocks parent thread

If you don't take a reference to Future, the parent thread will be blocked while the child runs because the Future is implicitly created and destroyed and in order to destroy it, it must wait for a value.

Solve this somehow ... nicely ...

Release v1.0.1

  • Fix #42 \parallel\run will choose wrong thread for scheduling
  • Fix #41 buffered channel could not be drained after close
  • Fix #40 class in FQN stops stubs being generated

Crash on `new Channel()`

$channel = new Channel();

causes a segfault.

This was auto-completed for me by an IDE using stubs I generated from the extension. It would probably be wise to mark the Channel constructor as private so this problem can't occur.

Bug in literals copying

Bug in copying literals may lead to corrupted heap.

For some reason I wrote this really strange, spotted errors during review ... the nature of this bug means it's somewhere between hard and impossible to write a test for it, it's definitely there though, I see it very clearly with fresh eyes.

Question: Anyone to share any documented example please?

I'm using Golang whenever I need concurrency and parallelism, but I would like to stick with PHP for achieving either the same results or the 85% the least of actual performance if possible.

Is there any project / example documented somewhere so I can read about it and teach myself how to use his extension extensively?

Release v1.0.0

WIP to make changelog for next release ...

  • Caching improvements
  • Channel comparison (==) fixed
  • Debug handler added for Channel
  • Debug handler for Future (just shows runtime)
  • Fix copying of interned strings
  • Object support
  • Functional API for parallel\Runtime

Will assign to you when ready @remicollet :)

Blocking when reusing or unsetting variable for Runtime

Script to replicate:

<?php

use parallel\{Runtime, Future};

$closure = function($n)
{
	$x = 0;
	for ($i = 0; $i < 99999999 - $n * 10000; $i++)
		$x++;
	
	echo "done {$n}\n";

	return [$n, $x];
};

$option = @$argv[1] ?? '0';

echo "Running option {$option}\n\n";

$timer = microtime(true);

$runtimes = [];
$futures = [];
foreach (range(1, 10) as $n)
{
	if ($option === '0')
	{
		$runtime = new Runtime();
		$futures[] = $runtime->run($closure, [$n]);
	}
	elseif ($option === '1')
	{
		$runtime = new Runtime();
		$futures[] = $runtime->run($closure, [$n]);
		unset($runtime);
	}
	elseif ($option === '2')
	{
		$runtimes[$n] = new Runtime();
		$futures[] = $runtimes[$n]->run($closure, [$n]);
	}
}

$count = count($futures);
while ($count > 0)
{
	$resolved = [];
	$errored = [];
	$count -= Future::select($futures, $resolved, $errored);
	
	foreach ($resolved as $future)
	{
		$a = $future->value();

		echo "{$a[0]} -> {$a[1]}\n";
	}
}

echo "\n", number_format(1000 * (microtime(true) - $timer)), " ms\n";
echo "Parallel version ", phpversion('parallel'), "\n";

When run with option 0, it's not run in parallel, with one child process doing the work, and the next one sleeping/waiting:

Running option 0

1 ms to init run 1
done 1
1,894 ms to init run 2
done 2
1,889 ms to init run 3
done 3
1,897 ms to init run 4
done 4
1,889 ms to init run 5
done 5
1,896 ms to init run 6
done 6
1,907 ms to init run 7
done 7
1,891 ms to init run 8
done 8
1,924 ms to init run 9
done 9
1,885 ms to init run 10
1 -> 99989999
2 -> 99979999
3 -> 99969999
4 -> 99959999
5 -> 99949999
6 -> 99939999
7 -> 99929999
8 -> 99919999
9 -> 99909999
done 10
10 -> 99899999

18,966 ms
Parallel version 0.8.3

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
    1 root      20   0   18132   3176   2692 S  0.0  0.0   0:00.04 bash
  413 root      20   0   41040   3240   2760 R  0.0  0.0   0:00.81  `- top
  800 root      20   0  332936  24284  18652 S  0.0  0.1   0:00.03 php
  812 root      20   0  332936  24284  18652 R 51.0  0.1   0:01.53  `- php
  813 root      20   0  332936  24284  18652 S  0.0  0.1   0:00.00  `- php

When run with option 1 (unsetting $runtime), same as above, but no longer shows a waiting process:

Running option 1

done 1
1,895 ms to init run 1
done 2
1,929 ms to init run 2
done 3
1,885 ms to init run 3
done 4
1,892 ms to init run 4
done 5
1,887 ms to init run 5
done 6
1,887 ms to init run 6
done 7
1,922 ms to init run 7
done 8
1,910 ms to init run 8
done 9
1,889 ms to init run 9
done 10
1,898 ms to init run 10
1 -> 99989999
2 -> 99979999
3 -> 99969999
4 -> 99959999
5 -> 99949999
6 -> 99939999
7 -> 99929999
8 -> 99919999
9 -> 99909999
10 -> 99899999

18,994 ms
Parallel version 0.8.3

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
    1 root      20   0   18132   3308   2824 S  0.0  0.0   0:00.03 bash
  409 root      20   0   41040   3096   2616 R  0.0  0.0   0:00.12  `- top
  517 root      20   0  256964  24184  18960 S  0.0  0.1   0:00.03 php
  529 root      20   0  256964  24184  18960 R 42.3  0.1   0:01.27  `- php

When run with option 2 (storing all the runtimes in an array), works as expected in parallel and much faster:

Running option 2

1 ms to init run 1
0 ms to init run 2
0 ms to init run 3
0 ms to init run 4
1 ms to init run 5
1 ms to init run 6
6 ms to init run 7
4 ms to init run 8
8 ms to init run 9
16 ms to init run 10
done 6
done 5
done 2
done 10
done 4
done 9
done 7
done 8
done 3
done 1
1 -> 99989999
2 -> 99979999
3 -> 99969999
4 -> 99959999
5 -> 99949999
6 -> 99939999
7 -> 99929999
8 -> 99919999
9 -> 99909999
10 -> 99899999

4,615 ms
Parallel version 0.8.3

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
    1 root      20   0   18132   3176   2692 S  0.0  0.0   0:00.04 bash
  413 root      20   0   41040   3240   2760 R  0.0  0.0   0:00.88  `- top
  849 root      20   0  940716  25836  18436 S  0.0  0.1   0:00.02 php
  856 root      20   0  940716  25836  18436 R 79.1  0.1   0:02.55  `- php
  857 root      20   0  940716  25836  18436 R 69.8  0.1   0:02.35  `- php
  858 root      20   0  940716  25836  18436 R 69.4  0.1   0:02.36  `- php
  859 root      20   0  940716  25836  18436 R 78.1  0.1   0:02.66  `- php
  860 root      20   0  940716  25836  18436 R 85.4  0.1   0:02.83  `- php
  861 root      20   0  940716  25836  18436 R 86.7  0.1   0:02.87  `- php
  862 root      20   0  940716  25836  18436 R 74.4  0.1   0:02.48  `- php
  863 root      20   0  940716  25836  18436 R 86.4  0.1   0:02.90  `- php
  864 root      20   0  940716  25836  18436 R 71.8  0.1   0:02.33  `- php
  865 root      20   0  940716  25836  18436 R 81.1  0.1   0:02.65  `- php

I'm assuming that this is because $runtime is a type of resource, and it needs to wait for it to finish when it it replaced or unset? At first this was very confusing and I was wondering why my code wasn't running more than one thread.

If this is the case, perhaps an E_NOTICE could be thrown when trying to replace a variable holding a running parallel\Runtime. That is, unless you could find a solution for this and not have to depend on the variable staying.

Or perhaps when on __unset() on a Runtime it should kill it like $runtime->kill()?

What do the Future|Runtime errors extend

Do they extend \Error or \Exception. Been trying to figure that out while reading ext-parallel/php C code but I'm not entire sure what the difference is zend_ce_error_exception and zend_ce_exception and zend_ce_error. My guess is that zend_ce_throwable maps to \Throwable, zend_ce_error to \Error. But I'm unsure about zend_ce_error_exception since it mentions both error and exception.

SAPI support?

Hi @krakjoe

Thank you for writing parallel! I just can't wait to try it.

pthreads v3 is restricted to operating in CLI only... So I'm promoting the advice to hard and fast fact: you can't use pthreads safely and sensibly anywhere but CLI.

and is parallel support SAPI (php-fpm)?

Thank you.

\parallel\Runtime\Class\Unavailable unusable

use parallel\Runtime\Class\Unavailable;

$thing = null;
if($thing instanceof Unavailable){
    var_dump("hi");
}else{
    var_dump("nope");
}
Parse error: syntax error, unexpected 'Class' (T_CLASS), expecting identifier (T_STRING) or '{'

It seems PHP isn't a fan of Class in the class FQN.

Trying to get get multiple values from a channel

Loving the channels so far, and been trying to get values from them non-blocking in the main thread. But for some reason I only get the first value. Any points on what I'm missing/doing wrong?

<?php

require 'vendor/autoload.php';

use \parallel\Runtime;
use \parallel\Channel;
use \parallel\Future;
use \parallel\Events;
use \parallel\Events\Event;
use \parallel\Events\Input;
use \parallel\Events\Timeout;
use React\EventLoop\Factory;

$parallels = [];
$parallels[] = new \parallel\Runtime();
//$parallels[] = new \parallel\Runtime();
//$parallels[] = new \parallel\Runtime();
//$parallels[] = new \parallel\Runtime();
$channel = Channel::make("channel", Channel::Infinite);
$channelTwo = Channel::make("channelTwo", Channel::Infinite);
$input = new Input();
$input->add("channelTwo", $channelTwo);
$events = new Events();
$events->setTimeout(1);
$events->setInput($input);
$events->addChannel($channel);

$f = function($channel, $channelTwo){
    $channel = Channel::open($channel);
    $channelTwo = Channel::open($channelTwo);

    while ($int = $channelTwo->recv()) {
//        sleep(1);
        echo PHP_EOL . $int . PHP_EOL;
        $channel->send(($int * 13));
    }

    echo 'true', PHP_EOL;
    return true;
};

$futures = [];
foreach ($parallels as $parallel) {
    $futures[] = $parallel->run($f, [(string)$channel, (string)$channelTwo]);
}

$loop = Factory::create();


$loop->futureTick(function () use ($channelTwo) {
    foreach (range(1, 10) as $i) {
        $channelTwo->send($i);
    }
});

// Call 1K times per second
$loop->addPeriodicTimer(0.001, function () use ($events, $channel, $loop, &$timer) {
    try {
        while ($event = $events->poll()) {
//        foreach ($events as $event) {
            //    echo microtime(true), 'tick', PHP_EOL;

            if (!($event instanceof Event)) {
                return;
            }

            echo microtime(true) . "event:" . var_export($event, true) . "\n";

            switch ($event->type) {
                case Event::Read:
//                    echo microtime(true), 'read:' . var_export($event->object->recv(), true) . "\n";
                    echo microtime(true) .  'read:' . var_export($event->value, true) . "\n";
                    //            }

                    //            if ($event->object instanceof Channel &&
                    //                $event->value == "input") {
                    //                echo microtime(true), "OK:Channel\n";
                    //            }
                    break;

                case Event::Write:
                    echo microtime(true) . 'write', PHP_EOL;
                    $events->addChannel($channel);
                    break;
            }
        }
    } catch (Timeout $timeout) {
        echo microtime(true) . 'timeout' . PHP_EOL;
        return;
    } catch (Throwable $throwable) {
        echo (string)$throwable;
    }

});

// Terminate on SIGINT
$loop->addSignal(SIGINT, function () use ($parallels, $loop, $channel, $channelTwo) {
    foreach ($parallels as $parallel) {
        $parallel->close();
    }
    $loop->stop();

    // There might be some cleanup issues here with the channels
    $channel->close();
    $channelTwo->close();
});

$loop->run();

Can be better simplified.

Instead of returning a future return null. Delete the future implementation and just take a Closure as an extra optional parameter for what to do in the event of the callback finishing.

Differentiate between return void and return value in parallel closures

<?php
$channel = parallel\Channel::make('http');
$runtime = new \parallel\Runtime();

$runtime->run(function ($string) {
    if (!$string) {
        return;
    }

    $channel = \parallel\Channel::open('http');
    $channel->send(strlen($string));
});

Throws the error: Uncaught Error: return on line 2 of entry point ignored by caller, caller must retain reference to Future.

I think this should only happen if return <value> is used, notably return null; return 0; return $value;

Not sure if its possible to detect this but maybe the rules for the exception could be:

  • don't throw exception if there are 0 return statements
  • don't throw exception if there are $n return statements and all of them return a value
  • don't throw exception if there are $n return statements and none of them return a value
  • throw exception if there are $n return statements returning a value and $m returning no value.

This is a convenience that affects channel based code more than the "standard" use-case returning futures. In case of a channel used for communication, the return value becomes uninteresting to the caller.

Unable to recv remaining contents of buffered channel after closing

$chan = Channel::make("hi", 10001);
for($i = 0; $i < 10000; ++$i){
    $chan->send($i);
}
$chan->close();
var_dump($chan->recv());

The recv() call will throw an exception even though there are values in the buffer to read.

In golang, close()ing a buffered channel only prevents sending further values into it, it does not prevent reading ("draining"). I expected to be able to drain a buffered channel before encountering exceptions. https://play.golang.org/p/ot87ro27tFk

A way to get thread ID

zend_thread_id() isn't available unless PHP was compiled in debug mode, so it isn't suitable for most practical use cases.

It would be nice to have API similar to pthreads' Thread::getCurrentThreadId() (and perhaps Thread->getThreadId()).

Pass environment/context variables to runtime

When you want to pass on environment information from the main thread to the parallel runtime, then this is currently only possible as arguments to run().

What if we could pass environment/context variables to the runtime? This would help for instrumentation (my use case) but in general any thing where the parent needs to pass more high level data.

$parallel = new \parallel\Runtime();
$parallel->addEnvironment('FOO', 'BAR');
$parallel->run(function () { var_dump($_SERVER['FOO']); });
// string(3) "BAR"

Current workaround is putenv() with all its problems :-)

Release v0.8.3

Changelog

  • Fix potential race condition for state on Future::value (where an exception was thrown)
  • Better compatibility with JIT

I think this can be pushed out as soon as your QA passes, the out of date travis builds are not effecting us here.

Check whether future has a value

Is there a way or could you add a method to check whether the Future has a value (i.e. is done or errored)? Currently you can only try to get the value and it will block until it has the value. For an asynchronous environment this is highly inefficient and lowers throughput.

Channels

We need bi-directional communication between threads ... we need channels ...

TODO:

  • select - implemented in Group
  • tests for edges
  • return false throw when send()/recv() on closed
  • not sure if timeouts make sense on recv()/send()

Started at cf2c909

Typo?

In practice this means Closures intended for parallel execution must not:

  • (...)
  • execute a limited set of instructions

It sounds like "it may execute an unlimited set of instructions".

Channel Roles ?

In go, a routine can enforce which side of a channel a routine should access, and it will throw errors when a channel accepted for recv is used for sending and vice versa ...

We could do this for tasks (I've done it already) ... it would work like this ...

$runtime->run(function(Channel\Read $channel) {
    $channel->send(false); // exception
}, [$channel])

$runtime->run(function(Channel\Write $channel)) {
    $channel->recv(); // exception
}, [$channel]);

Channel implements Channel\Read and Channel\Write, so you could use those types on any function parameter and it would pass, but specifically only for tasks, we can enforce the role.

It gets awkward because closures that we transmit over channels cannot have the same behaviour as tasks ... to overcome this, we could add a Channel::role allowing the caller to determine the role (instead of the callee) ...

Another option is to drop the magic "cast" for tasks and just have Channel::role which returns a Channel\Read or Channel\Write ...

Essentially, this is the cost of this feature:

parallel/src/scheduler.c

Lines 205 to 220 in 2485584

if ((info &&
ZEND_TYPE_IS_SET(info->type) &&
ZEND_TYPE_IS_CLASS(info->type) &&
Z_TYPE_P(slot) == IS_OBJECT) &&
(Z_OBJCE_P(slot) == php_parallel_channel_ce ||
Z_OBJCE_P(slot) == php_parallel_channel_read_ce ||
Z_OBJCE_P(slot) == php_parallel_channel_write_ce)) {
zend_string *name =
ZEND_TYPE_NAME(info->type);
if (zend_string_equals_literal_ci(name, "parallel\\channel\\read")) {
php_parallel_channel_role(param, PHP_PARALLEL_CHANNEL_READ);
} else if (zend_string_equals_literal_ci(name, "parallel\\channel\\write")) {
php_parallel_channel_role(param, PHP_PARALLEL_CHANNEL_WRITE);
}
}

It means every time we push a param onto the stack of a runtime, we have to travel that path ... that's not a particularly slow path, so it's quite a cheap check to make ...

Is there any interest in this, or is it too magical ?

Exceptions

Exceptions are a bit of a mess, with a couple of specialized types. and the rest general parallel\Exception.

I'm not a fan of specialized exceptions, however, since you are likely to have state that needs to be carefully managed in the case of failures, I'm going to introduce specialized exceptions for all errors.

This is just a heads up to readers reading ...

Todo:

  • Runtime
  • Future
  • Channel
  • Events

Notes:

The Error at the root of each namespace shall be thrown when an internal error occurs, of the kind you are not likely to be able to recover from - some underlying system error, or fundamental mistake in your code.

The errors under the Error namespace in each root namespace may also signify a fundamental mistake in your code, but of the kind you may be able to recover from at runtime.

Functional API

When writing real world code, you find yourself doing new Runtime at awkward times, like this.

Actually deciding what runtime object to use is a decision that can be made internally, so that we may just have:

\parallel\run(function() {
    echo "Hello Parallel\n";
});

I expect, that will become idiomatic parallel code ...

We will add the following API:

  • \parallel\bootstrap
  • \parallel\run

Note: all other API's remain unchanged.

\parallel\bootstrap(string)

Shall use the provided file for bootstrapping runtimes created via \parallel\run.

Only needs to be called once per process.

\parallel\run(Closure $task, array $argv = [])

Shall schedule task for execution in an available runtime, maintained by parallel.

parallel shall create a runtime where there is none available for execution (without limit).

Note: because runtimes have a thread local scope, so do available runtimes. This means parallel\run will only be able to use runtimes that were created by the current thread with \parallel\run.

Non-blocking Future::select()?

Is it possible to provide a non-blocking variant of the Future's select() method?
I've got a use case for this extension within an event-based loop (ext-uv for example), which would require polling the future state in a non-blocking manner.

By now I'd do a workaround and select with a timeout of 1. Would 0 work here as well and return immediately? If so, this issue can be closed, since non-blocking is possible.

Build with remi php-zts

Hello!

Install instruction (https://github.com/krakjoe/parallel/blob/develop/INSTALL.md#installation) is not correct.

I have CentOS 7 with remi php-cli build.

$ yum info php-cli
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.reconn.ru
 * epel: epel.mirror.far.fi
 * extras: mirror.logol.ru
 * remi-php72: mirror.reconn.ru
 * remi-safe: mirror.reconn.ru
 * updates: mirror.reconn.ru
Installed Packages
Name        : php-cli
Arch        : x86_64
Version     : 7.2.16
Release     : 1.el7.remi
Size        : 15 M
Repo        : installed
From repo   : remi-php72
Summary     : Command-line interface for PHP
URL         : http://www.php.net/
License     : PHP and Zend and BSD and MIT and ASL 1.0 and NCSA and PostgreSQL
Description : The php-cli package contains the command-line interface
            : executing PHP scripts, /usr/bin/php, and the CGI interface.

I got an error:

$ ./configure --enable-parallel
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -E
checking for icc... no
checking for suncc... no
checking whether cc understands -c and -o together... yes
checking for system library directory... lib
checking if compiler supports -R... no
checking if compiler supports -Wl,-rpath,... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for PHP prefix... /usr
checking for PHP includes... -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib
checking for PHP extension directory... /usr/lib64/php/modules
checking for PHP installed headers prefix... /usr/include/php
checking if debug is enabled... no
checking if zts is enabled... no
checking for re2c... re2c
checking for re2c version... 0.14.3 (ok)
checking for gawk... gawk
checking whether to enable parallel support... yes, shared
checking whether to enable parallel coverage support... no
checking whether to enable parallel developer build flags... no
checking for ZTS... configure: error: parallel requires ZTS, please use PHP with ZTS enabled

Correct way is:

...
$ zts-phpize
Configuring for:
PHP Api Version:         20170718
Zend Module Api No:      20170718
Zend Extension Api No:   320170718

$./configure --enable-parallel --with-php-config=/usr/bin/zts-php-config
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -E
checking for icc... no
checking for suncc... no
checking whether cc understands -c and -o together... yes
checking for system library directory... lib
checking if compiler supports -R... no
checking if compiler supports -Wl,-rpath,... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for PHP prefix... /usr
checking for PHP includes... -I/usr/include/php-zts/php -I/usr/include/php-zts/php/main -I/usr/include/php-zts/php/TSRM -I/usr/include/php-zts/php/Zend -I/usr/include/php-zts/php/ext -I/usr/include/php-zts/php/ext/date/lib
checking for PHP extension directory... /usr/lib64/php-zts/modules
checking for PHP installed headers prefix... /usr/include/php-zts/php
checking if debug is enabled... no
checking if zts is enabled... yes
checking for re2c... re2c
checking for re2c version... 0.14.3 (ok)
checking for gawk... gawk
checking whether to enable parallel support... yes, shared
checking whether to enable parallel coverage support... no
checking whether to enable parallel developer build flags... no
checking for ZTS... ok
checking parallel coverage... disabled
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by cc... /bin/ld
checking if the linker (/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B
checking the name lister (/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking for gawk... (cached) gawk
checking command to parse /bin/nm -B output from cc object... ok
checking for sysroot... no
checking for mt... no
checking if : is a manifest tool... no
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if cc supports -fno-rtti -fno-exceptions... no
checking for cc option to produce PIC... -fPIC -DPIC
checking if cc PIC flag -fPIC -DPIC works... yes
checking if cc static flag -static works... no
checking if cc supports -c -o file.o... yes
checking if cc supports -c -o file.o... (cached) yes
checking whether the cc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
...

Would you change documentation to clarify the moment?

Drop Future::value(int $timeout) in favour of Events ?

With the addition of Events API, I managed to drop a couple of methods from Future since it replaced Future::select.

I'd like to go further and drop the timeout variation of value from Future: It would be better to have fewer methods on Future, and it would be better to keep working with timeouts in Events only.

It's obviously a(nother) nasty BC break, but I care less about that right now than sticking to the mantra of having succinct API ...

Can anyone think of a use case that really requires a timeout on value that could not be supported by using the Events API ?

Getting the exception thrown inside a closure

Been trying to get the exception thrown inside the closure passed into the runtime. But the resulting parallel\Exception contains no reference in getPrevious to it, was expecting it to be that way:

try {
    $resolve($future->value());
} catch (\Throwable $throwable) {
    $reject($throwable->getPrevious());
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.