How to aggregate data processed in parallel with pcntl about event-loop HOT 4 CLOSED

Thanks for the help. Child-process looks like it runs things in a seperate thread which I'm getting the impression is just a safer, and therefore defacto, way to conduct processing in parallel. Its a shame we can't do this natively in PHP like truely asynchronous languages can (e.g. Javascript).

Yes, the Child Process component lets you spawn a new process you can use to defer blocking CPU intensive operations to. Last time I checked, and I can be wrong here, but JavaScript isn't a multithreaded language. NodeJS does do multi threading by offloading the blocking parts. Found this nice read up about it, for those interested: https://www.digitalocean.com/community/tutorials/how-to-use-multithreading-in-node-js

https://github.com/clue/reactphp-pq looks interesting though its just example code at the moment - doesn't appear to be any actual code that does this job. I suspect @clue would run into the same issue as me: Its easy enough to return simple data types (bool, int, string, float, array) but its difficult to return PHP objects unless there classes have serialize and unserialize magic methods which means you can't share objects easily between threads.

Over the years I've spent a significant amount of time on this. Used both processes and threads for this in PHP. Wrote an overview down in https://blog.wyrihaximus.net/2022/07/my-road-to-fibers-with-reactphp/. And to be honest child processes work, but they are tricky in non-Linux environments, plus they aren't. Wrote my fair share of packages for it, including one that lets you run a givinig callable. https://github.com/WyriHaximus?tab=repositories&q=child-process&type=&language=&sort= However, child processes are heavy, and you need to serialise whatever you're sending between processes. Or you can uses threads, used ext-parallel for that, and then you can toss objects between threads directly. However the bigger the object, the slower it becomes. Set up an entire org to do this with ReactPHP with a ton of different tooling doing a whole bunch of things with it https://github.com/reactphp-parallel/. And I still think it has a place, however, it isn't getting updated since PHP 8.1 got fibers.

TL;DR We provide the tools to make this work. And for all our components, including the filesystem we make sure it works non-blocking, which means that some of the extensions we suggest might use threads under the hood for you just like NodeJS does.

from event-loop.

Hey @fiasco, I think reactphp/child-process is what you're looking for.

Forking can be a possibility here but there are many things you need to be careful about. For example, when forking you create an exact copy of your process, but this doesn't mean that they share the same resources. This could potentially lead to race conditions or even unwanted behavior, all depends on what you're trying to do.

For now the child-process component should fit for your use case, you can also take a look at clue/reactphp-pq.

I hope this helps 👍

from event-loop.

Thanks for the help. Child-process looks like it runs things in a seperate thread which I'm getting the impression is just a safer, and therefore defacto, way to conduct processing in parallel. Its a shame we can't do this natively in PHP like truely asynchronous languages can (e.g. Javascript).

https://github.com/clue/reactphp-pq looks interesting though its just example code at the moment - doesn't appear to be any actual code that does this job. I suspect @clue would run into the same issue as me: Its easy enough to return simple data types (bool, int, string, float, array) but its difficult to return PHP objects unless there classes have serialize and unserialize magic methods which means you can't share objects easily between threads.

from event-loop.

I believe this has been answered, so I'm closing this for now. Please come back with more details if this problem persists and we can always reopen this 👍

from event-loop.