shaneharter / php-daemon Goto Github PK
View Code? Open in Web Editor NEWBuild production-ready Daemons using PHP 5.3+. Build fault-tolerant applications without the boilerplate.
License: Other
Build production-ready Daemons using PHP 5.3+. Build fault-tolerant applications without the boilerplate.
License: Other
I'm currently looking to migrate away from PEAR's System_Daemon and this looks awesome. We're using Composer (http://getcomposer.org/) for dependency management. Would you be open to adding a composer.json file (http://getcomposer.org/doc/02-libraries.md) to this project so it can be easily added to projects that would like to use it?
Share memory lock plugin was broken.
To verification, just change the Lock_File
in Example/LongPoll to Lock_Shm
. You will be able to run more than one instance.
I would like to start work on adding a Core_Worker_Via_Memcache class. It would make it much simpler to implement this new class if there were some unit tests to run to ensure the results from each method call are behaving as intended.
This is possibly related to #57.
When PHP Daemon tears itself down for an automatic reboot it invokes its own destruct method explicitly. The destruct method loops through worker and plugin aliases, invoking each of their teardown methods and removing the objects
foreach(array_merge($this->workers, $this->plugins) as $object) {
$this->{$object}->teardown();
unset($this->{$object});
}
However, the aliases are maintained in the arrays. I'm noticing that occasionally the PHP interpreter is invoking the destruct method again. In the second iteration, the workers and plugins are already unset so it attempts to call a method on an undefined property. The error that is raised causes PHP Daemon to attempt to restart itself.... Which calls the destruct method again and the whole party continues.
Because the original restart was actually successful in spawning a new instance of the daemon, you end up with one correctly functioning daemon and a few endless looping walking dead daemons.
I'm using version 2.1.1. Also, thanks for the library and the great documentation! Overall, this has been working great for me.
Hi,
I get this error when I run a example or when I execute my worker.
PHP Unknown: Declaration of Core_Worker_ObjectMediator::check_environment() should be compatible with Core_Worker_Mediator::check_environment(array $errors = Array) in lib/PHP-Daemon/Core/Worker/ObjectMediator.php on line 12 pid 2483
To reapir it I'have changed the file Daemon/Core/Worker/ObjectMediator.php
-- public function check_environment() {
-- $errors = array();
with
++ public function check_environment(Array $errors = array()) {
++ //$errors = array();
Considering the last paragraph of the wiki page:
If all of that was overly complex I apologize, but you can suffice to say this: Signal handlers can be complex and nuanced. But you should know that any code you create or spawn from within a signal handler will itself be unable to handle signals.
Would a possible solution be to create a callback of some sort and then toss it into an execution queue, to be run outside of the signal handler perhaps? It would introduce a slight delay to execution, but I believe it would allow working around that pitfall.
Please chmod it to 644 and update your master. Thanks!
If you have multiple workers their names can't start with the same letter. It's case sensitive. Took me all morning before I tracked down what was causing the error.
This stems from using ftok which takes a single character string on line 538 of Mediator.php. Some error checking should be added to check for this scenario and throw an exception.
$this->guid = ftok($ftok, $this->alias[0]);
$this->worker('RecordWorker', new RecordWorker(), new Core_Worker_Via_SysV(128 * 1024 * 1024));
$this->worker('RecordUpdater', new RecordUpdateWorker(), new Core_Worker_Via_SysV(110 * 1024 * 1024));
[2014-04-24 11:39:48] 11498 Application Startup Complete. Starting Event Loop.
[2014-04-24 11:39:48] 11503 RecordUpdater Worker Process Started
Warning: msg_send(): msgsnd failed: Identifier removed in /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Via/SysV.php on line 208
Call Stack:
0.0002 233184 1. {main}() /home/br64538/workspace/daemon/lib/daemon/run.php:0
0.9073 6718928 2. Core_Daemon->run() /home/br64538/workspace/daemon/lib/daemon/run.php:15
0.9075 6720472 3. SNUpdateDaemon->execute() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Daemon.php:414
0.9075 6720704 4. Core_Worker_ObjectMediator->getStalexChanges() /home/br64538/workspace/daemon/lib/daemon/SNUpdateDaemon.class.php:105
0.9075 6720992 5. Core_Worker_Mediator->__call() /home/br64538/workspace/daemon/lib/daemon/SNUpdateDaemon.class.php:105
0.9077 6760544 6. Core_Worker_Mediator->call() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Mediator.php:851
0.9077 6761112 7. Core_Worker_Via_SysV->put() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Mediator.php:824
0.9078 6763192 8. msg_send() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Via/SysV.php:208
[2014-04-24 11:39:48] 11498 RecordWorker SysV::put() Failed for call_id 2: Retrying. Error Code: 43
[2014-04-24 11:39:48] 11503 RecordUpdater getStalexChanges() is Not Callable.
The idea here is that we've learned from this first release of the Worker API that shared memory has faults and is, you could say, not the most rock solid PHP API available. And if Redis or Memcached are available on the server, one or both could be a better option than SHM.
Hello !
The extension PNCTL is installed on my server (apache2, debian) and work on my scripts but when i try to start run.php of examples (PrimeNumbers), i get this error :
Date PID Label Message
[2013-11-10 14:57:02] 4524 Checking Dependencies... Failed:
[2013-11-10 14:57:02] 4524 The PCNTL Extension is not installed
[2013-11-10 14:57:02] 4524 Examples\PrimeNumbers\Daemon is Shutting Down...
But effectively, in php -i and phpinfo() the PCNTL extension is enabled.
A little help or indication about this?
Thanks.
Core_Worker_Mediator::shm_init Failed. Could Not Read Header. If this problem persists, try running the daemon with the --resetworkers option
tried with the --resetworkers option
php run.php --resetworkers
but the error still persists ...
Any idea how it can be resolved?
Its running PHP5.4.8
Thanks
We encountered an issue when rotating and cleaning up log files.
It turned out that when a daemon (automatically) restarts, the log file handle is not closed.
This causes the new daemon process to inherit the handle and the file to remain open.
After the first restart the log file will be reopened causing the file to be opened twice.
Luckily on our system the new log file would get a handle value of 0 and be considered to be
STDIN and closed on subsequent restarts. We could not free up disk space by removing the file since it was still open. We had to truncate the file to free up disk space.
This can be easily fixed for the daemon. I will add a pull request for this.
In row 265 in Daemon.php the following seems to be mistaken: "if (is_numeric($this->loop_interval) == false)".
if loop_interval is NULL which is what is supposed to occur when working in blocking mode then this causes an exception. Removing this row works nicely so can you please check if this is indeed a typo or if I'm missing anything?
Thanks!
Hi,
I am building a solution that crawls some websites, and after a while the following messages are "dumped" in the log.
[2014-04-29 13:32:11] 6013 Workers SysV::put() Failed for call_id 2: Retrying. Error Code: 11
[29-Apr-2014 13:32:11 UTC] PHP Warning: msg_send(): msgsnd failed: Resource temporarily unavailable in /var/www/worker/includes/daemon/Core/Worker/Via/SysV.php on line 208 pid 6013
I am currently backtracing -and debugging the root cause, but also have tried to find an explanation of the Error code provided, so far I have not been able to find any documentation of the error code. Any ideas?
I have 2 Damons, for 2 differnet purposes.
From ObjectMediatior.php I see the ftok is always the same, this means they both seem to want to use the same msg_q
$tmp = sys_get_temp_dir();
$ftok = sprintf($tmp . '/%s_%s', str_replace('/', '_',
$this->daemon->get('filename')), $this->alias);
Am I missing something about configuring my env to allow multple seperate
Daemons run on the same box.
Hi i was wondering if it was possible to allow creating constructors for worker classes that implements Core_IWorker. Much like creating constructors for classes that implements \Core_ITask. It would be quite beneficial to have this feature.
Btw, i love the work you've done on PHP-Daemon. I always dreamt of an efficient creation and management of processes in PHP, and PHP-Daemon made that dream come true. Its a fantastic package overall. Many thanks for the hard work that you've done. Keep it alive and rocking !!
in
public function offsetSet($offset, $value)
{
if (is_scalar($offset))
$this->contents[$offset] = $value;
throw new Exception('Could not set INI value: $offset must be a scalar');
}
You need to make that an if else as setting an INI will always crash with the Exception.
I am using command line arguments to control the behavior of the daemon, but they are lost when the daemon restarts. This makes the auto-restart feature unusable for me.
The problem centers around Core_Daemon::command() where the filename of the script is built. Here some default arguments (-d --recoverworkers) are appended, but the original arguments are not mentioned at all.
I was thinking it would be nice to have an additional commandline option which would kill (or possibly restart) any processes (started with -d) that are currently running for a given daemon.
My use-case is because I'm still developing the code for my daemons. So occasionally I deploy new code to the server and have to do a kill pid
to get the new code read in and executed. It would be nice if there was a way to do this without first looking up the pid etc...
In my application I have registered a handler for the ON_SHUTDOWN hook, which cleans up some network connections. During regular shutdown, the hook is executed fine, but when the daemon is automatically restarted, the hook is not called, resulting in lingering connections.
The reason for this is that the hook is dispatched from the destructor of the Daemon class, which in the case of an automatic restart is called when the exit
call is made in the restart()
method. Unfortunately, for non-intuitive reasons, right before the exit
call is made, the array containing all the callbacks is cleared, so once the destructor runs, there aren't any hooks leftover to call.
It is possible to launch multiple instances of daemon even when using file lock plugin. For example I run one daemon instance, than after Core_Daemod::loop_interval seconds i can launch more instances of the same daemon. I don't understand the logic of TTL checking, till it equals to Core_Daemod::loop_interval, which is usually a few seconds... I commented out lines 85-87 of Core/Lock/File.php to use this plugin for now
Hi,
Since you store the handle to the log file in a static variable the file is only opened once per process. In my case I have a process which has run across a date boundary and I'm using an implementation of log_file()
based on date('Ymd')
.
Since the handle is already there the code never re-calls log_file()
and doesn't find out that there should be a new filename so it carries on writing to the old log file.
Possible solutions would be:
log_file()
then close the handle and open a new oneI think I prefer the second option and can implement it with a pull request if you like but maybe I'm missing something and there is a better third option?
Thanks,
Kelvin :)
Hi!
The last commit on this project was 8 months ago, there are numerous issues without responses, no apparent development. I am using this tool heavily and am waiting for the changes promised for v2.2.
@shaneharter are you still maintaining this project? We need you :)
Thanks
tobias
Some system information: this daemon is running on Mac OS X, using PHP 5.5 installed via homebrew. The current stable in homebrew is 5.5.23.
My daemon launches several tasks to download files from a remote host. I have a condition that occurs when, after whatever the first task is that finishes, the main runloop stops executing (or, at the very least, execute() isn't being called). All of the tasks eventually finish and I can still call log() on the parent, and if I look at a process list the daemon is still "running", but not actually doing anything and no new tasks are created.
Downgrading to PHP 5.5.20 fixes it.
I know this is a frustratingly vague bug report, so if there's anything else I can help you with I would be glad to provide assistance.
Hi,
I'm really sorry to post such a question as an 'issue' but I can't get it working…
I want to have a daemon that cares about my socket server (php) and in the docs you've written about socket server. So I think it should work but I don't know HOW.
If I don't set a call repeat time and leave out my while(true) nothing will matter. If I leave while(true) the server doesn't respond to inputs. And without the daemon it works well with the while(true) and responds to input.
Maybe someone could write the most important things I should care about to run my socket server with your daemon.
Jan
So it is :) It must be protected to allow setting any application env variables (shutdown status, for example).
Keep getting this one when a worker dies and the whole daemon dies.
README.md indicates that version 2.1 has been released, but 2.0 is the only stable version listed on Packagist. There are a couple development 2.1 releases, but it's unclear which one is the release (if any). Can you please tag the v2.1 commit?
If a worker process dies for some reason then after the timeout interval Daemon::__destruct() calls teardown() on a null object. The result is PHP Fatal error: Call to a member function teardown() on a non-object. This could be handled gracefully by checking that the worker object is non-null before calling teardown().
Bug:
$this->{$object}->teardown();
unset($this->{$object});
Solution:
if(isset($this->{$object})){
$this->{$object}->teardown();
unset($this->{$object});
}
Mediator::process() can return null and direct call made at line 724
$this->process($call->pid)->kill();
can raise a fatal error
Hi
I'm having problem of getting error:
"Dropped Call. Requeuing Call # To {$call->method}
"
And "Dropped Call. Requeue threshold reached.
I can't figure out what is the root cause for that behavior happening more over this is happening occasionally in non consistent way.
The function which triggers this is "public function garbage_collector()".
I tried to do restart when it is happening, but to put the problematic call back to daemon queue by the already written line code:
$this->call($call) (and)
$this->daemon->restart(); (my additional line code)
Questions is, if after daemon restart we already have the same daemon queue state as before or it is probably being initialized?
If we don't maintain daemon's queue state between daemon restart how can I return the call to rabbitMQ that will temporary store the call until daemon will be available again to handle the call.
This should happen somehow from this scope of code (Mediator.php)
Is this deamon suitable for irc applications, and if so, where to start would be appreciated
Refactor the Core_Daemon::show_help()
method to work like show_install_instructions()
.
Create two instance vars on Core_Daemon
:
protected $help_usage = array();
protected $help_options = array();
Change the Core_Daemon::getopt()
method to write to these arrays. Change the show_help()
method to just implode these arrays and print them with a similar format to the existing code.
This will enable application developers to parse their own command line options by overloading getopt (and calling parent::getopt() when they're done), while writing help instructions that get displayed alongside the library-specific instructions.
Hi,I want to try this php-daemon,but I am having difficulty on how to implement this,I created as socket listener in php and I want to apply this php-daemon to my project.because my listener will exit if there is no client connection and I re-run this by using task-scheduler in every 1 min.but I don't want to exit my script I want to run forever. I hope you can help me
Thank you in advance.
where i use -d ,it will log every work status message like 'Job x Is Running','Job x Is Running' in lo file,how can i disable it ,let it just log what i want?
Hi,
congrats on php-daemon! I've just started looking into the framework and find it quite cool and complete! Especially good the cluster-based functionality, very useful. A couple of questions/comments maybe you can help steer me:
Anyways, great work. I think this will be very useful in several jobs, and when combined with a message queue such as php-resque or whichever one anyone prefers, gives us a complete toolbox for work...
is_idle() is nice, but it would actually be nice to know how many tasks are running (or have a "worker slots available"), in order to more efficiently pop jobs off of a work queue in each execution state of the event loop.
inlcude php files one by one?
thanks
This is a large feature that I'm currently thinking about. I'm trying to sketch-out what the feature will look like, and how best to integrate it into the existing codebase.
Goals:
Implementation Thoughts:
Core/Worker/Queues
Core_Worker_Queues_SysV
Core_Daemon::create_worker()
method to choose a specific queue.IWorker
objects. You would instantiate a Queue object before you create the worker, configure it however you want, and then pass it to create_worker()
Any comment/input is appreciated. Is this a feature that interests you? Would you be interested in helping on any part of it? In beta testing?
I'd expect this feature to be on a Jan/Feb 2013 timeline.
Hi,
First up thanks for this code. Just playing around with it and it seems like a nice clear bit of code which does exactly what I need!
I've run into a problem because I am storing database config in a config file loaded by Core_Plugins_Ini. I want to create the database connection in my daemon's setup()
function. However, due to the order that things happen the setup()
in Core_Plugins_Ini hasn't been executed by the time my daemon's setup()
function is called.
How would you normally suggest avoiding this problem? One solution would be to init the plugins before the main daemon (around https://github.com/shaneharter/PHP-Daemon/blob/master/Core/Daemon.php#L269 ) but I'm not sure if this would have unwanted side effects?
The other solution (which I'm currently using as a workaround) is to override run() to look like this:
public function run()
{
$this->init_db();
parent::run();
}
This seems pretty hacky though. I guess the other solution is to check in execute()
whether the connection has been created and create it if now.
How do you recommend dealing with this situation?
Thanks!
Recently I need to write a program to pull message from SQS constantly. Then I found this library to do it. I extended the abstract class Core_daemon and wrote main function of my program in execute(). When I ran the daemon process, I found the daemon process produced a lots of defunct processes. Is it normal situation? or something I‘ve done wrong?
I thought I had reported this issue before but I can't find it now.
To reproduce run in daemon mode and set a low $auto_restart_interval. Every time the daemon restarts it creates a new msg_queue and new shm mallocs for each worker. This will quickly eat up all your memory and the daemon will eventually be unable to restart.
I've build a deamon with several workers with PHP-Daemon v2.0
At a certain point of time (after approximately 8 hours with 8 workers) the Daemon reaches an "Fatal Error" and tries to restart the worker processes. There's probably something wrong with the memory allocation. Because if I restart the Worker (and automaticly flush the memory) it works like before.
See in the detail here:
[2015-06-10 05:56:04] 13940 mitchell Message Queue Error 13: Permission denied]
[2015-06-10 05:56:04] 13940 mitchell Incrementing Error Count for catchall to 24
[2015-06-10 05:56:04] 13940 mitchell Message Queue Error 13: Permission denied
[2015-06-10 05:56:04] 13940 mitchell Incrementing Error Count for catchall to 25
[2015-06-10 05:56:04] 13940 mitchell Message Queue Error 13: Permission denied
[2015-06-10 05:56:04] 13940 mitchell IPC 'catchall' Error Threshold Reached
[2015-06-10 05:56:04] 13940 mitchell Fatal Error: Worker process will restart
[2015-06-10 05:56:04] 13940 WheresAppDaemon\Daemon is Shutting Down...
[2015-06-10 05:56:07] 13940 Restart Happening Now...
Date PID Label Message
[2015-06-10 05:56:32] 18615 Invalid Alias. Identifiers must be scalar.
[2015-06-10 05:56:32] 18615 WheresAppDaemon\Daemon is Shutting Down...
[2015-06-10 05:56:34] 18615 Restart Happening Now...
Date PID Label Message
[2015-06-10 05:56:36] 18620 Invalid Alias. Identifiers must be scalar.
[2015-06-10 05:56:36] 18620 WheresAppDaemon\Daemon is Shutting Down...
Also when the Daemon tries to restarts itself I got the message Identifiers must be scalar. Maybe this is related to the fact that there's no memory left for the Daemon. I run a Vagrant hashicorp/precise32 Ubuntu machine with 2048MB of memory.
Ok, to start off, I know this isn't an issue or something of that kind but since this package doesn't have a huge following on any forum or whatever, I figured I may aswel try here.
I want to make a daemon script in PHP that basically runs for every user I have in my application. I also want my users to have the ability to stop such a process whenever they want. And also start it back up again.
Is this the package I've been looking for? The average run time of the script is like 2 hours. Sometimes they run for 5 minutes and the user stops it, sometimes they run for a full week.
Thanks for those that considered answering and to those who read this unstructured message. I use everything ranging from Job Queue's to Cron Jobs but I can't seem to find something that completely suits my needs.
Using native getopt method from php it doesn`t let you alter the arguments list in order to get the corresponding arguments correctly.
For example, you may access the daemon from a framework, so "php bootstrap.php -d" won`t daemonize the process.
I am trying to create an application which reads messages off of a distributed queue and processes each of them. As there are very many messages coming in (thousands per second), high throughput and performance are essential for the app.
I looked into using this daemon framework as basis for the app, but found out that despite setting the $loop_interval to 0, the maximum number of iterations per second is 100. While this may be more than enough for other, less intense workloads, it is far too little for my use case.
The code comments suggest that setting the variable to 0 would result in the loop being executed continually without delay (except for the occasional ON_IDLE event), but I found out that even when with the value 0 there is a call to usleep() which wastes a huge amount of time.
The call is in Core_daemon::timer() on line 883 (as of this writing). There the process is set to sleep for 10ms, effectively limiting the maximum number of iterations per second to 100. The comments justify doing this by needing to "give the CPU a break", but 10ms is a looooong time in CPU terms.
Please reduce the length of the break or entirely remove it, so that high-throughput, high-performance apps can also benefit from this otherwise well-written framework. I suggest 100µs as delay in between iterations.
Bug: This foreach loop in Daemon::__destruct() does not behave as expected:
foreach($this->workers + $this->plugins as $object) {
$this->{$object}->teardown();
unset($this->{$object});
}
The '+' operator performs an array union using keys, not the values. In this case the array indices may conflict, causing some array values to be excluded. See the note here: http://php.net/manual/en/language.operators.array.php#86379.
Impact: This bug prevents the teardown() function from being called on the complete set of workers and plugins, for example on the Lock_File plugin object. As a result, the lock file may not be unlinked, so restart() fails after the auto_restart_interval with the message "Core_Lock_File::set Failed. Additional Lock Detected."
Solution 1:
foreach($this->workers as $object) {
$this->{$object}->teardown();
unset($this->{$object});
}
foreach($this->plugins as $object) {
$this->{$object}->teardown();
unset($this->{$object});
}
Solution 2:
foreach(array_unique(array_merge($this->workers,$this->plugins)) as $object){
$this->{$object}->teardown();
unset($this->{$object});
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.