Comments (6)
SimonFrings
commented on June 14, 2024
Hey @wnuQq, thanks for bringing this up 👍
I don't believe this is a memory leak, this is actually the expected behavior. When running your script, PHP starts to instantiate all classes, interfaces etc. needed for your script to run properly and will keep those in your memory. For example take the first two lines of your file output:
| 2097152 | 700432 | 0 |
| 2097152 | 1488680 | 0 |
You didn't even send out a request yet and memory is already growing, this is caused by PHP and the expected behavior in this case. In the third line (| 10485760 | 7584792 | 10 |) your memory grows, because all your planned requests are now in memory too (caused by the clue/reactphp-mq project, also expected). Progressing further your memory shrinks with every request being handled after a while. The rest you see at the end is all the PHP stuff again. You never see your memory dropping further down, because you only track your memory usage while your PHP script is running.
In conclusion, this is not a bug in clue/reactphp-mq or reactphp/http. You can also try out the same example with only one request instead of 500 and see the same amount of memory in the end.
I think this should answer your question, so I will close this ticket for now. If something comes up or needs some further investigation, we can always reopen.
I hope this helps!
from http.
wnuQq
commented on June 14, 2024
Hi @SimonFrings
Thanks for your answer. Unfortunately, I'm not convinced ;)
I will try to be more specific, maybe this will help you to understand my concerns, or you will be able to point out in which point I'm wrong.
| 2097152 | 700432 | 0 |
| 2097152 | 1488680 | 0 |
Everything as you said: PHP starts to instantiate all classes etc.
| 10485760 | 7584792 | 10 |
At this point - as you said: "all your planned requests are now in memory" - agreed! And it is no surprise that allocated memory depends on how many requests are in the queue. So when we have 500 PHP uses ~ 7584792 and this will be 10485760 of the total memory allocated.
| 10485760 | 6413880 | 141 |
| 10485760 | 6219192 | 160 |
| 10485760 | 6037112 | 180 |
| 10485760 | 5855032 | 200 |
| 10485760 | 5672952 | 220 |
"Progressing further your memory shrinks with every request being handled after a while" - yup. I can see that PHP does not require all this memory anymore and I'm waiting for it to release the allocated memory (first column: 10485760).
| 10485760 | 2520152 | 500 |
| 10485760 | 2520152 | 500 |
| 10485760 | 2520152 | 500 |
...but it looks like it's not going to happen. Allocated 10485760, when 2520152 is needed. I was testing for more extensive data sets. I'm running similar code on production and I can see clearly that allocated memory (first column) never goes down.
I think I was wrong when I called it a leak because memory is not leaking - it is needed at some point during the script is running but after processing messages which caused this spike I expect memory will be released, but it is not.
from http.
SimonFrings
commented on June 14, 2024
@wnuQq I'm not sure if this is something that regards ReactPHP. In my eyes your system and PHP decide no how much memory will be allocated to your process. I don't exactly know how they handle allocated memory in this case, but I can imagine that PHP needed more memory from your system after you surpassed a certain number of used memory (2097152 -> 10485760) and kept this amount after it dropped back down in case you need it at a later time again. This way PHP wouldn't have to communicate back and forth with your system, but that's just my assumption.
from http.
WyriHaximus
commented on June 14, 2024
This way PHP wouldn't have to communicate back and forth with your system, but that's just my assumption.
That assumption is correct, PHP will increase the allocated amount of memory with the system as your memory usage goes up. It tends to go in bigger blocks than you request and it indeed never goes down. Not sure why but it could be a relic from when PHP was mostly run as a request handle and there was no need to free memory because requests didn't last long.
from http.
wnuQq
commented on June 14, 2024
Hi @WyriHaximus,
Not sure what you mean by "never" :)
Take a look aaat this beauty ;)
$array = [];
$i = 0;
$string = 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvvvvvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvvvvvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvvvvvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvvvvvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavaaaaaaaaaaaaaaaaaaaaaaaavvvvvvaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';
$file = fopen(__DIR__ . '/memorySimple.csv', 'w');
fputcsv($file, [memory_get_usage(true), memory_get_usage()]);
while (true) {
$i++;
$array[$i] = $string . $string . $string . $string . $string . $string . $string . $string . $string . $i;
if ($i % 100 === 0) {
fputcsv($file, [memory_get_usage(true), memory_get_usage()]);
if ($i >= 1000) {
$array = array_slice($array, 150);
echo count($array) . PHP_EOL;
}
}
}
And results:
| 2097152 | 407384 | ***
| 6291456 | 5330832 |
| 12582912 | 10250128 |
| 16777216 | 15173520 |
| 20971520 | 20088720 |
| 27262976 | 25003920 |
| 31457280 | 29935504 |
| 35651584 | 34850704 |
| 41943040 | 39765904 |
| 46137344 | 44681104 |
| 52428800 | 49596304 |
| 48234496 | 47171504 |
| 46137344 | 44713904 |
| 44040192 | 42256304 |
| 41943040 | 39798704 |
| 39845888 | 37341104 |
| 37748736 | 34883504 |
| 33554432 | 32425904 |
| 31457280 | 29939632 |
| 29360128 | 27482032 |
| 27262976 | 25003952 |
| 27262976 | 22546352 |
| 25165824 | 20088752 |
| 23068672 | 17631152 |
| 18874368 | 15173552 |
| 14680064 | 12707760 |
| 12582912 | 10250160 |
| 8388608 | 7792560 |
| 6291456 | 5330864 |
| 2097152 | 407384 | ***
Memory was released. Every unset (array_splice) works as expected. You can try it also this way:
...
if ($i % 2 === 0) {
fputcsv($file, [memory_get_usage(true), memory_get_usage()]);
if ($i >= 1000) {
$array = array_slice($array, 3);
...
It shows that PHP releases memory in the same way it allocates it - in chunks. So my suspicion is that something in Queue is not releasing memory - allocated chunks cannot be released as something is still there.
from http.
WyriHaximus
commented on June 14, 2024
Hi @WyriHaximus, Not sure what you mean by "never" :)
As in never for the duration of the process is running. But that might have changed in recent PHP versions.
from http.
Related Issues (20)
- [Question] How to convert "Psr\Http\Message\StreamInterface" to "React\Stream\ReadableStreamInterface" HOT 1
- HTTP client: streamed request not triggering drain (perhaps) HOT 4
- Preserve method on redirect HOT 1
- Memory leak? HOT 8
- HTTP client: Expose maximum time to keep alive idle connection with persistent connections HOT 5
- Cannot create socket server HOT 3
- HTTP client on specific network shows poor performance HOT 5
- HTTP client: Simplify HTTP file uploads and submitting forms HOT 2
- EmptyBodyStream as request body not working HOT 3
- Regression: object with __toString no longer being accepted as request body in Browser post HOT 1
- getFragment() returns empty string, username:password@hostname not supported HOT 1
- Unable to properly cancel request (Browser) HOT 8
- URI Behind Reverse Proxy Incorrect Scheme HOT 5
- Laravel + websockets: Unhandled promise rejection with TypeError HOT 1
- MAX_CHUNK_HEADER_SIZE size increase HOT 4
- Random HTTP Requests don't resolve HOT 2
- Body Stream Closes when running Browser Request HOT 3
- The psr/http-message v2 is not supported HOT 1
- Add support for psr/http-message v2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from http.