Comments (15)
@KrishnakanthYachareni From the configuration I don't see any changes related to the memory management, which means that you are running with Pooled ByteBufs. Is it possible that you see this reserved memory?
Check this https://www.programmersought.com/article/9322400832/ ,
https://netty.io/wiki/reference-counted-objects.html and may be this https://projectreactor.io/docs/netty/release/reference/index.html#faq.memory-leaks
from reactor-netty.
@violetagg Yes, my application has been using default Pooled ByteBus, as I did not have any manual configuration related to mememory management. You can see below it is a reserved memory.
If you see the memory usage of the pod which is gradually increasing in the pod and memory is not collecting by GC.
Why does ByteBuf memory is not clearing, is the following snippet not releasing the buffer properly?
Flux<FcmResponseWrapper> response = fcmRestClient.sendPush(fluxRequests, appName, eventIdentifiers);
List<Tuple2<Long, FcmResponseWrapper>> listOfSendResponses = response.elapsed()
.collectList()
.block(); // Thread blocking.
from reactor-netty.
Is it possible to enable memory metrics provided by Reactor Netty
https://projectreactor.io/docs/netty/release/reference/index.html#_metrics_6
reactor.netty.bytebuf.allocator.active.heap.memory
reactor.netty.bytebuf.allocator.active.direct.memory
from reactor-netty.
@violetagg Just to clarify that in my code no other objects are allocating more memory and memeory is leaking from reactor related objects. That I verfied during JVM profiling.
bytebuf are not emitted but ConnectionProvider metrics are emitting. Following code used to enable the reactor metrics. am I missing anything?
Schedulers.enableMetrics(); // in Spring boot main method.
private ConnectionProvider getConnectionProvider() {
return ConnectionProvider.builder("fcmConnection")
.metrics(true)
.build();
}
@Bean
public NettyServerCustomizer nettyServerCustomizer(){
return httpServer -> httpServer.metrics(true, Function.identity());
}
My suspect is .subscribeOn(Schedulers.fromExecutor(executor)
method with reactor scheduler may causing the leak, Can you please validate the code?
messages.flatMap(msgPair -> webClient.post()
.uri(appURI)
.header(HttpHeaders.AUTHORIZATION, accessToken)
.body(Mono.just(msgPair.getFirst()), FirebaseRequest.class)
.exchangeToFlux(clientResponse -> this.handleResponse(clientResponse, msgPair.getSecond())))
.subscribeOn(Schedulers.fromExecutor(executor));
from reactor-netty.
Add this to the client httpClient.metrics(true, Function.identity());
from reactor-netty.
@violetagg I collected the byteBuff memory metric and both seems always constant and same value.
from reactor-netty.
@KrishnakanthYachareni You need the active
not the overal
memory that is reserved
reactor.netty.bytebuf.allocator.active.heap.memory
reactor.netty.bytebuf.allocator.active.direct.memory
vs
reactor.netty.bytebuf.allocator.used.heap.memory
reactor.netty.bytebuf.allocator.used.direct.memory
from reactor-netty.
@violetagg My bad, I miss understood the mertics name. Both memories are most of the time showing as 0
but some time direct memory have certain value and active heap memory is always 0. what does it mean?
Note: Just to mention Non-Heap memory is gradually increasing and not going down.
from reactor-netty.
Took this screenshot after few hours of running and later again both memories went down to 0
from reactor-netty.
@KrishnakanthYachareni So to summarise the active memory is not increasing constantly but it returns to 0?
from reactor-netty.
@violetagg Yes. that is right!
from reactor-netty.
@KrishnakanthYachareni So looks like no memory leaks?
from reactor-netty.
@violetagg I did these changes, this could be a reason memory leak got fixed.
From
webClient.subscribeOn(Schedulers.fromExecutor(executor));
To
webClient.publishOn(Schedulers.fromExecutor(executor));
from reactor-netty.
@KrishnakanthYachareni I cannot comment on the change from subscribeOn
/publishOn
without having a reproducible example. That said should I close this, if the above is working for you?
from reactor-netty.
@violetagg yes please, you can close it.
from reactor-netty.
Related Issues (20)
- Timeout leaves connection in the pool in configured state, blocking a pool slot HOT 17
- ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. HOT 3
- HTTP2 Client doesn't work/DefaultChannelPipeline wrongly configure for HTTP2 HOT 2
- Connection reset by peer; nested exception is io.netty.channel.unix.Errors$NativeIoException: readAddress(..) HOT 2
- springcloud gateway netty pool config HOT 1
- LEAK: ByteBuf.release() was not called before it's garbage-collected. HOT 17
- ClassNotFoundException for HttpDecoderConfig using reactor-netty-http HOT 3
- Mutable NewConnectionProvider HOT 10
- ByteBuf LEAK when PoolAcquirePendingLimitException is thrown HOT 9
- Consistent Memory Increase in Webflux Application HOT 5
- Build should not depend on `doc:antora`
- Ongoing connection reset by peer HOT 6
- IllegalArgumentException: Failed to parse a port from request HOT 2
- Wiremock proxy : Connection prematurely closed BEFORE response HOT 2
- Add ability to set idle-timeout for websockets HOT 3
- Micrometer Prometheus metrics export fails due to conditional `proxy.address` tag HOT 2
- Intermittent error with connection reset by peer. HOT 3
- recordServerConnectionInactive never invoked after WebSocket upgrade HOT 1
- Metrics not disposed for `GracefulShutdownInstrumentedPool` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from reactor-netty.