Debugging an issue on AWS ALB 502s about hyper-express HOT 13 CLOSED

I see, ok. Yeah hyper-express adds a default 500 error handler but not a not found handler. You can specify one with Server.set_not_found_handler() but I'll bump version so the native defaults work to handle those proxy errors. I've also noticed browsers don't like it when a webserver instantly closes a connection without any route found.

from hyper-express.

Hey, your write up seems to be great and detailed. While hyper-express adds a lot of features on top of uWebsockets.js, It is fundamentally just a wrapper that doesn't mess with the actual request lifecycle of a uWS.HttpRequest. For that reason, It is very likely that the bug you are running into is caused due to a problem with uWebsockets.js itself. I would highly recommend you make this issue in the uWebsockets.js repository so the maintainer of uWebsockets.js can provide more information on this problem.

I do recall that uWebsockets.js tends to timeout requests and connections in an interval that is a factor of 4. While I don't see why that would be a problem, It may be a starting point or in some way linked to the weird cancelation times ranging from 0.001s all the way up to 8 - 12s.

Hope the above helps and I'll be following you issue in uWebsockets.js closely in case any changes are needed to `hyper-express.

from hyper-express.

One additional follow up question I have is does the endpoint which suffers from the 502s consume the incoming request body (If any) as a stream?

from hyper-express.

@kartikk221 thanks for the input. No, we are buffering request bodies.

from hyper-express.

@kartikk221 thanks for the input. No, we are buffering request bodies.

I see. The only culprit I can possibly think of is there being back-pressure under load for incoming Request Body and so hyper-express automatically pauses/resumes the Request socket while downloading the body chunks into memory. But this is just a guess, there's a higher chance its a bug with timeouts than an error from the pausing/resuming.

from hyper-express.

back-pressure

I looked through some of the uws/usocket code and it looks like back-pressure only applies to ws connections. We're only serving http at this point, or do I misunderstand the behavior here?

from hyper-express.

back-pressure

I looked through some of the uws/usocket code and it looks like back-pressure only applies to ws connections. We're only serving http at this point, or do I misunderstand the behavior here?

So back-pressure can apply to HTTP as well. For example, let's say you are receiving a large request body and are further streaming it to your filesystem by piping the request to a writable. It is possible that this writable experiences back-pressure as the filesystem of your machine is stressed. In this scenario, hyper-express will pause the uWS.HttpRequest effectively pausing any more body chunks from being received as to not exhaust the memory while the back-pressure drains. Once the back-pressure has been drained, hyper-express will resume the Request to begin receiving and piping more body chunks until completion.

from hyper-express.

@kartikk221 Thanks for that explanation. To be clear, there's no back pressure setting for http routes that I need could tweak like there is for ws routes?

from hyper-express.

@jsonmorris No, this is not something you can tweak as it is implemented by hyper-express on top of uWebsockets.js to automatically minimize memory usage for large body streams.

from hyper-express.

@kartikk221 would be available to bump uws.js to v20.31.0?

from hyper-express.

@kartikk221 would be available to bump uws.js to v20.31.0?

Sure, any changes I should be aware that need to be implemented on my end or is this just a version bump to bugfix?

from hyper-express.

It's a similar change to v20.30.0, which I see you've already adopted. It adds an additional default 404 error code for unmatched routes.

from hyper-express.

@jsonmorris Try hyper-express version 6.7.3 to see If that resolves your issue.

from hyper-express.