"Unmasked client to server frame" causes 502s about http-kit HOT 9 OPEN

@alexandergunnarson Hi Alex,

A few questions so long-

• What version of http-kit are you running?
• I presume this behaviour is new? Did anything obvious change prior to the new behaviour?
• Are you able to easily check if http-kit is receiving https "Upgrade", and "Connection: upgrade" headers? Off-hand, I seem to recall having seen this error before when behind a proxy that wasn't forwarding the above headers.

Thanks

from http-kit.

Hey! Thanks for your quick response.

• Running 2.8.0-beta3. I believe there was some reason why we're using a beta version, but I don't remember precisely why. Could have been because the client used virtual threads, and that was a desirable feature for us? I'm going to upgrade now to 2.8.0-RC1 to rule out issues.
• This behavior has happened occasionally ever since we introduced WebSockets to our system back in December. So not new, just newly reported here :)
• CloudFront is forwarding all headers as far as I can tell (based on config and logs). However, it just occurred to me that because Upgrade, and Connection: upgrade is specific to HTTP/1.1 (not HTTP/2), maybe what's happening here is that Upgrade, and Connection: upgrade aren't actually being handled correctly somehow, and that the issue is being masked. That is, maybe it's the case that >99% of our clients are using HTTP/2 in the browser, not HTTP/1.1, and so when a user uses HTTP/1.1, it yields the "Unmasked client to server frame" error. But this is highly conjectural. I will see whether I can check in our logs.

from http-kit.

If CloudFront logs are to be believed, the following headers were forwarded for the most recent offending WS requests (all such requests yield 401 or 502 errors):

Accept-Encoding
Accept-Language
Cache-Control
Connection
Cookie
Host
Origin
Pragma
Sec-WebSocket-Extensions
Sec-WebSocket-Key
Sec-WebSocket-Version
Upgrade
User-Agent

The load balancer we have is "dumb" in that it forwards everything and is unlikely to affect anything.

from http-kit.

Just checked and the vast majority of our WebSocket requests (both failed and successful) are HTTP/1.1, so we can seemingly rule out the headers issue.

from http-kit.