Requests with too large headers fields are not logged about webp_server_go HOT 4 CLOSED

In what scenario is it necessary to send such a large header to the WebP Server Go?🤔

from webp_server_go.

The website, say example.com, has a large Cookie header. When visiting webp.example.com to request the webp server, the Cookie header is also sent by the browser. Our webp server is behind an nginx proxy and the status code 431 is prevented by not forwarding the Cookie header from nginx to the webp server.

Note, however, that this issue is not about the webp server returning 431. It is about the webp server not logging a request that results in status code 431.

from webp_server_go.

Okay, I got the reason for this.

WebP Server Go is using https://github.com/gofiber/fiber as HTTP framework, it has a constant called ReadBufferSize:

ReadBufferSize specifies the I/O buffer size in bytes for incoming messages.

It's reflecting the same variable in FastHTTP, with description below:

// Per-connection buffer size for requests' reading.
// This also limits the maximum header size.
// Increase this buffer if your clients send multi-KB RequestURIs
// and/or multi-KB headers (for example, BIG cookies).
//
// Default: 4096
ReadBufferSize int `json:"read_buffer_size"`

Since is by default 4096, a large cookie of more characters will fail on FastHTTP first, so Fiber and WebP Server Go won't be able to log this request.

A quick fix for this(if you can compile by yourself), is to change

var app = fiber.New(fiber.Config{
	ServerHeader:          "WebP Server Go",
	AppName:               "WebP Server Go",
	DisableStartupMessage: true,
	ProxyHeader:           "X-Real-IP",
})

to something like(You can adjust ReadBufferSize to something bigger that fits your needs):

var app = fiber.New(fiber.Config{
	ServerHeader:          "WebP Server Go",
	AppName:               "WebP Server Go",
	DisableStartupMessage: true,
	ProxyHeader:           "X-Real-IP",
	ReadBufferSize: 8192,
})

in webp-server.go file and then compile the server(https://docs.webp.sh/build/), we are planning to make it changeable by supplying ENV in future version, stay tuned.

from webp_server_go.

This issue should be fixed in 0.9.12 now. You need to customize your own env READ_BUFFER_SIZE

from webp_server_go.