curl 56 SSL_read error erno 0 with openssl >= 3.2 & curl 8.7.1 about curl HOT 10 CLOSED

We need more details. This is not happening for me:

This works fine: curl -1 https://curl.se

curl 8.8.0-DEV (x86_64-pc-linux-gnu) libcurl/8.7.1 OpenSSL/3.2.2 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 libssh2/1.11.0 nghttp2/1.61.0 librtmp/2.3 OpenLDAP/2.5.16

and

curl 8.7.1 (x86_64-pc-linux-gnu) libcurl/8.7.1 OpenSSL/3.2.2 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 libssh2/1.11.0 nghttp2/1.61.0 librtmp/2.3 OpenLDAP/2.5.16

from curl.

mm ok i used a small python web server based on baseHttpHandler,
here is code: (please adapt it as you want, gender file can be anything)
(i used ssl 3.3.1 & 3.2.0)

#!/usr/bin/python3

import http.server
import socketserver
import socket
import ssl

PORT = 8000

GENDER_FILE = 'tts_gender.json'
KEY = "df7a0aa5884e46a89b435e91ffe3c018"
SSL_CERT = 'tts_server.crt'
SSL_KEY = 'tts_server.key'

class MsnSynthesizerHandler(http.server.BaseHTTPRequestHandler):
    def __init__(self, request, client_address, server):
        super().__init__(request, client_address, server)

    def do_GET(self):
        assert self.headers.get("ocp-apim-subscription-key") == KEY, "missing or wrong sub key"
        print("path=%s", self.path)
        assert self.path == "/cognitiveservices/voices/list" , "missing or wrong sub key"
        self.send_response(200)
        self.send_header('Content-type', 'application/json')
        self.end_headers()
        with open(GENDER_FILE, "rb") as f:
            self.wfile.write(f.read())

Handler = MsnSynthesizerHandler

with socketserver.TCPServer(("", PORT), Handler) as httpd:
    httpd.socket = ssl.wrap_socket(httpd.socket, keyfile=SSL_KEY, certfile=SSL_CERT, server_side=True)
    print("serving at port", PORT)
    httpd.serve_forever()

The tls is handshaked properly and 200 ok sent back, this seems to be just a close con detection,
Note that we build our staic openssl libs, default option mostly,

from curl.

when linking (static) curl app with libcurl & openssl >= 3.2.0 , a simple https 1.0 (or 1.1 with Header connection : close trigger an curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0 when remote close the connection.

there needs to be a known termination point (like content length, close notify etc) otherwise curl will keep reading. try self.close_notify()

from curl.

@SpitchAG if you get a problem with your custom server only and not with any public sites, I think we can suspect that maybe the issue is in your server...

from curl.

We want to be sure we are not going to have regression issues here,

Do we know why there is a need to have a known termination point using openssl 3.2.x but not using openssl 3.1.x ? (using same curl base code)

i see that when using openssl 3.1, curl is able to deduce :

• no chunk, no close, no size. Assume close to signal end
• Closing connection

WiIth 3.2, (same curl code), this log is replaced by:

• OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0
• Closing connection

So there is a behavior change caused by openssl upgrade.

from curl.

OpenSSL changed its defaults in handling "unexpected" EOF, see php/php-src#8369 and openssl/openssl#11378, among others.

from curl.

But those threads are pretty old, prior to 3.1., or were they fixed after 3,1 ?

from curl.

AFAIK, the behaviour was changed in OpenSSL 3.2. The thing that seems to be missing in your sample code is the TLS shutdown at the end of the connection. Without a shutdown message from the server, the OpenSSL in curl reports an error when it sees the socket closed.

Since your code does not seem to send a Content-Length header, curl needs to read until the close of the connection. The TLS shutdown is then necessary for a clean end of the download or it becomes indistinguishable from an abort (e.g. the server just crashing in the middle) and the download being incomplete.

from curl.

Nice, ok yes it seems that this is the ssl change indeed,
so issue can be closed, good job,

from curl.

I can replicate this on openssl 3.3.0 and curl 8.6.0 too.

from curl.