http4s / http4s-jdk-http-client Goto Github PK

JDK 11+ http client implementation for http4s clients

Home Page: https://jdk-http-client.http4s.org

License: Apache License 2.0

Scala 98.67% Nix 1.30% Shell 0.02%

http4s-jdk-http-client's Introduction

Http4s

Http4s is a minimal, idiomatic Scala interface for HTTP services. Http4s is Scala's answer to Ruby's Rack, Python's WSGI, Haskell's WAI, and Java's Servlets.

val http = HttpRoutes.of {
  case GET -> Root / "hello" =>
    Ok("Hello, better world.")
}

Learn more at http4s.org.

If you run into any difficulties please enable partial unification in your build.sbt (not needed for Scala 2.13 and beyond, because Scala 2.13.0+ has partial unification switched on by default)

scalacOptions ++= Seq("-Ypartial-unification")

Requirements

Running the blaze backend requires a modern, supported version of the JVM to build and run, as it relies on server APIs unavailable before JDK8u252. Any JDK newer than JDK8u252, including 9+ is supported.

Code of Conduct

http4s is proud to be a Typelevel project. We are committed to providing a friendly, safe and welcoming environment for all, and ask that the community adhere to the Scala Code of Conduct.

License

This software is licensed under the Apache 2 license, quoted below.

Copyright 2013-2021 http4s [https://http4s.org]

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

[http://www.apache.org/licenses/LICENSE-2.0]

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Acknowledgments

Special thanks to YourKit for supporting this project's ongoing performance tuning efforts with licenses to their excellent product.

http4s-jdk-http-client's People

Contributors

Stargazers

Watchers

Forkers

rossabaker scala-steward sullis saraiva132 yilinwei amesgen jhnsmth billzabob isomarcte hamnis armanbilge mi-char isgasho rivero-ag hnaderi ken1ma ybasket

http4s-jdk-http-client's Issues

Deadlock eventually occurs in `JdkHttpClient` for certain requests

I am currently sharing the Java 8 HTTP client to create both the WebSocket client and the HTTP client. I make a single request with the HTTP client followed by connecting to a WebSocket and sending messages periodically. Eventually (always around 400 seconds later), everything hangs and CPU usage is locked at 100%.

This doesn't occur if:

I create a separate client and WebSocket client using .simple[IO]
I don't make the initial request with the client
I request other URLs with the client

Since it doesn't happen with every URL, I'm assuming it has something to do with the endpoint I'm hitting. It gives the following response:

Response(
  status=200,
  headers=Headers(
    cf-cache-status: DYNAMIC,
    cf-ray: 57d4a1eb4eaaa062-SLC,
    connection: keep-alive,
    content-type: application/json,
    date: Wed, 01 Apr 2020 19:14:20 GMT,
    expect-ct: max-age=604800,
    report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct",
    server: cloudflare,
    set-cookie: <REDACTED>,
    set-cookie: <REDACTED>,
    strict-transport-security: max-age=31536000; includeSubDomains,
    transfer-encoding: chunked,
    via: 1.1 google,
    x-envoy-upstream-service-time: 26,
    x-ratelimit-bucket: 41f9cd5d28af77da04563bcb1d67fdfd,
    x-ratelimit-limit: 2,
    x-ratelimit-remaining: 1,
    x-ratelimit-reset: 1585768466,
    x-ratelimit-reset-after: 5
  )
)

{
  "url": "wss://gateway.discord.gg",
  "shards": 1,
  "session_start_limit": {
    "total": 1000,
    "remaining": 992,
    "reset_after": 23572680
  }
}

I tried to put together a minimized example of what I'm seeing here:

import cats.effect._
import cats.implicits._
import fs2.Stream
import java.net.http.HttpClient
import org.http4s.client.jdkhttpclient._
import org.http4s.client.jdkhttpclient.WSFrame._
import org.http4s._
import org.http4s.headers._
import org.http4s.implicits._
import scala.concurrent.duration._

object Main extends IOApp {
  override def run(args: List[String]): IO[ExitCode] = {
    IO(HttpClient.newHttpClient).map(client => (JdkHttpClient[IO](client), JdkWSClient[IO](client))).flatMap {
      case (client, wsClient) =>
        client.expect[String](Request[IO](uri = uri"https://discordapp.com/api/gateway/bot", headers = Headers.of(headers))) >> connection(wsClient)
          .compile
          .drain
          .as(ExitCode.Success)
    }
  }

  // Removed since it's private info
  val token: String = ???

  val headers =
    Authorization(Credentials.Token("Bot".ci, token))

  def connection(wsClient: WSClient[IO]) =
    Stream
      .resource(wsClient.connectHighLevel(WSRequest(uri"wss://echo.websocket.org")))
      .flatMap { connection =>
        printResponses(connection).concurrently(
          Stream
            .awakeEvery[IO](1.seconds)
            .evalMap(time => connection.send(Text(s"${time.toSeconds} seconds")))
        )
      }

  def printResponses(connection: WSConnectionHighLevel[IO]): Stream[IO, Unit] = {
    connection.receiveStream
      .collect {
        case Text(data, _) => data
      }
      .evalMap(s => IO(println(s)))
  }

}

This prints out:

1 seconds
2 seconds
3 seconds
4 seconds
5 seconds
6 seconds
7 seconds
8 seconds
...
393 seconds
394 seconds
395 seconds
396 seconds
397 seconds

And then it hangs and CPU usage is stuck at 100%

JDK WebSocket is not sufficiently "low-level" to implement `WSClient`

Currently we are offering an implementation of the "low-level" WSClient instead of the WSClientHighLevel:

http4s-jdk-http-client/core/src/main/scala/org/http4s/jdkhttpclient/JdkWSClient.scala

Lines 45 to 48 in b360b86

  /** Create a new `WSClient` backed by a JDK 11+ http client. */ 

 def apply[F[_]]( 

 jdkHttpClient: HttpClient 

 )(implicit F: Async[F]): WSClient[F] =

It's not stated explicitly, but the implication is that when using the "low-level" interface the user is responsible for handling pings and close frames. For example, only the high-level interface promises to do this and does so in the default implementation.

However, the JDK WebSocket already handles those for us.

WebSocket handles received Ping and Close messages automatically (as per the WebSocket Protocol) by replying with Pong and Close messages. If the listener receives Ping or Close messages, no mandatory actions from the listener are required.

This defies expectations and causes bugs: in the high-level connection implementation pings and close frames are handled manually. But because the JDK client is also handling these, that causes them to be handled to twice, which leads to protocol violations and I/O errors.

My best proposal to fix this is to make a breaking bump and offer only the WSClientHighLevel going forward.

h/t @yurique for reporting and investigation.

Bug in multipart POST

In version 0.3.7, multipart POST does not seem to work. Using the standard http4s client everything is fine.

    val multipart = Multipart[IO](Vector(Part.formData("form", """foo""")))
    
    val request: Request[IO] = Request[IO](POST, url).withEntity(multipart).withHeaders(multipart.headers)

    // http4s standard client works fine

    BlazeClientBuilder[IO](global).resource.use { client0 =>
        client0.expect[String](request)
    }

    // http4s-jdk-http-client client 0.3.7  doesen't work

    val baseClient = HttpClient.newBuilder().version(HttpClient.Version.HTTP_2)
    val client0: Client[IO] = JdkHttpClient(baseClient.build())
    client0.expect[String](request)

Java 21+: Ensure `simple` client is closed

Java 21 adds methods to close a HttpClient and also makes it implement AutoCloseable: https://bugs.openjdk.org/browse/JDK-8267140

Ideally, http4s-jdk-http-client should leverage this and expose the simple client wrapped in a Resource to ensure the client is properly closed and cleaned up after use. Not sure how this can be achieved without requiring Java 21+ though, needs research.

Releasing from main as compatible with http4s 1.0.0-M31

Hi,

The current (March 1st 2022) main branch remains unreleased so we have no versions compatible with http4s 1.0.0-M31.

Can the project owners consider releasing from main a new milestone please?

The particular interest we have is support for proxies in the client. To c;confirm, our internal code works well with a custom local version built directly from main.

Cheers

JdkWSClient will occasionally try to send a CLOSED frame on an already closed connection

This is odd to me because it looks like there is logic to make sure that this doesn't happen.

http4s-jdk-http-client version 0.3.0
openjdk version "14" 2020-03-17
OpenJDK Runtime Environment (build 14+36-1461)
OpenJDK 64-Bit Server VM (build 14+36-1461, mixed mode, sharing)

Here's the stack trace I'm getting:

java.io.IOException: closed output
        at java.net.http/jdk.internal.net.http.RawChannelTube.write(RawChannelTube.java:340)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl.write(TransportImpl.java:114)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl$SendTask.tryCompleteWrite(TransportImpl.java:597)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl$SendTask.run(TransportImpl.java:547)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl.sendClose(TransportImpl.java:271)
        at java.net.http/jdk.internal.net.http.websocket.WebSocketImpl.sendClose0(WebSocketImpl.java:337)
        at java.net.http/jdk.internal.net.http.websocket.WebSocketImpl.sendClose(WebSocketImpl.java:307)
        at org.http4s.client.jdkhttpclient.JdkWSClient$.$anonfun$apply$17(JdkWSClient.scala:94)
        at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:87)
        at cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:359)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:380)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:323)
        at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:139)
        at cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:359)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:380)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:323)
        at cats.effect.internals.Callback$AsyncIdempotentCallback.run(Callback.scala:130)
        at cats.effect.internals.Trampoline.cats$effect$internals$Trampoline$$immediateLoop(Trampoline.scala:67)
        at cats.effect.internals.Trampoline.startLoop(Trampoline.scala:35)
        at cats.effect.internals.TrampolineEC$JVMTrampoline.super$startLoop(TrampolineEC.scala:89)
        at cats.effect.internals.TrampolineEC$JVMTrampoline.$anonfun$startLoop$1(TrampolineEC.scala:89)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)
        at cats.effect.internals.TrampolineEC$JVMTrampoline.startLoop(TrampolineEC.scala:89)
        at cats.effect.internals.Trampoline.execute(Trampoline.scala:43)
        at cats.effect.internals.TrampolineEC.execute(TrampolineEC.scala:42)
        at cats.effect.internals.Callback$AsyncIdempotentCallback.apply(Callback.scala:136)
        at cats.effect.internals.Callback$AsyncIdempotentCallback.apply(Callback.scala:125)
        at cats.effect.concurrent.Deferred$ConcurrentDeferred.$anonfun$unsafeRegister$1(Deferred.scala:201)
        at cats.effect.concurrent.Deferred$ConcurrentDeferred.$anonfun$unsafeRegister$1$adapted(Deferred.scala:201)
        at cats.effect.concurrent.Deferred$ConcurrentDeferred.$anonfun$notifyReadersLoop$1(Deferred.scala:236)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:87)
        at cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:359)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:380)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:323)
        at cats.effect.internals.IOShift$Tick.run(IOShift.scala:35)
        at cats.effect.internals.PoolUtils$$anon$2$$anon$3.run(PoolUtils.scala:52)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:832)

Re-enable websocket mdoc

Just a reminder to activate mdoc again for the websocket stuff. I had to disable it b/c mdoc doesn't work with package privates.

Java IOException

I'm getting a Java IOException when the websocket is closed from the server and I presume that the client is sending a close message. I'm haven't read the RFC to ascertain the correct behaviour (whether this is a misbehaving websocket server, or it can happen).

java.io.IOException: closed output
        at java.net.http/jdk.internal.net.http.RawChannelTube.write(RawChannelTube.java:340)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl.write(TransportImpl.java:114)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl$SendTask.tryCompleteWrite(TransportImpl.java:597)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl$SendTask.run(TransportImpl.java:547)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
        at java.net.http/jdk.internal.net.http.websocket.TransportImpl.sendClose(TransportImpl.java:271)
        at java.net.http/jdk.internal.net.http.websocket.WebSocketImpl.sendClose0(WebSocketImpl.java:337)
        at java.net.http/jdk.internal.net.http.websocket.WebSocketImpl.sendClose(WebSocketImpl.java:307)
        at org.http4s.client.jdkhttpclient.JdkWSClient$.$anonfun$apply$17(JdkWSClient.scala:91)
        at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:87)
        at cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:355)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:376)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:316)
        at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:136)
        at cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:355)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:376)
        at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:316)
        at cats.effect.internals.Callback$AsyncIdempotentCallback.run(Callback.scala:131)
        at cats.effect.internals.Trampoline.cats$effect$internals$Trampoline$$immediateLoop(Trampoline.scala:70)
        at cats.effect.internals.Trampoline.startLoop(Trampoline.scala:36)

Implement WebSocket Client API from http4s-client

It should be straightforward to implement, since the model/API was heavily based on the one currently the jdk-http-client.

@amesgen do you have any strategy in mind wrt compatibility?

sbt-release is not actually releasing

I thought I released last night, but I had do do an additional sonatypeBundleRelease. Would also be nice if it published the site, which is also a separate step.

Fs2 > 2.4.5 Binary Compatibility Issue

Using this project with a transitive dependency on fs2 >= 2.4.5 (current release uses 2.4.2) will create a runtime failure. This isn't an issue with the jdk client directly, but I thought it would be good to raise awareness here.

typelevel/fs2#2177

On the one hand, cutting a new release against 2.4.6 of fs2 would solve this issue, but on the other depending on what fs2 decides to do that might break again.

HTTP body not read to completion. Dropping connection.

A batch of POST requests cause connection to be dropped due to unconsumed request body:
HTTP body not read to completion. Dropping connection.
Full log:

[2022-08-04 11:36:06,274] INFO  [io-compute-7] o.h.b.c.n.NIO1SocketServerGroup:276 - Service bound to address /127.0.0.1:8080
[2022-08-04 11:36:06,286] INFO  [io-compute-7] o.h.b.s.BlazeServerBuilder:422 - 
  _   _   _        _ _
 | |_| |_| |_ _ __| | | ___
 | ' \  _|  _| '_ \_  _(_-<
 |_||_\__|\__| .__/ |_|/__/
             |_|
[2022-08-04 11:36:06,308] INFO  [io-compute-7] o.h.b.s.BlazeServerBuilder:425 - http4s v0.23.14 on blaze v0.23.12 started at http://127.0.0.1:8080/
[2022-08-04 11:36:07,271] INFO  [io-compute-7] o.h.b.s.Http1ServerStage$$anon$1:271 - HTTP body not read to completion. Dropping connection.
0: Response(status=200, httpVersion=HTTP/1.1, headers=Headers(content-length: 0, date: Thu, 04 Aug 2022 09:36:07 GMT))
[2022-08-04 11:36:07,340] INFO  [io-compute-7] o.h.b.s.Http1ServerStage$$anon$1:271 - HTTP body not read to completion. Dropping connection.
1: Response(status=200, httpVersion=HTTP/1.1, headers=Headers(content-length: 0, date: Thu, 04 Aug 2022 09:36:07 GMT))
[2022-08-04 11:36:07,354] INFO  [io-compute-6] o.h.b.s.Http1ServerStage$$anon$1:271 - HTTP body not read to completion. Dropping connection.
2: Response(status=200, httpVersion=HTTP/1.1, headers=Headers(content-length: 0, date: Thu, 04 Aug 2022 09:36:07 GMT))
[2022-08-04 11:36:07,367] INFO  [io-compute-8] o.h.b.s.Http1ServerStage$$anon$1:271 - HTTP body not read to completion. Dropping connection.
....
23: Response(status=200, httpVersion=HTTP/1.1, headers=Headers(content-length: 0, date: Thu, 04 Aug 2022 09:36:07 GMT))
[2022-08-04 11:36:07,638] INFO  [blaze-acceptor-0-0] o.h.b.c.ServerChannel:211 - Closing NIO1 channel /127.0.0.1:8080
[2022-08-04 11:36:07,639] INFO  [io-compute-2] o.h.b.c.n.SelectorLoop:72 - Shutting down SelectorLoop blaze-selector-0
...
java.io.IOException: HTTP/1.1 header parser received no bytes
	at java.net.http/jdk.internal.net.http.common.Utils.wrapWithExtraDetail(Utils.java:348)
	at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.onReadError(Http1Response.java:675)
	at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.checkForErrors(Http1AsyncReceiver.java:302)
	at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.flush(Http1AsyncReceiver.java:268)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:205)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:230)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
	at async_ @ org.http4s.jdkhttpclient.package$.$anonfun$fromCompletableFuture$1(package.scala:39)
	at delay @ org.http4s.jdkhttpclient.JdkHttpClient$.$anonfun$apply$21(JdkHttpClient.scala:243)
	at use @ Caused by: java.io.EOFException: EOF reached while reading
	at java.net.http/jdk.internal.net.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:596)
	at 
...
java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.java:640)
	at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:845)
	at java.net.http/jdk.internal.net.http.SocketTube$SocketFlowTask.run(SocketTube.java:181)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:230)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:303)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:256)
	at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:774)
	at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:957)
	at java.net.http/jdk.internal.net.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:253)
	at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:979)
	at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.lambda$run$3(HttpClientImpl.java:934)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:934)

And this - once in a while - makes client request lost on the wire.
For this to happen:

request should have non-empty body
response should be generated before body is consumed;

Checked with blaze and ember servers.

import cats.effect._
import cats.implicits._
import com.comcast.ip4s._
import org.http4s.blaze.server.BlazeServerBuilder
import org.http4s.client.dsl.Http4sClientDsl
import org.http4s.dsl.io._
import org.http4s.ember.server._
import org.http4s.implicits._
import org.http4s.jdkhttpclient.JdkHttpClient
import org.http4s.server.Router
import org.http4s.{HttpRoutes, Uri}

import java.net.http.HttpClient
import java.net.http.HttpClient.Version

object HttpConnectionCloseBodyDrain extends IOApp.Simple {

  def run: IO[Unit] = {
    val routes = HttpRoutes.of[IO] {
      case POST -> Root / "test" / "post" =>
        Ok()
    }

    val client = JdkHttpClient[IO](HttpClient.newBuilder().version(Version.HTTP_1_1).build())

    val postRequests = client.use { client =>
      val clientDsl = Http4sClientDsl[IO]
      import clientDsl._

      val req = POST("dummy body", Uri.fromString("http://localhost:8080/test/post?param=XXX)").toOption.get)
      val reqResource = client.run(req)
      List.from(0 until 100).traverse_ { i =>
        reqResource.use { response =>
          IO.println(s"$i: $response")
        }
      }
    }

    BlazeServerBuilder[IO]
      .bindHttp(8080, "localhost")
      .withHttpApp(Router("/" -> routes).orNotFound)
      .resource
      .use { _ => postRequests }

    /*
    EmberServerBuilder
      .default[IO]
      .withHost(ipv4"127.0.0.1")
      .withPort(port"8080")
      .withHttpApp(Router("/" -> routes).orNotFound)
      .build
      .use { _ => postRequests }
     */
  }
}

BodyPublishers.fromPublisher fails with Content-Length: 0

java.lang.IllegalArgumentException: non-positive contentLength: 0
at java.net.http.HttpRequest$BodyPublishers.fromPublisher(HttpRequest.java:539)
at org.http4s.client.jdkhttpclient.JdkHttpClient$.$anonfun$apply$4(JdkHttpClient.scala:67)
at org.http4s.client.jdkhttpclient.JdkHttpClient$.$anonfun$apply$4$adapted(JdkHttpClient.scala:67)
at scala.Option.fold(Option.scala:263)
at org.http4s.client.jdkhttpclient.JdkHttpClient$.$anonfun$apply$1(JdkHttpClient.scala:67)

Use `ClientRouteTestBattery`

see http4s/http4s#4140 (comment)

also add a regression test for #395

"java.io.IOException: HTTP/1.1 header parser received no bytes" over HTTPS Proxy

Hi, I encounter java.io.IOException: HTTP/1.1 header parser received no bytes when making an API call through an authenticated HTTPS proxy.

This is how I instantiated the client.

Resource
        .eval(F.delay {
          val proxyHost = config.proxyHost
          val proxyPort = 8443
          HttpClient
            .newBuilder()
            .version(HttpClient.Version.HTTP_1_1)
            .sslContext(buildSSLContext())
            .proxy(ProxySelector.of(new InetSocketAddress(proxyHost, proxyPort)))
            .build()
        })
        .flatMap(JdkHttpClient(_))

private def buildSSLContext(): SSLContext = {
    val keyStore = KeyStore.getInstance(KeyStore.getDefaultType())
    keyStore.load(null)

    val certFileStream = this.getClass.getResourceAsStream("/path/to/cert.pem")
    val certificate    = CertificateFactory.getInstance("X.509").generateCertificate(certFileStream)
    keyStore.setCertificateEntry("cert", certificate);

    val tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm())
    tmf.init(keyStore)
    val trustManagers = tmf.getTrustManagers()

    val sslContext = SSLContext.getInstance("TLS")
    sslContext.init(null, trustManagers, null)
    sslContext
  }

I have also tried removing TLS1.3 by following the this approach but to no avail.

This is largely how I set the header and make the API call

Uri.fromString(url).liftTo[F].flatMap { uri =>
            val reqEntity =
                    MyRequest(
                        ... (omitted)
                    )

            val requestHeaders = {
                    val proxyCredentials = BasicCredentials(
                        config.proxyUsername,
                        config.proxyPassword
                    )
                    Headers(
                        Header.Raw(ci"Proxy-Authorization", s"Basic ${proxyCredentials.token}"),
                        Header.Raw(ci"X-API-key", config.apiKey),
                    )
                }

            val request = Request[F](
                method = Method.POST,
                uri = uri,
                headers = requestHeaders
            )
            .withEntity(reqEntity)

            client.run(request).use { response =>
                ...
            }
            .recoverWith {
                case e =>
                    for {
                        _ <- Logger[F].error(s"Error when calling API: $e: ${e.getMessage}") // the exception is logged here
                        err <- Exception.APICallError.raise[F, MyResponse] // throw my custom exception
                    } yield err
            }
        }

Any help would be appreciated!

Fix snapshot publishing

Run echo $PGP_SECRET | base64 --decode | gpg --import
base64: invalid input

I exported a key and base64 encoded it, but apparently not in the format necessary to create something that works with that command. Does anybody remember the correct incantation?

When is a new version going to be released?

We need a newer version of fs2-core in our project and the older incompatible version is brought by http4s-jdk-http-client.

Migration back to gh-pages

The main site is down, and nobody else can see Netlify, so I migrated back to gh-pages so I'm not the single point of failure.

I've set up a CNAME, but GitHub still won't accept it:

> dig jdk-http-client.http4s.org +noall +answer -t A

; <<>> DiG 9.10.6 <<>> jdk-http-client.http4s.org +noall +answer -t A
;; global options: +cmd
jdk-http-client.http4s.org. 2894 IN	CNAME	http4s.github.io.
http4s.github.io.	2894	IN	A	185.199.108.153
http4s.github.io.	2894	IN	A	185.199.109.153
http4s.github.io.	2894	IN	A	185.199.110.153
http4s.github.io.	2894	IN	A	185.199.111.153

The /stable and /latest redirects will no longer work. I see a /latest directory in gh-pages.

If I ignore the security warning (which I expect will resolve itself), I get a 404. We might need an index.html or some other landing page that redirects to a particular version.

	/** Create a new `WSClient` backed by a JDK 11+ http client. */
	def apply[F[_]](
	jdkHttpClient: HttpClient
	)(implicit F: Async[F]): WSClient[F] =