Giter Club home page Giter Club logo

kitura-nio's Introduction

Kitura

Swift 5.0 APIDoc Build Status - Master macOS Linux Apache 2 Slack Status

Kitura-NIO

Kitura-NIO is a SwiftNIO based networking library for Kitura. Kitura-NIO adopts the same API as KituraNet, making the transition from KituraNet to Kitura-NIO seamless. While Kitura-NIO shares some code with Kitura-Net, the core comprising of HTTPServer, ClientRequest/ClientResponse and TLS support have been implemented using SwiftNIO. Kitura-NIO uses NIOSSL for TLS support.

We expect most of our users to require higher level concepts such as routing, templates and middleware. These are not provided in Kitura-NIO. If you want to use those facilities you should be coding at the Kitura level, for this please see the Kitura project. Kitura-NIO, like Kitura-net, underpins Kitura which offers a higher abstraction level to users.

Kitura-NIO 2 has been tested with Swift 5. If you are using Swift 4, please use Kitura-NIO 1. See the release history for details.

Features

  • Port Listening
  • HTTP Server support (request and response)
  • Basic HTTP client support

Using Kitura-NIO

With Kitura 2.5 and future releases, to run on top of Kitura-NIO (instead of Kitura-Net) all you need to do is set an environment variable called KITURA_NIO before building your Kitura application:

    export KITURA_NIO=1 && swift build

If you have already built your Kitura application using Kitura-Net and want to switch to using KITURA_NIO, you need to update the package before building:

    export KITURA_NIO=1 && swift package update && swift build

Using the environment variable we make sure that only one out of Kitura-NIO and Kitura-Net is linked into the final binary.

Please note that though Kitura-NIO has its own GitHub repository, the package name is KituraNet. This is because the Kitura-NIO and Kitura-Net are expected to provide identical APIs, and it makes sense if they share the package name too.

Getting Started

Visit www.kitura.io for reference documentation.

Contributing to Kitura-NIO

We'd be more than happy to receive bug reports, enhancement requests and pull requests!

  1. Clone this repository.

$ git clone https://github.com/Kitura/Kitura-NIO && cd Kitura-NIO

  1. Build and run tests.

$ swift test

You may also want to run the tests in parallel: $ swift test --parallel

In some Linux environments, a low open file limit could cause test failures. See this.

Community

We'd really love to hear feedback from you.

Join the Kitura on Swift Forums or our Slack to meet the team!

License

This library is licensed under Apache 2.0. The full license text is available in LICENSE.

kitura-nio's People

Contributors

bridger avatar dannys42 avatar djones6 avatar helenmasters avatar ianpartridge avatar kant avatar nethraravindran avatar saihemak avatar shial4 avatar swiftdevops avatar taborkelly avatar vipinmenon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kitura-nio's Issues

KituraKit test failures with Kitura-NIO

Instructions to reproduce the failure:

  1. git clone https://github.com/IBM-Swift/KituraKit
  2. cd KituraKit
  3. export KITURA_NIO=1
  4. swift test

A lot of tests fail on both macOS and Linux.

Improve Kitura-NIO's Jazzy docs

Kitura-NIO's docs must be identical to Kitura-net's, given that the APIs are expected to be identical. There are a lot of gaps to fill.

ClientResponse.status wrongly returns -1

Test used:

import Kitura
import KituraNet
import Dispatch

// configure the router
let router = Router()
router.get("/test") { request, response, _ in
    try response.status(.OK).end()
}

// start Kitura
DispatchQueue(label: "server").async {
    Kitura.addHTTPServer(onPort: 8080, with: router)
    Kitura.run()
}

//let the server come up
sleep(5)

//make a request
_ = HTTP.get("http://user:pass@localhost:8080/test") { response in
    print(response?.status ?? "no response")
}

Output with Kitura-net (which is expected):

200

Output with Kitura-NIO:

-1

KituraNIO should close the connection if it receives an error in its last ChannelHandler

In SwiftNIO it's generally a good idea that on errorCaught, the Channel should be closed to prevent leaking network connections that are in some error states. In KituraNIO this should be added to HTTPRequestHandler and HTTPClientHandler.

Why? Generally, ChannelHandlers fire errors in cases where they don't know how to recover themselves. If another, later ChannelHandler knows how to recover, that ChannelHandler would consume the error. In other words: If an error reaches the end of the pipeline, nobody knew how to handle that error and closing the Channel is a good default.

This has always been true in SwiftNIO but in NIO 2 this will become especially important because ByteToMessageDecoders (and the HTTP decoders) won't close the connections themselves anymore. Why is that? If the decoders themselves close the connection, then there's no way for the user to opt out.

The code to implement this is very straightforward:

public func errorCaught(ctx: ChannelHandlerContext, error: Error) {
    switch error {
    case let e as SomeErrorTypeIKnowHowToHandle where e.someCondition == expectedValue:
        // handle error
    case let e as SomeOtherErrorIKnowHowToHandle:
        // handle that error
    default:
        // log error?
        ctx.close(promise: nil)
    }
}

Differences in default request headers between Kitura-NIO and Kitura-net

I tried printing the default HTTP request headers using this simple app:

import Kitura
import KituraNet
import Dispatch

// configure the router
let router = Router()
router.get("/test") { request, response, _ in
    for header in request.headers {
        print(header)
    }
    try response.status(.OK).end()
}

// start Kitura
DispatchQueue(label: "server").async {
    Kitura.addHTTPServer(onPort: 8080, with: router)
    Kitura.run()
}

//let the server come up
sleep(2)

//make a request
_ = HTTP.get("http://user:pass@localhost:8080/test") { response in
    print("Response = ", response?.statusCode ?? "no response")
}

With Kitura-net I see:

("Host", Optional("localhost:8080"))
("Accept", Optional("*/*"))
("Authorization", Optional("Basic dXNlcjpwYXNz"))

and with Kitura-NIO I see:

("Authorization", Optional("Basic dXNlcjpwYXNz"))
("User-Agent", Optional("Kitura"))
("Host", Optional("localhost"))

This needs investigation.

Make the HTTPServer more configurable

Performance tests have shown that features that may be critical for certain workloads may prove to be an overhead for others. We need to make these features optional and provide a mechanism for users to opt in. Some of the configurable features:

  1. Idle socket detection and cleanup
  2. HTTP pipelining
  3. HTTP auto-error handling (e.g WebSocket upgrade workloads)

Secondly, performance tests have also shown that the optimum number of event loop threads at which the performance could peak is benchmark specific. This is because Kitura-NIO consciously choses to move the router handlers invocations to a Dispatch queue (and not on the event loop themselves). We then see two sets of tasks competing for CPU: one set are the event loops that accept incoming connections and deliver responses, the other set are the Dispatch threads that actually run the router handlers. Tests have proven that starving one set against the other slows down performance and it seems that, to maintain a balance, the number of event loop threads will have to be tuned experimentally by the user.

Of course, these allude to an API change.

cc @ianpartridge @djones6

Pipelining tests are flaky

Pipelining tests seem to be failing, with a timeout, more often than earlier, on Travis. These failures aren't reproducible locally.

regression: PR #143

Kitura tests fail with the changes related to #143

We need to do the following:

  1. Debug all Kitura failures and fix them
  2. Subsequently measure performance and decide if the new changes could be acceptable

I've reverted #143 using #153

Create a performance testing plan for Kitura-NIO

As discussed and decided upon, this is the priority in terms of performance testing for Kitura-NIO:

Measurements with microbenchmarks with Kitura-NIO vs. Kitura-net
Measurements with the same microbenchmarks on Express (All the above can be on Ubuntu 16.04)
Measurements of comparing 18.04 to 16.04
Getting TechEmpower running on Kitura-NIO and comparing to Kitura-net

PipeliningTests.testPipelining has a data race

TSan on Swift 4.2 on macOS complains about data races in testPipelining:

problem

Test Case '-[KituraNetTests.PipeliningTests testPipelining]' started.
==================
WARNING: ThreadSanitizer: data race (pid=75673)
  Read of size 8 at 0x7b080003f1d0 by thread T30:
    #0 Delegate.handle(request:response:) PipeliningTests.swift:126 (KituraNetPackageTests:x86_64+0x10c4f9)
    #1 protocol witness for ServerDelegate.handle(request:response:) in conformance Delegate <compiler-generated> (KituraNetPackageTests:x86_64+0x10ced6)
    #2 closure #1 in HTTPRequestHandler.channelRead(ctx:data:) HTTPRequestHandler.swift:86 (KituraNetPackageTests:x86_64+0x54f7a)
    #3 partial apply for closure #1 in HTTPRequestHandler.channelRead(ctx:data:) <compiler-generated> (KituraNetPackageTests:x86_64+0x5501d)
    #4 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x3a060)
    #5 __tsan::invoke_and_release_block(void*) <null>:10184352 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x685ab)
    #6 _dispatch_client_callout <null>:10184352 (libdispatch.dylib:x86_64+0x3dce)

  Previous write of size 8 at 0x7b080003f1d0 by thread T31:
    #0 Delegate.handle(request:response:) PipeliningTests.swift:130 (KituraNetPackageTests:x86_64+0x10c7e7)
    #1 protocol witness for ServerDelegate.handle(request:response:) in conformance Delegate <compiler-generated> (KituraNetPackageTests:x86_64+0x10ced6)
    #2 closure #1 in HTTPRequestHandler.channelRead(ctx:data:) HTTPRequestHandler.swift:86 (KituraNetPackageTests:x86_64+0x54f7a)
    #3 partial apply for closure #1 in HTTPRequestHandler.channelRead(ctx:data:) <compiler-generated> (KituraNetPackageTests:x86_64+0x5501d)
    #4 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x3a060)
    #5 __tsan::invoke_and_release_block(void*) <null>:10184352 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x685ab)
    #6 _dispatch_client_callout <null>:10184352 (libdispatch.dylib:x86_64+0x3dce)

  Location is heap block of size 24 at 0x7b080003f1c0 allocated by main thread:
    #0 malloc <null>:10184384 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x4adca)
    #1 swift_slowAlloc <null>:10184384 (libswiftCore.dylib:x86_64+0x3093f8)
    #2 PipeliningTests.testPipelining() PipeliningTests.swift:48 (KituraNetPackageTests:x86_64+0x106ba5)
    #3 @objc PipeliningTests.testPipelining() <compiler-generated> (KituraNetPackageTests:x86_64+0x109470)
    #4 __invoking___ <null>:10184384 (CoreFoundation:x86_64h+0x4ad1b)

  Thread T30 (tid=24724834, running) is a GCD worker thread

  Thread T31 (tid=24724833, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race PipeliningTests.swift:126 in Delegate.handle(request:response:)
==================
==================
WARNING: ThreadSanitizer: Swift access race (pid=75673)
  Modifying access of Swift variable at 0x7b10000ed470 by thread T24:
    #0 HTTPRequestHandler.updateKeepAliveState() HTTPRequestHandler.swift:135 (KituraNetPackageTests:x86_64+0x562f7)
    #1 HTTPServerResponse.sendResponse(channel:handler:status:withBody:) HTTPServerResponse.swift:193 (KituraNetPackageTests:x86_64+0x771b7)
    #2 closure #1 in HTTPServerResponse.end() HTTPServerResponse.swift:146 (KituraNetPackageTests:x86_64+0x75063)
    #3 partial apply for closure #1 in HTTPServerResponse.end() <compiler-generated> (KituraNetPackageTests:x86_64+0x7543e)
    #4 closure #1 in HTTPServerResponse.execute(on:_:) HTTPServerResponse.swift:106 (KituraNetPackageTests:x86_64+0x73762)
    #5 partial apply for closure #1 in HTTPServerResponse.execute(on:_:) <compiler-generated> (KituraNetPackageTests:x86_64+0x7888b)
    #6 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x8b1b4)
    #7 thunk for @escaping @callee_guaranteed () -> ()partial apply <compiler-generated> (KituraNetPackageTests:x86_64+0x21e598)
    #8 thunk for @escaping @callee_guaranteed (@in_guaranteed ()) -> (@out ()) <compiler-generated> (KituraNetPackageTests:x86_64+0x89f67)
    #9 partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed ()) -> (@out ()) <compiler-generated> (KituraNetPackageTests:x86_64+0x2111fb)
    #10 closure #3 in SelectableEventLoop.run() EventLoop.swift:705 (KituraNetPackageTests:x86_64+0x210fe2)
    #11 partial apply for closure #3 in SelectableEventLoop.run() <compiler-generated> (KituraNetPackageTests:x86_64+0x21108b)
    #12 thunk for @callee_guaranteed () -> (@error @owned Error) <compiler-generated> (KituraNetPackageTests:x86_64+0x143926)
    #13 thunk for @callee_guaranteed () -> (@error @owned Error)partial apply <compiler-generated> (KituraNetPackageTests:x86_64+0x21113e)
    #14 closure #1 in withAutoReleasePool<A>(_:) EventLoop.swift:430 (KituraNetPackageTests:x86_64+0x2087a8)
    #15 partial apply for closure #1 in withAutoReleasePool<A>(_:) <compiler-generated> (KituraNetPackageTests:x86_64+0x21e659)
    #16 autoreleasepool<A>(invoking:) <null>:11253008 (libswiftObjectiveC.dylib:x86_64+0x3f2e)
    #17 SelectableEventLoop.run() EventLoop.swift:704 (KituraNetPackageTests:x86_64+0x20f571)
    #18 closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:initializer:) EventLoop.swift:871 (KituraNetPackageTests:x86_64+0x215182)
    #19 partial apply for closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:initializer:) <compiler-generated> (KituraNetPackageTests:x86_64+0x21dc39)
    #20 thunk for @escaping @callee_guaranteed (@guaranteed Thread) -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x215b7b)
    #21 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Thread) -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x2fc528)
    #22 closure #1 in static Thread.spawnAndRun(name:body:) Thread.swift:105 (KituraNetPackageTests:x86_64+0x2fcee4)
    #23 @objc closure #1 in static Thread.spawnAndRun(name:body:) <compiler-generated> (KituraNetPackageTests:x86_64+0x2fcf90)

  Previous read of size 8 at 0x7b10000ed470 by thread T30:
    #0 HTTPRequestHandler.keepAliveState.getter <compiler-generated> (KituraNetPackageTests:x86_64+0x52d26)
    #1 HTTPServerResponse.end() HTTPServerResponse.swift:137 (KituraNetPackageTests:x86_64+0x73ed2)
    #2 HTTPServerResponse.end(text:) HTTPServerResponse.swift:116 (KituraNetPackageTests:x86_64+0x7388a)
    #3 protocol witness for ServerResponse.end(text:) in conformance HTTPServerResponse <compiler-generated> (KituraNetPackageTests:x86_64+0x7819f)
    #4 Delegate.handle(request:response:) PipeliningTests.swift:126 (KituraNetPackageTests:x86_64+0x10c6e0)
    #5 protocol witness for ServerDelegate.handle(request:response:) in conformance Delegate <compiler-generated> (KituraNetPackageTests:x86_64+0x10ced6)
    #6 closure #1 in HTTPRequestHandler.channelRead(ctx:data:) HTTPRequestHandler.swift:86 (KituraNetPackageTests:x86_64+0x54f7a)
    #7 partial apply for closure #1 in HTTPRequestHandler.channelRead(ctx:data:) <compiler-generated> (KituraNetPackageTests:x86_64+0x5501d)
    #8 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x3a060)
    #9 __tsan::invoke_and_release_block(void*) <null>:11253008 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x685ab)
    #10 _dispatch_client_callout <null>:11253008 (libdispatch.dylib:x86_64+0x3dce)

  Location is heap block of size 59 at 0x7b10000ed440 allocated by thread T24:
    #0 malloc <null>:11253040 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x4adca)
    #1 swift_slowAlloc <null>:11253040 (libswiftCore.dylib:x86_64+0x3093f8)
    #2 closure #1 in HTTPServer.listen(on:) HTTPServer.swift:172 (KituraNetPackageTests:x86_64+0x61479)
    #3 partial apply for closure #1 in HTTPServer.listen(on:) <compiler-generated> (KituraNetPackageTests:x86_64+0x61d38)
    #4 thunk for @escaping @callee_guaranteed (@guaranteed Channel) -> (@owned EventLoopFuture<()>) <compiler-generated> (KituraNetPackageTests:x86_64+0x16909b)
    #5 thunk for @escaping @callee_guaranteed (@guaranteed Channel) -> (@owned EventLoopFuture<()>)partial apply <compiler-generated> (KituraNetPackageTests:x86_64+0x179508)
    #6 thunk for @escaping @callee_guaranteed (@in_guaranteed Channel) -> (@out EventLoopFuture<()>) <compiler-generated> (KituraNetPackageTests:x86_64+0x1691b7)
    #7 partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed Channel) -> (@out EventLoopFuture<()>) <compiler-generated> (KituraNetPackageTests:x86_64+0x178e88)
    #8 closure #1 in setupChildChannel #1 () in ServerBootstrap.AcceptHandler.channelRead(ctx:data:) Bootstrap.swift:263 (KituraNetPackageTests:x86_64+0x16b85a)
    #9 partial apply for closure #1 in setupChildChannel #1 () in ServerBootstrap.AcceptHandler.channelRead(ctx:data:) <compiler-generated> (KituraNetPackageTests:x86_64+0x1795c7)
    #10 thunk for @escaping @callee_guaranteed () -> (@owned EventLoopFuture<()>) <compiler-generated> (KituraNetPackageTests:x86_64+0x356a4)
    #11 thunk for @escaping @callee_guaranteed () -> (@owned EventLoopFuture<()>)partial apply <compiler-generated> (KituraNetPackageTests:x86_64+0x179680)
    #12 closure #1 in EventLoopFuture.then<A>(file:line:_:) EventLoopFuture.swift:442 (KituraNetPackageTests:x86_64+0x225a74)
    #13 partial apply for closure #1 in EventLoopFuture.then<A>(file:line:_:) <compiler-generated> (KituraNetPackageTests:x86_64+0x225ffe)
    #14 EventLoopFuture._addCallback(_:) EventLoopFuture.swift:597 (KituraNetPackageTests:x86_64+0x226882)
    #15 EventLoopFuture._whenComplete(_:) EventLoopFuture.swift:603 (KituraNetPackageTests:x86_64+0x226ba8)
    #16 EventLoopFuture.then<A>(file:line:_:) EventLoopFuture.swift:439 (KituraNetPackageTests:x86_64+0x225634)
    #17 setupChildChannel #1 () in ServerBootstrap.AcceptHandler.channelRead(ctx:data:) Bootstrap.swift:261 (KituraNetPackageTests:x86_64+0x16b687)
    #18 closure #2 in ServerBootstrap.AcceptHandler.channelRead(ctx:data:) Bootstrap.swift:287 (KituraNetPackageTests:x86_64+0x16c385)
    #19 partial apply for closure #2 in ServerBootstrap.AcceptHandler.channelRead(ctx:data:) <compiler-generated> (KituraNetPackageTests:x86_64+0x1792cb)
    #20 thunk for @escaping @callee_guaranteed () -> (@owned EventLoopFuture<()>, @error @owned Error) <compiler-generated> (KituraNetPackageTests:x86_64+0x16c3fd)
    #21 partial apply for thunk for @escaping @callee_guaranteed () -> (@owned EventLoopFuture<()>, @error @owned Error) <compiler-generated> (KituraNetPackageTests:x86_64+0x17938e)
    #22 closure #1 in EventLoop.submit<A>(_:) EventLoop.swift:306 (KituraNetPackageTests:x86_64+0x20725d)
    #23 partial apply for closure #1 in EventLoop.submit<A>(_:) <compiler-generated> (KituraNetPackageTests:x86_64+0x207459)
    #24 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x8b1b4)
    #25 thunk for @escaping @callee_guaranteed () -> ()partial apply <compiler-generated> (KituraNetPackageTests:x86_64+0x21e598)
    #26 thunk for @escaping @callee_guaranteed (@in_guaranteed ()) -> (@out ()) <compiler-generated> (KituraNetPackageTests:x86_64+0x89f67)
    #27 partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed ()) -> (@out ()) <compiler-generated> (KituraNetPackageTests:x86_64+0x2111fb)
    #28 closure #3 in SelectableEventLoop.run() EventLoop.swift:705 (KituraNetPackageTests:x86_64+0x210fe2)
    #29 partial apply for closure #3 in SelectableEventLoop.run() <compiler-generated> (KituraNetPackageTests:x86_64+0x21108b)
    #30 thunk for @callee_guaranteed () -> (@error @owned Error) <compiler-generated> (KituraNetPackageTests:x86_64+0x143926)
    #31 thunk for @callee_guaranteed () -> (@error @owned Error)partial apply <compiler-generated> (KituraNetPackageTests:x86_64+0x21113e)
    #32 closure #1 in withAutoReleasePool<A>(_:) EventLoop.swift:430 (KituraNetPackageTests:x86_64+0x2087a8)
    #33 partial apply for closure #1 in withAutoReleasePool<A>(_:) <compiler-generated> (KituraNetPackageTests:x86_64+0x21e659)
    #34 autoreleasepool<A>(invoking:) <null>:11253040 (libswiftObjectiveC.dylib:x86_64+0x3f2e)
    #35 SelectableEventLoop.run() EventLoop.swift:704 (KituraNetPackageTests:x86_64+0x20f571)
    #36 closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:initializer:) EventLoop.swift:871 (KituraNetPackageTests:x86_64+0x215182)
    #37 partial apply for closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:initializer:) <compiler-generated> (KituraNetPackageTests:x86_64+0x21dc39)
    #38 thunk for @escaping @callee_guaranteed (@guaranteed Thread) -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x215b7b)
    #39 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Thread) -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x2fc528)
    #40 closure #1 in static Thread.spawnAndRun(name:body:) Thread.swift:105 (KituraNetPackageTests:x86_64+0x2fcee4)
    #41 @objc closure #1 in static Thread.spawnAndRun(name:body:) <compiler-generated> (KituraNetPackageTests:x86_64+0x2fcf90)

  Thread T24 (tid=24725091, running) created by main thread at:
    #0 pthread_create <null>:11253088 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x2936d)
    #1 static Thread.spawnAndRun(name:body:) Thread.swift:90 (KituraNetPackageTests:x86_64+0x2fb9ae)
    #2 static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:initializer:) EventLoop.swift:857 (KituraNetPackageTests:x86_64+0x214a5f)
    #3 closure #1 in MultiThreadedEventLoopGroup.init(threadInitializers:) EventLoop.swift:911 (KituraNetPackageTests:x86_64+0x2164f6)
    #4 partial apply for closure #1 in MultiThreadedEventLoopGroup.init(threadInitializers:) <compiler-generated> (KituraNetPackageTests:x86_64+0x216624)
    #5 thunk for @callee_guaranteed (@guaranteed @escaping @callee_guaranteed (@guaranteed Thread) -> ()) -> (@owned SelectableEventLoop, @error @owned Error) <compiler-generated> (KituraNetPackageTests:x86_64+0x21673f)
    #6 partial apply for thunk for @callee_guaranteed (@guaranteed @escaping @callee_guaranteed (@guaranteed Thread) -> ()) -> (@owned SelectableEventLoop, @error @owned Error) <compiler-generated> (KituraNetPackageTests:x86_64+0x2168b9)
    #7 Collection.map<A>(_:) <null>:11253088 (libswiftCore.dylib:x86_64+0x1f50e)
    #8 MultiThreadedEventLoopGroup.init(numberOfThreads:) EventLoop.swift:891 (KituraNetPackageTests:x86_64+0x215ac3)
    #9 MultiThreadedEventLoopGroup.__allocating_init(numberOfThreads:) EventLoop.swift (KituraNetPackageTests:x86_64+0x2159a8)
    #10 HTTPServer.init() HTTPServer.swift:69 (KituraNetPackageTests:x86_64+0x58f16)
    #11 HTTPServer.__allocating_init() HTTPServer.swift (KituraNetPackageTests:x86_64+0x58998)
    #12 PipeliningTests.testPipelining() PipeliningTests.swift:47 (KituraNetPackageTests:x86_64+0x106b5f)
    #13 @objc PipeliningTests.testPipelining() <compiler-generated> (KituraNetPackageTests:x86_64+0x109470)
    #14 __invoking___ <null>:11253088 (CoreFoundation:x86_64h+0x4ad1b)

  Thread T30 (tid=24724834, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: Swift access race HTTPRequestHandler.swift:135 in HTTPRequestHandler.updateKeepAliveState()
==================
Test Case '-[KituraNetTests.PipeliningTests testPipelining]' passed (0.406 seconds).

repro

# on macOS
cd Kitura-NIO
swift test --sanitize=thread

Feature Request: access to EventLoopGroup

According to documentation, an application should generally have a single EventLoopGroup. Kitura-NIO create its own group and threads, and provides no access. Performance will suffer if every library used creates its own group.

I'd like the options:

  1. To create my own group, and pass it into Kitura.
  2. If a group is not specified and Kitura creates one, n accessor to get that group.

Close active connections on HTTPServer.stop()

Work on Kitura/SwiftyRequest#69 revealed a basic flaw in Kitura-NIO's HTTPServer. On a stop() call to the HTTPServer we seem to be closing only the listening socket. The active connections aren't closed.

The connections are probably staying alive through some valid retain cycles (how this is happening needs investigation). We must close the active connections.

test failures

I ran the following configuration (all git hashes)

Kitura-NIO
106677541e0bcd13c3a631db15427e1e8c614235
swift-nio
7895bc88fda8e057cd0c95d23849204158b72f84
swift-nio-extras
ad4edc8cb59878f660a9e5fe89a39708f475d176
swift-nio-http2
b41ded5f2e05a1a1a6baa70453e69c06373c18bd
swift-nio-ssl
9e7e47c1ca7d6cd5108ce5f48082ee123a74c22e

my system is macOS 10.14 pre-release

I ran the unit tests and got a few test failures and eventually a crash. When I re-ran the tests I still got the failures but not the crash anymore.

Test Case '-[KituraNetTests.ClientE2ETests testErrorRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testErrorRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETest
s:261[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testErrorRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testErrorRequests]' failed (10.308 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testHeadRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testHeadRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:53[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testHeadRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testHeadRequests]' failed (10.008 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testKeepAlive]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testKeepAlive] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:72[0]", "ClientE2ETests:72[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testKeepAlive] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testKeepAlive]' failed (10.015 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPatchRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPatchRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:214[0]", "ClientE2ETests:214[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPatchRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPatchRequests]' failed (10.006 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPostRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPostRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:120[0]", "ClientE2ETests:120[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPostRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPostRequests]' failed (10.006 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPutRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPutRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:167[0]", "ClientE2ETests:167[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPutRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPutRequests]' failed (10.009 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testSimpleHTTPClient]' started.
Test Case '-[KituraNetTests.ClientE2ETests testSimpleHTTPClient]' passed (10.036 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testUrlURL]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testUrlURL] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:270[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testUrlURL] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testUrlURL]' failed (10.006 seconds).
Test Suite 'ClientE2ETests' failed at 2018-08-20 1:14:27.530 pm.

and

Test Case '-[KituraNetTests.LargePayloadTests testLargeGets]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.LargePayloadTests testLargeGets] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "LargePayloadTests:71[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.LargePayloadTests testLargeGets] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.LargePayloadTests testLargeGets]' failed (10.009 seconds).
Test Case '-[KituraNetTests.LargePayloadTests testLargePosts]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.LargePayloadTests testLargePosts] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "LargePayloadTests:43[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.LargePayloadTests testLargePosts] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.LargePayloadTests testLargePosts]' failed (10.004 seconds).

and

Test Case '-[KituraNetTests.MonitoringTests testStartedFinishedHTTP]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/MonitoringTests.swift:70: error: -[KituraNetTests.MonitoringTests testStartedFinishedHTTP] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "started", "finished".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/MonitoringTests.swift:72: error: -[KituraNetTests.MonitoringTests testStartedFinishedHTTP] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.MonitoringTests testStartedFinishedHTTP]' failed (10.012 seconds).
Test Suite 'MonitoringTests' failed at 2018-08-20 1:14:58.608 pm.

the full output was

Compile CNIOSHA1 c_nio_sha1.c
Compile CNIOHTTPParser c_nio_http_parser.c
Compile CNIOZlib empty.c
Compile CNIOLinux shim.c
Compile CNIODarwin shim.c
Compile CNIOOpenSSL helpers.c
Compile CNIOAtomics src/c-atomics.c
Compile Swift Module 'Socket' (3 sources)
Compile Swift Module 'NIOPriorityQueue' (2 sources)
Compile Swift Module 'LoggerAPI' (1 sources)
Compile Swift Module 'NIOConcurrencyHelpers' (2 sources)
Compile Swift Module 'NIO' (52 sources)
Compile Swift Module 'SSLService' (1 sources)
Compile Swift Module 'NIOHTTP1' (9 sources)
Compile Swift Module 'NIOTLS' (3 sources)
Compile Swift Module 'NIOFoundationCompat' (1 sources)
Compile Swift Module 'NIOOpenSSL' (14 sources)
Compile Swift Module 'NIOWebSocket' (9 sources)
/tmp/.nio-release-tools_jIe3dZ/swift-nio-ssl/Sources/NIOOpenSSL/OpenSSLHandler.swift:287:35: warning: 'changeCapacity(to:)' is deprecated: changeCapacity has been replaced by reserveCapacity
                    receiveBuffer.changeCapacity(to: receiveBuffer.capacity + 1024)
                                  ^
Compile Swift Module 'KituraNet' (26 sources)
Compile Swift Module 'KituraNetTests' (15 sources)
Linking ./.build/x86_64-apple-macosx10.10/debug/KituraNetPackageTests.xctest/Contents/MacOS/KituraNetPackageTests
Test Suite 'All tests' started at 2018-08-20 1:13:07.056 pm
Test Suite 'KituraNetPackageTests.xctest' started at 2018-08-20 1:13:07.056 pm
Test Suite 'BufferListTests' started at 2018-08-20 1:13:07.056 pm
Test Case '-[KituraNetTests.BufferListTests testAppendData]' started.
Test Case '-[KituraNetTests.BufferListTests testAppendData]' passed (0.068 seconds).
Test Case '-[KituraNetTests.BufferListTests testAppendUnsafePointerLength]' started.
Test Case '-[KituraNetTests.BufferListTests testAppendUnsafePointerLength]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testDataAndCount]' started.
Test Case '-[KituraNetTests.BufferListTests testDataAndCount]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillArray]' started.
Test Case '-[KituraNetTests.BufferListTests testFillArray]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillData]' started.
Test Case '-[KituraNetTests.BufferListTests testFillData]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillMutableData]' started.
Test Case '-[KituraNetTests.BufferListTests testFillMutableData]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillUnsafeMutablePointer]' started.
Test Case '-[KituraNetTests.BufferListTests testFillUnsafeMutablePointer]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testRewind]' started.
Test Case '-[KituraNetTests.BufferListTests testRewind]' passed (0.000 seconds).
Test Suite 'BufferListTests' passed at 2018-08-20 1:13:07.126 pm.
	 Executed 8 tests, with 0 failures (0 unexpected) in 0.070 (0.070) seconds
Test Suite 'ClientE2ETests' started at 2018-08-20 1:13:07.127 pm
Test Case '-[KituraNetTests.ClientE2ETests testEphemeralListeningPort]' started.
Test Case '-[KituraNetTests.ClientE2ETests testEphemeralListeningPort]' passed (0.009 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testErrorRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testErrorRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:261[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testErrorRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testErrorRequests]' failed (10.308 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testHeadRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testHeadRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:53[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testHeadRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testHeadRequests]' failed (10.008 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testKeepAlive]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testKeepAlive] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:72[0]", "ClientE2ETests:72[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testKeepAlive] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testKeepAlive]' failed (10.015 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPatchRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPatchRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:214[0]", "ClientE2ETests:214[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPatchRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPatchRequests]' failed (10.006 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPostRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPostRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:120[0]", "ClientE2ETests:120[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPostRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPostRequests]' failed (10.006 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPutRequests]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPutRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:167[0]", "ClientE2ETests:167[1]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPutRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPutRequests]' failed (10.009 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testSimpleHTTPClient]' started.
Test Case '-[KituraNetTests.ClientE2ETests testSimpleHTTPClient]' passed (10.036 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testUrlURL]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testUrlURL] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:270[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testUrlURL] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testUrlURL]' failed (10.006 seconds).
Test Suite 'ClientE2ETests' failed at 2018-08-20 1:14:27.530 pm.
	 Executed 9 tests, with 14 failures (0 unexpected) in 80.403 (80.404) seconds
Test Suite 'ClientRequestTests' started at 2018-08-20 1:14:27.531 pm
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsMisformattedPathCorrectly]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsMisformattedPathCorrectly]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPathCorrectly]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPathCorrectly]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPort]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPort]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestBasicAuthentcation]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestBasicAuthentcation]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultMethodIsGET]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultMethodIsGET]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultSchemaIsHTTP]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultSchemaIsHTTP]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestParse]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestParse]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSet]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSet]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSyncBehavior]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSyncBehavior]' passed (0.523 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithSimpleSchema]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithSimpleSchema]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithValidURL]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithValidURL]' passed (0.000 seconds).
Test Suite 'ClientRequestTests' passed at 2018-08-20 1:14:28.058 pm.
	 Executed 11 tests, with 0 failures (0 unexpected) in 0.526 (0.527) seconds
Test Suite 'HTTPResponseTests' started at 2018-08-20 1:14:28.058 pm
Test Case '-[KituraNetTests.HTTPResponseTests testContentTypeHeaders]' started.
Test Case '-[KituraNetTests.HTTPResponseTests testContentTypeHeaders]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPResponseTests testHeadersContainerHTTPHeaders]' started.
Test Case '-[KituraNetTests.HTTPResponseTests testHeadersContainerHTTPHeaders]' passed (0.001 seconds).
Test Suite 'HTTPResponseTests' passed at 2018-08-20 1:14:28.060 pm.
	 Executed 2 tests, with 0 failures (0 unexpected) in 0.002 (0.002) seconds
Test Suite 'HTTPStatusCodeTests' started at 2018-08-20 1:14:28.060 pm
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfEveryValidCode]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfEveryValidCode]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfStatusCode]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfStatusCode]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPStatusCodeTests testInvalidStatusCode]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testInvalidStatusCode]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPStatusCodeTests testStatusCodeCreation]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testStatusCodeCreation]' passed (0.000 seconds).
Test Suite 'HTTPStatusCodeTests' passed at 2018-08-20 1:14:28.061 pm.
	 Executed 4 tests, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'LargePayloadTests' started at 2018-08-20 1:14:28.061 pm
Test Case '-[KituraNetTests.LargePayloadTests testLargeGets]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.LargePayloadTests testLargeGets] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "LargePayloadTests:71[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.LargePayloadTests testLargeGets] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.LargePayloadTests testLargeGets]' failed (10.009 seconds).
Test Case '-[KituraNetTests.LargePayloadTests testLargePosts]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.LargePayloadTests testLargePosts] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "LargePayloadTests:43[0]".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.LargePayloadTests testLargePosts] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.LargePayloadTests testLargePosts]' failed (10.004 seconds).
Test Suite 'LargePayloadTests' failed at 2018-08-20 1:14:48.075 pm.
	 Executed 2 tests, with 4 failures (0 unexpected) in 20.014 (20.014) seconds
Test Suite 'LifecycleListenerTests' started at 2018-08-20 1:14:48.075 pm
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycle]' started.
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycle]' passed (0.004 seconds).
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycleWithState]' started.
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycleWithState]' passed (0.002 seconds).
Test Case '-[KituraNetTests.LifecycleListenerTests testServerFailLifecycle]' started.
Test Case '-[KituraNetTests.LifecycleListenerTests testServerFailLifecycle]' passed (0.513 seconds).
Test Suite 'LifecycleListenerTests' passed at 2018-08-20 1:14:48.594 pm.
	 Executed 3 tests, with 0 failures (0 unexpected) in 0.519 (0.519) seconds
Test Suite 'MiscellaneousTests' started at 2018-08-20 1:14:48.595 pm
Test Case '-[KituraNetTests.MiscellaneousTests testEscape]' started.
Test Case '-[KituraNetTests.MiscellaneousTests testEscape]' passed (0.000 seconds).
Test Case '-[KituraNetTests.MiscellaneousTests testHeadersContainers]' started.
Test Case '-[KituraNetTests.MiscellaneousTests testHeadersContainers]' passed (0.000 seconds).
Test Suite 'MiscellaneousTests' passed at 2018-08-20 1:14:48.595 pm.
	 Executed 2 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
Test Suite 'MonitoringTests' started at 2018-08-20 1:14:48.595 pm
Test Case '-[KituraNetTests.MonitoringTests testStartedFinishedHTTP]' started.
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/MonitoringTests.swift:70: error: -[KituraNetTests.MonitoringTests testStartedFinishedHTTP] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "started", "finished".
/private/tmp/.nio-release-tools_jIe3dZ/Kitura-NIO/Tests/KituraNetTests/MonitoringTests.swift:72: error: -[KituraNetTests.MonitoringTests testStartedFinishedHTTP] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.MonitoringTests testStartedFinishedHTTP]' failed (10.012 seconds).
Test Suite 'MonitoringTests' failed at 2018-08-20 1:14:58.608 pm.
	 Executed 1 test, with 2 failures (0 unexpected) in 10.012 (10.012) seconds
Test Suite 'ParserTests' started at 2018-08-20 1:14:58.608 pm
Test Case '-[KituraNetTests.ParserTests testParseComplexUrl]' started.
Test Case '-[KituraNetTests.ParserTests testParseComplexUrl]' passed (0.001 seconds).
Test Case '-[KituraNetTests.ParserTests testParserDescription]' started.
Test Case '-[KituraNetTests.ParserTests testParserDescription]' passed (0.001 seconds).
Test Case '-[KituraNetTests.ParserTests testParseSimpleUrl]' started.
Test Case '-[KituraNetTests.ParserTests testParseSimpleUrl]' passed (0.000 seconds).
Test Suite 'ParserTests' passed at 2018-08-20 1:14:58.611 pm.
	 Executed 3 tests, with 0 failures (0 unexpected) in 0.003 (0.003) seconds
Test Suite 'PipeliningTests' started at 2018-08-20 1:14:58.611 pm
Test Case '-[KituraNetTests.PipeliningTests testPipelining]' started.
Test Case '-[KituraNetTests.PipeliningTests testPipelining]' passed (0.010 seconds).
Test Case '-[KituraNetTests.PipeliningTests testPipeliningSpanningPackets]' started.
Fatal error: Can't remove first element from an empty collection
Test Case '-[KituraNetTests.PipeliningTests testPipeliningSpanningPackets]' passed (0.007 seconds).
Test Suite 'PipeliningTests' passed at 2018-08-20 1:14:58.628 pm.
	 Executed 2 tests, with 0 failures (0 unexpected) in 0.017 (0.017) seconds
Test Suite 'RegressionTests' started at 2018-08-20 1:14:58.628 pm
Test Case '-[KituraNetTests.RegressionTests testBadRequest]' started.
Exited with signal code 4

Actually, the failures don't seem to be intermittent. That's a second run

Compile CNIOSHA1 c_nio_sha1.c
Compile CNIODarwin shim.c
Compile CNIOZlib empty.c
Compile CNIOHTTPParser c_nio_http_parser.c
Compile CNIOLinux shim.c
Compile CNIOOpenSSL helpers.c
Compile CNIOAtomics src/c-atomics.c
Compile Swift Module 'Socket' (3 sources)
Compile Swift Module 'NIOPriorityQueue' (2 sources)
Compile Swift Module 'LoggerAPI' (1 sources)
Compile Swift Module 'NIOConcurrencyHelpers' (2 sources)
Compile Swift Module 'NIO' (52 sources)
Compile Swift Module 'SSLService' (1 sources)
Compile Swift Module 'NIOTLS' (3 sources)
Compile Swift Module 'NIOFoundationCompat' (1 sources)
Compile Swift Module 'NIOHTTP1' (9 sources)
Compile Swift Module 'NIOOpenSSL' (14 sources)
Compile Swift Module 'NIOWebSocket' (9 sources)
/tmp/.nio-release-tools_UwXYPU/swift-nio-ssl/Sources/NIOOpenSSL/OpenSSLHandler.swift:287:35: warning: 'changeCapacity(to:)' is deprecated: changeCapacity has been replaced by reserveCapacity
                    receiveBuffer.changeCapacity(to: receiveBuffer.capacity + 1024)
                                  ^
Compile Swift Module 'KituraNet' (26 sources)
Compile Swift Module 'KituraNetTests' (15 sources)
Linking ./.build/x86_64-apple-macosx10.10/debug/KituraNetPackageTests.xctest/Contents/MacOS/KituraNetPackageTests
Test Suite 'All tests' started at 2018-08-20 1:49:14.884 pm
Test Suite 'KituraNetPackageTests.xctest' started at 2018-08-20 1:49:14.885 pm
Test Suite 'BufferListTests' started at 2018-08-20 1:49:14.885 pm
Test Case '-[KituraNetTests.BufferListTests testAppendData]' started.
Test Case '-[KituraNetTests.BufferListTests testAppendData]' passed (0.070 seconds).
Test Case '-[KituraNetTests.BufferListTests testAppendUnsafePointerLength]' started.
Test Case '-[KituraNetTests.BufferListTests testAppendUnsafePointerLength]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testDataAndCount]' started.
Test Case '-[KituraNetTests.BufferListTests testDataAndCount]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillArray]' started.
Test Case '-[KituraNetTests.BufferListTests testFillArray]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillData]' started.
Test Case '-[KituraNetTests.BufferListTests testFillData]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillMutableData]' started.
Test Case '-[KituraNetTests.BufferListTests testFillMutableData]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testFillUnsafeMutablePointer]' started.
Test Case '-[KituraNetTests.BufferListTests testFillUnsafeMutablePointer]' passed (0.000 seconds).
Test Case '-[KituraNetTests.BufferListTests testRewind]' started.
Test Case '-[KituraNetTests.BufferListTests testRewind]' passed (0.000 seconds).
Test Suite 'BufferListTests' passed at 2018-08-20 1:49:14.957 pm.
	 Executed 8 tests, with 0 failures (0 unexpected) in 0.072 (0.072) seconds
Test Suite 'ClientE2ETests' started at 2018-08-20 1:49:14.957 pm
Test Case '-[KituraNetTests.ClientE2ETests testEphemeralListeningPort]' started.
Test Case '-[KituraNetTests.ClientE2ETests testEphemeralListeningPort]' passed (0.010 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testErrorRequests]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testErrorRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:261[0]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testErrorRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testErrorRequests]' failed (10.321 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testHeadRequests]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testHeadRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:53[0]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testHeadRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testHeadRequests]' failed (10.009 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testKeepAlive]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testKeepAlive] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:72[1]", "ClientE2ETests:72[0]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testKeepAlive] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testKeepAlive]' failed (10.006 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPatchRequests]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPatchRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:214[1]", "ClientE2ETests:214[0]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPatchRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPatchRequests]' failed (10.007 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPostRequests]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPostRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:120[0]", "ClientE2ETests:120[1]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPostRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPostRequests]' failed (10.009 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testPutRequests]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testPutRequests] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:167[0]", "ClientE2ETests:167[1]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testPutRequests] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testPutRequests]' failed (10.010 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testSimpleHTTPClient]' started.
Test Case '-[KituraNetTests.ClientE2ETests testSimpleHTTPClient]' passed (0.182 seconds).
Test Case '-[KituraNetTests.ClientE2ETests testUrlURL]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.ClientE2ETests testUrlURL] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "ClientE2ETests:270[0]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.ClientE2ETests testUrlURL] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.ClientE2ETests testUrlURL]' failed (10.008 seconds).
Test Suite 'ClientE2ETests' failed at 2018-08-20 1:50:25.521 pm.
	 Executed 9 tests, with 14 failures (0 unexpected) in 70.562 (70.563) seconds
Test Suite 'ClientRequestTests' started at 2018-08-20 1:50:25.521 pm
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsMisformattedPathCorrectly]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsMisformattedPathCorrectly]' passed (0.001 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPathCorrectly]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPathCorrectly]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPort]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestAppendsPort]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestBasicAuthentcation]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestBasicAuthentcation]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultMethodIsGET]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultMethodIsGET]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultSchemaIsHTTP]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestDefaultSchemaIsHTTP]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestParse]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestParse]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSet]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSet]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSyncBehavior]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestSyncBehavior]' passed (0.526 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithSimpleSchema]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithSimpleSchema]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithValidURL]' started.
Test Case '-[KituraNetTests.ClientRequestTests testClientRequestWhenInitializedWithValidURL]' passed (0.000 seconds).
Test Suite 'ClientRequestTests' passed at 2018-08-20 1:50:26.051 pm.
	 Executed 11 tests, with 0 failures (0 unexpected) in 0.529 (0.530) seconds
Test Suite 'HTTPResponseTests' started at 2018-08-20 1:50:26.051 pm
Test Case '-[KituraNetTests.HTTPResponseTests testContentTypeHeaders]' started.
Test Case '-[KituraNetTests.HTTPResponseTests testContentTypeHeaders]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPResponseTests testHeadersContainerHTTPHeaders]' started.
Test Case '-[KituraNetTests.HTTPResponseTests testHeadersContainerHTTPHeaders]' passed (0.001 seconds).
Test Suite 'HTTPResponseTests' passed at 2018-08-20 1:50:26.053 pm.
	 Executed 2 tests, with 0 failures (0 unexpected) in 0.001 (0.002) seconds
Test Suite 'HTTPStatusCodeTests' started at 2018-08-20 1:50:26.053 pm
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfEveryValidCode]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfEveryValidCode]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfStatusCode]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testClassOfStatusCode]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPStatusCodeTests testInvalidStatusCode]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testInvalidStatusCode]' passed (0.000 seconds).
Test Case '-[KituraNetTests.HTTPStatusCodeTests testStatusCodeCreation]' started.
Test Case '-[KituraNetTests.HTTPStatusCodeTests testStatusCodeCreation]' passed (0.000 seconds).
Test Suite 'HTTPStatusCodeTests' passed at 2018-08-20 1:50:26.054 pm.
	 Executed 4 tests, with 0 failures (0 unexpected) in 0.001 (0.002) seconds
Test Suite 'LargePayloadTests' started at 2018-08-20 1:50:26.054 pm
Test Case '-[KituraNetTests.LargePayloadTests testLargeGets]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.LargePayloadTests testLargeGets] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "LargePayloadTests:71[0]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.LargePayloadTests testLargeGets] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.LargePayloadTests testLargeGets]' failed (10.007 seconds).
Test Case '-[KituraNetTests.LargePayloadTests testLargePosts]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:179: error: -[KituraNetTests.LargePayloadTests testLargePosts] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "LargePayloadTests:43[0]".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:112: error: -[KituraNetTests.LargePayloadTests testLargePosts] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.LargePayloadTests testLargePosts]' failed (10.006 seconds).
Test Suite 'LargePayloadTests' failed at 2018-08-20 1:50:46.068 pm.
	 Executed 2 tests, with 4 failures (0 unexpected) in 20.013 (20.013) seconds
Test Suite 'LifecycleListenerTests' started at 2018-08-20 1:50:46.068 pm
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycle]' started.
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycle]' passed (0.005 seconds).
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycleWithState]' started.
Test Case '-[KituraNetTests.LifecycleListenerTests testLifecycleWithState]' passed (0.002 seconds).
Test Case '-[KituraNetTests.LifecycleListenerTests testServerFailLifecycle]' started.
Test Case '-[KituraNetTests.LifecycleListenerTests testServerFailLifecycle]' passed (0.012 seconds).
Test Suite 'LifecycleListenerTests' passed at 2018-08-20 1:50:46.088 pm.
	 Executed 3 tests, with 0 failures (0 unexpected) in 0.019 (0.020) seconds
Test Suite 'MiscellaneousTests' started at 2018-08-20 1:50:46.088 pm
Test Case '-[KituraNetTests.MiscellaneousTests testEscape]' started.
Test Case '-[KituraNetTests.MiscellaneousTests testEscape]' passed (0.000 seconds).
Test Case '-[KituraNetTests.MiscellaneousTests testHeadersContainers]' started.
Test Case '-[KituraNetTests.MiscellaneousTests testHeadersContainers]' passed (0.000 seconds).
Test Suite 'MiscellaneousTests' passed at 2018-08-20 1:50:46.088 pm.
	 Executed 2 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
Test Suite 'MonitoringTests' started at 2018-08-20 1:50:46.088 pm
Test Case '-[KituraNetTests.MonitoringTests testStartedFinishedHTTP]' started.
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/MonitoringTests.swift:70: error: -[KituraNetTests.MonitoringTests testStartedFinishedHTTP] : Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "started", "finished".
/private/tmp/.nio-release-tools_UwXYPU/Kitura-NIO/Tests/KituraNetTests/MonitoringTests.swift:72: error: -[KituraNetTests.MonitoringTests testStartedFinishedHTTP] : XCTAssertNil failed: "Error Domain=com.apple.XCTestErrorDomain Code=0 "(null)"" - 
Test Case '-[KituraNetTests.MonitoringTests testStartedFinishedHTTP]' failed (10.009 seconds).
Test Suite 'MonitoringTests' failed at 2018-08-20 1:50:56.097 pm.
	 Executed 1 test, with 2 failures (0 unexpected) in 10.009 (10.009) seconds
Test Suite 'ParserTests' started at 2018-08-20 1:50:56.097 pm
Test Case '-[KituraNetTests.ParserTests testParseComplexUrl]' started.
Test Case '-[KituraNetTests.ParserTests testParseComplexUrl]' passed (0.001 seconds).
Test Case '-[KituraNetTests.ParserTests testParserDescription]' started.
Test Case '-[KituraNetTests.ParserTests testParserDescription]' passed (0.000 seconds).
Test Case '-[KituraNetTests.ParserTests testParseSimpleUrl]' started.
Test Case '-[KituraNetTests.ParserTests testParseSimpleUrl]' passed (0.000 seconds).
Test Suite 'ParserTests' passed at 2018-08-20 1:50:56.099 pm.
	 Executed 3 tests, with 0 failures (0 unexpected) in 0.001 (0.002) seconds
Test Suite 'PipeliningTests' started at 2018-08-20 1:50:56.099 pm
Test Case '-[KituraNetTests.PipeliningTests testPipelining]' started.
Test Case '-[KituraNetTests.PipeliningTests testPipelining]' passed (0.009 seconds).
Test Case '-[KituraNetTests.PipeliningTests testPipeliningSpanningPackets]' started.
Test Case '-[KituraNetTests.PipeliningTests testPipeliningSpanningPackets]' passed (0.008 seconds).
Test Suite 'PipeliningTests' passed at 2018-08-20 1:50:56.116 pm.
	 Executed 2 tests, with 0 failures (0 unexpected) in 0.017 (0.017) seconds
Test Suite 'RegressionTests' started at 2018-08-20 1:50:56.116 pm
Test Case '-[KituraNetTests.RegressionTests testBadRequest]' started.
Test Case '-[KituraNetTests.RegressionTests testBadRequest]' passed (0.006 seconds).
Test Case '-[KituraNetTests.RegressionTests testBadRequestFollowingGoodRequest]' started.
Test Case '-[KituraNetTests.RegressionTests testBadRequestFollowingGoodRequest]' passed (0.006 seconds).
Test Case '-[KituraNetTests.RegressionTests testIssue1143]' started.
Test Case '-[KituraNetTests.RegressionTests testIssue1143]' passed (0.007 seconds).
Test Case '-[KituraNetTests.RegressionTests testServersCollidingOnPort]' started.
Test Case '-[KituraNetTests.RegressionTests testServersCollidingOnPort]' passed (0.003 seconds).
Test Case '-[KituraNetTests.RegressionTests testServersSharingPort]' started.
Test Case '-[KituraNetTests.RegressionTests testServersSharingPort]' passed (0.002 seconds).
Test Suite 'RegressionTests' passed at 2018-08-20 1:50:56.140 pm.
	 Executed 5 tests, with 0 failures (0 unexpected) in 0.024 (0.024) seconds
Test Suite 'KituraNetPackageTests.xctest' failed at 2018-08-20 1:50:56.140 pm.
	 Executed 52 tests, with 20 failures (0 unexpected) in 101.249 (101.256) seconds
Test Suite 'All tests' failed at 2018-08-20 1:50:56.140 pm.
	 Executed 52 tests, with 20 failures (0 unexpected) in 101.249 (101.256) seconds

Bind errors while running Kitura and Kitura-NIO tests

With SwiftNIO, Channel.close() calls close(socket) which moves the underlying socket to the TIMED_WAIT state. In this state, subsequent bind calls fail, until the socket moves out of the TIMED_WAIT state and is actually closed. Kitura tests fail frequently with this error. While using Kitura-net that uses BlueSocket, I can see a shutdown(socket) being issued before close(socket). That may be the reason why this failure isn't seen with KituraNet. Needs investigation.

We can workaround this using HTTPServer.allowPortReuse = true

possibly flaky test PipeliningTests.testPipeliningSpanningPackets

saw this today in docker, Swift 5.0. This looks like it's flaky.

Test Case 'PipeliningTests.testPipeliningSpanningPackets' started at 2019-04-12 16:58:14.053
/Users/johannes/tmp/tmp-for-release-tools/.nio-release-tools_DHLV8i/Kitura-NIO/Tests/KituraNetTests/PipeliningTests.swift:116: error: PipeliningTests.testPipeliningSpanningPackets : Asynchronous wait failed - Exceeded timeout of 10.0 seconds, with unfulfilled expectations: test pipelining spanning packets
Test Case 'PipeliningTests.testPipeliningSpanningPackets' failed (10.009 seconds)

package.resolved was

{
  "object": {
    "pins": [
      {
        "package": "Socket",
        "repositoryURL": "https://github.com/IBM-Swift/BlueSocket.git",
        "state": {
          "branch": null,
          "revision": "ccc671b022bc60e5fffcb2aa15f499c6331bfd18",
          "version": "1.0.46"
        }
      },
      {
        "package": "SSLService",
        "repositoryURL": "https://github.com/IBM-Swift/BlueSSLService.git",
        "state": {
          "branch": null,
          "revision": "0127cd4156bb0ba702894f2509199e4b65ca1db5",
          "version": "1.0.46"
        }
      },
      {
        "package": "LoggerAPI",
        "repositoryURL": "https://github.com/IBM-Swift/LoggerAPI.git",
        "state": {
          "branch": null,
          "revision": "e29073bb7cecf3673e56bcb16180e8fd0cb091f6",
          "version": "1.8.1"
        }
      },
      {
        "package": "OpenSSL",
        "repositoryURL": "https://github.com/IBM-Swift/OpenSSL.git",
        "state": {
          "branch": null,
          "revision": "6c7e2e3610037fbc05c0d846cec981518961d8bf",
          "version": "2.2.2"
        }
      },
      {
        "package": "swift-nio-ssl-support",
        "repositoryURL": "https://github.com/apple/swift-nio-ssl-support.git",
        "state": {
          "branch": null,
          "revision": "c02eec4e0e6d351cd092938cf44195a8e669f555",
          "version": "1.0.0"
        }
      },
      {
        "package": "swift-nio-zlib-support",
        "repositoryURL": "https://github.com/apple/swift-nio-zlib-support.git",
        "state": {
          "branch": null,
          "revision": "37760e9a52030bb9011972c5213c3350fa9d41fd",
          "version": "1.0.0"
        }
      }
    ]
  },
  "version": 1
}
  • swift-nio manually edited to ba7970fe396e8198b84c6c1b44b38a1d4e2eb6bd
  • swift-nio-ssl manually edited to 0f3999f3e3c359cc74480c292644c3419e44a12f

Linker error in libz on Ubuntu 16.04

Linking ./.build/x86_64-unknown-linux/debug/KituraPackageTests.xctest
/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:function _T08NIOHTTP122HTTPResponseCompressorC19deinitializeEncoder33_E8A369BA8CC72FD6D05117C456504CA2LLyyF: error: undefined reference to 'deflateEnd'
/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:function _T08NIOHTTP119PartialHTTPResponse33_E8A369BA8CC72FD6D05117C456504CA2LLV12compressBody3NIO10ByteBufferVSgSC10z_stream_sVz10compressor_AF0oP9AllocatorV9allocators5Int32V4flagtF: error: undefined reference to 'deflateBound'
/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:function _T0SC10z_stream_sV8NIOHTTP1E15deflateToBuffer33_E8A369BA8CC72FD6D05117C456504CA2LLs5Int32V3NIO04ByteE0Vz6buffer_AG4flagtFSis016UnsafeMutableRawE7PointerVcfU_: error: undefined reference to 'deflate'
/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:/root/Kitura/.build/x86_64-unknown-linux/debug/NIOHTTP1.build/HTTPResponseCompressor.swift.o:function CNIOZlib_deflateInit2: error: undefined reference to 'deflateInit2_'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

swift test -Xlinker -lz is used as a temporary work-around for the above issue.

Kitura tests on kitura-nio branch leak the MultiThreadedEventLoopGroup threads

This might really be a kitura issue but it probably also happens here: Kitura tests on kitura-nio branch leak the MultiThreadedEventLoopGroup threads, basically the MultiThreadedEventLoopGroup that are created in the tests are never torn down, ie. the threads are still kept alive.

Essentially it's the same issue that Vapor has too

MultiThreadedEventLoopGroup.syncShutdownGracefully should be used for example in the tearDown method of a test

Indicating a failed upgrade to WebSocket via `shouldUpgrade`

The shouldUpgrade function passed to NIOWebSocket.WebSocketUpgrader is expected to return an EventLoopFuture<HTTPHeaders?> to indicate a successful or failed upgrade. On a successful upgrade, we return eventLoop.makeSucceededFuture(httpHeaders). But on a failed upgrade (may be because the requested websocket service doesn't exist), it may make sense to return a eventLoop.makeFailedFuture(<error>). The questions here are:
a. Does the NIOWebSocket.WebSocketUpgrader handle an error wrapped in a failed future?
b. What error value do we return?

cc @ianpartridge

autobahn: compression tests are failing

Both, the 12.x.x and 13.x.x tests fail with this exception on the client side. The server side doesn't show any error symptoms:

Servers = [u'ws://127.0.0.1:9001']
Unhandled Error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite/wstest.py", line 346, in run
    start(options, spec)
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite/wstest.py", line 280, in start
    reactor.run()
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/base.py", line 1267, in run
    self.mainLoop()
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/base.py", line 1276, in mainLoop
    self.runUntilCurrent()
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/base.py", line 902, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 520, in connectionLost
    self.connector.connectionLost(reason)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/base.py", line 1160, in connectionLost
    self.factory.clientConnectionLost(self, reason)
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite/fuzzing.py", line 1292, in clientConnectionLost
    self.createReports()
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite/fuzzing.py", line 469, in createReports
    self.createMasterReportHTML(self.outdir)
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite/fuzzing.py", line 670, in createMasterReportHTML
    agent_case_report_file = self.makeAgentCaseReportFilename(agentId, caseId, ext = 'html')
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite/fuzzing.py", line 498, in makeAgentCaseReportFilename
    return self.cleanForFilename(agentId) + "_case_" + c + "." + ext
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite/fuzzing.py", line 487, in cleanForFilename
    s0 = ''.join([c if c in "abcdefghjiklmnopqrstuvwxyz0123456789" else " " for c in str.strip().lower()])
exceptions.AttributeError: 'NoneType' object has no attribute 'strip'

ClientRequest auth should use an Authorization header

In #171 we noted that currently, the username and password fields are being sent as part of the URL for basic authentication, and that they are currently not percent-escaped by the NIO client.

Rather than escaping them, I believe they should instead be sent in an Authorization header. This would alleviate the need to escape their values, besides being recommended practice. Normally the Authorization header is sent in response to a server sending a WWW-Authenticate header, but I believe it's valid to provide it outside of that flow.

This would be a behavioural (breaking?) change as the format of the URL would change where basic authentication is being used, and would require recipients to support parsing the Authorization header (which I believe they should do, but I can imagine there may be implementations out there that assumes they'll be tacked on to the URL).

Both Kitura-NIO and Kitura-net would need to be changed to behave consistently.

LargePayloadTest fails with SwiftNIO 1.7.0

LargePayloadTests.testLargeGets fails consistently with SwiftNIO 1.7.0 with:

*** Error in `/root/pushkar/Kitura-NIO/.build/x86_64-unknown-linux/debug/KituraNIOPackageTests.xctest': realloc(): invalid next size: 0x00007f5e4004c110 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f5ea5d447e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x834aa)[0x7f5ea5d504aa]
/lib/x86_64-linux-gnu/libc.so.6(realloc+0x179)[0x7f5ea5d51839]

KituraNIO's ClientRequest.end calls wait and may be called from an EventLoop

In SwiftNIO we improved the wait-on-EventLoop detection to now catch many more of those issues. In fact it hits one in KituraNIO itself

Steps to reproduce

git clone https://github.com/IBM-Swift/kitura.git
git clone https://github.com/apple/swift-nio.git
cd vapor && swift package edit --path ../swift-nio swift-nio
swift test

tests that repro this issue:

  • testRedirect
  • testDataIsNotCorrupted
  • testFileServer
  • testRangeRequestWithIfRangeHeaderAsLastModified
  • ...

Expected behavior

runs cleanly

Actual behavior

Test Case '-[KituraTests.TestResponse testRedirect]' started.
Precondition failed: BUG DETECTED: wait() must not be called when on an EventLoop.
Calling wait() on any EventLoop can lead to
- deadlocks
- stalling processing of other connections (Channels) that are handled on the EventLoop that wait was called on

Further information:
- current eventLoop: Optional(SelectableEventLoop { selector = Selector { descriptor = 328 }, scheduledTasks = PriorityQueue(count: 0): [] })
- event loop associated to future: SelectableEventLoop { selector = Selector { descriptor = 331 }, scheduledTasks = PriorityQueue(count: 0): [ScheduledTask(readyTime: 145590957924862)] }: file /Users/johannes/extsrc/Kitura/.build/checkouts/Kitura-NIO.git--476728271407110211/Sources/KituraNIO/ClientRequest.swift, line 384
2018-08-16 19:25:16.855518+0100 xctest[73265:3499090] Precondition failed: BUG DETECTED: wait() must not be called when on an EventLoop.
Calling wait() on any EventLoop can lead to
- deadlocks
- stalling processing of other connections (Channels) that are handled on the EventLoop that wait was called on

Further information:
- current eventLoop: Optional(SelectableEventLoop { selector = Selector { descriptor = 328 }, scheduledTasks = PriorityQueue(count: 0): [] })
- event loop associated to future: SelectableEventLoop { selector = Selector { descriptor = 331 }, scheduledTasks = PriorityQueue(count: 0): [ScheduledTask(readyTime: 145590957924862)] }: file /Users/johannes/extsrc/Kitura/.build/checkouts/Kitura-NIO.git--476728271407110211/Sources/KituraNIO/ClientRequest.swift, line 384

Analysis

it's this bit of code

        do {
            channel = try bootstrap.connect(host: hostName, port: Int(self.port ?? 80)).wait()
        } catch let error {
            Log.error("Connection to \(hostName):\(self.port ?? 80) failed with error: \(error)")
            callback(nil)
            return
        }

in ClientRequest's public func end(close: Bool = false).

Stack trace

    frame #0: 0x000000010ae39c60 libswiftCore.dylib`_swift_runtime_on_report
    frame #1: 0x000000010ae8b2bb libswiftCore.dylib`_swift_stdlib_reportFatalErrorInFile + 123
    frame #2: 0x000000010ab93b71 libswiftCore.dylib`closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 257
    frame #3: 0x000000010ae19982 libswiftCore.dylib`partial apply forwarder for closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 34
    frame #4: 0x000000010ae1bc49 libswiftCore.dylib`closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Neverpartial apply forwarder with unmangled suffix ".832" + 9
    frame #5: 0x000000010ab932ca libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #6: 0x000000010ae19893 libswiftCore.dylib`partial apply forwarder for closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 531
    frame #7: 0x000000010ab932ca libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #8: 0x000000010ad5d3f8 libswiftCore.dylib`function signature specialization <Arg[1] = Exploded> of Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 136
    frame #9: 0x000000010ab92b59 libswiftCore.dylib`Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 25
    frame #10: 0x000000010a7d6cd1 NIO`EventLoopFuture.wait(file="/Users/johannes/extsrc/Kitura/.build/checkouts/Kitura-NIO.git--476728271407110211/Sources/KituraNIO/ClientRequest.swift", line=384, self=0x00000001026b6350) at EventLoopFuture.swift:812
  * frame #11: 0x000000010972b581 KituraNIO`ClientRequest.end(close=false, self=0x0000000103079770) at ClientRequest.swift:384
    frame #12: 0x000000010973156d KituraNIO`HTTPClientHandler.channelRead(ctx=0x00000001026b3910, data=NIO.NIOAny @ 0x00007000152e1508, self=0x00000001026b5770) at ClientRequest.swift:572
    frame #13: 0x0000000109731f7e KituraNIO`protocol witness for _ChannelInboundHandler.channelRead(ctx:data:) in conformance HTTPClientHandler at <compiler-generated>:0
    frame #14: 0x000000010a79b80e NIO`ChannelHandlerContext.invokeChannelRead(data=NIO.NIOAny @ 0x00007000152e1508, self=0x00000001026b3910) at ChannelPipeline.swift:1284
    frame #15: 0x000000010a79fd01 NIO`ChannelHandlerContext.fireChannelRead(data=NIO.NIOAny @ 0x00007000152e1508, self=0x00000001026b3860) at ChannelPipeline.swift:1097
    frame #16: 0x000000010a634b86 NIOHTTP1`HTTPDecoder.firePendingInOut(ctx=0x00000001026b3860, self=0x00000001026b3360) at HTTPDecoder.swift:516
    frame #17: 0x000000010a6347c8 NIOHTTP1`HTTPDecoder.decodeHTTP(ctx=0x00000001026b3860, self=0x00000001026b3360) at HTTPDecoder.swift:504
    frame #18: 0x000000010a636a2a NIOHTTP1`HTTPDecoder.channelRead(ctx=0x00000001026b3860, data=NIO.NIOAny @ 0x00007000152e2188, self=0x00000001026b3360) at HTTPDecoder.swift:601
    frame #19: 0x000000010a640fb1 NIOHTTP1`protocol witness for _ChannelInboundHandler.channelRead(ctx:data:) in conformance HTTPDecoder<A> at <compiler-generated>:0
    frame #20: 0x000000010a79b80e NIO`ChannelHandlerContext.invokeChannelRead(data=NIO.NIOAny @ 0x00007000152e2188, self=0x00000001026b3860) at ChannelPipeline.swift:1284
    frame #21: 0x000000010a79b8a6 NIO`ChannelHandlerContext.invokeChannelRead(data=NIO.NIOAny @ 0x00007000152e2188, self=0x00000001026b37b0) at ChannelPipeline.swift:1286
    frame #22: 0x000000010a795c02 NIO`ChannelPipeline.fireChannelRead0(data=NIO.NIOAny @ 0x00007000152e2188, self=0x00000001026b5570) at ChannelPipeline.swift:808
    frame #23: 0x000000010a8423c6 NIO`SocketChannel.readFromSocket(self=0x00000001026b5400) at SocketChannel.swift:144
    frame #24: 0x000000010a74e860 NIO`BaseSocketChannel.readable0(self=0x00000001026b5400) at BaseSocketChannel.swift:958
    frame #25: 0x000000010a74fe7c NIO`BaseSocketChannel.readable(self=0x00000001026b5400) at BaseSocketChannel.swift:942
    frame #26: 0x000000010a7516b9 NIO`protocol witness for SelectableChannel.readable() in conformance BaseSocketChannel<A> at <compiler-generated>:0
    frame #27: 0x000000010a7c2401 NIO`SelectableEventLoop.handleEvent<A>(ev=(rawValue = 4), channel=0x0000000103475480, self=0x0000000103078820) at EventLoop.swift:613
    frame #28: 0x000000010a7c373b NIO`closure #1 in closure #1 in SelectableEventLoop.run(ev=NIO.SelectorEvent<NIO.NIORegistration> @ 0x00007000152e2b48, self=0x0000000103078820) at EventLoop.swift:662
    frame #29: 0x000000010a7cc886 NIO`partial apply for closure #1 in closure #1 in SelectableEventLoop.run() at <compiler-generated>:0
    frame #30: 0x000000010a7c3800 NIO`thunk for @callee_guaranteed (@guaranteed SelectorEvent<NIORegistration>) -> (@error @owned Error) at <compiler-generated>:0
    frame #31: 0x000000010a7cc8cb NIO`partial apply for thunk for @callee_guaranteed (@guaranteed SelectorEvent<NIORegistration>) -> (@error @owned Error) at <compiler-generated>:0
    frame #32: 0x000000010a82bd51 NIO`Selector.whenReady(strategy=block, body=0x000000010a7cc8b0 NIO`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed (@guaranteed NIO.SelectorEvent<NIO.NIORegistration>) -> (@error @owned Swift.Error) to @escaping @callee_guaranteed (@in_guaranteed NIO.SelectorEvent<NIO.NIORegistration>) -> (@error @owned Swift.Error) at <compiler-generated>, self=0x00000001030789b0) at Selector.swift:588
    frame #33: 0x000000010a7c3466 NIO`closure #1 in SelectableEventLoop.run(self=0x0000000103078820, nextReadyTask=nil) at EventLoop.swift:657
    frame #34: 0x000000010a7c354b NIO`partial apply for closure #1 in SelectableEventLoop.run() at <compiler-generated>:0
    frame #35: 0x000000010a743ec6 NIO`thunk for @callee_guaranteed () -> (@error @owned Error) at <compiler-generated>:0
    frame #36: 0x000000010a7c359b NIO`thunk for @callee_guaranteed () -> (@error @owned Error)partial apply at <compiler-generated>:0
    frame #37: 0x000000010a7be9f2 NIO`closure #1 in withAutoReleasePool<A>(execute=0x000000010a7c3580 NIO`reabstraction thunk helper from @callee_guaranteed () -> (@error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".40" at <compiler-generated>) at EventLoop.swift:427
    frame #38: 0x000000010a7cc846 NIO`partial apply for closure #1 in withAutoReleasePool<A>(_:) at <compiler-generated>:0
    frame #39: 0x000000010b586f2f libswiftObjectiveC.dylib`ObjectiveC.autoreleasepool<A>(invoking: () throws -> A) throws -> A + 47
    frame #40: 0x000000010a7be959 NIO`withAutoReleasePool<A>(execute=0x000000010a7c3580 NIO`reabstraction thunk helper from @callee_guaranteed () -> (@error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".40" at <compiler-generated>) at EventLoop.swift:426
    frame #41: 0x000000010a7c28f1 NIO`SelectableEventLoop.run(self=0x0000000103078820) at EventLoop.swift:656
    frame #42: 0x000000010a7c684e NIO`closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(t=(pthread = 0x00007000152e4000), initializer=0x000000010a7cc220 NIO`partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@in_guaranteed NIO.Thread) -> (@out ()) to @escaping @callee_guaranteed (@guaranteed NIO.Thread) -> () at <compiler-generated>, self=NIO.MultiThreadedEventLoopGroup, lock=(mutex = 0x000000010cc37060), _loop=0x0000000103078820, loopUpAndRunningGroup=0x000000010cc33760) at EventLoop.swift:856
    frame #43: 0x000000010a7cc30a NIO`partial apply for closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:initializer:) at <compiler-generated>:0
    frame #44: 0x000000010a7c6def NIO`thunk for @escaping @callee_guaranteed (@guaranteed Thread) -> () at <compiler-generated>:0
    frame #45: 0x000000010a858ca1 NIO`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Thread) -> () at <compiler-generated>:0
    frame #46: 0x000000010a859310 NIO`closure #1 in static Thread.spawnAndRun(p=(_rawValue = 0x000000010cc36620)) at Thread.swift:105
    frame #47: 0x000000010a859379 NIO`@objc closure #1 in static Thread.spawnAndRun(name:body:) at <compiler-generated>:0
    frame #48: 0x00007fff69553881 libsystem_pthread.dylib`_pthread_body + 126
    frame #49: 0x00007fff695537f3 libsystem_pthread.dylib`_pthread_start + 70
    frame #50: 0x00007fff695533bd libsystem_pthread.dylib`thread_start + 13

Add Unix Domain Socket support in ClientRequest

We're considering adding client-side Unix Domain Socket support with Kitura-net and Kitura-NIO. The change is intended to be added to ClientRequest and , in terms of the API, it will be purely additive for the time being.

TSan detected data-race in KituraNetTests.PipeliningTests testPipeliningSpanningPackets

hashes:

Kitura-NIO
214abc91b6fd51b7d2817175f0599ad120754ce6
swift-nio
176dd6e8564d60e936b76f3a896d667ae3acba31
swift-nio-extras
28154e042e1de46c0fb7da93f9b6d6518ef09156
swift-nio-http2
83d27d1dc3df4a6c0fbb97b82972695a756e36aa
swift-nio-ssl
db16c3a90b101bb53b26a58867a344ad428072e0

on macOS

log:

Test Case '-[KituraNetTests.PipeliningTests testPipeliningSpanningPackets]' started.
==================
WARNING: ThreadSanitizer: data race (pid=70301)
  Read of size 8 at 0x7b080002e6b0 by thread T31:
    #0 Delegate.handle(request:response:) PipeliningTests.swift:126 (KituraNetPackageTests:x86_64+0x10dd09)
    #1 protocol witness for ServerDelegate.handle(request:response:) in conformance Delegate <compiler-generated> (KituraNetPackageTests:x86_64+0x10e6e6)
    #2 closure #1 in HTTPRequestHandler.channelRead(ctx:data:) HTTPRequestHandler.swift:102 (KituraNetPackageTests:x86_64+0x5698a)
    #3 partial apply for closure #1 in HTTPRequestHandler.channelRead(ctx:data:) <compiler-generated> (KituraNetPackageTests:x86_64+0x56a2d)
    #4 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x3ab50)
    #5 __tsan::invoke_and_release_block(void*) <null>:8038048 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x685ab)
    #6 _dispatch_client_callout <null>:8038048 (libdispatch.dylib:x86_64+0x3dce)

  Previous write of size 8 at 0x7b080002e6b0 by thread T16:
    #0 Delegate.handle(request:response:) PipeliningTests.swift:130 (KituraNetPackageTests:x86_64+0x10dff7)
    #1 protocol witness for ServerDelegate.handle(request:response:) in conformance Delegate <compiler-generated> (KituraNetPackageTests:x86_64+0x10e6e6)
    #2 closure #1 in HTTPRequestHandler.channelRead(ctx:data:) HTTPRequestHandler.swift:102 (KituraNetPackageTests:x86_64+0x5698a)
    #3 partial apply for closure #1 in HTTPRequestHandler.channelRead(ctx:data:) <compiler-generated> (KituraNetPackageTests:x86_64+0x56a2d)
    #4 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (KituraNetPackageTests:x86_64+0x3ab50)
    #5 __tsan::invoke_and_release_block(void*) <null>:8038048 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x685ab)
    #6 _dispatch_client_callout <null>:8038048 (libdispatch.dylib:x86_64+0x3dce)

  Location is heap block of size 24 at 0x7b080002e6a0 allocated by main thread:
    #0 malloc <null>:8038080 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x4adca)
    #1 swift_slowAlloc <null>:8038080 (libswiftCore.dylib:x86_64+0x3093f8)
    #2 PipeliningTests.testPipeliningSpanningPackets() PipeliningTests.swift:86 (KituraNetPackageTests:x86_64+0x10ae15)
    #3 @objc PipeliningTests.testPipeliningSpanningPackets() <compiler-generated> (KituraNetPackageTests:x86_64+0x10d390)
    #4 __invoking___ <null>:8038080 (CoreFoundation:x86_64h+0x4ad1b)

  Thread T31 (tid=18612269, running) is a GCD worker thread

  Thread T16 (tid=18612242, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race PipeliningTests.swift:126 in Delegate.handle(request:response:)
==================
Test Case '-[KituraNetTests.PipeliningTests testPipeliningSpanningPackets]' passed (0.467 seconds).

Build failures on Darwin with SSLService changes

/Users/Nethra/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:54:45: error: 'init(withCipherSuite:clientAllowsSelfSignedCertificates:)' is unavailable: This API not supported on Apple platforms.
    static let clientSSLConfig = SSLService.Configuration(withCipherSuite: nil, clientAllowsSelfSignedCertificates: true)
                                            ^~~~~~~~~~~~~
SSLService.SSLService:20:16: note: 'init(withCipherSuite:clientAllowsSelfSignedCertificates:)' has been explicitly marked unavailable here
        public init(withCipherSuite cipherSuite: String? = default, clientAllowsSelfSignedCertificates: Bool = default)
               ^
/Users/Nethra/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:44:31: error: 'init(withCACertificateDirectory:usingCertificateFile:withKeyFile:usingSelfSignedCerts:cipherSuite:)' is unavailable: This API not supported on Apple platforms.
            return SSLService.Configuration(withCACertificateDirectory: nil, usingCertificateFile: certificatePath,
                              ^~~~~~~~~~~~~
SSLService.SSLService:28:16: note: 'init(withCACertificateDirectory:usingCertificateFile:withKeyFile:usingSelfSignedCerts:cipherSuite:)' has been explicitly marked unavailable here
        public init(withCACertificateDirectory caCertificateDirPath: String?, usingCertificateFile certificateFilePath: String?, withKeyFile keyFilePath: String? = default, usingSelfSignedCerts selfSigned: Bool = default, cipherSuite: String? = default)
               ^
/Users/Nethra/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:44:31: error: 'init(withCACertificateDirectory:usingCertificateFile:withKeyFile:usingSelfSignedCerts:cipherSuite:)' is unavailable: This API not supported on Apple platforms.
            return SSLService.Configuration(withCACertificateDirectory: nil, usingCertificateFile: certificatePath,
                              ^~~~~~~~~~~~~
SSLService.SSLService:28:16: note: 'init(withCACertificateDirectory:usingCertificateFile:withKeyFile:usingSelfSignedCerts:cipherSuite:)' has been explicitly marked unavailable here
        public init(withCACertificateDirectory caCertificateDirPath: String?, usingCertificateFile certificateFilePath: String?, withKeyFile keyFilePath: String? = default, usingSelfSignedCerts selfSigned: Bool = default, cipherSuite: String? = default)
               ^
/Users/Nethra/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:54:45: error: 'init(withCipherSuite:clientAllowsSelfSignedCertificates:)' is unavailable: This API not supported on Apple platforms.
    static let clientSSLConfig = SSLService.Configuration(withCipherSuite: nil, clientAllowsSelfSignedCertificates: true)
                                            ^~~~~~~~~~~~~
SSLService.SSLService:20:16: note: 'init(withCipherSuite:clientAllowsSelfSignedCertificates:)' has been explicitly marked unavailable here
        public init(withCipherSuite cipherSuite: String? = default, clientAllowsSelfSignedCertificates: Bool = default)
               ^
/Users/Nethra/Kitura-NIO/Tests/KituraNetTests/KituraNIOTest.swift:54:45: error: 'init(withCipherSuite:clientAllowsSelfSignedCertificates:)' is unavailable: This API not supported on Apple platforms.
    static let clientSSLConfig = SSLService.Configuration(withCipherSuite: nil, clientAllowsSelfSignedCertificates: true)
                                            ^~~~~~~~~~~~~

ThinkTime test route /think/async fails with KituraNIO

ThinkTime test with the route /think/async crashes in Kitura-NIO.

The crash is because the thread that initiated the I/O task is not the same thread that performs the I/O operation. ie, the current thread that tries to perform the operation is not as the same thread that was tied with EventLoop.

Hence we must ensure that we perform the I/O task on the same thread or we submit the task to be executed to the EventLoop.

testSimpleHTTPClient is flaky and buggy

testSimpleHTTPClient does a GET to http://www.ibm.com and asserts that the HTTP status returned is OK. In reality, we sometimes seem to be getting a movedTemporarily, the reason for which is beyond the scope of this project. I have #20 raised for this.

There's another problem in the test case. The test makes the request but does not wait for expectations to be fulfilled. In the case of failure, some other test appears to have failed. This needs to be fixed, but it may increase the test times. I propose we watch the behaviour with #20 and take a call later, accordingly.

Test LifecycleListenerTests.testLifecycle is flaky

This failure shows up once in a while:

Test Suite 'LifecycleListenerTests' started at 2018-07-28 05:12:50.104
Test Case 'LifecycleListenerTests.testLifecycle' started at 2018-07-28 05:12:50.104
/home/travis/build/IBM-Swift/Kitura-NIO/Tests/KituraNIOTests/LifecycleListenerTests.swift:72: error: LifecycleListenerTests.testLifecycle : Asynchronous wait failed - Exceeded timeout of 5.0 seconds, with unfulfilled expectations: stop

Poor performance with the web-frameworks benchmark

Kitura with Kitura-NIO seems to performs badly on the web-frameworks benchmarks

Reported over slack. Copied from there:

Language (Runtime) Framework (Middleware) Average 50th percentile 90th percentile 99th percentile 99.9th percentile Standard deviation
swift (4.2) perfect (3.1) 22.11 ms 21.81 ms 26.35 ms 33.46 ms 56.41 ms 3607.00
swift (5.0) vapor (3.3) 65.25 ms 27.59 ms 49.31 ms 1354.51 ms 3496.16 ms 228498.67
swift (5.0) kitura (2.6) 35.61 ms 33.10 ms 48.56 ms 74.07 ms 647.15 ms 26 433.67
swift (5.0) kitura (2.7) 62.88 ms 61.14 ms 83.65 ms 115.44 ms 533.98 ms 21219.00
swift (5.0) kitura-nio (2.7) 177.09 ms 65.25 ms 115.28 ms 3343.02 ms 6831.05 ms 555543.00
Language (Runtime) Framework (Middleware) Requests / s Throughput
swift (4.2) perfect (3.1) 44599.00 41.93 MB
swift (5.0) vapor (3.3) 32438.67 54.37 MB
swift (5.0) kitura (2.6) 28 626.67 53.09 MB
swift (5.0) kitura (2.7) 15597.00 29.04 MB
swift (5.0) kitura-nio (2.7) 13049.00 24.67 MB

CI failures on kitura-2.6 branch

https://travis-ci.org/IBM-Swift/Kitura/jobs/487043108

Compile CNIOOpenSSL shims.c
In file included from /Users/travis/build/IBM-Swift/Kitura/.build/checkouts/swift-nio-ssl.git--2353935232896716629/Sources/CNIOOpenSSL/shims.c:16:
/Users/travis/build/IBM-Swift/Kitura/.build/checkouts/swift-nio-ssl.git--2353935232896716629/Sources/CNIOOpenSSL/include/c_nio_openssl.h:17:10: fatal error: 'openssl/conf.h' file not found
#include <openssl/conf.h>
         ^~~~~~~~~~~~~~~~
1 error generated.

I suspect this is because Kitura/Kitura@68e9893 needs backporting to the kitura-2.6 branch.

Spaces in URL query parameters not parsing correctly

When switching from KituraNet to Kitura-NIO URLs errors occur with spaces in query parameters.

A URL such as http://www.something.com/hi/there?parameter=Hi%20There will either throw an exception (SwiftMetrics):

*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** +[NSURLComponents setPercentEncodedQuery:]: invalid characters in percentEncodedQuery'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff2bd8ccf9 __exceptionPreprocess + 256
    1   libobjc.A.dylib                     0x00007fff56921a17 objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff2bd8cb2b +[NSException raise:format:] + 201
    3   libswiftFoundation.dylib            0x00007fff57dc3ba2 $s10Foundation13URLComponentsV19percentEncodedQuerySSSgvsTf4gn_n + 402
    4   libswiftFoundation.dylib            0x00007fff57d5a0c2 $s10Foundation13URLComponentsV19percentEncodedQuerySSSgvs + 18
    5   KituraNet                           0x000000010122285d $s9KituraNet17HTTPServerRequestC6urlURL10Foundation0F0Vvg + 5597
    6   KituraNet                           0x0000000101227141 $s9KituraNet17HTTPServerRequestCAA06ServerD0A2aDP6urlURL10Foundation0G0VvgTW + 17
    7   SwiftMetricsKitura                  0x0000000100a124c3 $s18SwiftMetricsKitura11HttpMonitor33_AF26E906DB1EA1A8FEA139730DBA0D88LLC7started7request8responsey0C3Net13ServerRequest_p_AH0R8Response_ptF + 307
    8   SwiftMetricsKitura                  0x0000000100a137b5 $s18SwiftMetricsKitura11HttpMonitor33_AF26E906DB1EA1A8FEA139730DBA0D88LLC0C3Net06ServerE0AaeFP7started7request8responseyAE0O7Request_p_AE0O8Response_ptFTW + 21
    9   KituraNet                           0x0000000101210fe5 $s9KituraNet18HTTPRequestHandlerC11channelRead7context4datay3NIO07ChannelD7ContextC_AG6NIOAnyVtFyycfU_ + 837
    10  KituraNet                           0x00000001012110fc $s9KituraNet18HTTPRequestHandlerC11channelRead7context4datay3NIO07ChannelD7ContextC_AG6NIOAnyVtFyycfU_TA + 12
    11  KituraNet                           0x00000001011fe11d $sIeg_IeyB_TR + 45
    12  libdispatch.dylib                   0x0000000101dd8e7c _dispatch_call_block_and_release + 12
    13  libdispatch.dylib                   0x0000000101dd9f1b _dispatch_client_callout + 8
    14  libdispatch.dylib                   0x0000000101deca06 _dispatch_root_queue_drain + 816
    15  libdispatch.dylib                   0x0000000101ded2da _dispatch_worker_thread2 + 125
    16  libsystem_pthread.dylib             0x0000000101e530b7 _pthread_wqthread + 583
    17  libsystem_pthread.dylib             0x0000000101e52e01 start_wqthread + 13
)
libc++abi.dylib: terminating with uncaught exception of type NSException

OR have a Router error without Swift Metrics:

[ERROR] [Router.swift:613 process(request:response:callback:)] request.parsedURLPath.path is nil. Failed to process request

RegressionTests.testBadRequestFollowingGoodRequest fails with #78

With the router invocation moved to a DispatchQueue as in #78 we see a new failure in RegressionTests.testBadRequestFollowingGoodRequest. A work-around for this failure was included in #78.

Initially, with the changes for #78, the test first started failing with a precondition failure at HTTPServerPipelineHandler.swift, line 108. This was fixed in swift-nio through this PR. After that, the same test starts failing with no activity on the client.

The test kicks off an HTTP server and makes a good request (GET /) followed by a bad request to an invalid route (GET #/). The first request has a normal response and the second one has an error response triggered from NIOHTTP1.HTTPServerProtocolErrorHandler. We only check for a badRequest response from the second request. With #78 we seem to be receiving neither of the responses at the client.

I printed some diagnostics as suggested by @weissi using a ChannelHandler. I added this to the server pipeline:

final class PrintEverythingHandler: ChannelDuplexHandler {
    typealias OutboundIn = HTTPServerResponsePart
    typealias InboundIn = HTTPServerRequestPart
    
    func channelRead(ctx: ChannelHandlerContext, data: NIOAny) {
        print("Server <- \(self.unwrapInboundIn(data))")
        ctx.fireChannelRead(data)
    }
    
    func write(ctx: ChannelHandlerContext, data: NIOAny, promise: EventLoopPromise<Void>?) {
        print("Server -> \(self.unwrapOutboundIn(data))")
        ctx.write(data, promise: promise)
    }
}

... and this to the client pipeline

final class PrintEverythingHandler: ChannelDuplexHandler {
    typealias OutboundIn = HTTPClientRequestPart
    typealias InboundIn = HTTPClientResponsePart

    func channelRead(ctx: ChannelHandlerContext, data: NIOAny) {
        print("Client <- \(self.unwrapInboundIn(data))")
        ctx.fireChannelRead(data)
    }

    func write(ctx: ChannelHandlerContext, data: NIOAny, promise: EventLoopPromise<Void>?) {
        print("Client -> \(self.unwrapOutboundIn(data))")
        ctx.write(data, promise: promise)
    }
}

Server <- means "server received" and Server -> means the "server sent". Client <- and Client -> have similar meanings.

Now, I see two types of intermittent problems running RegressionTests.testBadRequestFollowingGoodRequest

Problem 1

Client -> head(HTTPRequestHead { method: GET, uri: "/", version: HTTP/1.1, headers: [] })
Client -> end(nil)

Client -> head(HTTPRequestHead { method: GET, uri: "#/", version: HTTP/1.1, headers: [] })
Client -> end(nil)

Server <- head(HTTPRequestHead { method: GET, uri: "/", version: HTTP/1.1, headers: [] })
Server <- end(nil)

Server -> head(HTTPResponseHead { version: HTTP/1.1, status: notFound, headers: [("Date", "Sun, 26 Aug 2018 18:37:21 GMT"), ("Content-Type", "text/plain"), ("Connection", "Keep-Alive"), ("Keep-Alive", "timeout=60.0"), ("Content-Length", "14")] })
Server -> body(NIO.IOData.byteBuffer(ByteBuffer { readerIndex: 0, writerIndex: 14, readableBytes: 14, capacity: 16, slice: _ByteBufferSlice(upperBound: 16, _begin: 0), storage: 0x00007f6be0027400 (16 bytes)}))
Server -> end(nil)

So, the server sends a response to the first request, no error response to the second request and nothing is received at the client.

Problem 2

Client -> head(HTTPRequestHead { method: GET, uri: "/", version: HTTP/1.1, headers: [] })
Client -> end(nil)

Client -> head(HTTPRequestHead { method: GET, uri: "#/", version: HTTP/1.1, headers: [] })
Client -> end(nil)

Server <- head(HTTPRequestHead { method: GET, uri: "/", version: HTTP/1.1, headers: [] })
Server <- end(nil)

Here, the server send neither of the responses.

Expected output

Client -> head(HTTPRequestHead { method: GET, uri: "/", version: HTTP/1.1, headers: [] })
Client -> end(nil)

Client -> head(HTTPRequestHead { method: GET, uri: "#/", version: HTTP/1.1, headers: [] })
Client -> end(nil)

Server <- head(HTTPRequestHead { method: GET, uri: "/", version: HTTP/1.1, headers: [] })
Server <- end(nil)
Server -> head(HTTPResponseHead { version: HTTP/1.1, status: notFound, headers: [("Connection", "Keep-Alive"), ("Content-Type", "text/plain"), ("Date", "Sun, 26 Aug 2018 18:38:38 GMT"), ("Keep-Alive", "timeout=60.0"), ("Content-Length", "14")] })
Server -> body(NIO.IOData.byteBuffer(ByteBuffer { readerIndex: 0, writerIndex: 14, readableBytes: 14, capacity: 16, slice: _ByteBufferSlice(upperBound: 16, _begin: 0), storage: 0x00007f2a1c02d030 (16 bytes)}))
Server -> end(nil)

HTTPServerProtocolErrorHandler returning an error response

Client <- head(HTTPResponseHead { version: HTTP/1.1, status: notFound, headers: [("Connection", "Keep-Alive"), ("Content-Type", "text/plain"), ("Date", "Sun, 26 Aug 2018 18:38:38 GMT"), ("Keep-Alive", "timeout=60.0"), ("Content-Length", "14")] })

Client <- body(ByteBuffer { readerIndex: 0, writerIndex: 14, readableBytes: 14, capacity: 14, slice: _ByteBufferSlice(upperBound: 173, _begin: 159), storage: 0x00007f2a14031c60 (1024 bytes)})
Client <- end(nil)

Client <- head(HTTPResponseHead { version: HTTP/1.1, status: badRequest, headers: [("Connection", "close"), ("Content-Length", "0")] })
Client <- end(nil)

Here, the server sent a normal response followed by an error response and both of them were received on the client.

If the two requests are separated by a time gap, we always get the expected output. This is how we are currently working around the problem.

Swift 5 performance: remove NSString methods

String bridging to NSString needs to be avoided on Swift 5 given the performance penalty that it currently comes with. This issue relates to the removal of all NSString uses (or NSString methods called on Strings via bridging).

write errors while running the TechEmpower/Updates benchmark

When the Updates benchmark is run with Kitura-NIO, we see some intermittent write errors:

[2018-11-20T19:39:41.296Z] [ERROR] [HTTPServerResponse.swift:90 write(from:)] No channel available to write
[2018-11-20T19:39:41.297Z] [ERROR] [HTTPServerResponse.swift:123 end()] No channel available to end the response
[2018-11-20T19:39:41.297Z] [ERROR] [HTTPServerResponse.swift:90 write(from:)] No channel available to write
[2018-11-20T19:39:41.297Z] [ERROR] [HTTPServerResponse.swift:123 end()] No channel available to end the response

The errors come from here and here.

Ability to limit incoming request size (Kitura#1384)

Copied from Kitura/Kitura#1384

Kitura currently does not have the concept of a limit on incoming request size. It is possible to destabilize a server by sending (accidentally or maliciously) a large request payload which either impairs performance, or exhausts available memory.

A limit should be configurable on a server which causes requests above that size to be rejected, and the connection closed.

ClientE2ETests failing with ioOnClosedChannel

If IPv6 is disabled, the IPv6Tests are bound to fail. In addition to that, if the ClientE2ETests are run subsequently, most of them fail with NIO.ChannelError.ioOnClosedChannel or NIOOpenSSL.OpenSSLError.sslError.

Please note the strict conditions for these failures to occur:

  1. IPv6 is disabled
  2. The IPv6Tests are run before the failing ClientE2ETests

flaky test ClientE2ETests.testSimpleHTTPClient

failure

Test Case 'ClientE2ETests.testSimpleHTTPClient' started at 2018-10-19 15:44:10.365
/tmp/.nio-release-tools_g7eLXu/Kitura-NIO/Tests/KituraNetTests/ClientE2ETests.swift:113: error: ClientE2ETests.testSimpleHTTPClient : XCTAssertEqual failed: ("Optional(KituraNet.HTTPStatusCode.serviceUnavailable)") is not equal to ("Optional(KituraNet.HTTPStatusCode.OK)") - HTTP Status code was Optional(KituraNet.HTTPStatusCode.serviceUnavailable)
Test Case 'ClientE2ETests.testSimpleHTTPClient' failed (30.195 seconds)

OS

Ubuntu 14.04 in Docker

code

KituraNIO hash: 214abc9
swift-nio: 9bae007a15b49d7fe685b6b33308822c6d0536c0
swift-nio-ssl: db16c3a90b101bb53b26a58867a344ad428072e0

$ cat Package.resolved 
{
  "object": {
    "pins": [
      {
        "package": "Socket",
        "repositoryURL": "https://github.com/IBM-Swift/BlueSocket.git",
        "state": {
          "branch": null,
          "revision": "fb4e74cd8eb3ce4a2b860fbf727b2dada796d77d",
          "version": "1.0.22"
        }
      },
      {
        "package": "SSLService",
        "repositoryURL": "https://github.com/IBM-Swift/BlueSSLService.git",
        "state": {
          "branch": null,
          "revision": "e2af36601f91b3cefa371c00ecdcb62fef1d000a",
          "version": "1.0.22"
        }
      },
      {
        "package": "LoggerAPI",
        "repositoryURL": "https://github.com/IBM-Swift/LoggerAPI.git",
        "state": {
          "branch": null,
          "revision": "5041f2673aa75d6e973d9b6bd3956bc5068387c8",
          "version": "1.7.3"
        }
      },
      {
        "package": "OpenSSL",
        "repositoryURL": "https://github.com/IBM-Swift/OpenSSL.git",
        "state": {
          "branch": null,
          "revision": "ba74a683141fde66d4bf88358b6a678d7e5a27ce",
          "version": "1.0.1"
        }
      },
      {
        "package": "swift-nio-ssl-support",
        "repositoryURL": "https://github.com/apple/swift-nio-ssl-support.git",
        "state": {
          "branch": null,
          "revision": "c02eec4e0e6d351cd092938cf44195a8e669f555",
          "version": "1.0.0"
        }
      },
      {
        "package": "swift-nio-zlib-support",
        "repositoryURL": "https://github.com/apple/swift-nio-zlib-support.git",
        "state": {
          "branch": null,
          "revision": "37760e9a52030bb9011972c5213c3350fa9d41fd",
          "version": "1.0.0"
        }
      }
    ]
  },
  "version": 1
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.