vapor / websocket-kit Goto Github PK

Describe the bug

A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:

1. Add package with configuration '...'
2. Send request with options '...'
3. See error

Expected behavior

A clear and concise description of what you expected to happen.

Environment

• Vapor Framework version: 4.0
• Vapor Toolbox version:
• OS version: ubuntu18.04

Additional context

Add any other context about the problem here.

my configure.swift file

import Vapor

public func configure(_ app: Application) throws {
    try routes(app)
    let webs = Wbs()
    webs.echoTest(app)
}

class Wbs: NSObject {
    func echoTest(_ app: Application) {
        let url = "wss://fstream.binance.com/stream?streams=btcusdt@aggTrade/btcusdt@kline_1m/btcusdt@bookTicker"
        WebSocket.connect(to: url, on: app.eventLoopGroup) { ws in
            ws.onText { (ws, text) in
                debugPrint("rec:\(text)")
            }
        }
    }
}

i run it,then crashed, output info:

[ NOTICE ] Server starting on http://127.0.0.1:10010
[ ERROR ] bind(descriptor:ptr:bytes:): Address already in use (errno: 98)
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
Fatal error: leaking promise created at (file: "/root/websocket-master/.build/checkouts/websocket-kit/Sources/WebSocketKit/WebSocketClient.swift", line: 61): file /root/websocket-master/.build/checkouts/websocket-kit/Sources/WebSocketKit/WebSocketClient.swift, line 61
Current stack trace:
0    libswiftCore.so                    0x00007f670ee4e990 swift_reportError + 50
1    libswiftCore.so                    0x00007f670eec2260 _swift_stdlib_reportFatalErrorInFile + 115
2    libswiftCore.so                    0x00007f670eba7925  + 1399077
3    libswiftCore.so                    0x00007f670eba7567  + 1398119
4    libswiftCore.so                    0x00007f670eba7b02  + 1399554
5    libswiftCore.so                    0x00007f670eba5fa0 _assertionFailure(_:_:file:line:flags:) + 517
6    Run                                0x0000557935092ac2  + 6871746
7    Run                                0x0000557935176f6e  + 7806830
8    Run                                0x0000557935176ed8  + 7806680
9    Run                                0x00005579350924e4  + 6870244
10   Run                                0x0000557935092cf5  + 6872309
11   libswiftCore.so                    0x00007f670ee5058b  + 4187531
12   Run                                0x000055793560f0bc  + 12624060
13   libswiftCore.so                    0x00007f670ee5058b  + 4187531
14   Run                                0x0000557934ffe502  + 6264066
15   Run                                0x0000557934ffe5e9  + 6264297
16   libswiftCore.so                    0x00007f670ee5058b  + 4187531
17   Run                                0x00005579350b3c55  + 7007317
18   Run                                0x00005579350b3e59  + 7007833
19   libswiftCore.so                    0x00007f670ee5058b  + 4187531
20   Run                                0x00005579350b60e5  + 7016677
21   libswiftCore.so                    0x00007f670ee5058b  + 4187531
22   Run                                0x000055793511a79a  + 7427994
23   libswiftCore.so                    0x00007f670ee5058b  + 4187531
24   Run                                0x000055793508a8e1  + 6838497
25   Run                                0x000055793508a919  + 6838553
26   libswiftCore.so                    0x00007f670ee5058b  + 4187531
27   Run                                0x0000557935117b25  + 7416613
28   Run                                0x0000557934b19a6f  + 1133167
29   Run                                0x000055793518a944  + 7887172
30   Run                                0x000055793518a7d4  + 7886804
31   Run                                0x000055793518a8b1  + 7887025
32   Run                                0x0000557935115d3f  + 7408959
33   Run                                0x000055793508691b  + 6822171
34   Run                                0x00005579350870df  + 6824159
35   Run                                0x000055793508dda3  + 6852003
36   Run                                0x00005579350876ff  + 6825727
37   Run                                0x00005579351706c1  + 7780033
38   Run                                0x0000557935173c4e  + 7793742
39   Run                                0x0000557935173df9  + 7794169
40   libpthread.so.0                    0x00007f670e83a6db  + 30427
41   libc.so.6                          0x00007f670c9c36e0 clone + 63
0x557934b85bb9, closure #1 (Swift.Int32) -> () in static Backtrace.Backtrace.install() -> () at /root/websocket-master/.build/checkouts/swift-backtrace/Sources/Backtrace/Backtrace.swift:67
0x557934b85bc8, @objc closure #1 (Swift.Int32) -> () in static Backtrace.Backtrace.install() -> () at /root/websocket-master/:0
0x7f670e84597f
0x7f670eba61b2
0x557935092ac1, closure #1 () -> () in NIO.EventLoopFuture.deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:419
0x557935176f6d, closure #1 () -> Swift.Bool in implicit closure #1 () -> Swift.Bool in NIO.debugOnly(() -> ()) -> () at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/Utilities.swift:44
0x557935176ed7, NIO.debugOnly(() -> ()) -> () at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/Utilities.swift:44
0x5579350924e3, NIO.EventLoopFuture.deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:415
0x557935092cf4, NIO.EventLoopFuture.__deallocating_deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:0
0x7f670ee5058a
0x55793560f0bb, objectdestroy at /root/websocket-master/:0
0x7f670ee5058a
0x557934ffe501, NIO.ClientBootstrap.deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/Bootstrap.swift:0
0x557934ffe5e8, NIO.ClientBootstrap.__deallocating_deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/Bootstrap.swift:0
0x7f670ee5058a
0x5579350b3c54, NIO.HappyEyeballsConnector.deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/HappyEyeballs.swift:0
0x5579350b3e58, NIO.HappyEyeballsConnector.__deallocating_deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/HappyEyeballs.swift:0
0x7f670ee5058a
0x5579350b60e4, objectdestroy.43 at /root/websocket-master/:0
0x7f670ee5058a
0x55793511a799, objectdestroy at /root/websocket-master/:0
0x7f670ee5058a
0x55793508a8e0, NIO.ScheduledTask.deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/EventLoop.swift:0
0x55793508a918, NIO.ScheduledTask.__deallocating_deinit at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/EventLoop.swift:0
0x7f670ee5058a
0x557935117b24, closure #3 () -> () in NIO.SelectableEventLoop.run() throws -> () at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/SelectableEventLoop.swift:441
0x557934b19a6e, reabstraction thunk helper from @callee_guaranteed () -> (@error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error) at /root/websocket-master/:0
0x55793518a943, partial apply forwarder for reabstraction thunk helper from @callee_guaranteed () -> (@error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error) at /root/websocket-master/:0
0x55793518a7d3, NIOConcurrencyHelpers.Lock.withLock(() throws -> A) throws -> A at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIOConcurrencyHelpers/lock.swift:105
0x55793518a8b0, NIOConcurrencyHelpers.Lock.withLockVoid(() throws -> ()) throws -> () at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIOConcurrencyHelpers/lock.swift:111
0x557935115d3e, NIO.SelectableEventLoop.run() throws -> () at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/SelectableEventLoop.swift:424
0x55793508691a, static NIO.MultiThreadedEventLoopGroup.(runTheLoop in _D5D78C61B22284700B9BD1ACFBC25157)(thread: NIO.NIOThread, canEventLoopBeShutdownIndividually: Swift.Bool, selectorFactory: () throws -> NIO.Selector, initializer: (NIO.NIOThread) -> (), _: (NIO.SelectableEventLoop) -> ()) -> () at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/EventLoop.swift:851
0x5579350870de, closure #1 (NIO.NIOThread) -> () in static NIO.MultiThreadedEventLoopGroup.(setupThreadAndEventLoop in _D5D78C61B22284700B9BD1ACFBC25157)(name: Swift.String, selectorFactory: () throws -> NIO.Selector, initializer: (NIO.NIOThread) -> ()) -> NIO.SelectableEventLoop at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/EventLoop.swift:871
0x55793508dda2, partial apply forwarder for closure #1 (NIO.NIOThread) -> () in static NIO.MultiThreadedEventLoopGroup.(setupThreadAndEventLoop in _D5D78C61B22284700B9BD1ACFBC25157)(name: Swift.String, selectorFactory: () throws -> NIO.Selector, initializer: (NIO.NIOThread) -> ()) -> NIO.SelectableEventLoop at /root/websocket-master/:0
0x5579350876fe, reabstraction thunk helper from @escaping @callee_guaranteed (@guaranteed NIO.NIOThread) -> () to @escaping @callee_guaranteed (@in_guaranteed NIO.NIOThread) -> (@out ()) at /root/websocket-master/:0
0x5579351706c0, partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@guaranteed NIO.NIOThread) -> () to @escaping @callee_guaranteed (@in_guaranteed NIO.NIOThread) -> (@out ()) at /root/websocket-master/:0
0x557935173c4d, closure #1 (Swift.Optional) -> Swift.Optional in static NIO.ThreadOpsPosix.run(handle: inout Swift.Optional, args: NIO.Box<(body: (NIO.NIOThread) -> (), name: Swift.Optional)>, detachThread: Swift.Bool) -> () at /root/websocket-master/.build/checkouts/swift-nio/Sources/NIO/ThreadPosix.swift:105
0x557935173df8, @objc closure #1 (Swift.Optional) -> Swift.Optional in static NIO.ThreadOpsPosix.run(handle: inout Swift.Optional, args: NIO.Box<(body: (NIO.NIOThread) -> (), name: Swift.Optional)>, detachThread: Swift.Bool) -> () at /root/websocket-master/:0
0x7f670e83a6da
0x7f670c9c371e
0xffffffffffffffff
Illegal instruction

Connection establishing is currently spread out over three files.

WebSocketHandler is basically an implementation detail of WebSocket and should be part of it. The channel is already passed to the ctor.

Would be great to be able to specify a generic Message type on client.connect, that would auto-encode/decode if Codable conformance is present. This type could default to String.

onText could also be renamed onMessage.

Just a thought.

Related to discussion here: vapor/vapor#2451 (comment)

It could make sense to have WebSocketHandler listen for a quiesce request and close the WebSocket automatically.

Describe the bug

A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:

1. Add package with configuration '...'
2. Send request with options '...'
3. See error

Expected behavior

A clear and concise description of what you expected to happen.

Environment

• Vapor Framework version: 4.39.2
• Vapor Toolbox version: 18.3.0
• OS version:macOS 11.1

Additional context

Add any other context about the problem here.

Hi,
i try connect to a websocket,but onText function no get any data?

I found it's hard to do unit test in Vapor with WebSocket.
I have a echo websocket set at "ws://localhost/"
I tried to use
try app.client().webSocket("ws://localhost/").wait(),
And tried
HTTPClient.webSocket(hostname: "localhost",port: 8080, on: worker).wait()

Both of them throw error: The operation couldn’t be completed. (NIO.ChannelError error 1.) when i try to connect.

The detailed error is

ChannelError
  ▿ connectFailed : NIOConnectionError
    - host : "localhost"
    - port : 80
    - dnsAError : nil
    - dnsAAAAError : nil
    ▿ connectionErrors : 2 elements
      ▿ 0 : SingleConnectionFailure
        ▿ target : [IPv6]localhost/::1:80
          ▿ v6 : IPv6Address
            ▿ _storage : <Box<(address: sockaddr_in6, host: String)>: 0x100c7d3b0>
        ▿ error : connection reset (error set): Connection refused (errno: 61)
          - errnoCode : 61
          ▿ reason : FailureDescription
            - reason : "connection reset (error set)"
      ▿ 1 : SingleConnectionFailure
        ▿ target : [IPv4]localhost/127.0.0.1:80
          ▿ v4 : IPv4Address
            ▿ _storage : <Box<(address: sockaddr_in, host: String)>: 0x100c57ec0>
        ▿ error : connection reset (error set): Connection refused (errno: 61)
          - errnoCode : 61
          ▿ reason : FailureDescription
            - reason : "connection reset (error set)"

Can anyone help me?

Some APIs require that each frame is sent with a maskKey. It would be awesome if the library could do this automatically, or if we could opt in for a random maskKey for each frame.

Additionally, NIO's default maxFrameSize is 1 << 14 which is not large enough to decode some very large frames. I saw that your current PR #6 has support for this in the server upgrader, but it would be awesome to also get this support for clients.

Is there any way to guarantee delivery via WebSocket.send, or catch an error when not delivering,

i registered performance problems if i send lot of data from a client to the server. maximum throughput i can reach is about 20 Mbyte/s and CPU Usage is about 99-100%. Is this normal ?!

When the server send a frame immediately after being connected it's possible to lose it and never have the onTextCallback called depending on how threads are executed.

Below is an exemple with the Discord gateway API:

import Foundation
import Async
import WebSocket

// TL;DR Discord socket flow:
// 1. open socket
// 2. server immediately send a "hello" opcode
// 3.1 client send a "heartbeat" opcode every time interval given by the gateway in "hello"
// 3.2 server send an "ack" after each heartbeat
// 4. client send an "identify" opcode (not show here)
// 5. server send "dispatch" opcodes depending on events

let worker = MultiThreadedEventLoopGroup(numberOfThreads: System.coreCount)

print("connecting...")
let socket = try HTTPClient.webSocket(scheme: .wss, hostname: "gateway.discord.gg", path: "/v=6&encoding=json", on: worker).wait()
print("connected!")

socket.onText({ (ws, text) in
    // first frame expected from the socket is a "hello" and looks like:
    //   {"t":null,"s":null,"op":10,"d":{"heartbeat_interval":41250,"_trace":["gateway-prd-main-v1h0"]}}
    // instead the first frame received is an "ack" frame in response to the "heartbeat" sent below
    //   {"t":null,"s":null,"op":11,"d":null}
    // we have missed the "hello" frame
    print("received", text)
    socket.close(code: .normalClosure)
})

socket.onBinary({ print("data", $1) })
socket.onError({ print("error", $1) })
socket.onCloseCode({ print("code", $0) })

// Discord protocol requires to send "heartbeat" periodically
// this is a valid heartbeat request to send after the "hello"
socket.send("{\"op\":1,\"d\":null}")

try socket.onClose.wait()
print("disconnected.")

Expected output:

connecting...
connected!
received {"t":null,"s":null,"op":10,"d":{"heartbeat_interval":41250,"_trace":["gateway-prd-main-04vj"]}}
code protocolError
error uncleanShutdown
disconnected.

Actual output:

connecting...
connected!
received {"t":null,"s":null,"op":11,"d":null}
code protocolError
error uncleanShutdown
disconnected.

Because it's related to both network and threading, you might not encounter the issue each run.

By design in Vapor there is no call to connect in the user implementation, so I can't control the exact moment the WebSocket is opened and register my callback before.

I think the only possible solution is to move to a delegate pattern that would be set during init and guarantee to be there right from the start.

I can provide a PR but it dramatically change the current design.

A Client receives only first n bytes from the first frame and skip other continuation frames, as a result, received message is broken. Client needs to implement Data Framing(RFC 6455 - Section 5).

According to rfc:

In the WebSocket Protocol, data is transmitted using a sequence of
frames.
The base framing protocol defines a frame type with an opcode, a
payload length, and designated locations for "Extension data" and
"Application data", which together define the "Payload data".
Certain bits and opcodes are reserved for future expansion of the
protocol.
A data frame MAY be transmitted by either the client or the server at
any time after opening handshake completion and before that endpoint
has sent a Close frame (Section 5.5.1).

There are some circumstances in my app where I'd like to send data directly from a file to a WebSocket. Is it possible to support that, without loading the file contents into memory?

I see that Channel.writeAndFlush can take a FileRegion, which conforms to NIOAny. Using that type can allow SwiftNIO to send the file without loading it into memory. However, the docs for FileRegion say:

depending your 'ChannelPipeline' setup it may not be possible to use a FileRegion as a ChannelHandler may need access to the bytes (in a ByteBuffer) to transform these.

Does anyone have a good guess (or know where to look) to see if the ChannelHandlers in Websocket-Kit and SwiftNIO need access directly to a ByteBuffer?

Without any additional port specification, a secure websocket connection is supposed to run over port 443. Websocket-kit currently falls back to port 80 when the URI has no port component, even when the protocol is wss://.

This results in connects failing when an API provider just specifies wss://abc.foo/endpoint as address without additional port information.

pingAndScheduleNextTimeoutTask in WebSocket should not call close (as this assumes the connection is still open), and should rather just immediately close the channel.

When a client sends a ping, it will be masked. The pong response should send back the unmasked value as its payload, not the masked value. Not doing so affects a client that is tracking the pings it sent out by indexing on the payload it sent.
Solution is to unmask the input frame data in the pong function.

Hi, how could I use this websocket on iOS client app without entire Vapor stack? I'm looking for alternative light weight SocketIO implementation

Servers close a connection with an error if receives not a masked frame from Client. Currently, WebSocket supports masking but does not mask all frames, it needs to mask pond and close frame also.

According to RFC6455 section 5.3:

a client MUST mask all frames that it
sends to the server

Running socket.close(code: .normalClosure) should trigger a close of the socket on the server end.

Attaching .wait() resolves immediately on the client end. But whatever I do, the socket is still marked as open by the server. This was working at some point and now it's broken.

Is your feature request related to a problem? Please describe.

The close method waits for acknowledgement from the server. In some situations we already know the connection is closed (e.g. if the reachability state to the server has changed), and would rather just force a disconnected state immediately.

Describe the solution you'd like

Introduce forceClose method to WebSocket, that just terminates the channel.

I'm using vapor/websocket in an app where main logic is implemented on DispatchQueue.main.

When WebSocket connection is established, I track the connection like this:

        }, onUpgrade: { ws, req in
            DispatchQueue.main.async {
                myApp.register(webSocket: ws)
            }

When the app has to send some data to websocket, can it call ws.send() directly from main queue? Or should it dispatch to websocket's queue first (ws.eventLoop.execute { ws.send(...) })?

In
https://github.com/vapor/websocket/blob/master/Sources/WebSocket/WebSocket.swift
I see that data is ultimately sent to a channel so as I understand eventLoop.execute{} is not required except that there's isClosed variable which can potentially be accessed from different threads.
How this should be done correctly?

In some situations the promise passed to WebSocket.close is not being completed.

Add support for close codes that are required for some API implementations.

https://tools.ietf.org/html/rfc6455#section-7.1.5

Amazon's Application Load Balancers are configured to shut down connections that haven't had traffic.

Sending a ping on the connection every so often is enough to keep the connection alive. I plan to implement this functionality on the WebSocket class and will submit a PR.

I have created a WebSocket route in Vapor 4. The client is HTML/JS. The web browser connects fine but once the server (Vapor 4) sends a message to the browser (Chrome and Safari) the following error occurs:

A server must not mask any frames that it sends to the client.

Apparently, Vapor is masking all frame sent from the server. According to the documentation, that appears to be incorrect:
ror:

In the WebSocket Protocol, data is transmitted using a sequence of frames. To avoid confusing network intermediaries (such as intercepting proxies) and for security reasons that are further discussed in Section 10.3, a client MUST mask all frames that it sends to the server (see Section 5.3 for further details). (Note that masking is done whether or not the WebSocket Protocol is run over TLS.) The server MUST close the connection upon receiving frame that is not masked. In this case, a server MAY send a Close frame with a status code of 1002 (protocol error) as defined in Section 7.4.1. A server MUST NOT mask any frames that it sends the client. A client MUST close a connection if it detects a masked
frame. In this case, it MAY use the status code 1002 (protocol error) as defined in Section 7.4.1. (These rules might be relaxed in a future specification.)

Once a changed WebSocket.makeMaskKey to return nil. The web application worked fine.

Steps to reproduce

1. Create a websocket route in Vapor.
2. Connect to the websocket from HTML/JS.
3. Send data from the Vapor 4 server to the client web browser (Safari or Chrome)

Expected behavior

You will receive the following Javascript error:

A server must not mask any frames that it sends to the client.

Actual behavior

Data sent to the browser and received with onMessage

Environment

• Vapor Framework version: 4.0.0-alpha3.1.1
• Vapor Toolbox version: 3.1.10
• OS version: macOS Catalina 10.15 Beta (19A546d)

Describe the bug

When running the client (or server) and the running app receives a Websocket message that is split over many frames CPU load is massive and it takes a very long time to parse the incoming frames concatenating the buffer and contract the message.

To Reproduce

1. create a simple Vapor app with a websocket server
2. create a WebSocket client (using this branch to that is can send large amounts of data #96)
3. send a large amount of data to the server (a few MB is enough)
4. Notice how the CPU load on the server spikes and it takes ages to handle the incoming data even through the client was able to send it all very quickly without any real CPU load.

Expected behavior

The server should be able to handle a multi frame ws message without completely locking up the cpu core at 100%.

Environment

MacOS (this happens with or without the #95)

• Vapor Framework version: 2.1.3
• Vapor Toolbox version: 18.3.3
• OS version: macOS 12.0 Beta (21A5284e)

Additional context

Running a profile suggesss that 71% of the cpu time is spent moving memory around.

and 28% is spend relocating the buffer

This seems related to the WebSocketFrameSequence.append method. It might well be better to make the WebSocketFrameSequence is it possible to create a ByteBuffer like object that just points to the underlying existing buffers rather than copies and decloates them?

/WebSocketKit/WebSocketClient.swift:39:22
warning: 'Atomic' is deprecated: please use NIOAtomic instead

let isShutdown = Atomic<Bool>(value: false)

Hi,
I've tried to modify the example in tests in order to send a small image as NSData.

This is the change on server side:

return WebSocket.server(on: channel) { ws in
  let data = UIImage(named: "test")!.pngData()!
  let byteArray = [UInt8](data)
  ws.send(raw: byteArray, opcode: .binary)
}

And this insie the portion of the client

WebSocket.connect(to: "ws://localhost:\(port)", on: self.elg) { ws in
  ws.onBinary { (socket, bytes) in
    let data = bytes.getData(at: 0, length: bytes.capacity)
    let imageFromData = UIImage(data: data!)
    ws.close(promise: nil)
}
}.cascadeFailure(to: promise)

However message is never received anything inside ws.onBinary. Where am I wrong?

I ran into an issue where a NIOWebSocketError.invalidFrameLength error was thrown, but WebSocket.closeCode is nil and WebSocket.isClosed is false.

It seems that errors thrown and caught by the NIO framework don't make their way back to the WebSocket class. The WebSocket.onClose Future is triggered, but I don't know how to get the error code at that point.

I also see that when the onClose future is fulfilled I also need to manually close the websocket, or else this assertion will trigger:

deinit {
    assert(self.isClosed, "WebSocket was not closed before deinit.")
}

I see that when the error is thrown the WebSocketProtocolErrorHandler in NIO catches and sends the appropriate close code back over the channel. Is there some way to surface this to the WebSocket class as well?

Sorry if I'm just misunderstanding the API. I'm switch to Vapor from Kitura, but I'm really liking what I see so far!

This project and vapor core don't really seem to be compatible with swift 5/NIO 2.0, whereas your http project (now http-kit) is... Are you planning on updating this one? If not, could you please put something in the readme to indicate that it is deprecated.

[Feature] Implementation of Channels With :

Authentication

• JWT
• Http Web-hook

Presence

• Active Users

Real-time Database

• Records/Lists

Emit Message

• to all channels
• to single channels

Examples :
Deepstream.io
Apache Kafka
Pusher Channels

Implementation over Vapor Websocket (NIO)

Describe the bug

An URL which contains "query" part, it is silently omitted by the websocket-kit. So it is possible to not be able to connect to a WebSocket because it can contain essential information to connect.

To Reproduce

    let wss = "wss://wssserver.com/endpoint/?essentialinfo=here"
   WebSocket.connect(to: wss, headers: headers, on: app.eventLoopGroup, onUpgrade: {...})

Steps to reproduce the behavior:

1. Add package with default configuration
2. Send request with options above
3. See error

Expected behavior

It should use the query part of the URL (why don't u use simply URL and URLComponents from the Foundation lib instead of the hackery done in this lib? :))

Environment

Xcode: Version 13.0 beta (13A5154h)

• Vapor Framework version: 4.47.0
• Vapor Toolbox version: -
• OS version: 12.0 (21A5248p)

Additional context

As per https://tools.ietf.org/html/rfc6455 section 5.5.1 closing code may be absent. In particular, Safari/Chrome/Firefox seem not to send it thus onCloseCodeCallback is not getting called at all:

        if let closeCode = data.readInteger(as: UInt16.self)
            .map(Int.init)
            .flatMap(WebSocketErrorCode.init(codeNumber:))
        {
            webSocket.onCloseCodeCallback(closeCode)
        }

It would be nice to have that callback called even if there's no code so app could stop sending data to client at this point without waiting for onClose {}.

Also there's reason field which the lib does not currently fetch which also belongs to that callback imo.

I propose renaming it to onClosingHandshakeBegin or onCloseFrameReceived with params (code: WebSocketErrorCode?, reason: String?). If it sounds good, I can make a PR.

When using vapor web socket-kit, my swift app crashes at line 341 in swift-nio/NIOHTTP1/HTTPDecoder with a nil optional.
Debugging the code, the 'swap' function did not work.
This is a Linux only issue. It works fine on MacOS.
Swift version 5.2.4 and 5.1.5 tested.
websocket-kit version 2.1.0 and master tested.

All combinations fail on Linux.

I was building Websocket and HTTP on Circle CI without Vapor installed.

After having this issue I found from vapor/vapor#1562 that I need to install libressl with brew install libressl

If someone would help me I would like to try to fix myself this.

Error from Circle CI with Xcode 9.3:

Compile Swift Module 'NIOOpenSSL' (11 sources)
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h"
        ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h:17:10: error: 'openssl/conf.h' file not found
#include <openssl/conf.h>
         ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/NIOOpenSSL/OpenSSLHandler.swift:16:8: error: could not build Objective-C module 'CNIOOpenSSL'
import CNIOOpenSSL
       ^
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h"
        ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h:17:10: error: 'openssl/conf.h' file not found
#include <openssl/conf.h>
         ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/NIOOpenSSL/OpenSSLHandler.swift:16:8: error: could not build Objective-C module 'CNIOOpenSSL'
import CNIOOpenSSL
       ^
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h"
        ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h:17:10: error: 'openssl/conf.h' file not found
#include <openssl/conf.h>
         ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/NIOOpenSSL/OpenSSLHandler.swift:16:8: error: could not build Objective-C module 'CNIOOpenSSL'
import CNIOOpenSSL
       ^
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h"
        ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/CNIOOpenSSL/include/c_nio_openssl.h:17:10: error: 'openssl/conf.h' file not found
#include <openssl/conf.h>
         ^
/Users/distiller/project/TestWebsockets/.build/checkouts/swift-nio-ssl.git-1370587408992578247/Sources/NIOOpenSSL/OpenSSLHandler.swift:16:8: error: could not build Objective-C module 'CNIOOpenSSL'
import CNIOOpenSSL
       ^
error: terminated(1): /Applications/Xcode-9.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-build-tool -f /Users/distiller/project/TestWebsockets/.build/debug.yaml main output:

If you connect to Slack RTM API, as long as you don't write a message to the socket everything is fine. But the moment you write a message the connection is dropped and no callback gets called. You can check that the connection is effectively dropped because messages from Slack are no longer received and the connection status of the Slack bot appears as disconnected.

Here is an example project that reproduces the issue. It has an alternative implementation using https://github.com/daltoniam/Starscream to prove that this is an issue with this library.

Websocket-Kit requires a deployment target of macOS 10.15+

I read that Vapor 4 requires 10.15+ because it replaced OpenSSL with the Apple's new CryptoKit library, but I was hoping to use Websocket-Kit as a stand-alone package (outside of Vapor) because I cannot yet limit my app's deployment target to 10.15+.

Is that possible? Looking at the source, I see some TLSConfiguration in websocketClient and the extension that adds the various connect() functions, but it looks like the server side of things is in the clear for older macOS versions? Is that a safe conclusion?

(Apologies for asking a question as an issue; it's just VERY hard to reach anyone with appropriate expertise any other way. I DID search!)

I'm using the WebSocketClient to start the connection, and want to define functions that use WebSocket.send outside of the onUpgrade callback.

WebSocketClient doesn't expose the websocket or a send() function.

When I create a package.swift with vapor web socket dependency and run swift build, I get errors, such as:
.build/checkouts/core.git-9210800844849382486/Sources/Async/Future+Flatten.swift:50:51: error: missing argument label 'file:' in call
let promise = worker.eventLoop.newPromise([Element.Expectation].self)

which comes from the package(url: "https://github.com/vapor/core.git", from: "3.0.0"). I checked the vapor core and it has a lot of new versions. So it the package setup still valid or what I am doing wrong? I use Xcode 9.2.

Ubuntu 18.04, websocket-kit 2.1.1

Swift for Tensorflow 0.10

$ which swift
home/xander/swift-tensorflow-RELEASE-0.10-cuda10.2-cudnn7-ubuntu18.04/usr/bin/swift
$ swift --version
Swift version 5.3-dev (LLVM 55d27a5828, Swift 6a5d84ec08)
Target: x86_64-unknown-linux-gnu

I have been successfully using the onText callback to parse the messages my web socket client receives. However, when I replace this with onBinary, I never receive anything. It's never called. I expected onBinary would be called with the same data simply as a ByteBuffer rather than first decoding it to a String. What is the expected usage of onBinary? If this is not a bug, in what cases is onBinary expected to be called?

I would prefer to access the data via the original ByteBuffer rather than as a converted String because I will have to convert a String back to bytes/Data to do my decoding.

I'm curious if compression support has been considered in this package. I'd be interested in using it in my server-side app.

Kitura developed this package for websocket compression which is also using NIO's websockets. Maybe it could be adapted to work with Vapor? https://github.com/IBM-Swift/Kitura-WebSocket-Compression

Once upgrade is complete, there is no socket error handling being performed. channel.closeFuture is not sufficient. As a simple test: disable your interface when you have an active websocket connection and try to handle this event in consumer code.

Besides that I would also in general expose channel to give consumers a chance to modify the pipeline to their needs, no reason to be super restrictive.

Describe the bug

(Apologies in advance: I'm not sure which layer is potentially failing, or if I'm failing to interact with a layer properly. There is a good chance that this isn't a bug with websocket-kit)

WebSocket.connect does not appear to notify the user if the server decides not to upgrade the request. Specifically, .whenFailure is never triggered on the resulting EventLoopFuture.

To Reproduce

I'm using a vapor server, and my webSocket route defines a shouldUpgrade function that returns nil (under certain conditions):

shouldUpgrade: { req in
    ...
    req.eventLoop.makeSucceededFuture(nil)
}

I believe that returning nil is the correct use of the vapor API, but I also tried returning a failed future without any success:

req.eventLoop.makeFailedFuture(Abort(.forbidden))

On the client side, no code is triggered to alert the client that the upgrade failed

let result = WebSocket.connect(...)
result.whenFailure { error in
    // not triggered
}

Expected behavior

I would expect .whenFailure to be executed. Or, I would expect the API to support a closure for when the upgrade was rejected.

Environment

• Vapor websocket-kit: 2.1.2
• Vapor Framework version: 4.35.0
• Vapor Toolbox version: 18.0.0
• OS version: Mac OS 11.0.1 (20B50)

Currently send always sends unmaked frames:

send(WebSocketFrame(fin: true, opcode: opcode, data: buffer), promise: promise)

but as of RTC-6455, section 5.3 it must be masked when acting as a client

Mask: 1 bit

 Defines whether the "Payload data" is masked.  If set to 1, a
 masking key is present in masking-key, and this is used to unmask
 the "Payload data" as per Section 5.3.  All frames sent from
 client to server have this bit set to 1.

The way we test Vapor as on onward package doesn't work when the PR is from a fork because it tries to check out a revision that doesn't exist. The main Vapor package checks out both the package and forward provider and uses them both so we should copy that

websocket-kit is currently pegged to swift-nio-ssl 2.0.0. Could you please update this to 2.10.2, which includes a fix that allows swift-nio-ssl to build with Swift For TensorFlow and latest Swift trunk. See apple/swift-nio-ssl#259.

Currently websocket-kit fails to build on Swift For TensorFlow and latest Swift trunk as a result of its swift-nio-ssl dependency version.

Thanks!

Currently when building the package with the latest AHC we get build warnings as we're pointing at deprecated code. We should update our code to point at the new functions and remove the build warnings

How to change maxFrameSize to bigger value than default one (16384 bytes)?
Due to private modification can't set my own, is the way to solve this?

When closing WS session initiated with wscat, ws.onClose.always { } is fired instantly.

But when using WebSocket() in browsers and calling webSocket.close(), ws.onClose is not getting fired until Browser's tab is refreshed or closed even though server receives 'close' packet. Tested on Safari, Chrome and Firefox - all behave the same.

Further debugging shown that it's getting stuck in private func receivedClose:

            // This is an unsolicited close. We're going to send a response frame and
            // then, when we've sent it, close up shop. We should send back the close code the remote
            // peer sent us, unless they didn't send one at all.
            let closeFrame = WebSocketFrame(fin: true, opcode: .connectionClose, data: data)
            _ = ctx.write(wrapOutboundOut(closeFrame)).always { // <---------- HERE
                _ = ctx.close(promise: nil)
            }

ctx.write does not return.

Steps to reproduce:
1.

git clone https://github.com/zmeyc/ws-close-bug
cd ws-close-bug
swift build
swift run
OR
swift package generate-xcodeproj

2. Put breakpoint in ws.onClose { }.

3. Run ./test_wscat.sh, hit CTRL-C. ws.onClose { } will fire instantly.

Open test.html (in the repo above)
Press "Connect button" two times.
On second press previous connection will be closed, but ws.onClose won't be fired until the page is refreshed.

The fix #89 for issue #86 was merged before a test was written to expose the issue. So, I think this repo would benefit from a test so that the issue doesn't reemerge in the future. Specifically, there should be a close test (similar to testServerClose) with a TLS enabled server using websocket-kit which communicates with a websocket-kit client.