Giter Club home page Giter Club logo

flyingfox's Introduction

Build Codecov Platforms Swift 5.10 License Twitter

Introduction

FlyingFox is a lightweight HTTP server built using Swift Concurrency. The server uses non blocking BSD sockets, handling each connection in a concurrent child Task. When a socket is blocked with no data, tasks are suspended using the shared AsyncSocketPool.

Installation

FlyingFox can be installed by using Swift Package Manager.

Note: FlyingFox requires Swift 5.8 on Xcode 14.3+. It runs on iOS 13+, tvOS 13+, macOS 10.15+ and Linux. Windows 10 support is experimental.

To install using Swift Package Manager, add this to the dependencies: section in your Package.swift file:

.package(url: "https://github.com/swhitty/FlyingFox.git", .upToNextMajor(from: "0.14.0"))

Usage

Start the server by providing a port number:

import FlyingFox

let server = HTTPServer(port: 80)
try await server.start()

The server runs within the the current task. To stop the server, cancel the task terminating all connections immediatley:

let task = Task { try await server.start() }
task.cancel()

Gracefully shutdown the server after all existing requests complete, otherwise forcefully closing after a timeout:

await server.stop(timeout: 3)

Wait until the server is listening and ready for connections:

try await server.waitUntilListening()

Retrieve the current listening address:

await server.listeningAddress

Note: iOS will hangup the listening socket when an app is suspended in the background. Once the app returns to the foreground, HTTPServer.start() detects this, throwing SocketError.disconnected. The server must then be started once more.

Handlers

Handlers can be added to the server by implementing HTTPHandler:

protocol HTTPHandler {
  func handleRequest(_ request: HTTPRequest) async throws -> HTTPResponse
}

Routes can be added to the server delegating requests to a handler:

await server.appendRoute("/hello", to: handler)

They can also be added to closures:

await server.appendRoute("/hello") { request in
  try await Task.sleep(nanoseconds: 1_000_000_000)
  return HTTPResponse(statusCode: .ok)
}

Incoming requests are routed to the handler of the first matching route.

Handlers can throw HTTPUnhandledError if after inspecting the request, they cannot handle it. The next matching route is then used.

Requests that do not match any handled route receive HTTP 404.

FileHTTPHandler

Requests can be routed to static files with FileHTTPHandler:

await server.appendRoute("GET /mock", to: .file(named: "mock.json"))

FileHTTPHandler will return HTTP 404 if the file does not exist.

DirectoryHTTPHandler

Requests can be routed to static files within a directory with DirectoryHTTPHandler:

await server.appendRoute("GET /mock/*", to: .directory(subPath: "Stubs", serverPath: "mock"))
// GET /mock/fish/index.html  ---->  Stubs/fish/index.html

DirectoryHTTPHandler will return HTTP 404 if a file does not exist.

ProxyHTTPHandler

Requests can be proxied via a base URL:

await server.appendRoute("GET *", to: .proxy(via: "https://pie.dev"))
// GET /get?fish=chips  ---->  GET https://pie.dev/get?fish=chips

RedirectHTTPHandler

Requests can be redirected to a URL:

await server.appendRoute("GET /fish/*", to: .redirect(to: "https://pie.dev/get"))
// GET /fish/chips  --->  HTTP 301
//                        Location: https://pie.dev/get

WebSocketHTTPHandler

Requests can be routed to a websocket by providing a WSMessageHandler where a pair of AsyncStream<WSMessage> are exchanged:

await server.appendRoute("GET /socket", to: .webSocket(EchoWSMessageHandler()))

protocol WSMessageHandler {
  func makeMessages(for client: AsyncStream<WSMessage>) async throws -> AsyncStream<WSMessage>
}

enum WSMessage {
  case text(String)
  case data(Data)
}

Raw WebSocket frames can also be provided.

RoutedHTTPHandler

Multiple handlers can be grouped with requests and matched against HTTPRoute using RoutedHTTPHandler.

var routes = RoutedHTTPHandler()
routes.appendRoute("GET /fish/chips", to: .file(named: "chips.json"))
routes.appendRoute("GET /fish/mushy_peas", to: .file(named: "mushy_peas.json"))
await server.appendRoute(for: "GET /fish/*", to: routes)

HTTPUnhandledError is thrown when it's unable to handle the request with any of its registered handlers.

Routes

HTTPRoute is designed to be pattern matched against HTTPRequest, allowing requests to be identified by some or all of its properties.

let route = HTTPRoute("/hello/world")

route ~= HTTPRequest(method: .GET, path: "/hello/world") // true
route ~= HTTPRequest(method: .POST, path: "/hello/world") // true
route ~= HTTPRequest(method: .GET, path: "/hello/") // false

Routes are ExpressibleByStringLiteral allowing literals to be automatically converted to HTTPRoute:

let route: HTTPRoute = "/hello/world"

Routes can include a specific method to match against:

let route = HTTPRoute("GET /hello/world")

route ~= HTTPRequest(method: .GET, path: "/hello/world") // true
route ~= HTTPRequest(method: .POST, path: "/hello/world") // false

They can also use wildcards within the path:

let route = HTTPRoute("GET /hello/*/world")

route ~= HTTPRequest(method: .GET, path: "/hello/fish/world") // true
route ~= HTTPRequest(method: .GET, path: "/hello/dog/world") // true
route ~= HTTPRequest(method: .GET, path: "/hello/fish/sea") // false

Trailing wildcards match all trailing path components:

let route = HTTPRoute("/hello/*")

route ~= HTTPRequest(method: .GET, path: "/hello/fish/world") // true
route ~= HTTPRequest(method: .GET, path: "/hello/dog/world") // true
route ~= HTTPRequest(method: .POST, path: "/hello/fish/deep/blue/sea") // true

Specific query items can be matched:

let route = HTTPRoute("/hello?time=morning")

route ~= HTTPRequest(method: .GET, path: "/hello?time=morning") // true
route ~= HTTPRequest(method: .GET, path: "/hello?count=one&time=morning") // true
route ~= HTTPRequest(method: .GET, path: "/hello") // false
route ~= HTTPRequest(method: .GET, path: "/hello?time=afternoon") // false

Query item values can include wildcards:

let route = HTTPRoute("/hello?time=*")

route ~= HTTPRequest(method: .GET, path: "/hello?time=morning") // true
route ~= HTTPRequest(method: .GET, path: "/hello?time=afternoon") // true
route ~= HTTPRequest(method: .GET, path: "/hello") // false

HTTP headers can be matched:

let route = HTTPRoute("*", headers: [.contentType: "application/json"])

route ~= HTTPRequest(headers: [.contentType: "application/json"]) // true
route ~= HTTPRequest(headers: [.contentType: "application/xml"]) // false

Header values can be wildcards:

let route = HTTPRoute("*", headers: [.authorization: "*"])

route ~= HTTPRequest(headers: [.authorization: "abc"]) // true
route ~= HTTPRequest(headers: [.authorization: "xyz"]) // true
route ~= HTTPRequest(headers: [:]) // false

Body patterns can be created to match the request body data:

public protocol HTTPBodyPattern: Sendable {
  func evaluate(_ body: Data) -> Bool
}

Darwin platforms can pattern match a JSON body with an NSPredicate:

let route = HTTPRoute("POST *", body: .json(where: "food == 'fish'"))
{"side": "chips", "food": "fish"}

WebSockets

HTTPResponse can switch the connection to the WebSocket protocol by provding a WSHandler within the response payload.

protocol WSHandler {
  func makeFrames(for client: AsyncThrowingStream<WSFrame, Error>) async throws -> AsyncStream<WSFrame>
}

WSHandler facilitates the exchange of a pair AsyncStream<WSFrame> containing the raw websocket frames sent over the connection. While powerful, it is more convenient to exchange streams of messages via WebSocketHTTPHandler.

Preview Macro Handler

The branch preview/macro contains an experimental preview implementation where handlers can annotate functions with routes:

@HTTPHandler
struct MyHandler {

  @HTTPRoute("/ping")
  func ping() { }

  @HTTPRoute("/pong")
  func getPong(_ request: HTTPRequest) -> HTTPResponse {
    HTTPResponse(statusCode: .accepted)
  }

  @JSONRoute("POST /account")
  func createAccount(body: AccountRequest) -> AccountResponse {
    AccountResponse(id: UUID(), balance: body.balance)
  }
}

let server = HTTPServer(port: 80, handler: MyHandler())
try await server.start()

The annotations are implemented via SE-0389 Attached Macros available in Swift 5.9 and later.

Read more here.

FlyingSocks

Internally, FlyingFox uses a thin wrapper around standard BSD sockets. The FlyingSocks module provides a cross platform async interface to these sockets;

import FlyingSocks

let socket = try await AsyncSocket.connected(to: .inet(ip4: "192.168.0.100", port: 80))
try await socket.write(Data([0x01, 0x02, 0x03]))
try socket.close()

Socket

Socket wraps a file descriptor and provides a Swift interface to common operations, throwing SocketError instead of returning error codes.

public enum SocketError: LocalizedError {
  case blocked
  case disconnected
  case unsupportedAddress
  case failed(type: String, errno: Int32, message: String)
}

When data is unavailable for a socket and the EWOULDBLOCK errno is returned, then SocketError.blocked is thrown.

AsyncSocket

AsyncSocket simply wraps a Socket and provides an async interface. All async sockets are configured with the flag O_NONBLOCK, catching SocketError.blocked and then suspending the current task using an AsyncSocketPool. When data becomes available the task is resumed and AsyncSocket will retry the operation.

AsyncSocketPool

protocol AsyncSocketPool {
  func prepare() async throws
  func run() async throws

  // Suspends current task until a socket is ready to read and/or write
  func suspendSocket(_ socket: Socket, untilReadyFor events: Socket.Events) async throws
}

SocketPool

SocketPool<Queue> is the default pool used within HTTPServer. It suspends and resume sockets using its generic EventQueue depending on the platform. Abstracting kqueue(2) on Darwin platforms and epoll(7) on Linux, the pool uses kernel events without the need to continuosly poll the waiting file descriptors.

Windows uses a queue backed by a continuous loop of poll(2) / Task.yield() to check all sockets awaiting data at a supplied interval.

SocketAddress

The sockaddr cluster of structures are grouped via conformance to SocketAddress

  • sockaddr_in
  • sockaddr_in6
  • sockaddr_un

This allows HTTPServer to be started with any of these configured addresses:

// only listens on localhost 8080
let server = HTTPServer(address: .loopback(port: 8080))

It can also be used with UNIX-domain addresses, allowing private IPC over a socket:

// only listens on Unix socket "Ants"
let server = HTTPServer(address: .unix(path: "Ants"))

You can then netcat to the socket:

% nc -U Ants

Command line app

An example command line app FlyingFoxCLI is available here.

Credits

FlyingFox is primarily the work of Simon Whitty.

(Full list of contributors)

flyingfox's People

Contributors

andrejacobs avatar huwr avatar sergiocampama avatar stackotter avatar swhitty avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flyingfox's Issues

URLs with spaces do not work

Hello,

for one of my usecases i need url which uses percent encoded spaces, eg: http://host/some%20folder,
currently such url wont get matched and handled, far from ideal but i got around it for now by editing HTTPDecoder and replacing the "%20" with "+", then i get the route matched and handle it accordingly.


let comps = status
            .trimmingCharacters(in: .whitespacesAndNewlines)
            .replacingOccurrences(of: "%20", with: "+")
            .split(separator: " ", maxSplits: 2, omittingEmptySubsequences: false)

any ideas on how to solve it nicely?

thanks for the great work!

Cross-Origin

When it comes to cross-origin requests, I've noticed that the requests don't even reach the server, so I can't set the necessary headers for cross-origin as I usually would. How can I handle this?

private func startListener() async{
    do {
        let headers = [
                       HTTPHeader("Access-Control-Allow-Origin"):"*",
                       HTTPHeader("Access-Control-Allow-Headers"): "*",
                       HTTPHeader("Access-Control-Allow-Methods"): "OPTIONS, GET, POST",
                       HTTPHeader("Access-Control-Allow-Credentials") : "true"
                      ]

          await server.appendRoute("/hook") { request in
          //  Don't receive any message.
          return HTTPResponse(statusCode: .ok,headers: headers)
        }

        try await server.start()

        print("Server is running on port xxxx")

    } catch {
        print("Server start error: \(error)")
    }
}

Error when targeting macOS 12.0

Hi Simon,

I'm running into an issue with FlyingFox when building my project. The problem only arises when I specifically direct Swift to build it for macOS 12.0 (the reason I'm doing that is because I'm installing my tool through mint, which automatically targets the machine's os version). Here's the error:

$ swift build -c release -Xswiftc -target -Xswiftc x86_64-apple-macosx12.0
Building for production...
/Users/stackotter/Desktop/Projects/Swift/Scute/.build/checkouts/FlyingFox/Sources/Handlers/ProxyHTTPHandler.swift:51:42: error: ambiguous use of 'data'
        let (data, response) = try await session.data(for: req)
                                         ^
/Users/stackotter/Desktop/Projects/Swift/Scute/.build/checkouts/FlyingFox/Sources/URLSession+Async.swift:42:10: note: found this candidate
    func data(for request: URLRequest, forceFallback: Bool = false) async throws -> (Data, URLResponse) {
         ^
Foundation.URLSession:3:17: note: found this candidate
    public func data(for request: URLRequest, delegate: URLSessionTaskDelegate? = nil) async throws -> (Data, URLResponse)
                ^
[3/7] Compiling FlyingFox AsyncSequence+Extensions.swift

It seems like you may have to rename your method or change the function signature in some other way so that it doesn't clash with the macOS 12.0 API additions,

Cheers,
stackotter

Does it have TLS/SSL support?

Althought we can use nginx/apache with a SSL certificate I am wondering if there are plans to add it here in the future.

Security issue in FlyingSocks `Socket.write`

Overview

If the data passed to Socket.write is a slice with a non-zero startIndex, memory after the end of the data buffer will be leaked to the recipient.

Cause

The issue is on line 229 of Socket.swift:

let sent = try write(buffer.baseAddress! + index, length: data.endIndex - index)

The code assumes that buffer.baseAddress! + index correctly gets the byte at index in the data, however baseAddress points to the byte at startIndex not at index 0. For example, consider the following code:

let data = "abcd".data(using: .utf8)!
let slice = data[2...] // contains "cd"

let index = slice.startIndex
try slice.withUnsafeBytes { buffer in               
    _ = try write(buffer.baseAddress! + index, length: data.endIndex - index)
    //                   (1)            (2)            (3)
    //
    // 1. baseAddress points to "c"
    // 2. after adding startIndex (which is 2), the pointer points to the byte after the end of the buffer
    // 3. length is 2 (4 - 2)
    //
    // In this example scenario, the server accidentally sends two bytes of
    // the memory after the end of the data buffer to the client, which could
    // lead to sensitive data being leaked in certain setups. It could also potentially
    // be combined with certain other types of vulnerabilities to execute arbitrary code.
}

Proof of concept

First, run the following command to start a tcp listener that simply prints out any data it receives.

nc -l 8080

Next, run this code snippet with swift run for the highest chance of reproducing, because that's how I ran it:

@main
struct FlyingFoxDataTest {
    static func main() async throws {
        // Generate some dummy data and make a slice
        let data = String(repeating: "abcd", count: 32).data(using: .utf8)!
        let slice = data[64...]
        
        // This length of string seems to work consistently on my laptop
        let secretPassword = "thisismyverylongandsecuresecretpasswordthisismyverylongandsecuresecretpasswordthisismyverylongandsecuresecretpassworditissogood!".data(using: .utf8)!
        
        // Attempt to send the slice through the socket
        let socket = try await AsyncSocket.connected(to: .inet(ip4: "127.0.0.1", port: 8080), pool: .polling)
        try await socket.write(slice)
    }
}

When I run that snippet, I get the following output:

$ nc -l 8080
thisismyverylongandsecuresecretpasswordthisismyverylongandsecure

As you can see, it sent the first half of secretPassword instead of the contents of the data slice. This bug could have pretty bad side-effects if it appeared in any unfortunate situations.

Mitigation

Make the following change:

- let sent = try write(buffer.baseAddress! + index, length: data.endIndex - index)
+ let sent = try write(buffer.baseAddress! + index - data.startIndex, length: data.endIndex - index)

I'll make a PR to fix this soon.

And I know this full on bug report might be a bit overkill, but I had fun getting that proof of concept to work, so I did it anyway.

Crash on Socket.close()

First, thank you for developing this fantastic lightweight server!

We've encountered a recurring crash, primarily when the iOS app is running in the background. The crash log is as follows:

Crashed: com.apple.root.user-initiated-qos.cooperative
0  libsystem_kernel.dylib         0x1e64 close + 8
1  App                          0x71c300 Socket.close() + 4384424704
2  App                          0x6f7eb8 HTTPServer.start() + 4384276152
3  libswift_Concurrency.dylib     0x41948 swift::runJobInEstablishedExecutorContext(swift::Job*) + 416
4  libswift_Concurrency.dylib     0x42868 swift_job_runImpl(swift::Job*, swift::ExecutorRef) + 72
5  libdispatch.dylib              0x15944 _dispatch_root_queue_drain + 396
6  libdispatch.dylib              0x16158 _dispatch_worker_thread2 + 164
7  libsystem_pthread.dylib        0xda0 _pthread_wqthread + 228
8  libsystem_pthread.dylib        0xb7c start_wqthread + 8

To provide further context, here are relevant code snippets related to how we start and stop the server:

 func start() {
        Task {
            do {
                if await !server.isListening {
                    try await server.start()
                }
            } catch (_ as CancellationError) {
                print("Task cancelled")
            } catch {
                logger.error("Error local server: \(error.localizedDescription)")
            }
        }
    }

    func stop() {
        let task = Task {
            await server.stop()
        }
        task.cancel()
    }

Thanks!

Handy JSON response

JSON responses are common for a web server. With this extension (serializing dates as millis by default to ease JS clients):

extension HTTPResponse {
    static let jsonTimestampMillisEncoder: JSONEncoder = {
        let jsonEncoder = JSONEncoder()
        jsonEncoder.dateEncodingStrategy = .millisecondsSince1970

        return jsonEncoder
    }()

    static func json(_ encodable: Encodable, jsonEncoder: JSONEncoder = jsonTimestampMillisEncoder) throws -> HTTPResponse {
        HTTPResponse(statusCode: .ok, body: try jsonEncoder.encode(encodable))
    }
}

one can easily return a JSON ready to be consumed by another application:

struct Cat: Codable {
    let birthDate: Date
    let name: String
}

struct MyHandler : HTTPHandler {
    func handleRequest(_ request: HTTPRequest) async throws -> HTTPResponse {
        try .json(Cat(birthDate: Date(), name: "Loris"))
    }
}

If you think it's a valuable addition, feel free to add it in the library ๐Ÿป

Readme improvement

I was migrating an HTTP Server to FlyingFox and I encountered a client side error "Socket write failed". I spent some time to discover this line:

if !request.shouldKeepAlive {

The client app didn't send the "keepalive" header in requests. I think it would be great to mention "keepalive" in readme.

Thank you for this great lightweight server :)

Multiple Handlers

When multiple endpoints are defined, one ends up with:

await server.appendRoute(HTTPRoute(.GET, "/cat"), to: MyCatGetHandler())
await server.appendRoute(HTTPRoute(.GET, "/dog"), to: MyDogGetHandler())
await server.appendRoute(HTTPRoute(.GET, "/fish"), to: MyFishGetHandler())

and then implementations contains only the handler:

struct MyCatGetHandler : HTTPHandler {
    func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse { }
}

struct MyDogGetHandler : HTTPHandler {
    func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse { }
}

struct MyFishGetHandler : HTTPHandler {
    func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse { }
}

Maybe it's my personal taste, but I find it better to have both the route and the handler in the same file, so by adding those extensions:

protocol RouteHandler : HTTPHandler {
    func route() -> HTTPRoute
}

extension HTTPServer {
    func appendRoutes(_ routeHandlers: RouteHandler...) {
        routeHandlers.forEach {
            appendRoute($0.route(), to: $0)
        }
    }
}

I'm able to write each handler like this (which I find convenient, specially when the route has placeholders, so I can write a single common function and define shared constants between the route and the request handler):

struct MyCatGetHandler : RouteHandler {

    func route() -> HTTPRoute {
        HTTPRoute(method: .GET, path: "/cat")
    }

    func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse {
        // implementation
    }
}

and also be able to add them all like this:

await server.appendRoutes(
    MyCatGetHandler(),
    MyDogGetHandler(),
    MyFishGetHandler()
)

If you think it's a valuable addition, feel free to add it in the library ๐Ÿป

This is my third proposal so far (#50 #49), and the fact I was able to write everything I needed as an extension denotes really good design choices on your side, so kudos for the great work! ๐Ÿ’ฏ

starting server port

I'm using this in a UI test environment. It seems like there's a silent issue at times after starting a server for UI testing where continued execution ceases at the log "starting server port". I have tried:

  1. Starting a server once in a task as documented in readme and stopping it once with the test bundle for the entire run. Issues launching the test app persist using this method.

  2. Starting a server directly using the async setup methods for a given test case. In this case it hangs up on execution.

  3. Starting a server in a task alongside launching the app in a setup method. This method works when I launch the app first, although not always.

Is anyone else using FF as a dependency of UI testing and have found a happiest path for using in such an environment?

Abilities to insert or delete the handlers in RoutedHTTPHandler

We currently only support for

public mutating func appendRoute(_ route: HTTPRoute, to handler: HTTPHandler) {
        handlers.append((route, handler))
    }
    public mutating func appendRoute(_ route: HTTPRoute,
                                     handler: @Sendable @escaping (HTTPRequest) async throws -> HTTPResponse) {
        handlers.append((route, ClosureHTTPHandler(handler)))
    }

Can we also have the function that inserts at 0 or remove all handlers.
The scenario we encounter is that we use it on the UI test, while we need to mock a response change, we will need to change the handlers array to make it happen, if append is the only function, we will need to rebuild a server again to make the change.

File Uploads

Is FlyingFox able to handle large file uploads?

Directory Handler for static files

First of all, thanks for this really good library!

With this setup:

let server = HTTPServer(port: 8080)

let root = URL(fileURLWithPath: "/Users/myUser/myStaticDir")

let route = HTTPRoute(method: .GET, path: "/static/*")
let handler = DirectoryHTTPHandler(root: root, serverPath: "static")

await server.appendRoute(route, to: handler)

try await server.start()

When I perform:

curl -L -vv http://localhost:8080/static/index.html

I'm always getting a 404.

That's because:

guard request.path.hasPrefix(serverPath) else {

never matches, because request.path will always be /static/index.html, so I find this example to be wrong: https://github.com/swhitty/FlyingFox#directoryhttphandler

To make it work I had to write the handler like this:

let handler = DirectoryHTTPHandler(root: root, serverPath: "/static/")

But then I found out that if I want to also handle the default file to be index.html when not explicitly specifying it:

curl -L -vv http://localhost:8080/static/

or

curl -L -vv http://localhost:8080/static

I have to also add:

let fileRoute = HTTPRoute(method: .GET, path: "/static")
let fileHandler = FileHTTPHandler(path: root.appendingPathComponent("index.html"), contentType: "text/html")
await server.appendRoute(fileRoute, to: fileHandler)

Then I thought this is a very common scenario one would expect to work with minimal work, so I written this extension:

extension HTTPServer {
    func appendStaticRoute(_ route: String, root: URL, defaultFileName: String = "index.html", withContentType contentType: String = "text/html") {
        appendRoute(HTTPRoute(method: .GET, path: "/\(route)/*"), to: DirectoryHTTPHandler(root: root, serverPath: "/\(route)/"))
        appendRoute(HTTPRoute(method: .GET, path: "/\(route)/"), to: FileHTTPHandler(path: root.appendingPathComponent(defaultFileName), contentType: contentType))
        appendRoute(HTTPRoute(method: .GET, path: "/\(route)"), to: .redirect(to: "/\(route)/"))
    }

    func appendStaticRoute(_ route: String, bundle: Bundle = .main, defaultFileName: String = "index.html", withContentType contentType: String = "text/html") throws {
        guard let root = bundle.resourceURL else { throw HTTPUnhandledError() }
        appendStaticRoute(route, root: root, defaultFileName: defaultFileName, withContentType: contentType)
    }
}

Which allows you to achieve the same with only:

let server = HTTPServer(port: 8080)

let root = URL(fileURLWithPath: "/Users/myUser/myStaticDir")

await server.appendStaticRoute("static", root: root)

try await server.start()

And so here I am, posting this. If you think it's a valuable addition, feel free to add it in the library ๐Ÿป

Security vulnerability: Path traversal in `DirectoryHTTPHandler`

What is a path traversal vulnerability?

A path traversal vulnerability occurs when paths aren't properly normalised by the server, allowing attackers to access any file on the server, not just files within the served directory.

Proof of concept

The following code snippet for creating a simple file server is vulnerable to path traversal (and so is any other code that uses DirectoryHTTPHandler).

let directoryHandler = DirectoryHTTPHandler(root: URL(fileURLWithPath: "."), serverPath: "/")
let server = HTTPServer(port: 80)
await server.appendRoute("GET *", to: directoryHandler)
try await server.start()

To observe the effect of this vulnerability, run the code snippet above and then run the following command in terminal to see that the server has exposed the contents of your machine's /etc/passwd file (and all other files for that matter):

curl --path-as-is http://127.0.0.1/../../../../../../../../../../../etc/passwd

Any user that can access the web server can run a similar command on their machine to access any file on the server (within the limits of the privileges that the server is running with).

Mitigation

Option 1 (easy, but isn't as nice as option 2)

Abort any requests to the directory handler that contain ../ in their path. This is how vapor's file server middleware avoids path traversal (the following snippet is taken from vapor source). This only fixes the issue for the built-in directory handler, but the check could be applied to all requests (not just those handled by the directory handler) to fix that.

// protect against relative paths
guard !path.contains("../") else {
    return request.eventLoop.makeFailedFuture(Abort(.forbidden))
}

I don't like this solution because it feels like a bit of a quick fix, but another way of looking is that its simplicity makes it very hard to mess up.

Option 2

Normalize request paths in HTTPDecoder to remove any .. components. Refer to the RFC specification on removing dot components (https://www.rfc-editor.org/rfc/rfc3986#section-5.2.4).

This solution is a bit more involved, but it results in nicer behaviour in my opinion, and I think it's the approach that most http servers take.

Conclusion

Let me know which solution you prefer. If you would like me to implement the fix, I'd be happy to implement either solution. Personally I prefer the behaviour of the second solution but the simplicity of the first, so it's up to you.

Allow the system to choose a port number

I'd like to use this server in some tests. The tests might be executed in parallel, so I'm not able to hard-code specific port numbers.

If you bind a socket address with port number 0, the system will assign a unique port number for you, which is ideal for these situations. Unfortunately, AFAICT, there is no way to get the port number or socket from HTTPServer, so I wouldn't be able to actually make requests to that server instance.

isRunning: Ability to see if the server instance is currently running or not.

Hi! First of all, You've done an amazing job with this library. Thank you very much.

I was wondering if there's a cleaner way to see if HTTPServer is active or not. Current work around seems to be either using a custom wrapper around HTTPServer or use self.listeningAddress != nil.

Let me know what you think

Small leak in makeAddress

Thanks for the great software. A small leak found with leaks:

   public static func makeAddress(from addr: sockaddr_storage) throws -> Address {
       switch Int32(addr.ss_family) {
       case AF_INET:
           var addr_in = try sockaddr_in.make(from: addr)
           let maxLength = socklen_t(INET_ADDRSTRLEN)
           let buffer = UnsafeMutablePointer<CChar>.allocate(capacity: Int(maxLength))
           try Socket.inet_ntop(AF_INET, &addr_in.sin_addr, buffer, maxLength)
           return .ip4(String(cString: buffer), port: UInt16(addr_in.sin_port).byteSwapped)

       case AF_INET6:
           var addr_in6 = try sockaddr_in6.make(from: addr)
           let maxLength = socklen_t(INET6_ADDRSTRLEN)
           let buffer = UnsafeMutablePointer<CChar>.allocate(capacity: Int(maxLength))
           try Socket.inet_ntop(AF_INET6, &addr_in6.sin6_addr, buffer, maxLength)
           return .ip6(String(cString: buffer), port: UInt16(addr_in6.sin6_port).byteSwapped)
...
    }

buffer is leaked. Should be an easy fix.

Does it have HTTP/2 support?

First of all thanks for developing this amazing package!

I wanted to ask if it is possible to use http 2 with this package as I see the server is currently using HTTP/1.1.

ContentType for CSS Files

If I try to use FlyingFox and embed some HTML with CSS in my app and then load the content via a WebKit browser instance, the HTML content does not load correctly because the CSS does not get loaded.

It appears that the makeContentType method in FileHTTPHandler does not return the right type for CSS files. If you add the following case to the method, the HTML content renders correctly:

case "css":
	return "text/css"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.