Hello,

I am trying to figure out from the documentation if its possible with one server instance on a port to handle both normal http requests and websocket requests.

Can you please point me out a sample (if any) , documentation or maybe a pseudocode on how to achieve this?

Thank you for your time

I want to use something like socket.io or sock.js or signalr in project but I'm newbie in clojure and jvm. As I know there is no implementations of socket.io server for http-kit or aleph. Also I know that Atmosphere has integration with play and vertx frameworks but not with any clojure frameworks. So is it possible to ceate something like ws fallback to long polling and see some example code for js client and http-kit server side?

I'm getting the following when I connect a few clients to an http-kit server via WebSockets and start throwing data at it. (I can be more specific if necessary.) This error seems to cause all WebSockets connections to be dropped and disallows future connections to the process.

Fri Aug 16 14:51:09 PDT 2013 [server-loop] ERROR - http server loop error, should not happen
java.nio.BufferUnderflowException
    at java.nio.Buffer.nextGetIndex(Buffer.java:478)
    at java.nio.DirectByteBuffer.getInt(DirectByteBuffer.java:665)
    at org.httpkit.server.WSDecoder.decode(WSDecoder.java:86)
    at org.httpkit.server.HttpServer.decodeWs(HttpServer.java:109)
    at org.httpkit.server.HttpServer.doRead(HttpServer.java:143)
    at org.httpkit.server.HttpServer.run(HttpServer.java:245)
    at java.lang.Thread.run(Thread.java:680)

I'm using http-kit 2.1.8 on Clojure 1.5.1.

What's the best way to start troubleshooting the issue?

I have this as my main method:

(defn -main [& args]
  (org.httpkit.server/run-server (reload/wrap-reload (compojure.handler/site #'rag-handler)) {:port 8080}))

But since run-server returns immediately that doesn't keep the server open. How can I fix that?

Is there a way currently to specify a WebSocket subprotocol during the connection handshake? I would like to define a list of subprotocols that can be sent back in a Sec-WebSocket-Protocol header:

http://tools.ietf.org/html/rfc6455#section-1.3

Additional header fields are used to select options in the WebSocket
Protocol.  Typical options available in this version are the
subprotocol selector (|Sec-WebSocket-Protocol|), list of extensions
support by the client (|Sec-WebSocket-Extensions|), |Origin| header
field, etc.  The |Sec-WebSocket-Protocol| request-header field can be
used to indicate what subprotocols (application-level protocols
layered over the WebSocket Protocol) are acceptable to the client.
The server selects one or none of the acceptable protocols and echoes
that value in its handshake to indicate that it has selected that
protocol.

Sec-WebSocket-Protocol: chat

Also to note that might be of interest: I've created a HTTP Kit websocket handler for the WAMP subprotocol at http://cljwamp.us and I've been really been enjoying using HTTP Kit - thanks for sharing it!

I'm using org.httpkit.client to access rabbitmq-management HTTP API.
Default vhost name in RabbitMQ is '/', which means that some of the URLs look like this: http://localhost:15672/api/exchanges/%2F/test.
org.httpkit.client converts it to http://localhost:15672/api/exchanges///test.
It happens because of this line: HttpUtils.java#L219.
As far as i understand, the problem isn't in the httpkit, the problem is in java.net.URI:

user> (.getPath (java.net.URI. "http://localhost:15672/api/exchanges/%2F/test"))
"/api/exchanges///test"

I don't really know how to solve this problem (besides forking and replacing line 219 of HttpUtils.java with something like this: String path = uri.getRawPath();)

Do you have any idea how i can make it work?

Thanks,

Hello there,

This is a question that may have a trivial resolution, as I am pretty new to Clojure. I have an app running on ring-jetty that I am switching to http-kit to handle websockets.

Before I run the server, I open a levelDB connection:

(with-open [main-db (db/open "db/main")]
    (println "Running server on port 8080")
    (run-jetty (web/handler-with-db main-db) { :port 8080 }))

Now, by switching to http-kit, this code breaks:

(with-open [main-db (db/open "db/main")]
    (println "Running server on port 8080")
    (run-server (web/handler-with-db main-db) {:port 8080}))

The reason is that with-open closes the file/db descriptor once the block finishes, run-jetty would hold until the server is closed and so the db would remain open as requests come in. With http-kit's run-server, it seems that a thread is started and the blocks immediately finishes, effectively closing the database when the server was just started.

Any idea on a workaround to this?

Add client/server SPDY support for TLS and clear (for backend RPC).

@(http/get "http://www3.westfalia.de/")

=>

Exception in thread "main" java.io.IOException: Corrupt GZIP trailer
    at java.util.zip.GZIPInputStream.readTrailer(GZIPInputStream.java:200)
    at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:92)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)

Under heavy load, I am seeing what I believe to be channels being re-used. I am getting an IllegalStateException from this line of http-kit code when I try to set a close handler on what I think should be a new channel.

Here are the details:

I am using http-kit with a REST API I am building to return large amounts of data in JSON, CSV, and XML. Specifically, I am using it so that I can stream these responses, as they may be many megabytes and can take time to assemble on the server.

You can see the code I am using here: https://github.com/cndreisbach/qu/blob/streaming/src/cfpb/qu/views.clj#L368

In this code, I am:

1. Creating a PipedInputStream and PipedOutputStream and connecting them.
2. In a separate thread, calling generate-stream from Cheshire to generate a stream of JSON and send it into the PipedOutputStream.
3. In the original thread, creating a channel from the request.
4. I read from the PipedInputStream. As long as I can keep reading from this stream, I send chunks over my channel. As soon as I can't, I close the channel.

Running with normal load, this works great! As soon as I put JMeter on it for some load testing, it goes bananas. I am getting this IllegalStateException from trying to set the close handler. Even if I stop setting the close handler, I am getting a different error (org.apache.http.MalformedChunkCodingException: Bad chunk header from JMeter, not from http-kit, but I think this is related.)

Has anybody seen this type of behavior before? Am I crazy for using multiple threads in my response and using PipedInput/OutputReader for managing streaming data?

Hi,
I am seeing this exception periodically when the server is under load. I believe it might happen when the handler is slow (maybe a queueing problem?).

Tue Jul 23 11:42:37 MDT 2013 [server-loop] ERROR - http server loop error, should not happen
java.nio.channels.CancelledKeyException
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
    at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)
    at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:105)
    at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:74)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)   
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
    at org.httpkit.server.HttpServer.run(HttpServer.java:232)
    at java.lang.Thread.run(Thread.java:680)

I am working on a light Server Sent Events API on top of the channel API.

To close a connection from server side and inform the client not to try to reconnect a 204 response code must be sent.
As http-kit will strip :status after the first response I am not sure if this can be done.

Any other option?

I unfortunately don't have any details yet on what the requests looked like, but wanted to start a dialog in the meantime...

Using [http-kit "2.1.5"], I'm seeing several of these exceptions in my logs, some that happen just after the app restarts:

Exception in thread "server-loop" java.lang.OutOfMemoryError: Java heap space
        at org.httpkit.server.HttpDecoder.readHeaders(HttpDecoder.java:173)
        at org.httpkit.server.HttpDecoder.decode(HttpDecoder.java:84)
        at org.httpkit.server.HttpServer.decodeHttp(HttpServer.java:80)
        at org.httpkit.server.HttpServer.doRead(HttpServer.java:141)
        at org.httpkit.server.HttpServer.run(HttpServer.java:245)
        at java.lang.Thread.run(Thread.java:722)

My suspicions are that this happened during a websocket connection, but I'll try to add more debugging to get a better idea of what's going on. I'll also try upgrading to 2.1.7.

Thanks!

G'day. Following my best guess at the API for client HTTPS requests - which you explicitly identify as modelled on clj-http - I tried using :insecure? true in the options to allow communication with an HTTPS server using a self-signed certificate.

This didn't function; without that at all, and with it present, I got:

{:opts {:url "https://localhost:8140/census/certificate_statuses/mechanical-noise", :method :get, :insecure? true}, :error #}

Looking into the code, it doesn't look like there is actually support in the code for "insecure" communication of this sort. For now I have stepped back to clj-http, which works correctly.

It would be nice to be able to use one library for both WebSocket servers and clients. It looks like currently only creating WebSocket servers is supported by http-kit.

Is there some way to turn gzip compression on in the lower-level Java stack (i.e., Netty)? Or is this something that should be handled by a reverse-proxy?

It is common to streamparse json in the body of requests.

I use cheshire to acomplish this. It works with the jetty adapter but when I attempt with httpkit i get the acception above.

It would be great if your org.httpkit.BytesInputStream could fill the reader interface

Server throws exception when serving an input-stream to a file that is bigger than the heap.

java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2798)
at me.shenfeng.http.DynamicBytes.expandIfNessarry(DynamicBytes.java:20)
at me.shenfeng.http.DynamicBytes.append(DynamicBytes.java:44)
at me.shenfeng.http.HttpUtils.readAll(HttpUtils.java:228)
at me.shenfeng.http.server.ClojureRing.encode(ClojureRing.java:80)
at me.shenfeng.http.server.RingHandler$1.run(RingHandler.java:65)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)

Hi,

I've search everywhere but couldn't find a hint about how to get a binary file, i.e. an array of bytes instead of a body.

When doing it the normal way, the body's size isn't correct; it's approximatively twice as big as it should. Probably due to the encoding.

My code at the moment:

(let [content-in-bytes (. (:body @p) getBytes)]
  (. file-channel write
    (ByteBuffer/wrap content-in-bytes)
    (:start-offset promises-hash)))

Can someone suggest how I should deal with this?

Thanks!

Is there any limit on the websocket message size?

Here is a sample request that fails:
https://www.dropbox.com/s/mu70eecfexgiemr/fail.txt

On the site: http://http-kit.org/client.html
In the example with: synchronous with @promise
It says: :form-parmas
I'm pretty sure it should say: :form-params

Wow, the formatting in Githubs issue tracking is absolutely crap, I can't figure out how to get rid of it. Hope it makes sense anyway. If you can, look at the source

When doing a standard request against google, i get the following problem:

HTTPS:

=>(require '[org.httpkit.client :as http])
nil
    => @(http/get "https://google.com")
    {:opts {:url "https://google.com", :method :get}, :error #SSLHandshakeException javax.net.ssl.SSLHandshakeException: General SSLEngine problem}
=> (.printStackTrace (:error *1))
javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1362)
    at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513)
    at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1177)
    at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1149)
    at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
    at org.httpkit.client.HttpsRequest.doHandshake(HttpsRequest.java:112)
    at org.httpkit.client.HttpClient.doRead(HttpClient.java:130)
    at org.httpkit.client.HttpClient.run(HttpClient.java:367)
    at java.lang.Thread.run(Thread.java:722)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1683)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:808)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:806)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1299)
    at org.httpkit.client.HttpsRequest.doHandshake(HttpsRequest.java:89)
    ... 3 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
    at sun.security.validator.Validator.validate(Validator.java:260)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328)
    ... 10 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
    ... 16 more


HTTP request gives a denial response from the proxy, the same way a curl does which http_proxy unset:

=> @(http/get "http://google.com")
{:opts {:url "http://google.com", :method :get}, :body "\n<TITLE>Access Denied</TITLE>\n\n\n\n
\n\n
\n\n\n\n\n\n
\n\nAccess Denied (authentication_failed)\n
\n
\n\n
\n\nYour credentials could not be authenticated: \"Credentials are missing.\". You will not be permitted access until your credentials can be verified.\n\n
\n\nThis is typically caused by an incorrect username and/or password, but could also be caused by network problems.\n\n
\n\n
\nFor assistance, contact your network support team.

Your request was categorized by Blue Coat Web Filter as 'Search Engines/Portals'. 
If you wish to question or dispute this result, please click here.\n\n
\n
\n\n\n", :headers {:set-cookie "BCSI-CS-46f392e86967d19b=2; Path=/", :proxy-authenticate "BASIC realm=\"ACCDOM01_iwa\"", :pragma "no-cache", :content-type "text/html; charset=utf-8", :content-length "1069", :connection "close", :cache-control "no-cache"}, :status 407}

For comparison, slurp works fine with both of them:

=> (slurp "https://google.com")
"<title>Google</title><script>(function(){\nwindow.google={kEI:\"3rD7Ueb8OcSXtQba-oCYAg\",getEI:function(a){for(var b;a....

=> (slurp "http://google.com")
"<title>Google</title><script>(function(){\nwindow.google={kEI:\"BLH7UcyuJ4n6sga-rYGwCw\",getEI:function(a){for(var b;a&&(!a.getAttribute||!(b=a.getAttribute(\"eid\")));)a=a.parentNode;return b||google.kEI},https:function(){return\"https:\"==window.location.protocol},kEXPI:\"17259,4000116,4002855,4003242,400388....

This is using https.proxyHost and http.proxyHost to configure the proxy. Is there any way to configure the proxy per request that I am missing?

Thanks for http-kit, so far it's been really useful.

I'm trying to use http-kit in conjunction with Carmine to do Pub/Sub over Websockets.

The problem is, I think the code that sets up my Redis subscription isn't being fired properly from with-channel in http-kit. Basically what I do is set up an http-kit endpoint that establishes pub/sub events based on the request. Here is a simplified version:

(ns app.redis
  (:use org.httpkit.server)
  (:require [taoensso.carmine :as car]))

(def redis-pool (car/make-conn-pool))
(def redis-server (car/make-conn-spec))

(defn async-handler [request] 
  (let [sub-channel "channel*"
        pub-channel "channel-foo"]
    (with-channel request connection
      (car/with-new-pubsub-listener redis-server 
        {"channel*" (fn [msg]
                       (println "received" msg ", sending to" connection)
                       (send! connection msg))}
        (car/psubscribe "channel*"))

      (on-receive connection 
        (fn [data]
          (car/with-conn redis-pool redis-server
            (do (println "publishing" data "to" pub-channel)
             (car/publish pub-channel data))))))))

Now, I know the Carmine subscription handler works -- in fact, if I start another subscriber on the same channel in a REPL, that does receive all events that I'm firing in http-kit's on-receive handler. I think the problem is that the subscriber that I set up at the beginning of the with-channel call is never actually set up. (I do think the with-new-pubsub-listener call happens in another thread.)

Am I doing something obviously wrong? Is there a better way to intermingle imperative calls with the on-receive event listener setup?

Thanks.

Hi
I am working on a fire and forget API, where I can just give a http request and forget about it. I was hoping to use http-kit for it. I wanted to understand what happens if I do not deref a request.

Following happens

(dothings)                       ; works
(def a (dothingslater))       ; nothing happens
(count a)                         ; a few calls work
(dothings)                        ; hangs. on server restarts shows connect exception.

I understand that hanging happens because my server threads are blocked. But I still would like to know what happens to those requests.

Code

(require '[org.httpkit.client :as hk-client])
(defn dothingslater []
  (map #(hk-client/get (str "http://localhost/v1/pubapis/auth/ping?ping=" %))
       (take 100 (range))))

(defn dothings []
  (->>
   (map #(hk-client/get (str "http://localhost/v1/pubapis/auth/ping?ping=" %))
        (take 100 (range)))
   (map deref)))

Thanks
Anand

(ns sis.core
  (:import [java.lang.management ManagementFactory])
  (:require [clojure.edn :as edn])
  (:use [compojure.route :only [files not-found]]
      [compojure.handler :only [site]] ; form, query params decode; cookie; session, etc
      [compojure.core :only [defroutes GET POST DELETE ANY context]])
  (:require [org.httpkit.client :as client]
            [org.httpkit.server :as server] ))

(declare close-server-fns)

(defroutes close-server-app 
      (GET  "/cs/shutdown" [] 
            (fn [req]  (future (Thread/sleep 2000) (for [f close-server-fns] (f))) "shutdown ok.")
            ))

(defroutes app 
      (GET  "/" []  "hello"))

(defn startup []
  (let  [conf {:port 8080}, control-conf {:port 8081}]
    (def close-server-fns [(server/run-server app conf)
                           (server/run-server close-server-app  control-conf)
                           ])))

=> (startup)
#'sis.core/close-server-fns

In my browser, go to "http://localhost:8080", I can see "hello" message. But
after goting to "http://localhost:8081/cs/shutdown" , the servers doesn't shutdown.
I also try this method which ends with exception or no effect:

(defroutes close-server-app 
      (GET  "/cs/shutdown" [] 
            (fn [req]   (for [f close-server-fns] (f)) "shutdown ok.")
            ))

Hello,
A response header with multiple values (say, several Set-Cookie values) is reported by http-kit client as being unique (it appears to only keep the last one available).

Compare for example the output of:
curl www.google.com -v > /dev/null

=> two cookies (PREF and NID) are retrieved

with

(require '[org.httpkit.client :as http-kit])
(let [resp (http-kit/get "http://www.google.com")](println %28get-in @resp [:headers :set-cookie]%29))

=> Only one cookie (NID) found in :set-cookie

Hello,

I was wondering if there is an on-error event which can be used on server side to capture any websockets error and send a message back to the client.

If not, will a try-catch be ok?

It seems that http-kit client does not support HTTPS protocol.

A lot of the sites now automatically redirect to a HTTPS address.

Is support forthcoming, or else is there anything I can do to help?

Hey there,

I'm accessing a the facebook api from within my compojure app using the http-kit client and server. At the moment, I have the following function that I use to abstract the facebook requests

(defn api-request
  "Perform a get request on the Facebook graph api"
  [route method params]
  (let [req-func (case method
                       :get http/get
                       :post http/post
                       :delete http/delete)
        uri (str api-base route)
        opts (merge {:method method} {:query-params params})
        {:keys [status headers body error]} @(req-func uri opts)
        response (parse-string body true)]
    (if error ; HTTP error
      (throw (Exception. error))
      (if (:error response) ; Facebook error
        (throw+ {:message (-> response :error :message)
                 :type (-> response :error :type)
                 :code (-> response :error :code)
                 :subcode (-> response :error :error_subcode)})
        response))))

(defn me
  "Get the 'me' object from facebook"
  [access-token]
  (api-request "/me" :get {:access_token access-token}))

and i'm using it like so in a test route (that takes a facebook token as a parameter

(defn me-route
  [request]
  "Facebook api test route"
  (with-channel request channel
    (let [resp (fb/me (-> request :params :token))]
      (send! channel (response (json/generate-string resp)) true))))

The problem I'm having is that it only works for about 8 concurrent requests at a time on my dev machine. I get the feeling that I'm going about the whole async thing the wrong way. Could anyone describe what I'm doing wrong and how I can improve?

Cheers,
Tom

Hi guys, I would really like to use http-kit.... but i am stuck!. How do i make "lein ring server" work with anything other than jetty? .... Let me put it differently, How do i develop with http-kit while still supporting hot code reload ( i.e without server restart after every edit) ?

I couldnt find a mailing list... so i posted it here.

Kind regards.
Josh.

hey, i'm trying to do some basic performance testing on OSX 10.6.8, Java 1.6.0_45. here's the basic program.

(ns hkt.core
  (:require [org.httpkit.server :refer [run-server with-channel send!]]))

(def response {:status 200
               :body "ok"})

(defn async-handler [req]
  (with-channel req ch
    (future
      (send! ch response))))

(defn -main []
  (run-server
    async-handler
    {:port 8080}))

And then pointing ab at it:

ab -n 10000 -c 10 http://localhost:8080/

I haven't altered any of the max files settings, just running with stock config. Initially this gets through the first run at about 10000 r/s, but on subsequent runs it usually blocks at some point and times out. For example...

➜  ~  ab -n 10000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
apr_poll: The timeout specified has expired (70007)
Total of 6372 requests completed

I'm also seeing this occuring in an application I'm writing (which does much the same but offloads the requests to a Lamina channel for later processing).

I'm not sure where to look to debug this, or if it needs debugging at all. Are open file limits being hit? Or some JVM limits? Is GC getting in the way? Like I said, any advice to explain this appreciated. Thanks!

When you use compojure and don't do any sort of with-channel stuff, does that mean that requests are handled synchronously? In other words, we can never have more than num-worker-threads simultaneous requests?

I see the example for compojure. Is the implication that making the body of the function a reference to a function, e.g.,

(defn name-of-fn [req] ...)

(GET "/route/:id" [] 
   name-of-fm)

allows requests to be processed asynchronously?

If you don't do that, e.g.,

(GET "/route/:id" [id] 
   (this id) 
   (that id) 
   "<p>okay</p>")

then the request is processed synchronously?

I expected this to be converted to:

(defn create-stripe-test-token []
  (let [form-params {:card {:number 4242424242424242 :exp_month 12 :exp_year 2015 :cvc 123}}]
    (println (assoc stripe-options :form-params form-params))
    @(http/post "https://api.stripe.com/v1/tokens" (assoc stripe-options :form-params form-params)
          (fn [{:keys [status headers body error]}] (json/parse-string body)))))

curl https://api.stripe.com/v1/tokens \
   -d "card[number]=4242424242424242" \
   -d "card[exp_month]=12" \
   -d "card[exp_year]=2014" \
   -d "card[cvc]=123"

I have a use case where I would like to timeout async requests.

Is there any plan to support the deflate extension?

Sec-WebSocket-Extensions:x-webkit-deflate-frame

i've been using http-kit with compojure (just as a drop in replacement for jetty, no special considerations), but have come across some memory issues when trying to stream large responses (15mb+ email attachments from an imap connection).

jetty handles this fine with the ring response...

{:body (io/input-stream content-stream)}

but http-kit blows the heap (limited to 32mb).

should i look into debugging this further, or is this outside the scope of what http-kit is targeted at?

thanks

Doesn't seems to be possible to stream responses, like for instance server sent event responses.

Should it be possible to send websocket clients messages within the body of when-ws-request?
I tried to do that:

(when-ws-request req ws
  (send-mesg ws (str "hello")))

but the client never receives anything.

I don't think it has to do with any kind of delay on the client; forcing a serverside sleep:

(when-ws-request req ws
  (Thread/sleep 500)
  (send-mesg ws (str "hello")))

still drops the message.

If there's some stuff that needs to run after the body of when-ws-request before messages can be sent, can http-kit queue up messages sent within the body somehow?

I've noticed that when I make a POST request to my local Jetty server and I get a 204 No Content response back, http-kit's client times out.

I suspect the issue is in /src/java/org/httpkit/client/Decoder.java in the readHeaders method. Specifically 204 responses may omit the Content-Length header and always lack a response body so state should be set to ALL_READ, not READ_VARIABLE_LENGTH_CONTENT after reading the headers (similar to HEAD requests).

Hello!

I use Autobahn.Android with http-kit's websockets. When i disconnect socket on client side I have infinity stream of Close frames in both directions.

Autobahn send close frame, http-kit receive close frame, http-send close frame, Autobahn receive close frame, Autobahn send close frame etc. http-kit consume all heap, all cpu, and trying to live in this state.

I see this comment in http-kit.

// even though the logic connection is closed. the socket
// did not, if client willing to reuse it, http-kit is more
// than happy

I'm happy too :) Autobahn can be more smart (like web-browser's native implementations), but by RFC[7.1.1] http-kit SHOULD close tcp connection I think.

Also I can DOS application based on http-kit with one android with one WS connection :)

Thank you!

tests / examples could include PATCH with an updated compojure dependency

• https://github.com/weavejester/compojure/blob/master/HISTORY.md#110-2012-05-24
• https://github.com/weavejester/compojure/blob/596a0daa4f79da695394bc9b44e7e44c06cfb6ca/src/compojure/core.clj#L144

I had a question. I setup two servers on ec2 c1.medium machines in N. Virginia. They use they same compojure web app code, but one serves with http-kit and the other deploys to tomcat7 as a war. Both servers have nginx in front. I ran some simple tests from a box in ec2 Singapore of the same size. They repeatedly hit the same url that made a large db query, did some processing and returned a json response.

Tomcat7 could only handle up to a concurrency value of 1000, while http-kit could handle up to a concurrency value of 1600 using ApacheBench. However Tomcat7 was able to consistently get a higher number of requests/sec. Tomcat7 was getting around 900 requests/sec while http-kit was only getting 450-500 requests/sec. Is this a typical experience? Http-kit can handle more requests but will serve them a little slower?

Thanks

Hi shenfeng,

最近在看这个项目的源码，class的设计看得有些疑问，请教一下。
比如：

public abstract class ServerAtta {
    final LinkedList<ByteBuffer> toWrites = new LinkedList<ByteBuffer>();

    protected AsyncChannel channel;

    public abstract boolean isKeepAlive();
}

ServerAtta抽象类的toWrites和channel是所有Attachment都需要的，可以理解。

public class HttpAtta extends ServerAtta {

    public HttpAtta(int maxBody, int maxLine) {
        decoder = new HttpDecoder(maxBody, maxLine);
    }

    public final HttpDecoder decoder;


    boolean keepalive;

    public boolean isKeepAlive() {
        return keepalive;
    }
}

public class WsAtta extends ServerAtta {

    final public WSDecoder decoder;

    public WsAtta(AsyncChannel channel) {
        this.decoder = new WSDecoder();
        this.channel = channel;
    }

    public boolean isKeepAlive() {
        return true; // always keep-alived, wait other close it
    }
}

我理解decoder虽然Attachment也需要，然是Decode的处理方式和返回类型不同，没有共同接口，所有没有放在抽象基类里面。是这个原因？

还有decoder, channel, toWrites不使用getters/setters方法，是为了效率考虑？

然后关于channel的赋值，HttpAtta 是通过直接访问成员赋值，WsAtta是通过构造函数，是不是不太统一？

最后一个问题，AsyncChannel类的作用是作为一个输出缓冲？使得handle能异步写入并快速结束？

谢谢

After getting a request with Keep-Alive header http-kit doesn't reply with Keep-Alive header in the response, however it neither closes the connection.

I'm new to clojure and can't figure out how to :require and :import all that modules for hello world app.

Simple "copy-pastable" examples would be great :-)

According to this post[1] the Transfer-Encoding header should not be set to chunked. I tried to override in the send! call, but it seems hard-coded:

(server/send!
    channel
    {:headers {"Content-Type" "text/event-stream; charset=utf-8"
                    "Cache-Control" "no-cache"
                    "Transfer-Encoding" "identity"}
     :status 200
     :body (str "data: message from server #" id "\n\n")} false)

Also, documentation is not very clear, IMHO, because it doesn’t say anything about the text/event-stream header. Granted, http-kit are not the place to explain to Server-Sent events[2] but a pointer to some docs could be helpful.

• [1] http://stackoverflow.com/questions/13590755/eventsource-disconnects-after-first-message
• [2] http://dev.w3.org/html5/eventsource/

Hi,

http-kit client currently doesn't offer an option to automatically follow 30x responses.

I have a patch I can offer that intercepts 30x and automatically follows them. I need to add protection against redirect loops and then im ready to offer it if you're interested.

thx

准备给http-kit加上websocket的二进制数据帧支持。初步读了一下代码（protocol-api分支），发现代码其实基本已经完成了。貌似只需要decode时加上对应case，并且在AsyncChannel.send里稍作修改，去掉限制即可。但直觉告诉我事情可能不那么简单，是否什么地方还有坑呢？想在动手前先有些心理准备。

http-kit reads data and decodes HTTP in main thread, but invoke clojure function and encode HTTP response in worker thread.

After encoded HTTP response, worker thread invokes HttpServer.tryWrite, this function is synced per connection, which is right, but I'm wondering what happens when connection is keepalive and request is pipelined.

for example, we have following page:

<body>
  <img src="1.jpg" />
  <img src="2.jpg" />
</body>

and browser connect using keepalive, so there'll be two request in connection, and suppose server is slower handling the first request, which leads to the second worker thread invokes tryWrite first, isn't this wrong?

I can't find any code maintains this order.

I'm still struggling to get http.async.client working with http-kit. To try to narrow the problem down, I've switched to Java and async-http-client (which is what http.async.client is based on). With this test app, compiled against async-http-client 1.7.8:

package com.paulbutcher;

import com.ning.http.client.*;
import com.ning.http.client.websocket.*;

class Client {
  public static void main(String[] args) {
    try {
      AsyncHttpClient c = new AsyncHttpClient();
      Request r = c.prepareGet("http://localhost:8080").build();
      WebSocketUpgradeHandler h = new WebSocketUpgradeHandler.Builder().
        addWebSocketListener(
          new WebSocketTextListener() {
            @Override public void onMessage(String message) {}
            @Override public void onOpen(WebSocket websocket) {}
            @Override public void onClose(WebSocket websocket) {}
            @Override public void onError(Throwable t) {}
            @Override public void onFragment(String fragment, boolean last) {}
        }).build();
      WebSocket websocket = c.executeRequest(r, h).get();
    } catch (Exception e) {
      System.out.println(e);
    }
  }
}

I get:

java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.IllegalStateException: WebSocket is null

I'm not familiar enough with either async-http-client or http-kit to know where the problem lies, but I currently suspect some disagreement between them on the websocket protocol.