ring-clojure / ring Goto Github PK

The spec contains this right now:

The HTTP request method, must be one of :get, :head, :options, :put, :post, or :delete.

HTTP 1.1 allows extensions (note "extension-method"). WebDAV has been around since 1999, and it has a lot of them.

The PATCH method is very popular now, it's the primary method for updates in Rails and it's used in the GitHub API.

I see no reason why Ring should only allow the standard HTTP methods.

ring-core depends on [org.clojure/tools.reader "0.7.3"], while clojurescript (starting with -1909 I think) depends on tools.reader 0.7.8. In some configurations the ring-core dependency overrides the other, which produces hard-to-track-down error messages in ClojureScript.

Excluding the ring-core dependency and using 0.7.8 instead doesn't seem to cause problems in my project. If no tests fail I'd like to suggest updating the dependency on tools.reader to 0.7.8 to play fine with ClojureScript again.

On Windows, the resource and static file handling (.js .css files) in the development server is serving files that have not changed relative the the browsers If-Modified-Since date.

Headers:

GET /js/jquery.iframe-transport.js HTTP/1.1
Host: localhost:3000
Connection: keep-alive
Cache-Control: max-age=0
Accept: */*
If-Modified-Since: Thu, 02 May 2013 19:40:39 +0000
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Referer: http://localhost:3000/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

HTTP/1.1 200 OK
Date: Wed, 15 May 2013 15:48:46 GMT
Content-Length: 8902
Last-Modified: Thu, 02 May 2013 19:40:39 +0000
Content-Type: text/javascript
Server: Jetty(7.6.1.v20120215)

Stack trace:

java.lang.NullPointerException
at ring.middleware.multipart_params$request_context$reify__1899.getContentLength(multipart_params.clj:23)
at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.(FileUploadBase.java:936)
at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331)
at ring.middleware.multipart_params$file_item_seq.invoke(multipart_params.clj:38)
at ring.middleware.multipart_params$parse_multipart_params.invoke(multipart_params.clj:54)
at ring.middleware.multipart_params$wrap_multipart_params$fn__1919.invoke(multipart_params.clj:98)

In the 1.2.0 documentation, the doctoring for ring.util.response.set-cookies? says it "Requires the handler to be wrapped in the wrap-cookies middleware" [here: http://ring-clojure.github.io/ring/ring.util.response.html#var-set-cookie ], but there is no namespace for ring.middleware.cookies or ring.middleware.wrap-cookies or anything else that seems relevant.

Not trying to pick nits. I'm new to ring and am genuinely confused about it's cookie-handling.

(V 1.1.0, Clojure 1.3)

Here's a minimal failing test case:

(percent-decode "%24")

or

(percent-decode (percent-encode "$"))

And here's a monkey patch to demonstrate a fix:

(with-redefs [ring.util.codec/double-escape (ƒ [x] (.replace (.replace x "\\" "\\\\") "$" "\\$"))  ] (percent-decode "%24"))

Exception from calling url-decode:

Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 3
at java.lang.String.charAt(String.java:686)
at java.util.regex.Matcher.appendReplacement(Matcher.java:711)
at clojure.string$replace_by.invoke(string.clj:58)
at clojure.string$replace.invoke(string.clj:82)
at ring.util.codec$percent_decode.doInvoke(codec.clj:31)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at ring.util.codec$url_decode.doInvoke(codec.clj:51)
at clojure.lang.RestFn.invoke(RestFn.java:410)

Thanks to Chris Hapgood with whom I worked to figure this out.

Nested JavaScript arrays in AJAX call data to a ring application fail to parse. I think I can reproduce the error as far down as form-decode.

Disclaimer: I'm new to Ring and web programming in general. Please correct me if this ticket should be filed elsewhere.

AJAX call

$( document ).ready( function() {
    $.ajax({
        url: "/query/",
        data: {want: "C", constraints: [["A", "B"]]},
        success: function(data, aStatus, dummy){
            alert(data);
        }
    });
});

Result on Server

{:scheme :http, :request-method :get, 
 :query-string want=C&constraints%5B0%5D%5B%5D=A&constraints%5B0%5D%5B%5D=B, 
 :route-params {}, :content-type nil, :params {}, 
 :body #<HttpInput org.eclipse.jetty.server.HttpInput@1bd06bf>
 ....}

(clojure.pprint/pprint (-> result :query-string form-decode))

{"constraints[0][]" ["A" "B"], "want" "C"}

Desired Result

{"constraints" [["A" "B"]], "want" "C"}

While ring spec says that header names in request from ring adapter are lower-cased it says nothing about headers returned in responses. Actually response headers are usually written Camel-Cased. This is a problem for middlewares that may wish to post-process response headers after a handler.

For example code that reads and writes Last-Modified header should account for header name differences in request and response. Other use case would be a middleware that sets header only if it is not already present - if character cases are not considered header might be set twice.
See https://github.com/PetrGlad/ring/commit/get-header/ for a hack that fixes comparison in not-modified-response. I suspect that similar problems might also be elsewhere.

I am not sure what a better solution would be. My idea is that it would be better to have standard lowercase headers everywhere and prettify response headers with Camel-Case in ring adapter.

clj-stacktrace 2.5.0, the version Ring 1.2.0 uses, has a bug that raises an IndexOutOfBoundsException for exceptions with long messages, so if you use wrap-stacktrace some exceptions will be seen incorrectly. See this:

mmcgrana/clj-stacktrace#20
mmcgrana/clj-stacktrace#20 (comment)

Don't know if this bug is still present in clj-stacktrace 2.6.0.

Still has http://github.com/mmcgrana/ring

why is stacktrace throwing this error on a wrong route, how should it be handled

java.lang.IndexOutOfBoundsException
RT.java:784 clojure.lang.RT.nthFrom
RT.java:753 clojure.lang.RT.nth
utils.clj:19 clj-stacktrace.utils/quartile1
utils.clj:36 clj-stacktrace.utils/fence
repl.clj:95 clj-stacktrace.repl/find-source-width
repl.clj:107 clj-stacktrace.repl/pst-on
repl.clj:123 clj-stacktrace.repl/pst-str
RestFn.java:408 clojure.lang.RestFn.invoke
stacktrace.clj:17 ring.middleware.stacktrace/wrap-stacktrace-log[fn]
stacktrace.clj:79 ring.middleware.stacktrace/wrap-stacktrace-web[fn]
jetty.clj:18 ring.adapter.jetty/proxy-handler[fn](Unknown Source) ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle
HandlerWrapper.java:111 org.eclipse.jetty.server.handler.HandlerWrapper.handle
Server.java:349 org.eclipse.jetty.server.Server.handle
AbstractHttpConnection.java:452 org.eclipse.jetty.server.AbstractHttpConnection.handleRequest
AbstractHttpConnection.java:884 org.eclipse.jetty.server.AbstractHttpConnection.headerComplete
AbstractHttpConnection.java:938 org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete
HttpParser.java:634 org.eclipse.jetty.http.HttpParser.parseNext
HttpParser.java:230 org.eclipse.jetty.http.HttpParser.parseAvailable
AsyncHttpConnection.java:76 org.eclipse.jetty.server.AsyncHttpConnection.handle
SelectChannelEndPoint.java:609 org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle
SelectChannelEndPoint.java:45 org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run
QueuedThreadPool.java:599 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
QueuedThreadPool.java:534 org.eclipse.jetty.util.thread.QueuedThreadPool$3.run
Thread.java:679 java.lang.Thread.run

(ns ring-session-test.core)

(use 'ring.middleware.session)

(defn handler [{session :session uri :uri}]
  (let [res {:status 200
             :headers {"Content-Type" "text/plain"}
             :body (str session)
             :session {:a 1}}]
    (if (= (session :a) 1)
      (assoc res :session-cookie-attrs {:max-age 3600})
      res)))

(def app
  (wrap-session handler))

What I wanted is that the session cookie is sent on the first request
and then set the expiry date on another one. But on the second
request there is no cookie sent - I copy the headers below. If I add
:session-cookie-attrs on the first response - then it gets there in
the Cookie:

Set-Cookie:ring-session=00bd9357-a388-4714-ac1b-4e66def7e2f7;Max-Age=3600;Path=/

But on the second not.

Request URL:http://localhost:3000/che
Request Method:GET

When a cookie-attrs :path is specified and no session :root is specified, the default session :root "/" replaces the specified cookie-attrs :path.

(wrap-session {:cookie-attrs {:path "/mypath"}})

produces a cookie with a path of "/" rather than "/mypath".

(defn wrap-session [options]
"snipped code from wrap-session"
(let [session-root (options :root "/")
      cookie-attrs (merge (options :cookie-attrs) {:path session-root})]
  (println (str "cookie-attrs: " cookie-attrs))))

I came across this by chance when I mistyped the name of an input field. Here's my set of tests, all of them provide the expected response except for the last one.

The order is important to generate this error. All the tests below pass unless you create the value as a string first (passing in val before any val[]'s). I can provide more information if necessary, using ring 1.1.8.

These return the expected results

    localhost/url/?val=three
    {:val "three"} 

    localhost/url/?val[]=one&val[]=two
    {:val ["one" "two"]}

    localhost/url/?val=one&val=two&val=three
    {:val "three"}  

    localhost/url/?val[]=one&val[]=two&val=three
    {:val "three"}

This one throws an error

    localhost/url/?val=three&val[]=one&val[]=two    
    Server error

Stacktrace

    java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IPersistentCollection
    at clojure.core$conj.invoke(core.clj:83)
    at ring.middleware.nested_params$assoc_nested.invoke(nested_params.clj:23)
    at ring.middleware.nested_params$assoc_nested.invoke(nested_params.clj:22)
    at ring.middleware.nested_params$nest_params$fn__824.invoke(nested_params.clj:44)
    at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
    at clojure.core.protocols$fn__6041.invoke(protocols.clj:98)
    at clojure.core.protocols$fn__6005$G__6000__6014.invoke(protocols.clj:19)
    at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
    at clojure.core.protocols$fn__6026.invoke(protocols.clj:54)
    at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
    at clojure.core$reduce.invoke(core.clj:6177)
    at ring.middleware.nested_params$nest_params.invoke(nested_params.clj:46)
    at clojure.lang.AFn.applyToHelper(AFn.java:163)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.core$apply.invoke(core.clj:619)
    at clojure.core$update_in.doInvoke(core.clj:5587)
    at clojure.lang.RestFn.invoke(RestFn.java:467)
    at ring.middleware.nested_params$wrap_nested_params$fn__830.invoke(nested_params.clj:64)
    at ring.middleware.params$wrap_params$fn__749.invoke(params.clj:55)
    at ring.adapter.jetty$proxy_handler$fn__1310.invoke(jetty.clj:18)
    at ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle(Unknown Source)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
    at org.eclipse.jetty.server.Server.handle(Server.java:349)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:884)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:938)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
    at java.lang.Thread.run(Thread.java:679)

In SPEC, the :content-type key is defined to be the mime-type of the request.

Most ring adapters (like jetty, http-kit and http-core) just copy the contents of the Content-Type header verbatim.

The netty ring adapter however lowercases this and strips charset and multipart-boundary information.

It would seem to me netty-ring-adapter does the right thing according to SPEC: it computes just the mime type of the request.
However: multipart-params middleware requires the boundary, but doesn't (get-in request [:headers "content-type"]) to get the raw header.

• Should SPEC change or adapters change?
• What is the point of :content-type in the request map if it's just a direct copy out of :headers? Convenience? Then you still have to remove ";" etc if you want to get the mime-type.

So, I have a toy clojure program using Compojure routes and Ring Middleware ([ring/ring-core "1.2.0"]). The wrap-keyword-params doesn't seem to like arrays very much.

I have some client-side javascript which calls:

$.get('/test', {"array": [1,2,3], "egg": "spam"});

The toy clojure code:

(ns toy
  (:require [ring.middleware.keyword-params :refer :all]
            [compojure.core :refer :all]))

(defn wrap-routes [all-routes]
  (-> all-routes
      wrap-keyword-params))

(defn test-route [{params :params}]
  (println params)
  {:status 200})

(defroutes all-my-routes
  (GET "/test" r (test-route r))
  ;; snip
  )

(def all (wrap-routes all-my-routes))

;; snip

And the output is precisely {array[] [1 2 3], :egg "spam"}.

I suspect this is because in ring.middleware.keyword-params the test (keyword-syntax? k) on line 12 is incorrect for arrays. The keyword may resemble "array[]", as in my minimal code example, which then fails the test for keyword syntax.

Is this a bug, or am I doing something horribly wrong?

Addendum. The "obvious" solution, wrappying my routes with:

(defn wrap-routes [all-routes]
  (-> all-routes
    wrap-nested-params
    wrap-keyword-params))

Well, it produces {array [1 2 3], :egg "spam"}, which is a step in the right direction, but still off.

I would recommend using the latest tools.reader contrib library at https://github.com/clojure/tools.reader, functions clojure.tools.reader.edn/read or read-string, since they only require Clojure 1.3, unlike Clojure 1.5's clojure.edn namespace. Check their APIs before just doing a simple replacement, since there are slight differences, but perhaps not for the one-arg calls you are using in ring.

The use in ring-core/src/ring/middleware/cookies.clj function normalize-quoted-strs might be safe already because it is only done on strings that begin with a double-quote, but it would still be better to replace that call, too.

Other occurrences that should be replaced:

File ring-core/src/ring/middleware/session/cookie.clj:
function serialize postcondition
function unseal

Motivation: With Clojure 1.3 and 1.4, even with read-eval bound to false, clojure.core/read and read-string can cause arbitrary Java constructors to be executed, a few of which have potentially dangerous side effects that an attacker could use:

;; This causes precious-file.txt to be created if it doesn't exist, or
;; if it does exist, its contents will be erased (given appropriate
;; JVM sandboxing permissions, and underlying OS file permissions).
(defn read-string-unsafely [s](binding [read-eval false]
%28read-string s%29))

(read-string-unsafely "#java.io.FileWriter["precious-file.txt"]")

In a project I happened to have both and index.html and index.js file, and find-index-file always returned the javascript. Perhaps this should be more configurable, or prefer html over anything else.

Typically you only want wrap-stacktrace to take effect in a development setting. Doing this with environ looks something like this:

(jetty/run-jetty (-> #'app
                     ((if (env/env :dev)
                        trace/wrap-stacktrace
                        identity))
                     [...])
                 {:port port :join? false})

If wrap-stacktrace took a second argument to turn it into a no-op, it would be easier to use in the context of ->.

(jetty/run-jetty (-> #'app
                     (trace/wrap-stacktrace (env/env :production))
                     [...])
                 {:port port :join? false})

What do you think?

I'm using Ring through Compojure 1.1.5, with the lein-ring 0.8.2 plugin. My handler is set up to look something like this:

(defroutes app-routes
(GET "/info/:item" [item](-> %28info/get-data item endpoint%29
u/json-response))
(GET "/" [](-> %28io/resource)
(response/render {"Content-Type" "text/html"})))
(route/resources "/")
(route/not-found "Not Found"))

(def app
(handler/site app-routes))

The problem is, when I call 'lein ring uberwar', I get a WAR file with the index, js, and related front-end files stuck way back in WEB-INF/classes/public. Which isn't what I expect compared to java-generated WAR files, where those front-end resources are put in the root of the WAR.
This doesn't cause an issue with locally-run Tomcats, but deploying to Amazon's elastic beanstalk results in my index.html file being recognized as text/plain. So I end up having to explicitly call the service at http://what.ever/index.html, which is kind of hokey, right?

This might be an issue better aimed at lein-ring, but I thought I'd start here first.

(the project I'm seeing this problem with is sourced at http://github.com/ryankohl/nasa-data-browser)

ring-jetty-adapter still uses Jetty 7 which has been superseded by Jetty 8 & 9. Is it desirable to upgrade ring-jetty-adapter to Jetty 9 and Servlet API 3.0? If so, I will be more than happy to create a patch for this.

I recently got bit by the fact that :body is an input string. Here was what happened:

(def site-routes
  (-> (routes about-routes
              legal-routes
              paste-routes
              user-routes
              home-routes
              login-routes) 
      (api)
      (wrap-noir-flash)
      (wrap-noir-session {:store (monger-store "sessions")}) 
      (wrap-strip-trailing-slash)
      (wrap-prod-middleware)))

(def a-routes
  (-> api-routes
      (api)
      (wrap-noir-session {:store (monger-store "sessions")})
      (wrap-strip-trailing-slash)
      (wrap-prod-middleware)))

(defroutes handler
  site-routes
  a-routes
  (resources "/")
  (not-found (four-zero-four)))

This is denser than necessary, but allow me to explain.

I have two sets of routes that are both wrapped with api and thus wrap-params. I noticed that my a-routes never had any :form-params. I spent a while trying to debug this and figure out why with @brehaut and we came to the conclusion that the wrap-params in site-routes was reading the :body of the request, so a-routes's wrap-params was unable to do so.

I'm not sure these read-once semantics make sense. @brehaut was proposing that it could instead be like a lazy cons where the tail is either pending or realized, or there could perhaps be a middleware for that. At the very least, this should be documented. It feels very wrong to be bit by mutability in a functional framework!

(also posted to http-kit as I'm not sure what is the right "layer" to solve this problem)

This isn't so much a "here's a bug in the code" but more of a "is it possible to do X with ring."

I'm using
{:client-side [clojurescript]
:server-side [clojure, ring, http-kit]}

Now, I want my code to be "robust" vs dropped connections (phone / wifi that often times out.)

Is there a way to simulate this with ring middleware?

Basically, I need it to do things like:

(1) arbitrarily ignore requests
(2) arbitrarily delay requests
(3) arbitrarily kill web-sockets / comet / long-polling connections

I'd like this to be done in an as "unclean" way as possible -- i.e. instead of calling close on a websocket, have it abruptly killed / time out.

Thanks!

(PS, if this question doesn't belong here / should be posted in a forum somewhere, please let me know.)

wrap-multipart-params is different from other wrap-* functions

(defroutes all-routes
  ; start a thread per run
  (POST "/" [] (wrap-multipart-params handler))
  )

temp-file-store start a thread when called: https://github.com/ring-clojure/ring/blob/master/ring-core/src/ring/middleware/multipart_params/temp_file.clj#L39

In ring-core 1.2.1, when compiling with lein check, or running a repl in lein, I'm getting this explosion:

Exception in thread "main" java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest, compiling:(ring/middleware/multipart_params.clj:39:5)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6567)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyze(Compiler.java:6322)
    at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3624)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6562)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyze(Compiler.java:6322)
    at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5708)
    at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5139)
    at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3751)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6558)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.access$100(Compiler.java:37)
    at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:529)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6560)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyze(Compiler.java:6322)
    at clojure.lang.Compiler.eval(Compiler.java:6623)
    at clojure.lang.Compiler.load(Compiler.java:7064)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5018.invoke(core.clj:5530)
    at clojure.core$load.doInvoke(core.clj:5529)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5336)
    at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
    at clojure.core$load_lib.doInvoke(core.clj:5374)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:619)
    at clojure.core$load_libs.doInvoke(core.clj:5417)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:621)
    at clojure.core$use.doInvoke(core.clj:5507)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at compojure.handler$eval11534$loading__4910__auto____11535.invoke(handler.clj:1)
    at compojure.handler$eval11534.invoke(handler.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6619)
    at clojure.lang.Compiler.eval(Compiler.java:6608)
    at clojure.lang.Compiler.load(Compiler.java:7064)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5018.invoke(core.clj:5530)
    at clojure.core$load.doInvoke(core.clj:5529)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5336)
    at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
    at clojure.core$load_lib.doInvoke(core.clj:5374)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:619)
    at clojure.core$load_libs.doInvoke(core.clj:5413)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:619)
    at clojure.core$require.doInvoke(core.clj:5496)
    at clojure.lang.RestFn.invoke(RestFn.java:3894)

The stack starts at the point in my code where I wrap the app in:

(->> app
      .. stuff... 
      ring.middleware.multipart-params/wrap-multipart-params 
      ... more stuff ...
)

What's baffling to me is that the ring-core line cited in the above stacktrace is the last line of this function:

(defn- file-item-seq
  "Create a seq of FileItem instances from a request context."
  [context]
  (file-item-iterator-seq
    (.getItemIterator (FileUpload.) context)))

What that has to do with HttpServletRequest is utterly opaque to me.

Sadly, I haven't been able to isolate it any further, because I have very little understanding of the underlying Java stuff, and no idea what a HttpServletRequest is, why it's in ring, or why it could be exploding.

FWIW, the server I'm using here is aleph, not jetty. If this is a problem with aleph, please let me know and I'll ask Zach about it instead

Hi James,

I think I've just come across a problem with the wrap-params middleware.

A minimal reproduction:

(ns ring-bug
  (:require [compojure.handler]
            [compojure.core :as comp :refer (defroutes routes)]
            [ring.adapter.jetty :as jetty]))

(defroutes my-app
  ;;(compojure.handler/api (comp/GET  "/foo" {} "foo"))
  (compojure.handler/site (comp/POST "/bar" {p :params} (str "Params: " p))))

(defonce server
  (jetty/run-jetty #'my-app {:join? false :port 8086}))

;; When /api is commented out:
;; curl -d "name=steve" http://localhost:8086/bar => Params: {:name "steve"}

;; When /api is uncommented:
;; curl -d "name=steve" http://localhost:8086/bar => Params: {}

It looks to me like wrap-params is mutating something in place (perhaps the org.eclipse.jetty.server.HttpInput?), which breaks the second call to wrap-params.

Any ideas?

Thanks!

I understand why it does not do the early-exit method of comparing two strings, but instead a constant time algorithm, to avoid a timing attack. That is good.

However, I don't think it is implemented correctly. By calling String's .getBytes method, two strings that have equal length could each be transformed into a byte array with different lengths from each other, if they have different data. For example:

user=> (defn utf-16-code-units-to-str [code-units]
             (apply str (map char code-units)))
user=> (def s1 (utf-16-code-units-to-str [65 66 67 6500]))
#'user/s1
user=> (def s2 (utf-16-code-units-to-str [65 66 67 100]))
#'user/s2
user=> (.length s1)
4
user=> (.length s2)
4
user=> (seq (.getBytes s1))
(65 66 67 -31 -91 -92)
user=> (seq (.getBytes s2))
(65 66 67 100)
user=> (map bit-xor (.getBytes s1) (.getBytes s2))
(0 0 0 -123)

Thus secure-compare could return true for two different strings, as long as the arrays returned by .getBytes for the two were equal in the elements of the shorter one.

I would recommend replacing the two calls to (.getBytes) there with (map int s1) and (map int s2). Those will always return sequences containing the same number of integers as the original strings have Java characters.

In addition, there seem to be several calls to .getBytes throughout ring that do not specify a character encoding. This could lead to different behavior on different systems with different default character encodings. Not sure if that is desired or not, but it seems like it could lead to a lack of portability if not bugs.

A content-type header with a value like "vnd.example-com.foo+json; version=1.0" will lose the additional parameter when it passes through ring.util.servlet/set-content-type

In multipart_params there is a reference to assoc-conj in ring.util.codec. As far as I can tell there is no such namespace anymore. Maybe the function has moved to another namespace?

A HEAD request to a static file or resource should return the same headers and response code as a GET request.

Hi,

When given params map with a native clojure array behind one of the keys, wrap-nested-params corrupts it.

Request (POST body):

{"Platforms": ["Android", "iOS"]}

Parsed request (with ring-json/wrap-params):

{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {"Platforms" ["Android" "iOS"]}
...}

After ring.middleware.params/wrap-params:

{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {"Platforms" ["Android" "iOS"]}
...}

After ring.middleware.nested-params/wrap-nested-params:

{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {"Platforms" "iOS"}
...}

After ring.middleware.keyword-params/wrap-keyword-params:

{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {:Platforms "iOS"}
...}

Is it desired behaviour or not? Or is it some kind of interference with ring-json?

P.S. Why I've used those 3 wrappers? They're part of compojure.handler/api.

When using wrap-nested-params , parameters like:
http://someurl?param=value1&param=value2
are mapped to
{:params {:param "value2"}} instead of {:params {:param [ "value1" "value2"]}} in the request map.

The term stacktrace could be colored using the pst+ function in clj-stacktrace, and some work could be done to improve the HTML version as well.

The 1.1.6 changelog says "Removed default charset being incorrectly set on images," however, this is still happening.

Code to reproduce:

(ns testcase.core
  (:require [ring.util.response :as resp]))

(defn handler [request]
  (-> (resp/response "")
      (resp/header "Content-Type" "image/png")))

project.clj:

(defproject testcase "1.0.0-SNAPSHOT"
  :description "FIXME: write description"
  :dependencies [[org.clojure/clojure "1.3.0"]
                 [ring/ring-jetty-adapter "1.1.6"]
                 [ring/ring-core "1.1.6"]
                 [ring/ring-servlet "1.1.6"]]
  :dev-dependencies [[lein-ring "0.7.5"]]
  :ring {:handler testcase.core/handler})

I ran this with lein ring server and executed:

$ curl -i localhost:3000
HTTP/1.1 200 OK
Date: Wed, 26 Sep 2012 17:02:11 GMT
Content-Type: image/png;charset=ISO-8859-1
Content-Length: 0
Server: Jetty(7.6.1.v20120215)

$

It's also set for many other types it shouldn't be, including application/pdf, application/octet-stream, application/x-gzip, etc.

The Ring specification states that the response body can be an ISeq of strings. Wouldn't it make sense to let it be an ISeq of {String, ISeq, File, InputStream} instead? This would make composition of response bodies much simpler; a middleware that wishes to decorate a response body can then simply do (vector "callback(" body ");") without caring about whether body is a string or not.

wrap-resource is broken when used from a JAR file created with Leiningen 2.3.3. When the root URL is requested, wrap-resource thinks it can find an empty resource called "" and serves that instead of passing the request on to the handler.

I've created a SSCCE demonstrating the problem: https://github.com/noidi/wrap-resource-lein-2.3.3

I suspect the bug is caused by this change made in Leiningen 2.3.3: "Add directory entries to jar files."

ring.middleware.keyword-params has a fixed regex for what it considers legal keywords and there is no way to configure or override that.

I want to have params that include "." dots as keywords, but it seems that I need to copy the rest of this file to accomplish that.

I'm not sure if you will consider that as a bug, but in tests you submit a GET request with body, and then check that body is echoed by a handler.

Jetty passes body to your handler, so that works. But Undertow does not and I've spent quite a bit of time trying to understand what's going on until I noticed it's a get. :-) So while it works, maybe it's worth changing it to http/post, what do you think?

ring.middleware.stacktrace/wrap-stacktrace-web and wrap-stacktrace-log wraps the handler inside a (try handler (catch Exception ex ..) block. But if the handler contains assert expressions and some of they fail, these exceptions will not be catched by Ring, because assert throws AssertionError's and they aren't derived from java.lang.Exception. So nothing will be printed to the log, and Ring will return an empty 500 Server Error.

Here is a test:

;; http://localhost:8081/zerodiv will show a backtrace
;; http://localhost:8081/assert will not show it
(def my-handler
  (-> 
   #(case (:uri %)
      "/assert"    (assert false)
      "/zerodiv" (/ 1 0))
   ring.middleware.stacktrace/wrap-stacktrace
   ))
(ring.adapter.jetty/run-jetty #'my-handler {:port  8081     
                                            :join? false})

I think wrap-stracktrace-* must be modified to catch Throwable or maybe add an (catch AssertionError block if you don't want to catch all the Throwable~s/Error's exceptions.

I use wrap-session to manage cookie-based sessions for my users and many times when running unit tests I noticed that a certain test would stall for over 10s. I narrowed it down to one specific situation:

I make a request to the ring app. The session map is empty as there is no ring-session= cookie associated with it. Say a new :session map is associated with the response. wrap-response will occasionally (not always, maybe once every 3-5 test runs) will stall for 10+s. The other times there is no delay.

  (-> routes
       [...]
       (print-response "above wrap session")
       (wrap-session {:store (cookie-store {:key (env :cookie-key)})
                        :cookie-attrs {:domain (env :cookie-domain)
                                            :max-age 31536000
                                            :secure true
                                            :http-only true}})
        (print-response "below wrap session")
        [...])

Here's the output of the above:

2013-Jul-19 13:38:19 -0700 alex-haswell DEBUG [classroom.middleware] - above wrap session response: {:session {:teacher-id 1}, :status 200, :headers {}, :body {}}
2013-Jul-19 13:38:33 -0700 alex-haswell DEBUG [classroom.middleware] - below wrap session response: {:status 200, :headers {"Set-Cookie" ("ring-session=IkPHd4p0KNxCQj3o%2BqkXLoqQOlF%2BDOpYm%2FZGd2zo6BY%3D--JNyHj01WUANP5kDrFDJ5k6gqj2Hsemzy4CbhHP9suHA%3D;HttpOnly;Secure;Max-Age=31536000;Domain=localhost.com;Path=/")}, :body {}}

If you look at the timbre timestamps, you can see operation took a considerable while to complete. I'm wondering if I'm doing something wrong here that's causing the slow-down. Is this by design?

Documentation for ring.middleware.cookies (https://github.com/ring-clojure/ring/wiki/Cookies) states that by using :port attribute handler may restrict cookies to a specific port, nevertheless browsers (tested with Chrome 26 and Firefox 24) pass cookies to servers listening on different ports.

ring-core/src/ring/middleware/cookies.clj uses `Port=' attribute from RFC2965 obsoleted by RFC6265 (https://tools.ietf.org/html/rfc6265) which manifest:

Cookies do not provide isolation by port. If a cookie is readable by a service running on one port, the cookie is also readable by a service running on another port of the same server.

Please see
http://stackoverflow.com/questions/14309400/a-directory-explorer-middleware-for-ring-clojure

I find that listing directories like apache does is something super useful to me when debugging and exploring my projects.

https://github.com/ring-clojure/ring/blob/master/ring-devel/src/ring/middleware/reload.clj

It appears wrap-reload needs to be passed a handler as normal, but with the caveat that its referenced through a var (see mmcgrana#72).

Something along the lines of:

(def app
  (-> (compojure.core/routes my-routes)
      (compojure.handler/api)))

(def app-with-reload
  (ring.middleware.reload/wrap-reload #'app))

(defn -main []
  (ring.adapter.jetty/run-jetty #'app-with-reload {:port 8081 :join? false})
  (ring.adapter.jetty/run-jetty #'app-with-reload {:port 8082 :join? false}))

If instead wrap-reload is treated as a normal middleware—placed in the chain for app—it seems to work but the updated behavior is actually delayed by one request.

Raising this issue to hopefully get some documentation in wrap-reload that makes note that it needs the var reference; also it may be helpful for it to throw an exception if it's not passed a var to prevent accidental use as typical middleware.

ring/ring-core/test/ring/util/test/response.clj
line 126: (let [resp (file-response "backlink/foo.html"
backlink is not directory.

The current behaviour of the jetty adapter is to set charset=UTF-8 on every response. It turns out that doing this on image responses when HTTPS is being used causes IE7/8 (I think 9 as well, though didn't verify) to sporadically not display images (or very consistently in the case of IE7).

My temporary solution to this was to disable the charset in ring-jetty-adapter and instead use the following middleware:

(defn wrap-utf8
    "Wrap UTF-8 encoding only on text responses."
    [handler]
    (fn [req]
        (let [resp  (handler req)
              ctype (get-in resp [:headers "Content-Type"])]
                (if (and (> (count ctype) 5) (= (subs ctype 0 5) "text/" ))
                    (charset resp "UTF-8")
                    resp))))

(def -main
(run-jetty #'app {:port 3000 :join? false}))

and this is the line that gives me the warning, but server still works.

Implementation of temp-file-store in the ring.middleware.multipart-params.temp-file creates too many futures objects and this leads to too many threads running on web services that handles many multipart uploads...

P.S. I'll try to refactor code to use single expiration thread or something like...

P.P.S. Initial discussion was at mmcgrana#74

In Windows 7 + JVM(1.6.0_35) environment, download site using ring-server has been set-up.
archive.zip exceeding 100Mbyte has been used.
If archive.zip is downloaded using browser or wget then file transfer stops halfway.
But, it works fine in ubuntu environment.

project.clj

(defproject sample4 "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.4.0"]
                 [org.clojure/tools.cli "0.2.2"]
                 [ring/ring-core "1.1.6"]
                 [ring/ring-jetty-adapter "1.1.6"]
                 [compojure "1.1.3"]]
  :plugins [[lein-ring "0.7.5"]]
  :ring {:handler sample4.route/all-routes}
  :profiles {:dev {:dependencies [[ring-mock "0.1.3"]]}})

route.clj

(defroutes app-routes
  (GET "/" [] "Hello World")
  (GET "/archive.zip" []
    (let [archive-file (java.io.File. "c:\\home\\sample\\archive.zip")]
      {:status 200
       :body archive-file
       :headers {"Content-Type" "application/zip"
                 "Content-Length" (str (.length archive-file))
                 "Cache-Control" "no-cache"
                 "Content-Disposition" (str "attachment; filename=" (.getName archive-file))}}
      )
    )
  (route/not-found "Not Found"))

What bug is this?

a reference to assoc-conj can be found in ring-core/src/ring/middleware/multipart_params.clj in 1.2.0-beta1

wrap-resource doesn't make an attempt to set any response headers, such as last-modified and content-length; and wrap-file-info isn't compatible with wrap-resource unless the resource came from a file on the class-path. If the resource came from a jar file, we end up without the response headers set.

I package my app as an uber-jar, so I'd like wrap-resource to support conditional gets.

It is easy enough to get the content-length and last-modified date for a resource URL, using something like the following. This works whether the resource comes from a file or a jar file (or anywhere else for that matter)

(defn last-modified-and-content-length
  [^java.net.URL res-url]
  (let [ucon (.openConnection res-url)]
    [(.getLastModified ucon) (.getContentLengthLong ucon)]))

Could we incorporate something like this into wrap-resource?