"413 Request Entity Too Large nginx/1.9.13" when uploading 10.6 megabyte file about docstore HOT 11 CLOSED

Should be all set. Messed it up when I commented out some of the ssl stuff.

from docstore.

There was a report from Kai about some kind of "file too big" problem, investigating. That was a 135MB document.

from docstore.

I double-checked and Nginx is still 2G. If you have additional info on what the error was I can dig into it. Could be a timeout or something else

from docstore.

Aha! Tornado has a maximum upload size, but you can now customize it for the docstore app via max_file_size: in settings.yml. Note that the higher this is, the more memory the server will use!

I also updated docstore to use just Python 3. @eby if this is an issue I could add some backwards compatibility. Python 2 is end of life in 5 months, but I know all too well there's lots of Python 2 still out there!

from docstore.

Alright. It is up now with a 500M max_file_size to see how it does with memory. Also switched to python3. Let me know if you notice any issues @vielmetti

from docstore.

So again I'm not a python programmer. It looks like this has come up quite a bit. Not sure if the latest tornado already fixes the memory use but I found that there is a module to help buffer straight to disk so that memory isn't filled up: https://pythonhosted.org/tornadostreamform/

Supposedly bottle and flask already moved to a buffer to disk model when uploads are > small limit.

from docstore.

Yeah, tornado.web.stream_request_body looks promising, but from some quick investigation it looks like it might have some compatibility issues with XSRF cookies (at least in tornado 5).

from docstore.

I was able to upload http://a2docs.org/view/539 (about 130M) now with no issues.

I'll leave this issue open for you all to sort out if there should be any code changes (especially re tornado.web.stream_request_body) but am happy otherwise to close as "solved".

from docstore.

@vielmetti / @eby, can this ticket be closed out now?

from docstore.

I think someone should take a look at if tornado.web.stream_request_body (or something similar) could be used before closing this out.

from docstore.

Started a new #28 so this one can close.

from docstore.