Giter Club home page Giter Club logo

koa-body's Introduction

koa-body

CI KoaJs Slack


A full-featured koa body parser middleware. Supports multipart, urlencoded, and json request bodies. Provides the same functionality as Express's bodyParser - multer.

Install

Install with npm

npm install koa-body

Features

  • can handle requests such as:
    • multipart/form-data
    • application/x-www-form-urlencoded
    • application/json
    • application/json-patch+json
    • application/vnd.api+json
    • application/csp-report
    • text/xml
  • option for patch to Koa or Node, or either
  • file uploads
  • body, fields and files size limiting

Hello World - Quickstart

npm install koa koa-body # Note that Koa requires Node.js 7.6.0+ for async/await support

index.js:

const Koa = require('koa');
const { koaBody } = require('koa-body');

const app = new Koa();

app.use(koaBody());
app.use((ctx) => {
  ctx.body = `Request Body: ${JSON.stringify(ctx.request.body)}`;
});

app.listen(3000);
node index.js
curl -i http://localhost:3000/users -d "name=test"

Output:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 29
Date: Wed, 03 May 2017 02:09:44 GMT
Connection: keep-alive

Request Body: {"name":"test"}%

For a more comprehensive example, see examples/multipart.js

Usage with koa-router

It's generally better to only parse the body as needed, if using a router that supports middleware composition, we can inject it only for certain routes.

const Koa = require('koa');
const app = new Koa();
const router = require('koa-router')();
const { koaBody } = require('koa-body');

router.post('/users', koaBody(), (ctx) => {
  console.log(ctx.request.body);
  // => POST body
  ctx.body = JSON.stringify(ctx.request.body);
});

app.use(router.routes());

app.listen(3000);
console.log('curl -i http://localhost:3000/users -d "name=test"');

Usage with unsupported text body type

For unsupported text body type, for example, text/xml, you can use the unparsed request body at ctx.request.body. For the text content type, the includeUnparsed setting is not required.

// xml-parse.js:
const Koa = require('koa');
const { koaBody } = require('koa-body');
const convert = require('xml-js');

const app = new Koa();

app.use(koaBody());
app.use((ctx) => {
  const obj = convert.xml2js(ctx.request.body);
  ctx.body = `Request Body: ${JSON.stringify(obj)}`;
});

app.listen(3000);
node xml-parse.js
curl -i http://localhost:3000/users -H "Content-Type: text/xml" -d '<?xml version="1.0"?><catalog id="1"></catalog>'

Output:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 135
Date: Tue, 09 Jun 2020 11:17:38 GMT
Connection: keep-alive

Request Body: {"declaration":{"attributes":{"version":"1.0"}},"elements":[{"type":"element","name":"catalog","attributes":{"id":"1"}}]}%

Options

Options available for koa-body. Four custom options, and others are from raw-body and formidable.

  • patchNode {Boolean} Patch request body to Node's ctx.req, default false
  • patchKoa {Boolean} Patch request body to Koa's ctx.request, default true
  • jsonLimit {String|Integer} The byte (if integer) limit of the JSON body, default 1mb
  • formLimit {String|Integer} The byte (if integer) limit of the form body, default 56kb
  • textLimit {String|Integer} The byte (if integer) limit of the text body, default 56kb
  • encoding {String} Sets encoding for incoming form fields, default utf-8
  • multipart {Boolean} Parse multipart bodies, default false
  • urlencoded {Boolean} Parse urlencoded bodies, default true
  • text {Boolean} Parse text bodies, such as XML, default true
  • json {Boolean} Parse JSON bodies, default true
  • jsonStrict {Boolean} Toggles co-body strict mode; if set to true - only parses arrays or objects, default true
  • includeUnparsed {Boolean} Toggles co-body returnRawBody option; if set to true, for form encoded and JSON requests the raw, unparsed request body will be attached to ctx.request.body using a Symbol (see details), default false
  • formidable {Object} Options to pass to the formidable multipart parser
  • onError {Function} Custom error handle, if throw an error, you can customize the response - onError(error, context), default will throw
  • parsedMethods {String[]} Declares the HTTP methods where bodies will be parsed, default ['POST', 'PUT', 'PATCH']. Replaces strict option.

A note about parsedMethods

see http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-19#section-6.3

  • GET, HEAD, and DELETE requests have no defined semantics for the request body, but this doesn't mean they may not be valid in certain use cases.
  • koa-body is strict by default, parsing only POST, PUT, and PATCH requests
  • you may use either the enumeration or strings to chose which methods to parse: For example, HttpMethodEnum.PATCH

File Support

Uploaded files are accessible via ctx.request.files.

A note about unparsed request bodies

Some applications require cryptographic verification of request bodies, for example webhooks from slack or stripe. The unparsed body can be accessed if includeUnparsed is true in koa-body's options. When enabled, import the symbol for accessing the request body from unparsed = require('koa-body/unparsed.js'), or define your own accessor using unparsed = Symbol.for('unparsedBody'). Then the unparsed body is available using ctx.request.body[unparsed].

Some options for formidable

See node-formidable for a full list of options

  • maxFields {Integer} Limits the number of fields that the querystring parser will decode, default 1000
  • maxFieldsSize {Integer} Limits the amount of memory all fields together (except files) can allocate in bytes. If this value is exceeded, an 'error' event is emitted, default 2mb (2 * 1024 * 1024)
  • uploadDir {String} Sets the directory for placing file uploads in, default os.tmpDir()
  • keepExtensions {Boolean} Files written to uploadDir will include the extensions of the original files, default false
  • hashAlgorithm {String} If you want checksums calculated for incoming files, set this to either 'sha1' or 'md5', default false
  • multiples {Boolean} Multiple file uploads or no, default true
  • onFileBegin {Function} Special callback on file begin. The function is executed directly by formidable. It can be used to rename files before saving them to disk. See the docs

Changelog

Please see the Changelog for a summary of changes.

Tests

$ npm test

License

The MIT License, 2014 Charlike Mike Reagent (@tunnckoCore) and Daryl Lau (@daryllau)

koa-body's People

Contributors

adamkdean avatar amauryd avatar baoshan avatar booxood avatar changchanghwang avatar crobinson42 avatar damianb avatar davidlondono avatar davidtanner avatar dlau avatar egilsster avatar erikaugust avatar eusen avatar fabienjuif avatar forestmist avatar ivan-kleshnin avatar jarvisprestidge avatar markherhold avatar michaelnwani avatar naxmefy avatar necaris avatar robertherhold avatar simonbrunel avatar stafyniaksacha avatar thedadi avatar thuitaw avatar tommoor avatar trufae avatar undemian avatar zevisert avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

koa-body's Issues

How can I get actual downloaded path?

I'm wondering how I can get actual downloaded path. because when I print out ctx.request.body.files.pic.path, I get something like '/var/folders/ls/75t48c651sb1pt749vj3wz4m0000gn/T/upload_dc1a848b5aa6da5ff38bbb21d610199e' but actually It's supposed to be /uploads/filename. you can see this in my code below

import IncomingForm from "formidable";
import shortid from "shortid";
import path from "path";

let form = new IncomingForm();
form.encoding = "utf-8";

form.on("fileBegin", function(name, file) {
  let regex = /[^.]*/;
  let fileName = file.name.replace(regex, shortid());
  file.path = path.join(__dirname + "/../uploads/", fileName);
});

export default form;

Seeking maintainer

Hi everyone,

Apologies for the slow responses as of recent. Life has been hectic for me, and I haven't been able to take the time properly maintain this module.

I'm looking for someone to take over this project. The only condition I am stipulating is that I would like it to continue along the lines of a small lightweight middleware that does one thing very well.

Best,
Daryl

Example with koa-router doesn't work

The code example using koa-router doesn't works neither the files koa-router.js and multipart.js in the examples directory. All generates the error below:

assert.js:86
  throw new assert.AssertionError({
        ^
AssertionError: app.use() requires a generator function

Screenshot:
2015-08-24-221441_605x292_scrot

Intermittent failure - `koa-body` attaches empty object

About 50% of my POST requests result in koa-body attaching an empty object (i.e this.request.body = {}), as opposed to a set of files uploaded.

Here's a request that failed:
headers
body

And one that succeeded:
headers
body

I can't see anything notably different between these. Digging into Formidable now — I assume there's a decent chance it's coming from there — but figured it was worth documenting here.

Where can I handle error when file size exceeds maxFieldsSize?

I set koa-body options to

{
    formidable:{
        uploadDir: './static/uploads',
        keepExtensions:true,
        maxFieldsSize :1024*1024,  //1M
    },

    multipart: true,
    urlencoded: true,
}

when I upload a file with size larger than 1M, I get a 0 size file in my disk, but koa-body do not throw this error. How I can handle it ?

type check before uploading?

I wander if I can check the file's type before uploading it to the disk. So that I can interrupt the job with the wrong type of file.
For example, formidable expose IncomingForm.prototype.onPart to users so that we can define our own function to handle it.

form.onPart = function(part){
  if(part.mime.indexOf('image') != -1){
    this._error(new Error('wrong file type'));
  } else {
    this.handlePart(part); // default work of formidable
  }
}

keepExtensions default false and blank file generated for pure text field post

Hi, I am using koa-body 1.4.0, and have two issues:
1, In the doc the keepExtensions says default to true, but my test is false.
2, If in my form both text input and fileupload present, and if I just input text with no file, after submitting, koa-body will still generate a blank file into my dir
Here are my related codes:

const koaBody = require('koa-body')({
  multipart:true,
  formidable:{
    uploadDir: path.join(__dirname, '../public/audio'),
    keepExtensions: true
  }
});
...
...
/**
 * get upload page
 */
router.get('/upload', co.wrap(function* (ctx, next) {
  ctx.body = `
    <!doctype html>
    <html><body>
      <form action="/upload" enctype="multipart/form-data" method="post">
      <input type="text" name="username" placeholder="username"><br>
      <input type="text" name="title" placeholder="tile of film"><br>
      <input type="file" name="uploads" multiple="multiple"><br>
      <button type="submit">Upload</button>
    </body></html>
  `;
}));
/**
 * upload interface, img, audio, etc.
 *
 */
router.post('/upload', convert(koaBody), co.wrap(function* (ctx, next) {
  ctx.body = ctx.request.body;
  console.log(ctx.request.body);
}));

Multipart array

When sending an array in a simple urlencoded POST, it gets converted into an object or an array of objects thanks to the qs lib used by the co-body lib.

Example:

 books[new-1][is_new]=1
 books[new-1][title]=test

Becomes:

 {
     "books": {
         "new-1": {
             "is_new": "1",
             "title":"test"
         }
     }
 }

But when the same field is sent in a multipart POST (e.g. with a picture). Then the formidable lib is used, and because this lib only processes the fields independantly, it doesn't do the conversion and we end up having the following:

 {
     "books[new-1][is_new]":"1",
     "books[new-1][title]":"test"
 }

A simple fix I have found is to ask qs to rebuild a query string and reparse it instantly like so:

qs.parse(qs.stringify(body.fields))

It probably is not the best solution but it does the job

Error handling

Is there any way to handle the Error. For example, if I set the uploadDir as a path which doesn't exist, the app will crash.

Causes 'Not Found'

Whenever I add koaBody as a middleware, no matter what order, koa returns Not Found all the time

Create new NPM release

I'd love to use this package with the recent type definition files you've added without having to add them manual.

Please update the NPM package 🙂

Does not parse nested objects when uploading multipart form data

Request Object

{
	"title" : "Coca Cola",
	"details" : "More and More countries",
	"content" : {
		"options" : {
			"background" : {
				"color" : "#443344"
			}
		},
		"data" : "Hello world"
	}
}

Expected

{
	"title" : "Coca Cola",
	"details" : "More and More countries",
	"content" : {
		"options" : {
			"background" : {
				"color" : "#443344"
			}
		},
		"data" : "Hello world"
	}
}

Actual

{
	"title" : "Coca Cola",
	"details" : "More and More countries",
	"content[options][background][color]" : "#443344",
        "data" : "Hello world"
}

Partial upload, request needs to wait for upload

It looks like my request handler is getting called before the upload is complete. Is this how it works? I'm using the formidable: {uploadDir}.

I need to wait for this to complete (aka the formidable onEnd event). Is there a way to wait on this in my request handler?

automatically create the uploadDir folder

it will throw an error if the uploaddir folder does not exist, although i just need to write one line, i think it's more convenient that koabody can automatically create the uploadDir folder. would you please add the code?

form-data not supper object ??

today,I found a issue when using form-data,this middleware not parse object like
qq20180315-132319 2x
i found cource code
qq20180315-133527 2x

Seems to only support

Excuse me, my English is poor.

v1.6.0 deps co-body version

The deps version of co-body now is *, and just hours ago co-body release a new version(6.0.0), our Node server(6.10) down because of syntax error, so I think to lock the version.

Unhandled rejection error when using koa-body

Hi,

I'm experiencing something strange with error handling when using koa-body.

Errors thrown in my middleware results in unhandled rejections when i have koa-body registered as a middleware.

I tried to reduce this to a minimal use case:

  import koaBody from 'koa-body';

  router.post('/noKb', async (ctx, next) => {
    ctx.throw(500);
    await next();
  });

  router.post('/kb', koaBody(), async (ctx, next) => {
    ctx.throw(500);
    await next();
  });

The first example handles the error as I would expect, but the second route now throws this:

Unhandled rejection InternalServerError: Internal Server Error

As I'm quite new to Koa, I might be doing something wrong here, please let me know if thats the case.

Error forces Koa instance to die and need a restart

Context:
koaBody is configured with file uploads and custom uploadsDir. If the directory does not exist, an error is thrown that forces the process to require a restart.

Error: ENOENT: no such file or directory, open

Current local setup uses pm2 to manage process that restarts it, although it would be nice if that kind of things would not force the Koa server process to halt.

Seems like this is being propagated from Formidable.

supertest devDependency generating security warning

Upon installation, npm is generating this warning:

$ npm i
npm WARN notice [SECURITY] superagent has 1 low vulnerability. Go here for more details: https://nodesecurity.io/advisories?search=superagent&version=2.3.0 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info.

Details: https://nodesecurity.io/advisories?search=superagent&version=2.3.0

ED: Appears to be a sub-dep of supertest, so not as critical of an issue.

formidable event never gets triggered

fileBegin event from formidable never gets triggered when using koa-body(works fine with koa-better-body) It's really weird that It worked until yeserday and it randomly stopped working. file path should be /uploads/file_name but since fileBegin event never get triggered, file path is always like PROJECT_FOLDER/upload_3b1ae29425ef0e8f45a121173cec0d27/. Does anyone have any idea about this?

import Router from 'koa-router'
import IncomingForm from 'formidable'
import shortid from 'shortid'
import path from 'path'

const router = new Router()

router.post('/image/cover', async (ctx, next) => {
    let form = new IncomingForm();
form.encoding = "utf-8";

form.on("fileBegin", function(name, file) {
    let regex = /[^.]*/
    console.log("file name is", file.name)
    let fileName = file.name.replace(regex, shortid())
    file.path = path.join(__dirname + '/../uploads/', fileName)
});
    console.log(ctx.request.body.files.cover.path)
    //ctx.body = { url: URLpath }
})

Can I get blob file?

Hi! Can I get blob file in koa?
I send data from browser jQuery

$.ajax({
                    url: '/api/attachments/',
                    method: 'POST',
                    data: blob, // result FileReader.readAsArrayBuffer => blob
                    processData: false,
                    headers : {
                        'Content-Type': file.type != '' ? file.type : 'application/octet-stream' //  => image/jpeg
                    }
                });

Big security vulnerability

Hi,

I've just realised that it is very easy to create a big security vulnerability when using this package to upload files (I almost did this on my site).

Suppose you have JSON body parsing enabled on a POST or PUT route, say '/upload-files', as well as multipart parsing. This is quite easy to do e.g. if you add JSON parsing as global middleware. Suppose it expects the files to be in a field named 'uploads'. Then you can make a POST or PUT request to '/upload-files' with a JSON body that looks something like {"files": {"uploads": [{"path": "/any/file/path"}]}}, making the request handler think a file has been uploaded to /any/file/path. Now suppose that the handler is set up to copy uploaded files into a public uploads folder. By choosing the path appropriately I can make the server copy any file I like on the server into the public uploads folder and then view its contents. So by using well-known paths of sensitive files I can read private keys, passwords etc. and maybe even gain full access to the server this way.

I haven't tried actually doing this, but I think it's correct (sorry if I'm wrong). In my opinion, it would be better to put the files object on ctx.request instead of ctx.request.body, so that we know it can be trusted.

Thanks.

[suggest] Does not intercept the get request

app.use(
    (function(){
        var origin_bodyparser = require('koa-bodyparser')();

        return function* (next){
            if(this.request.method === "GET"){
                return yield* next;
            }

            yield* origin_bodyparser.apply(this, arguments);
        };
    })()
);

can't generate the folder for the upload files automaticly

@dlau

Hi, thanks for your awesome koa2 middleware. It is really powerful.

Now i have an advice for upload files.

when user specify a folder for the upload files, generate the specified folder automatically if the folder not existed


app.use(koaBody({
    multipart: true,
    formidable: {
        uploadDir: './uploads',
        keepExtensions: true,
        onFileBegin: function(name, file) {
            file.path = 'uploads/' + file.name
        }
    }
}))
➜  koa2-test node "/Users/percy/working/koa2-test/app.js"
server at http://localhost:3333 ~
请求: GET /
请求: POST /page/upload
events.js:182
      throw er; // Unhandled 'error' event
      ^

Error: ENOENT: no such file or directory, open 'uploads/'

If i do not create the folder manually, i will get the error message as shown above.

Doc issue?

In the section "Some options for formidable" of documentation, it indicates:

maxFields {Integer} Limits the number of fields that the querystring parser will decode, default 10
maxFieldsSize {Integer} Limits the amount of memory a field (not file) can allocate in bytes, default 2mb

For "maxFields", it's 1000 in formidable.
For "maxFieldsSize", here you indicated the amount of a field, but actually it is the amount of all form fields.

README typo

Hi

there is a typo in README at Some options for formidable section

maxFieldsSize {Integer} Limits the amount of memory all fields together (except files) can allocate in bytes. If this value is exceeded, an 'error' event is emitted, default 2mb (2 * 2 * 1024)

It seems the last part should be

2mb (2 * 1024 * 1024)

multipart

Hi :)

When will the multipart support be ready? Or is there any other way to let users upload files?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.