I'm not sure what to do about this one...

Right now, it looks like this library is always assuming that parameter values are case-insensitive. This is technically incorrect, as RFC 7231 says that "Parameter values might or might not be case-sensitive, depending on the semantics of the parameter name." And, in fact, there are some media types that demand case sensitivity (for example, security-related types like "application/cms").

That said, being fully spec compliant here would require a ton of work (reading every registered media type's specification?) so it probably isn't worth it. Maybe the right answer is to continue to assume case-insensitivity and then keep a list of media type parameters whose values are case-sensitive, and add to that list only as real bugs arise?

Suppose I have an Accept header that looks like:

Accept: application/vnd.api+json; ext=bulk

The output of new Negotiator(request).mediaType() would be application/vnd.api+json as expected. But when I specify an array of media types:

var negotiator = new Negotiator(request);
negotiator.mediaType(['application/vnd.api+json']); // returns `undefined`
negotiator.mediaType(['application/vnd.api+json; ext=bulk,patch']); // returns `undefined`
negotiator.mediaType(['application/vnd.api+json; ext=patch,bulk']); // returns `undefined`
negotiator.mediaType(['application/vnd.api+json; ext=bulk; supported-ext=bulk']); // returns `undefined`
negotiator.mediaType(['application/vnd.api+json; ext=bulk']); // returns `application/vnd.api+json; ext=bulk`

Which is kind of WTF because it only matches the exact parameters and I get the parameters in the media type as well. Is this expected behavior? I can see that it may be useful in extracting parameters but I expect that it should match the media type without the parameters, otherwise permutations of parameters may get huge. Possible dupe of #30.

I'm writing a Next.JS application that uses negotiator to find the best language to use from what we have translated.

This is our code currently:

  let lang = cookies.get("lang")?.value; // use the user's set language first

  if (!lang) { // if the user has not set their language, negotiate
    const acceptLanguageHeader = headers.get("Accept-Language") ?? "en-US,en;q=0.9";
    console.log(acceptLanguageHeader);
          // ^ "en-US,en;q=0.9"
    const negotiator = new Negotiator({ headers: { "Accept-Language": acceptLanguageHeader } });

    lang = negotiator.language(ALL_LOCALES) ?? "en";
                             // ^ ['cs', 'de', 'en', 'es', 'fil', 'fr', 'sv']
    console.log(lang);
          // ^ will always the first item of the array (cs) no matter what headers were given
  }

Am I doing something wrong?

I realise this is largely superficial but is there any plan to support the es6 import syntax?

Nice lib! Thanks for sharing it.

is there any reason it doesn't support a charset parameter on the accept header?

eg. Accept: 'application/json;charset=utf-8'

I might be able to put something together but I don't want to waste my time if there's a good reason it's not supported!

Thanks

Hey there,
Could you please make a release for v0.6.3? I'm trying to use the latest release of @angular/cli (13.2.0) and npm can't find this dependency.

Thanks a lot.

I was just curious about why parseMediaType() doesn't parse parameters after the q value,

//key before q
preferredMediaTypes("a/a;q=1, b/b;key=*;q=1", ["a/a","b/b"])
//b/b more preferred
//["b/b", "a/a"]

//key after q
preferredMediaTypes("a/a;q=1, b/b;q=1;key=*", ["a/a","b/b"])
//a/a more preferred
//["a/a", "b/b"]

Hi
this is typically done with

git tag 0.3.0
git push --tags

Thank you.

This was the only fully complete content negotiation library I could find. However, it's in CoffeeScript, and not precompiled, and also uses underscore a bit unnecessarily, making it not optimal for my use.

So, I ported it to JS. The result is not so bad. There are some parts that could still be a bit more abstracted out for greater DRY-ness, but they're pretty minimal.

I know sometimes people convert libs back and forth out of trollish desires, but that's not my goal here at all. I really just wanted to use this library without having to load coffee-script at run time. I changed the name, and republished it as "negotiator.js" to differentiate it: https://github.com/isaacs/negotiator.js

If you are interested in merging the two, I will be happy to continue maintaining it. If not, then feel free to close this issue, and we can live side by side in peace :)

Though * is not a valid value for an Accept header according to the HTTP spec, it is one I have encountered in the wild which caused problems for our service. The test case below shows the difference in the way negotiator handles */* and *. I would expect both values to be treated the same, the intent of * is obvious despite its divergence from spec.

accept.js

var Negotiator  = require('negotiator');
var restify     = require('restify');

var server = restify.createServer();

server.get('/', function(req, res, next){
    var neg = new Negotiator(req);
    res.json({mediaTypes: neg.mediaTypes()});
    next();
});

server.listen(9001);

test

$ node accept.js &
$ curl http://localhost:9001/ -H 'Accept: */*' && echo ""
{"mediaTypes":["*/*"]}
$ curl http://localhost:9001/ -H 'Accept: *' && echo ""
{"mediaTypes":[]}

This was originally reported to restify (restify/node-restify#1009) as an issue with their formatters and they pointed me to you as the source.

I am baffled why this is happening, but negotiator.mediaType() is returning */*. Here is my test code:

const Negotiator = require('negotiator');

const newHeader = new Headers({'Accept': 'text/turtle'});
const negotiator = new Negotiator(new Request('https://example.com/foo', { headers: newHeader}));
const mimetype = negotiator.mediaTypes();
console.log(mimetype);

The output should be ['text/turtle'] but it is ['*/*']. What's happening?

Node: v18.13.0
OS: Windows 10 x64

See #46

This comes from RFC 7231, which says: "The type, subtype, and parameter name tokens are case-insensitive."

Code to reproduce the issue:

var request = {
  headers: {
    accept: 'text/html;Charset="utf-8'
  }
};

var Negotiator = require('negotiator');
var negotiator = Negotiator(request);
var availableMediaTypes = ['text/html;charset=utf-8'];

// Expected result below: 'text/html;charset=utf-8'
// Actual result: undefined
console.log(negotiator.mediaType(availableMediaTypes))

In the example of Accept-Language Negotiation, the example is:

negotiator = new Negotiator(request)

availableLanguages = 'en', 'es', 'fr'

// Let's say Accept-Language header is 'en;q=0.8, es, pt'

negotiator.languages()
// -> ['es', 'pt', 'en']

negotiator.languages(availableLanguages)
// -> ['es', 'en']

language = negotiator.language(availableLanguages)
// -> 'es'

Now, I see that the user's most-preferred language is English (en), and English is an availableLanguage, so why would we resolve to Spanish (es)?

this is the only module in this org without 100% coverage haha

See this new failing test:

diff --git a/test/mediaType.js b/test/mediaType.js
index f20c58e..5845880 100644
--- a/test/mediaType.js
+++ b/test/mediaType.js
@@ -71,6 +71,10 @@
       accept: 'application/json, */*; q=0.01',
       provided: ['text/html', 'application/json'],
       selected: ['application/json', 'text/html']
+    }, {
+      accept: 'application/vnd.example;attribute=value',
+      provided: ['application/vnd.example;attribute=value', 'application/vnd.example;attribute=other'],
+      selected: ['application/vnd.example;attribute=value']
     }
   ];

which gives:

✖ Should return application/vnd.example;attribute=value for access header application/vnd.example;attribute=value with provided types application/vnd.example;attribute=value,application/vnd.example;attribute=other

AssertionError: [ 'application/vnd.example;attribute=other' ] deepEqual [ 'application/vnd.example;attribute=value' ]
    at Object.deepEqual (/Users/adam/Code/DVL/negotiator/node_modules/nodeunit/lib/types.js:83:39)
    at Object._this.(anonymous function) (/Users/adam/Code/DVL/negotiator/test/mediaType.js:24:12)
    at Object.<anonymous> (/Users/adam/Code/DVL/negotiator/node_modules/nodeunit/lib/core.js:236:16)
    at /Users/adam/Code/DVL/negotiator/node_modules/nodeunit/lib/core.js:236:16
    at Object.exports.runTest (/Users/adam/Code/DVL/negotiator/node_modules/nodeunit/lib/core.js:70:9)
    at /Users/adam/Code/DVL/negotiator/node_modules/nodeunit/lib/core.js:118:25
    at /Users/adam/Code/DVL/negotiator/node_modules/nodeunit/deps/async.js:513:13
    at iterate (/Users/adam/Code/DVL/negotiator/node_modules/nodeunit/deps/async.js:123:13)
    at /Users/adam/Code/DVL/negotiator/node_modules/nodeunit/deps/async.js:134:25
    at /Users/adam/Code/DVL/negotiator/node_modules/nodeunit/deps/async.js:515:17

The check seems to have been negated in the refactoring contained in

8ed7607#diff-c04cb3318b4409a34795c0b057ba0c33R73

Taking a look at our server logs and see this error pop up quite a bit:

00:41:48 web.1  | error: uncaughtException: Cannot call method 'toLowerCase' of undefined date=Wed Oct 15 2014 00:41:48 GMT+0000 (UTC), pid=1325, uid=0, gid=0, version=v0.10.32, rss=121901056, heapTotal=99441152, heapUsed=53938544, loadavg=[0.05419921875, 0.04833984375, 0.046875], uptime=13299.326321529, trace=[column=54, file=/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js, function=null, line=81, method=null, native=false, column=null, file=null, function=Array.every, line=null, method=every, native=true, column=14, file=/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js, function=specify, line=80, method=null, native=false, column=12, file=/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js, function=accepted.map.filter.sort.s, line=48, method=map.filter.sort.s, native=false, column=null, file=null, function=Array.map, line=null, method=map, native=true, column=20, file=/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js, function=getMediaTypePriority, line=47, method=null, native=false, column=21, file=/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js, function=provided.map.filter.sort.pa, line=101, method=map.filter.sort.pa, native=false, column=null, file=null, function=Array.map, line=null, method=map, native=true, column=21, file=/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js, function=preferredMediaTypes, line=100, method=null, native=false, column=12, file=/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/negotiator.js, function=Negotiator.(anonymous function), line=26, method=(anonymous function), native=false], stack=[TypeError: Cannot call method 'toLowerCase' of undefined,     at /app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js:81:54,     at Array.every (native),     at specify (/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js:80:14),     at accepted.map.filter.sort.s (/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js:48:12),     at Array.map (native),     at getMediaTypePriority (/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js:47:20),     at provided.map.filter.sort.pa (/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js:101:21),     at Array.map (native),     at preferredMediaTypes (/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/mediaType.js:100:21),     at Negotiator.(anonymous function) (/app/node_modules/express/node_modules/accepts/node_modules/negotiator/lib/negotiator.js:26:12)]

We have noticed that some browsers will send the Accept-Language header all in lower case (e.g.: fr-fr), some others will send it as fr-FR. This put a borden on the user to deal with these differences when it could be done at the lower level. Here is a repro case using npm REPL:

var n = require('negotiator/lib/language');
undefined
> n('en-GB', ['en-US', 'en-GB', 'fr-FR'])
[ 'en-GB' ]
> n('en-US', ['en-US', 'en-GB', 'fr-FR'])
[ 'en-US' ]
> n('en-us', ['en-US', 'en-GB', 'fr-FR'])
[]  // FAILS
> n('en-gb', ['en-US', 'en-GB', 'fr-FR'])
[]  // FAILS

what is the recommendation to solve this problem? can this be covered by negotiator/lib/language directly?

/cc @ericf

in middleware.js:

const locales = ["en", "tr"];
const negotiator = new Negotiator(request);
const locale = negotiator.language(locales);

curl -v --header "Accept-Language: tr" localhost:3000 redirects me to /en regardless of the accept-language header.

The whole middleware:

import Negotiator from "negotiator";

const locales = ["en", "tr"];

export function middleware(request) {
  const { pathname } = request.nextUrl;
  const pathnameHasLocale = locales.some(
    locale => pathname.startsWith(`/${locale}/`) || pathname === `/${locale}`
  );

  if (pathnameHasLocale) return;

  const negotiator = new Negotiator(request);
  const locale = negotiator.language(locales);
  request.nextUrl.pathname = `/${locale}${pathname}`;
  return Response.redirect(request.nextUrl);
};

export const config = {
  matcher: ["/((?!_next).*)"],
};

I don't really understand what I'm getting wrong here. Excuse me if I'm missing something very simple.

The JSON-LD spec uses the following accept header:

Accept: application/ld+json;profile=http://www.w3.org/ns/json-ld#expanded

When I try to use negotiator with this accept header, I'm seeing the following behaviour:

negotiator.mediaType() is application/ld+json
negotiator.mediaType(["application/ld+json"]) is undefined

negotiator.mediaType without the parameter is correct, but negotiator.mediaType with the parameter is incorrect.

This is probably linked to the profile parameter, as adding another MIME type after that works correctly:

Accept: application/ld+json;profile=http://www.w3.org/ns/json-ld#expanded, application/n-quads

negotiator.mediaTypes() is [ 'application/ld+json', 'application/n-quads' ]
negotiator.mediaTypes(["application/n-quads"]) is [ 'application/n-quads' ]

According to the spec, the value for a media type parameter can be a quoted string, but parsing Accept headers with quoted parameter values doesn't work as expected:

1. Specifically, using the internal parseMediaType function, calling parseMediaType('application/mytype+json;test="bob"') returns a parameters hash that looks like this {"test": '"bob"'}, where "bob" is quoted, even though it shouldn't be. (The spec says that the including the quotes or not in the Accept header shouldn't have an effect: "The quoted and unquoted values are equivalent.")
2. The bigger problem, though, comes when the quoted value contains a comma (which is valid under the definition of qdtext). Consider: parseAccept('application/x+json;test="bob,jill",*/*'). Currently, parseAccept function naively splits this at the commas, which leads it to try to parse it as three media types: 'application/x+json;test="bob', 'jill", and '*/*'. The second string is an invalid type, so it gets ignored, so the parsed value for the test parameter ends up as '"bob', with an extra left quote, instead of 'bob, jill'.

In https://github.com/jshttp/negotiator/blob/master/lib/negotiator.js#L21, would you be open to using more straightforward require statements to allow static tracing support?

Hello,

currently the incomming request is given to the constructur of 'Negotiator' and the 'availableMediaTypes' to 'negotiator.mediaTypes'. So for each request a new Negotiator instances needs to be created and the availableMediaTypes are prepared (at https://github.com/jshttp/negotiator/blob/master/lib/mediaType.js#L174)

Think changing the API constructor and availableMediaTypes is not possible due to backward compatibility. But maybe a new function which allows reusing the perpared priorities of the availableMediaTypes would be nice.

Thanks and kind regards,
Sven

Accept-Encoding: gzip, deflate is giving me ['deflate', 'gzip'] with:

  get acceptedEncodings() {
    var n = new Negotiator(this.req);
    return n.preferredEncodings();
  },

What is the expected behavior if there are duplicate content-types inside the Accept header.

var n = Negotiator({ headers: { accept: "application/json, application/xml, application/json" } });
n.mediaTypes();
// [ 'application/json', 'application/xml', 'application/json' ]
n.mediaTypes(['application/json', 'application/xml']);
// ['application/xml', 'application/json']
n.mediaTypes(['application/*']);
// []
n.mediaTypes(['*/*']);
// []

Does the preference of application/json decreases due to duplication? And the availableMediaTypes given to the mediaTypes() should be concrete concrete types?

The readme suggests that it should be possible to call negotiator.preferredEncoding(availableEncodings)

However, when calling preferredEncoding(['gzip']) with Accept-Encoding: deflate the return value is deflate instead of identity

Hi,

I'm from debian/ubuntu world. We are tracking this module using tags as version number for our package. In somehow you have released version 0.4.3, with no github tags, so.. we were not able to automatically recognize it.

Can you please consider to use tags when release a new version of your software?

This is really important for us.

preferredCharsets("first,second,third")
//Actual Output: ["first", "second", "third"]

preferredCharsets("first,second,third", [])
//Actual Output: []
//Should the output, instead be, ["first", "second", "third"] ?

I feel like passing undefined and passing an empty array should mean the same thing. But, I'm not sure how charset negotiation works.

This other question is more contrived,

preferredCharsets("SECOND,first,second,third", ["SeCoNd", "first"])
//Actual Output: ["first", "SeCoNd"]
//Should the output, instead be, ["SeCoNd", "first"] ?

I'm looking at this line, in particular,

It says, priority.o - spec.o. So, getCharsetPriority() will set the o value of "SeCoNd" to 2, the index of "second", instead of 0, the index of "SECOND".

If we change that line to spec.o - priority.o, the o value of "SeCoNd" becomes 0, and will come before "first".

But, of course, realistically, who's going to send such a header where there are duplicate charsets, and they're not consecutive, with lower-priority available charsets requested in between?

You shouldn't have to do anything about the second question. I just like reading random pieces of code and poking at them :x

more of a minor issue, I don't the spec mentions any sort of precedence for this but currently this fails since text/plain will come before text/html:

req.headers.accept = 'application/*;q=0.2, image/jpeg;q=0.8, text/html, text/plain';
req.accepted.should.eql(['text/html',  'text/plain', 'image/jpeg', 'application/*']);

same issue with the others

This was an issue I raised before with Express when the sorting of accept headers was still done there. Since v8 uses quicksort (which is not a stable sort) for arrays longer than 10 elements, the sort order of 'equal' elements is not guaranteed to be the same as the original array. If you put this to the test in for example preferred languages, you get the following results:

nl;q=0.5,fr,de,en,it,es,pt,no,se,fi

should sort as

fr,de,en,it,es,pt,no,se,fi,nl

likewise,

nl;q=0.5,fr,de,en,it,es,pt,no,se,fi,ro (which has 11 elements)

should sort as

fr,de,en,it,es,pt,no,se,fi,ro,nl

but instead sorts as

ro,de,en,it,fr,pt,no,se,fi,es,nl

Not sure if this should be a real problem, but I think people assume that the order of the preferred languages in their browser is honored. But of course only if they have more than 10 preferred languages.

Any thoughts on whether this is a problem or not? I see a closed issue from TJ from last year.

not sure if I'm doing something wrong, shouldn't these be sorted?

    var n = new Negotiator(this.req);
    return n.preferredMediaTypes();

where Accept is "application/*;q=0.2, image/jpeg;q=0.8, text/html" I get:

[ 'application/*', 'image/jpeg', 'text/html' ]

Hey,

I'm trying to use the Accepts module via Express (req.accepts) and thereby Negotiator to match some media types. They're in the form of application/vnd.foo.bar; v=1.

I'd like to cover the use cases of where the v is missing (to default to the latest version) and where the optional charset parameter is given. I do see some wildcard matching in lib/mediaType.js, but that only works from the requester side (HTTP Accept header in my case).

How would one do this? If it's necessary to enumerate all permutations of parameters, that sounds like an awfully lot of parsing work to be done for every request.

Thanks!

I'm curious about parseCharset(str, i) and this i parameter.
https://github.com/jshttp/negotiator/blob/master/lib/charset.js#L53

I see we have var i=0 and i++ here,
https://github.com/jshttp/negotiator/blob/master/lib/charset.js#L61

And at the very end, we have,
https://github.com/jshttp/negotiator/blob/master/lib/charset.js#L73

return {
  charset: charset,
  q: q,
  i: i,
};

It seems to me like it's possible for the i at the return statement to have a different value from the i given in the parameter. It's different because var i=0 and i++ modify the value of the parameter.

For example,

function parseCharset (str , i) {
    for (var i = 0; i < 4; i ++) {
    	console.log(i);
    }

    return {
    	i: i
    };
}
//The output is {i: 4}, not {i: 99}
parseCharset("", 99)

If we replace var i with let i, we get {i: 99}

Right now, I'm seeing i's value change between the start and the end of the function. Is this intentional? Or is this a mistake?

If it's intentional, my question is, what does it do, exactly? The code's a bit hard for me to read so I can't quite get the intent of it just yet.

I'd really like to be able to use this module by just using its pure functions.

It is feasible by requiring those functions directly:

const preferredEncodings = require('negociator/lib/encoding');

Do you think it is safe or should we add a standard way to use those functions?

Let's make the request

Accept-Language: zh, zh-CN;q=0.9

and
We provided backlist: ['zh-CN', 'zh-TW']

Coding:

const n = new require('negotiator')({ headers: { 'accept-language': 'zh, zh-CN;q=0.9' } })
n.language(['zh-CN', 'zh-TW'])

Actual:
'zh-TW'

Expect:
'zh-CN' for sure

Analysis:
In the code, we get two comparing code snippets:

function getLanguagePriority(language, accepted, index) {
    var priority = {
        o: -1,
        q: 0,
        s: 0
    };

    for (var i = 0; i < accepted.length; i++) {
        var spec = specify(language, accepted[i], index);

        // this means `s`(the matching level) > `q`(the quality) > `o`(the index of accepted)
        if (spec && (priority.s - spec.s || priority.q - spec.q || priority.o - spec.o) < 0) {
            priority = spec;
        }
    }

    return priority;
}

But when codes come here

function compareSpecs(a, b) {
    // `q`(the quality) > `s`(the matching level) > `o`(the accepted index) > `i`(comparing index)
    return (b.q - a.q) || (b.s - a.s) || (a.o - b.o) || (a.i - b.i) || 0;
}

WHICH LEADS:
when comparing zh-CN to accepted zh, zh-CN;q=0.9,
we get a high matching level zh-CN with a lower quality weight 0.9

and then comparing zh-TW to accepted zh, zh-CN;q=0.9,
we get the the only half-matched zh with a high quality weight 1

finally, we use compareSpecs to select a higher quality language zh-TW rather than the lower but 100% percent matched one zh-CN

Read the rfc4647, it is not defined whether this expectation should be reasonable.
It only reads:

If the user cannot be sure which scheme is being used (or
if more than one might be applied to a given request), the user
SHOULD specify the most specific (largest number of subtags) range
first and then supply shorter prefixes later in the list to ensure
that filtering returns a complete set of tags.

So the accepted language with zh, zh-CN;q=0.9 is not comformed to this user decision.
But I think, the comparing logic should be the same ( q > s > o), such that

n.languages(['zh-CN', 'zh-HK']) // returns ['zh-CN', 'zh-HK']
n.languages(['zh-HK', 'zh-CN']) // returns ['zh-HK', 'zh-CN']

Hello,

I found a bug of your regex.

If I give some additional attribute in the accept header like the following, the header can not be correctly parsed.
e.g.,

application/xhtml+xml;profile="http://www.wapforum.org/xhtml"

The result of regex match is:

type = application/xhtml+xml;profile="http://www.wapforum.org
subtype = xhtml"

Of course, it must not happen.
However, if a whitespace exists before the attribute, then regex works correctly again.

application/xhtml+xml; profile="http://www.wapforum.org/xhtml"

Best regards,
Kevin

In this repo, the box in the top right says Used by: 4.4m or 4.4 million packages. That plus the fact that this has been around for years suggests to me that it's ready for a stable API. Like many, I rely on this indirectly from expressjs in various projects. It's one of the few dependencies of express which hasn't hit 1.0 per http://npm.broofa.com/?q=express.

Thoughts on blockers?

probably gonna do this eventually. i want to cleanup the code so we're not creating closures everywhere. i.e. filter( e => e) can just be filter(Boolean)