Giter Club home page Giter Club logo

http2-wrapper's Introduction

http2-wrapper

HTTP/2 client, just with the familiar https API

Node CI codecov npm install size

This package was created to support HTTP/2 without the need to rewrite your code.
I recommend adapting to the http2 module if possible - it's much simpler to use and has many cool features!

Tip: http2-wrapper is very useful when you rely on other modules that use the HTTP/1 API and you want to support HTTP/2.

Pro Tip: While the native http2 doesn't have agents yet, you can use http2-wrapper Agents and still operate on the native HTTP/2 streams.

Installation

$ npm install http2-wrapper
$ yarn add http2-wrapper

Usage

const http2 = require('http2-wrapper');

const options = {
	hostname: 'nghttp2.org',
	protocol: 'https:',
	path: '/httpbin/post',
	method: 'POST',
	headers: {
		'content-length': 6
	}
};

const request = http2.request(options, response => {
	console.log('statusCode:', response.statusCode);
	console.log('headers:', response.headers);

	const body = [];
	response.on('data', chunk => {
		body.push(chunk);
	});
	response.on('end', () => {
		console.log('body:', Buffer.concat(body).toString());
	});
});

request.on('error', console.error);

request.write('123');
request.end('456');

// statusCode: 200
// headers: [Object: null prototype] {
//   ':status': 200,
//   date: 'Fri, 27 Sep 2019 19:45:46 GMT',
//   'content-type': 'application/json',
//   'access-control-allow-origin': '*',
//   'access-control-allow-credentials': 'true',
//   'content-length': '239',
//   'x-backend-header-rtt': '0.002516',
//   'strict-transport-security': 'max-age=31536000',
//   server: 'nghttpx',
//   via: '1.1 nghttpx',
//   'alt-svc': 'h3-23=":4433"; ma=3600',
//   'x-frame-options': 'SAMEORIGIN',
//   'x-xss-protection': '1; mode=block',
//   'x-content-type-options': 'nosniff'
// }
// body: {
//   "args": {},
//   "data": "123456",
//   "files": {},
//   "form": {},
//   "headers": {
//     "Content-Length": "6",
//     "Host": "nghttp2.org"
//   },
//   "json": 123456,
//   "origin": "xxx.xxx.xxx.xxx",
//   "url": "https://nghttp2.org/httpbin/post"
// }

API

Note: The session option was renamed to tlsSession for better readability.

Note: The timeout option applies to HTTP/2 streams only. In order to set session timeout, pass an Agent with custom timeout option set.

http2.auto(url, options, callback)

Performs ALPN negotiation. Returns a Promise giving proper ClientRequest instance (depending on the ALPN).

Note: The agent option represents an object with http, https and http2 properties.

const http2 = require('http2-wrapper');

const options = {
	hostname: 'httpbin.org',
	protocol: 'http:', // Try changing this to https:
	path: '/post',
	method: 'POST',
	headers: {
		'content-length': 6
	}
};

(async () => {
	try {
		const request = await http2.auto(options, response => {
			console.log('statusCode:', response.statusCode);
			console.log('headers:', response.headers);

			const body = [];
			response.on('data', chunk => body.push(chunk));
			response.on('end', () => {
				console.log('body:', Buffer.concat(body).toString());
			});
		});

		request.on('error', console.error);

		request.write('123');
		request.end('456');
	} catch (error) {
		console.error(error);
	}
})();

// statusCode: 200
// headers: { connection: 'close',
//   server: 'gunicorn/19.9.0',
//   date: 'Sat, 15 Dec 2018 18:19:32 GMT',
//   'content-type': 'application/json',
//   'content-length': '259',
//   'access-control-allow-origin': '*',
//   'access-control-allow-credentials': 'true',
//   via: '1.1 vegur' }
// body: {
//   "args": {},
//   "data": "123456",
//   "files": {},
//   "form": {},
//   "headers": {
//     "Connection": "close",
//     "Content-Length": "6",
//     "Host": "httpbin.org"
//   },
//   "json": 123456,
//   "origin": "xxx.xxx.xxx.xxx",
//   "url": "http://httpbin.org/post"
// }

http2.auto.protocolCache

An instance of quick-lru used for ALPN cache.

There is a maximum of 100 entries. You can modify the limit through protocolCache.maxSize - note that the change will be visible globally.

http2.auto.createResolveProtocol(cache, queue, connect)

cache

Type: Map<string, string>

This is the store where cached ALPN protocols are put into.

queue

Type: Map<string, Promise>

This is the store that contains pending ALPN negotiation promises.

connect

Type: (options, callback) => TLSSocket | Promise<TLSSocket>

See https://github.com/szmarczak/resolve-alpn#connect

http2.auto.resolveProtocol(options)

Returns a Promise<{alpnProtocol: string}>.

http2.request(url, options, callback)

Same as https.request.

options.h2session

Type: Http2Session

The session used to make the actual request. If none provided, it will use options.agent to get one.

http2.get(url, options, callback)

Same as https.get.

new http2.ClientRequest(url, options, callback)

Same as https.ClientRequest.

new http2.IncomingMessage(socket)

Same as https.IncomingMessage.

new http2.Agent(options)

Note: this is not compatible with the classic http.Agent.

Usage example:

const http2 = require('http2-wrapper');

class MyAgent extends http2.Agent {
	createConnection(origin, options) {
		console.log(`Connecting to ${http2.Agent.normalizeOrigin(origin)}`);
		return http2.Agent.connect(origin, options);
	}
}

http2.get({
	hostname: 'google.com',
	agent: new MyAgent()
}, response => {
	response.on('data', chunk => console.log(`Received chunk of ${chunk.length} bytes`));
});

options

Each option is an Agent property and can be changed later.

timeout

Type: number
Default: 0

If there's no activity after timeout milliseconds, the session will be closed. If 0, no timeout is applied.

maxSessions

Type: number
Default: Infinity

The maximum amount of sessions in total.

maxEmptySessions

Type: number
Default: 10

The maximum amount of empty sessions in total. An empty session is a session with no pending requests.

maxCachedTlsSessions

Type: number
Default: 100

The maximum amount of cached TLS sessions.

agent.protocol

Type: string
Default: https:

agent.settings

Type: object
Default: {enablePush: false}

Settings used by the current agent instance.

agent.normalizeOptions(options)

Returns a string representing normalized options.

Agent.normalizeOptions({servername: 'example.com'});
// => ':::::::::::::::::::::::::::::::::::::'

agent.getSession(origin, options)

Type: string URL object

Origin used to create new session.

Type: object

Options used to create new session.

Returns a Promise giving free Http2Session. If no free sessions are found, a new one is created.

A session is considered free when pending streams count is less than max concurrent streams settings.

agent.getSession(origin, options, listener)

listener

Type: object

{
	reject: error => void,
	resolve: session => void
}

If the listener argument is present, the Promise will resolve immediately. It will use the resolve function to pass the session.

Returns a Promise giving Http2Stream.

agent.createConnection(origin, options)

Returns a new TLSSocket. It defaults to Agent.connect(origin, options).

agent.closeEmptySessions(count)

count

Type: number Default: Number.POSITIVE_INFINITY

Makes an attempt to close empty sessions. Only sessions with 0 concurrent streams will be closed.

agent.destroy(reason)

Destroys all sessions.

agent.emptySessionCount

Type: number

A number of empty sessions.

agent.pendingSessionCount

Type: number

A number of pending sessions.

agent.sessionCount

Type: number

A number of all sessions held by the Agent.

Event: 'session'

agent.on('session', session => {
	// A new session has been created by the Agent.
});

Proxy support

Currently http2-wrapper provides support for these proxies:

  • HttpOverHttp2
  • HttpsOverHttp2
  • Http2OverHttp2
  • Http2OverHttp
  • Http2OverHttps

Any of the above can be accessed via http2wrapper.proxies. Check out the examples/proxies directory to learn more.

Note: If you use the http2.auto function, the real IP address will leak. http2wrapper is not aware of the context. It will create a connection to the end server using your real IP address to get the ALPN protocol. Then it will create another connection using proxy. To migitate this, you need to pass a custom resolveProtocol function as an option:

const resolveAlpnProxy = new URL('https://username:password@localhost:8000');
const connect = async (options, callback) => new Promise((resolve, reject) => {
	const host = `${options.host}:${options.port}`;

	(async () => {
		try {
			const request = await http2.auto(resolveAlpnProxy, {
				method: 'CONNECT',
				headers: {
					host
				},
				path: host,

				// For demo purposes only!
				rejectUnauthorized: false,
			});

			request.end();

			request.once('error', reject);

			request.once('connect', (response, socket, head) => {
				if (head.length > 0) {
					reject(new Error(`Unexpected data before CONNECT tunnel: ${head.length} bytes`));

					socket.destroy();
					return;
				}

				const tlsSocket = tls.connect({
					...options,
					socket
				}, callback);

				resolve(tlsSocket);
			});
		} catch (error) {
			reject(error);
		}
	})();
});

// This is required to prevent leaking real IP address on ALPN negotiation
const resolveProtocol = http2.auto.createResolveProtocol(new Map(), new Map(), connect);

const request = await http2.auto('https://httpbin.org/anything', {
	agent: {},
	resolveProtocol
}, response => {
	// Read the response here
});

request.end();

See unknown-over-unknown.js to learn more.

Mirroring another server

See examples/proxies/mirror.js for an example.

See examples/ws for an example.

Push streams

See examples/push-stream for an example.

Related

  • got - Simplified HTTP requests
  • http2-proxy - A simple http/2 & http/1.1 spec compliant proxy helper for Node.

License

MIT

http2-wrapper's People

Contributors

ambujsahu81 avatar pimterry avatar szmarczak avatar xamgore avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

http2-wrapper's Issues

Breaking builds

Latest version is breaking builds. I've installed got, not using http2-wrapper directly, and nodejs15 is required.

`http2.auto.protocolCache` should be a Map

It needs to be a Map (or something else) so it's easier to manage the cache. For example, tests

  • host option is an alternative to hostname option
  • cache hostname defaults to localhost

should be serial. At first they should wipe the cache. Then, it should make a request and make sure the ALPN protocol has been cached.

Additionally, users may want to clear the cache or increase the limit. https://github.com/sindresorhus/quick-lru looks good.

H2-over-H1 Proxy question

Run the example code h2-over-h1.js

Result:

Error: Requested origin https://httpbin.org does not match server https://httpbin.org:8765

at ClientHttp2Session.<anonymous> (/home/chandler/projects/seckill/http2-wrapper/source/agent.js:535:22)

at Object.onceWrapper (events.js:422:26)

at ClientHttp2Session.emit (events.js:315:20)

at Http2Session.onSettings (internal/http2/core.js:536:11) {stack: 'Error: Requested origin https://httpbin.org d…on.onSettings (internal/http2/core.js:536:11)', message: 'Requested origin https://httpbin.org does not match server https://httpbin.org:8765'}

the port 8765 is Charles Proxy port

``got`` example does not work

const got = require('got');
const {request} = require('http2-wrapper');

const h2got = got.extend({request});

(async () => {
	const {body} = await h2got('https://nghttp2.org/httpbin/headers');
	console.log(body);
})();

err

RequestError: Cannot read property 'on' of undefined
at EventEmitter.cacheReq.on.error (...node_modules/got/source/request-as-event-emitter.js:155:27)
at EventEmitter.emit (events.js:182:13)
at makeRequest (...node_modules/cacheable-request/src/index.js:126:9)
at get (...node_modules/cacheable-request/src/index.js:136:14)

Tests sometimes time out at the very beginning

Commit: 0eb1f3f

Logs:

szm@solus ~/Desktop/http2-wrapper $ ava test/agent.js -v -s

(node:18356) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
  
  ✖ Timed out while running tests

  54 tests were pending in test/agent.js

  ◌ caches a TLS session when successfully connected
  ◌ reuses a TLS session
  ◌ purges the TLS session cache on session error
  ◌ getSession() - passing string as `origin`
  ◌ getSession() - passing URL as `origin`
  ◌ `timeout` option
  ◌ `timeout` option - endless response
  ◌ `.settings` property
  ◌ `session` event
  ◌ `protocol` property
  ◌ `.sessions` may contain destroyed sessions
  ◌ `.sessions` may contain closed sessions
  ◌ sessions are grouped into options and authorities`
  ◌ prevents session overloading #1
  ◌ prevents session overloading #2
  ◌ prevents session overloading #3
  ◌ sessions can be manually overloaded
  ◌ respects the `maxSessions` option
  ◌ creates new session if there are no free sessions
  ◌ free sessions can become suddenly covered by shrinking their current streams count
  ◌ busy sessions can become suddenly covered by shrinking their current streams count
  ◌ session can cover another session by increasing its streams count limit
  ◌ closes covered sessions - `origin` event
  ◌ closes covered sessions - session no longer busy
  ◌ graceful close works
  ◌ does not close covered sessions if the current one is full
  ◌ no negative session count
  ◌ properly calculates session count #1
  ◌ properly calculates session count #2
  ◌ properly calculates session count #3
  ◌ properly calculates session count #4
  ◌ uses sessions which are more loaded to use fewer connections
  ◌ sessions are picked in an optimal way #1
  ◌ sessions are picked in an optimal way #2
  ◌ picks sessions with the highest stream capacity (single origin)
  ◌ picks sessions with the highest stream capacity (many origins)
  ◌ does not create a new session if there exists an authoritive one
  ◌ session may become free on maxConcurrentStreams update
  ◌ gives free sessions if available
  ◌ throws on servername mismatch
  ◌ throws if session is closed before receiving a SETTINGS frame
  ◌ newly queued sessions should not throw after `agent.destroy()`
  ◌ errors on failure
  ◌ catches session.request() errors
  ◌ no infinity loop on endpoint mismatch
  ◌ `.closeEmptySessions()` works
  ◌ respects `.maxEmptySessions` changes
  ◌ closes empty sessions automatically
  ◌ `maxEmptySessions` set to 0 causes to close the session after running through the queue
  ◌ gives the queued session if exists
  ◌ processes session queue on session close
  ◌ `agent.destroy()` destroys free sessions
  ◌ `agent.destroy()` destroys busy sessions
  ◌ `agent.destroy()` makes pending sessions throw

Should Push responses be cached?

Push streams are very useful while using a browser. We receive stylesheets, scripts, HTML content simultaneously.

Should Push responses be cached? Is there any real use case?

Http2 and cache issue

I don't know if I had to put this on Got or here but there is still a problem with http2 and cache option

/Users/devchris/Desktop/project/node_modules/responselike/src/index.js:18
			throw new TypeError('Argument `url` should be a string');
         ^
TypeError: Argument `url` should be a string
    at new Response (/Users/devchris/Desktop/project/node_modules/responselike/src/index.js:18:10)
    at ClientRequest.handler (/Users/devchris/Desktop/project/node_modules/cacheable-request/src/index.js:89:19)
    at Object.onceWrapper (events.js:421:26)
    at ClientRequest.emit (events.js:326:22)
    at ClientRequest.EventEmitter.emit (domain.js:486:12)
    at ClientRequest.origin.emit (/Users/devchris/Desktop/project/node_modules/@szmarczak/http-timer/dist/source/index.js:39:20)
    at /Users/devchris/Desktop/project/node_modules/http2-wrapper/source/client-request.js:319:16
    at ClientHttp2Stream.<anonymous> (/Users/devchris/Desktop/project/node_modules/http2-wrapper/source/client-request.js:262:7)
    at Object.onceWrapper (events.js:421:26)
    at ClientHttp2Stream.emit (events.js:314:20)

Provide equivalent of Agent to manage ClientHttp2Sessions across requests

To get to a fully transparent solution, on par with Agent for h1, we should look at providing a ClientHttp2Session manager that takes on the following responsibilities:

  1. Establishing new connections when no connection is currently established or when the maximum concurrent streams has been reached on an existing ClientHttp2Session
  2. Performing ALPN negotiation so we can transparently support both h1 and h2, without knowing upfront what the remote host supports
  3. Detecting failed or closed sessions, and re-establishing them when needed
  4. Have some forms of policing, like maximum simultaneous outgoing connections, socket timeouts etc
  5. It would be great if it could also expose some metrics and statistics: how many h1 vs h2 connections, idle vs in use connections, multiplexing rate etc

We should expose both an Agent and new H2-oriented interface, so this connection pool can transparently support both protocols. This is necessary when we're connecting to a remote host and determine that it does not support h2, then we like to reuse that existing TLS connection over h1. If we expose an Agent interface`, we can reuse the TLS connection.

Ideally, this component is also DNS-aware, meaning that it could do the following:

  • If DNS resolution indicates the remote host has multiple IP addresses, re-attempt connecting to a different IP if one connection fails. This makes sure that remote outages are handled more gracefully.
  • If multiple connections need to be established to a remote host, do so across multiple of the available IPs so the load gets distributed.

Broken with Node.js 14.0.0

 node -v
v14.0.0

package.json

{
  "name": "temp",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "http2-wrapper": "^1.0.0-beta.4.4"
  }
}

Use http2-wrapper to send the request.After that, there is no response and the nodejs program exits directly.

const http2 = require("http2-wrapper");

const options = {
    hostname: "translate.google.cn",
    protocol: "https:", // Note the `http:` protocol here
    path: "/",
    method: "GET",
    headers: {},
};
(async () => {
    try {
        const request = await http2.auto(options, (response) => {
            console.log("statusCode:", response.statusCode);
            console.log("headers:", response.headers);

            const body = [];
            response.on("data", (chunk) => body.push(chunk));
            response.on("end", () => {
                console.log("body:\n", Buffer.concat(body).toString());
            });
        });
        console.log(request);
        request.on("error", console.error);

        request.end();
    } catch (error) {
        console.error(error);
    }
})();

Use the native http2 module as follows. Successful response.

const http2 = require("http2");

const {
    HTTP2_HEADER_STATUS,
    HTTP2_HEADER_PATH,
    HTTP2_HEADER_METHOD,
} = http2.constants;
const client = http2.connect("https://translate.google.cn", {});
client.on("error", (err) => console.error(err));

const req = client.request({
    [HTTP2_HEADER_PATH]: "/",
    [HTTP2_HEADER_METHOD]: "GET",
});
console.log(req);
req.on("response", (headers, flags) => {
    console.log("statusCode:", headers[HTTP2_HEADER_STATUS]);
    console.log("headers:", headers);
});

req.setEncoding("utf8");
let data = "";
req.on("data", (chunk) => {
    data += chunk;
});
req.on("end", () => {
    console.log(`body:\n${data}`);
    client.close();
});
req.end();
Http2Stream {
  id: '<pending>',
  closed: false,
  destroyed: false,
  state: {},
  readableState: ReadableState {
    objectMode: false,
    highWaterMark: 16384,
    buffer: BufferList { head: null, tail: null, length: 0 },
    length: 0,
    pipes: [],
    flowing: null,
    ended: false,
    endEmitted: false,
    reading: false,
    sync: true,
    needReadable: false,
    emittedReadable: false,
    readableListening: false,
    resumeScheduled: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    destroyed: false,
    errored: false,
    closed: false,
    defaultEncoding: 'utf8',
    awaitDrainWriters: null,
    multiAwaitDrain: false,
    readingMore: true,
    decoder: null,
    encoding: null,
    [Symbol(kPaused)]: null
  },
  writableState: WritableState {
    objectMode: false,
    highWaterMark: 16384,
    finalCalled: true,
    needDrain: false,
    ending: true,
    ended: true,
    finished: false,
    destroyed: false,
    decodeStrings: false,
    defaultEncoding: 'utf8',
    length: 0,
    writing: false,
    corked: 0,
    sync: true,
    bufferProcessing: false,
    onwrite: [Function: bound onwrite],
    writecb: null,
    writelen: 0,
    afterWriteTickInfo: null,
    bufferedRequest: null,
    lastBufferedRequest: null,
    pendingcb: 1,
    prefinished: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    errored: false,
    closed: false,
    bufferedRequestCount: 0,
    corkedRequestsFree: {
      next: null,
      entry: null,
      finish: [Function: bound onCorkedFinish]
    }
  }
}
statusCode: 200
headers: [Object: null prototype] {
  ':status': 200,
  date: 'Sat, 25 Apr 2020 01:52:08 GMT',
  pragma: 'no-cache',
  expires: 'Fri, 01 Jan 1990 00:00:00 GMT',
  'cache-control': 'no-cache, must-revalidate',
  'x-frame-options': 'DENY',
  'content-type': 'text/html; charset=GB2312',
  'content-language': 'zh-CN',
  p3p: 'CP="This is not a P3P policy! See g.co/p3phelp for more info."',
  'x-content-type-options': 'nosniff',
  server: 'HTTP server (unknown)',
  'x-xss-protection': '0',
  'set-cookie': [
    'NID=203=NUm7GCDy0VPapvOEizE7Pt62uYET8mFwLWNM227xdaILpjliZaKQmEDchyBbrxVLZ0fIbKEBNUOcfXKuYmhrsHcl7mY2TCHD5koo83FgCSGfYqneog86gQtmmL9N9YBcMx8CeV2uMaU5SlpXKduR_O2aArx62n1uLlu9N21nkQ0; expires=Sun, 25-Oct-2020 01:52:08 GMT; path=/; domain=.google.cn; HttpOnly'
  ],
  'alt-svc': 'quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000',
  'accept-ranges': 'none',
  vary: 'Accept-Encoding'
}
body:

Proxy Error

I tried this example https://github.com/szmarczak/http2-wrapper/blob/master/examples/proxy/h2-over-h2.js
and I got this error.

(node:77358) UnhandledPromiseRejectionWarning: Error: Session closed without receiving a SETTINGS frame
    at ClientHttp2Session.<anonymous> (/Users/devchris/Desktop/project/node_modules/http2-wrapper/source/agent.js:396:22)
    at Object.onceWrapper (events.js:420:28)
    at ClientHttp2Session.emit (events.js:314:20)
    at ClientHttp2Session.EventEmitter.emit (domain.js:486:12)
    at emitClose (internal/http2/core.js:999:8)
    at internal/http2/core.js:1036:7
    at TLSSocket.onfinish (_stream_writable.js:677:5)
    at TLSSocket.emit (events.js:314:20)
    at TLSSocket.EventEmitter.emit (domain.js:486:12)
    at finish (_stream_writable.js:645:10)

I thought it was the proxy that doesn't support http2 but the provider tells me that the proxy actually supports it.

What do you think?

Thanks

Don't use H2 detection preflights & caching in Node versions where it's not required

As far as I can tell, the TLS preflight requests and detected protocol caching in https://github.com/szmarczak/http2-wrapper/blob/master/source/auto.js exists purely because of nodejs/node#33343. Is that right?

This bug doesn't exist in many of the recent Node versions (e.g. in my case I'm using Node 14.6). It would be nice to skip all that logic in those cases. That has two main benefits:

  • Saves a duplicate TLS handshake and the associated RTT for the first connections to each host and for hosts that fall out of the cache.
  • No cache invalidation issues. In my case, my tests just started failing because they reuse a port for some test servers, not all of which have the same H2 configuration, so cached initial values broke later requests.
  • Simplifies the code in those cases, and starts towards a path where this can be dropped completely and simplify the code fully (although I know that can't happen any time soon).

I've added some notes on the node issue about the fix. I think this is unnecessary from Node 14.3+, and I'm hopeful we can get it sorted in an v12 release sometime soon too. What do you think?

resolve-alpn

resolve-alpn should be installed as dependencies not dev one

It takes a long time to make the first request

With got v11:

> console.time('done');got('https://facebook.com').then(x => console.timeEnd('done'))
Promise { <pending> }
> done: 302.025ms
> console.time('done');got('https://facebook.com', { http2: true }).then(x => console.timeEnd('done'))
Promise { <pending> }
> done: 60188.968ms

> console.time('done');got('https://twitter.com').then(x => console.timeEnd('done'))
Promise { <pending> }
> done: 2343.526ms

> console.time('done');got('https://twitter.com', { http2: true }).then(x => console.timeEnd('done'))
Promise { <pending> }
> done: 5333.237ms

> console.time('done');got('https://google.com').then(x => console.timeEnd('done'))
Promise { <pending> }
> done: 983.180ms
> console.time('done');got('https://google.com', { http2: true }).then(x => console.timeEnd('done'))
Promise { <pending> }
> done: 1520.592ms

Not sure what's causing it. Pages are speedy when I load them in Chrome.

`agent.request()` is unsafe

resolve(session.request(headers, streamOptions));

session.request(headers, streamOptions) may emit an error event, which will be unhandled if resolve(...) has been called but it didn't go past await agent.request() yet. I have no idea how to reproduce this yet, but happened to me yesterday.

Prevent overloading sessions

Assume the server accepts only 10 requests per session. You make 20 requests at once - the first 10 are successful and the other 10 are rejected (each of them would emit an error).

We can prevent this from happening in future by caching session.remoteSettings.maxConcurrentStreams.

The other solution would be to move the queued requests once we receive the SETTINGS frame, we wouldn't have to cache anything this way.

RangeError: Invalid WebSocket frame: RSV2 and RSV3 must be clear

Follow up #46

CONNECTED!
RangeError: Invalid WebSocket frame: RSV2 and RSV3 must be clear
    at Receiver.getInfo (/Users/devchris/Library/Mobile Documents/com~apple~CloudDocs/Documents/code/nodejs/http2-wrapper/node_modules/ws/lib/receiver.js:171:14)
    at Receiver.startLoop (/Users/devchris/Library/Mobile Documents/com~apple~CloudDocs/Documents/code/nodejs/http2-wrapper/node_modules/ws/lib/receiver.js:131:22)
    at Receiver._write (/Users/devchris/Library/Mobile Documents/com~apple~CloudDocs/Documents/code/nodejs/http2-wrapper/node_modules/ws/lib/receiver.js:78:10)
    at writeOrBuffer (node:internal/streams/writable:395:12)
    at Receiver.Writable.write (node:internal/streams/writable:340:10)
    at ClientHttp2Stream.socketOnData (/Users/devchris/Library/Mobile Documents/com~apple~CloudDocs/Documents/code/nodejs/http2-wrapper/node_modules/ws/lib/websocket.js:900:35)
    at ClientHttp2Stream.emit (node:events:376:20)
    at addChunk (node:internal/streams/readable:311:12)
    at readableAddChunk (node:internal/streams/readable:286:9)
    at ClientHttp2Stream.Readable.push (node:internal/streams/readable:225:10) {
  [Symbol(status-code)]: 1002
}
DISCONNECTED!

Uncaught TypeError: Cannot read property 'node' of undefined

node -v
v12.8.1

npm -v
6.12.0

webpack:///./node_modules/http2-wrapper/source/utils/is-compatible.js
const version = process.versions.node.split('.');

is-compatible.js:3 Uncaught TypeError: Cannot read property 'node' of undefined
    at Object.<anonymous> (is-compatible.js:3)
    at Object../node_modules/http2-wrapper/source/utils/is-compatible.js (is-compatible.js:5)
    at __webpack_require__ (bootstrap:723)
    at fn (bootstrap:100)
    at Object../node_modules/http2-wrapper/source/index.js (index.js:2)
    at __webpack_require__ (bootstrap:723)
    at fn (bootstrap:100)
    at Object../src/utils/request2.js (request2.js:11)
    at __webpack_require__ (bootstrap:723)
    at fn (bootstrap:100)

to Add support for keepalive option for HTTP2.request

to Add support for keepalive option for HTTP2.request

In a period of time, if you initiate multiple requests to the same server, you can reuse the same http2 session.

const http2 = require('http2-wrapper');

const options = {
keepAlive:true,
	hostname: 'nghttp2.org',
	protocol: 'https:',
	path: '/httpbin/post',
	method: 'POST',
	headers: {
		'content-length': 6
	}
};
const timer=setInterval(()=>{
const request = http2.request(options, response => {})

},1000)

Versioning

Should we go with semantic versioning or mirror Node.js versions? E.g. if we know that the current state is fully compatible with 14.11.0, release @szmarczak/[email protected]. DefinitelyTyped does this with @types/node.

/cc @sindresorhus

Don't wait for 'finish' before 'response'

H2 client requests wait for the request body writing to finish before they fire response events (or headers or trailers events):

// Wait for the `finish` event. We don't want to emit the `response` event
// before `request.end()` is called.
const waitForEnd = fn => {
return (...args) => {
if (!this.writable && !this.destroyed) {
fn(...args);
} else {
this.once('finish', () => {
fn(...args);
});
}
};
};
// This event tells we are ready to listen for the data.
stream.once('response', waitForEnd((headers, flags, rawHeaders) => {

(Implemented in 2d6279a, apparently for proxying purposes).

This doesn't match Node's HTTP/1 behaviour, and that's breaking my application. Servers can respond before the body for all sorts of reasons, like rejecting uploads before the body is completed. In my case, I'm proxying mixed HTTP/1 & HTTP/2 traffic, and some of that traffic talks to Google's firebase API, which does some interesting two-way communication by reading & writing response bodies, and requires clients to receive response headers before the request is fully completed.

Here's a pure HTTP/1 example where response fires without finish:

const http = require('http');

const server = http.createServer((req, res) => {
    console.log('got request');

    // Don't wait for the full body - reply immediately:
    res.writeHead(200);
    res.flushHeaders(); // .end() also works, but stops the request completely
});

server.listen(5050, () => {
    const req = http.request('http://localhost:5050', {
        method: 'POST'
    });

    req.on('response', (res) => console.log('response:', res.statusCode));
    req.on('finish', () => console.log('req finished'));

    req.write('start writing');
    setTimeout(() => {
        console.log('end');
        req.end('end');
    }, 100);
});

It outputs:

got request
response: 200
end
req finished

You can fix this very easily by just removing waitForEnd, but based on the original commit that presumably breaks something related to proxying? I'm not sure about the details of that, but in my application with that change it all seems to work great.

Add tests for proxies

TODO before writing tests:

  • Make the constructor of Http2OverHttpX use the common initialize function
  • getSession() should create new proxy stream ONLY when needed, not on every call!
  • Rethink the structure of this.queue and this.sessions

This dependency was not found:

ERROR  Failed to compile with 3 errors                                                       17:41:26

This dependency was not found:

* http2 in ./node_modules/http2-wrapper/source/index.js, ./node_modules/http2-wrapper/source/client-re
quest.js and 1 other

To install it, you can run: npm install --save http2
 

`ClientRequest` incorrectly guesses the request origin

options.origin = `https://${options.host}:${options.port}`;

As you can see the port is always visible, while it shouldn't e.g. https://google.com:443.

This is one of the right ways to normalize it (the following is very slow):

static normalizeOrigin(url, servername) {
if (typeof url === 'string') {
url = new URL(url);
}
if (servername && url.hostname !== servername) {
url.hostname = servername;
}
return url.origin;
}

Agent vs Pool

A Pool would be a bit simpler than an Agent. One queue, one sessions array. That means we wouldn't have to normalize connect options - these would be passed in the constructor options. We could go for that approach if Agent turns out to be slow in production. On the other hand, Agent looks much more like an HTTP/1 Agent.

No HTTP (without TLS) support

I used it like this:

const got = require('got');
const {request} = require('http2-wrapper');
const h2got = got.extend({request});

const currentRequestPromise = h2got.post("http://enhcvrwpf7kw6.x.pipedream.net", {
  headers: {
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ query: "Hello" }),
})

currentRequestPromise
      .then(({ body, json }) => {
        console.log({"body": body})
        console.log({"json": json})
      })
      .catch(error => {
        console.log({"error": error})
      })

And I found the error from the code here:

if (options.protocol && options.protocol !== 'https:') {
throw new ERR_INVALID_PROTOCOL(options.protocol, 'https:');
}

As http is mentioned in the README I thought it would be possible but it looks like https is required. Is there a reason for that?

Agent tests sometimes time out (never resolve?)

Example: https://travis-ci.org/szmarczak/http2-wrapper/jobs/558875630

  ✖ Timed out while running tests
  12 tests were pending in /home/travis/build/szmarczak/http2-wrapper/test/agent.js
  ◌ agent › `timeout` option
  ◌ agent › passing string as `authority`
  ◌ agent › passing options as `authority`
  ◌ agent › sessions are not busy if still can make requests
  ◌ agent › sessions are busy when cannot make requests
  ◌ agent › gives free sessions if available
  ◌ agent › gives the queued session if exists
  ◌ agent › `maxSessions` option
  ◌ agent › can destroy free sessions
  ◌ agent › creates new session if there are no free sessions
  ◌ agent › can destroy busy sessions
  ◌ agent › `closeFreeSessions()` closes sessions with 0 pending streams only

I couldn't reproduce the issue yet.

This happens only on Node 11:

$ node --version
v11.15.0
$ npm --version
6.7.0
$ nvm --version
0.34.0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.