Node.js framework agnostic library that enables you to forward an http request to another HTTP server. Supported protocols: HTTP, HTTPS, HTTP2
Home Page: https://www.npmjs.com/package/fast-proxy
License: MIT License
As this module is now maintained by the fastify organisation members. We should remove the .github folder including the FUNDING.yml configuration.
undici timeout err.code=UND_ERR_REQUEST_TIMEOUT is not handled properly, therefore instead of getting error 504 I get error 500
We need to add in index.js line 88 the undici timeout error code as well:
} else if (err.code === 'ECONNRESET' || err.code === 'UND_ERR_REQUEST_TIMEOUT' ) {
Steps to reproduce the behavior:
in undici set the time out to a low number:
undiciOptions: {
connections: 4000,
pipelining: 20,
proxyCacheSize: 50,
proxyCacheTtl: 1000 * 60 * 60 * 24,
requestTimeout: 1000
}
in test server open a route with sleep:
for example:
.get('/api/v1/my-route/timeout', async (req, res) => {
await sleep(3000);
res.end();
})Should handle timeout code from undici
Note: I think the following error codes should also be added: ESOCKETTIMEDOUT and ETIMEDOUT
Server crash when aborting wrk when using fast-proxy with 0http with low server
I have the following code:
(Happens also without undici)
const { proxy } = fastProxy({
base: 'http://localhost:4000',
undiciOptions: {
connections: 4000,
pipelining: 20,
requestTimeout: 5000,
},
});
const { router, server } = cero({
server: low(),
});
router.all('/*', (req, res) => proxy(req, res, req.url, {}));
server.listen(3000, (socket) => {
if (socket) {
console.log('HTTP server ready!');
}
});With an additional restana server listening on port 4000
Then run
wrk -t8 -c50 -d20s http://localhost:3000/api/v1/test
hit CTRL-C to stop wrk.
Result:
process abort on with the following error:
Invalid access of discarded (invalid, deleted) uWS.HttpResponse/SSLHttpResponse.
There was an uncaught exception. I'm now going to commit suicide; farewell all.
When I'm using 0http without low it doesn’t happen
The process should not abort if I stop wrk
2.1.0
No response
16.x
Linux
N/A
In #42 a scenario was given where a user might be able to perform an SSRF attack by using a specially crafted URL with two leading forward slashes. The implemented fix has the side effect of stripping all double slashes from the entire request URL.
For example, given the following URL:
https://chicken.com/duck?redirect_uri=//quack-a-doodle-doo.com
The proxy will then strip the additional / off of the URL there.
The slash in a query parameter does not need to be url encoded.
The characters slash ("/") and question mark ("?") may represent data within the query component.
If I am understanding the vulnerability originally reported in #42 , it only exists if the // is at the beginning of the URL.
What I would like to propose is to only convert // if they are at the beginning of the URL.
Pass a request to fast-proxy with double slashes in the URL, such as /duck?redirect_uri=//quack-a-doodle-doo.com.
No response
Currently for POST actions fast-proxy support plain/text and application/json.
Adding support for additional application/x-www-form-urlencoded
curl -i -H "content-type: application/x-www-form-urlencoded" -X POST -d 'a=1' http://localhost:4000/api/v1/service
Handling the body is likewise:
if (req.body instanceof Stream) {
body = req.body
} else if (typeof req.body === 'string') {
body = req.body
populateHeaders(headers, body, 'text/plain')
} else {
body = JSON.stringify(req.body)
populateHeaders(headers, body, 'application/x-www-form-urlencoded')
}
It should be:
const qs = require('querystring')
if (req.body instanceof Stream) {
body = req.body
} else if (typeof req.body === 'string') {
body = req.body
populateHeaders(headers, body, 'text/plain')
} else if (headers['content-type'] === 'application/x-www-form-urlencoded') {
const qs = require('querystring');
body = qs.stringify(req.body)
populateHeaders(headers, body, 'application/x-www-form-urlencoded')
} else {
body = JSON.stringify(req.body)
populateHeaders(headers, body, 'application/x-www-form-urlencoded')
}
3.24.1
2.1.0
16.8.0
Windows
Windows 10 21H1
If you set a base option to the Proxy function, the header "Location" will not be rewrite because of the Undici Pool not updated.
At the start of the plugin, we provide a base option, this option will provide the base URL for the Undici Pool creation under the "buildRequest" function in request.js
So if you have "http://google.com" as your default base option and under the Proxy function you have "http://127.0.0.1:3001" the location header will be "https://google.com/3001", so the request will "fail".
const { proxy } = require('../index')({ base: 'https://google.com', undici: { connections: 100, pipelining: 10 } }) const service = require('restana')() service.all('/*', (req, res) => proxy(req, res, req.url, { base: 'http://127.0.0.1:3001', rewriteRequestHeaders (req, headers) { delete headers.connection return headers } })) service.start(3000)
The request should go trough with the location header set by the base option under the Proxy function.
We would like to use fast-proxy in our koa application.
Is there a proxy call option that is compatible with kos?
I think we should be archiving this module, because it was born out of the will of @jkyberneees and myself to merge it and fastify-reply-from, creating a generic module for proxying in node.js.
However, such goals never materialized.
I would like to transfer this module back to @jkyberneees or archive this module if he is not interested in maintaining it anymore.
For some strange reasons I cannot add other people to do releases on this module, and I have no time to do them.
add an optional flag not to use caching
In my application 99% of the url are dynamically routes. i.e. /users/:userId/orders/:orderId and also use query params
therefore the cache is not required.
const { proxy, close } = fastProxy({ base: null, useCache: false });
Thanks,
Yohay
If I get the chance to PR this I will, but did want to mention the use case and see if there is perhaps something obvious I'm missing. Also wanted to add a workaround which may or may not work (just started writing some basic tests).
Presently you can only specify a single base URL, which is required using undici or http2. I see that for undici that is a constraint in using a Pool. Perhaps then instead of a shared pool there could be a per base pool instantiated.
As a workaround I've decached the module and initialized it for each target.
const proxyAgents = {}
const getProxy = (base) => {
if (proxyAgents[base]) return proxyAgents[base].proxy
delete require.cache[require.resolve('fast-proxy')]
proxyAgents[base] = require('fast-proxy')(Object.assign({ base }, proxyOpts))
return proxyAgents[base].proxy
}🚨 You need to enable Continuous Integration on Greenkeeper branches of this repository. 🚨
To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.
Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet.
We recommend using:
If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.
Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please click the 'fix repo' button on account.greenkeeper.io.
I reran the benchmarks, my results are way lower than yours.
Rerun the benchmarks, with the following bugfix:
-router.on(['GET', 'POST', 'PUT', 'PATCH', 'OPTIONS', 'DELETE'], '/service/*', (req, res) => {
+router.get('/service/*', (req, res) => {wrk -t8 -c50 -d20s http://127.0.0.1:8080/service/hiSomewhat similar benchmarks.
My benchmarks:
fast-proxy-undici/0http: Requests/sec 10259.59 (HTTP pipelining = 10)
fast-proxy/0http: Requests/sec 6773.80
fast-proxy/restana: Requests/sec 6460.21
fast-proxy-undici/0http: Requests/sec 9448.67 (HTTP pipelining = 1)
fastify-reply-from: Requests/sec 5635.55
http-proxy: Requests/sec 3105.40
As you can see I'm getting max 3.3x performance gain instead of your 46.6x.
Without the above mentioned bugfix, the first test clocks in at 39305.96 Requests/sec (12x faster than http-proxy). Even then it is WAY slower compared to your benchmarks.
I don't know what is exactly going on, but I think it's fair to say that your benchmarks are wrong and misleading.
I believe there is some overlap here between this library and https://github.com/nxtedition/node-http2-proxy and just wanted to give a ping and see if there is any potential to combine efforts?
I'm looking to spend some extra time with it or something corresponding at some point in order to support undici.
Before you submit an issue we recommend you drop into the Gitter community or Fastify Help and ask any questions you have or mention any problems you've had getting started with Fastify.
Please read this entire template before posting any issue. If you ignore these instructions
and post an issue here that does not follow the instructions, your issue might be closed,
locked, and assigned the missing discussion label.
Enable changing the base on each request will enable dynamic routing without the need to preconfigure proxies for different targets
proxy(req, res, req.url, {base: route})
Line 47 in d9717ce
change:
url = new URL(source, base)
to
url = new URL(source, opts.base)
I am not 100% sure what correct behavior should be, but the host on the header of a proxied request looses its port.
Proxy server:
const { proxy } = require('fast-proxy')({
base: 'http://localhost:9600'
})
const gateway = require('restana')()
gateway.all('/', (req, res) => {
proxy(req, res, req.url, {})
})
gateway.start(8080)Target server:
const server = require('restana')();
server.get('/', (req, res) => {
console.log(req.headers.host);
res.send('Hello from simple http service');
});
server.start(9600);When the proxy server is requested and proxied to the target server, the target server will print just localhost. I expect it to print localhost:8080.
Is current behavior correct or should the port be forwarded too?
It would also be nice to have a way to override the upstream http headers from the proxy to the target.
I'm happy to provide a PR to fix the missing port or add a solution to be able to set http headers on the upstream request.
A thing to be aware of when undici is used.
In a CORS situation, preflight requests from Safari lead to 500 error.
Indeed, Safari sends an OPTIONS request with a content-length: 0 header, which is not well handled by undici in that situation.
It can be fixed by removing that content-length header, if present (Chrome doesn't send it), before calling the proxy function:
if(req.method === 'OPTIONS' && req.headers.hasOwnProperty('content-length')){
delete req.headers['content-length'];
}
proxy(...
?
No response
14.17.6
macOS
Proxy crashes server when proxying multiple similar requests.
err: {
"type": "TypeError",
"message": "Cannot read property 'upgrade' of undefined",
"stack":
TypeError: Cannot read property 'upgrade' of undefined
at Parser.2 (/[email protected]/node_modules/undici/lib/core/client.js:546:17)
}
There is a bug fixed in undici, which seem fixed in later versions. nodejs/undici#756
Confirmed on my machine that the error does not occur on undici=4.9.5
nodejs/undici#756 has a comment on how to reproduce:
No response
I want to proxy on a per - route basis. I tried but keep getting an error in kubernetes containers and is behind an nginx ingress.
// issue ref: https://github.com/fastify/fast-proxy/issues/42
function buildURL (source, reqBase) {
const dest = new URL(source, reqBase)
// if base is specified, source url should not override it
if (reqBase) {
if (!reqBase.endsWith('/') && dest.href.length > reqBase.length) {
reqBase = reqBase + '/'
}
if (!dest.href.startsWith(reqBase)) {
throw new Error('source must be a relative path string')
}
}
return dest
}this works:
upstream: 'https://example.com',
prefix: '/example',
rewritePrefix: 'https://example.com/',
http2: false,
type: 'JWT',
beforeHandler: [Function: beforeHandler] // auth works this way
}
this does not work, when I put it into kubernetes, but seems to work when I directly access on my local machine instance:
upstream: 'https://examplek8-service',
prefix: '/example',
rewritePrefix: 'http://examplek8-service:1111/health/,
http2: false,
type: 'JWT',
beforeHandler: [Function: beforeHandler] // auth works this way
}
error I get is
{"statusCode":500,"error":"Internal Server Error","message":"source must be a relative path string"}
{success:true} // health endpoint on kubernetes.
Please read this entire template before posting any issue. If you ignore these instructions
and post an issue here that does not follow the instructions, your issue might be closed,
locked, and assigned the missing discussion label.
^^ affirmative.
The type for the option undici can only be satisfied by undefined or Undici.Pool.Options, in contrast of what the documentation says:
Set to true to use undici instead of
require('http').
And what the code does:
Line 13 in a44be16
Lines 41 to 43 in a44be16
My proposal is to update the type as such:
export interface FastProxyOptions {
base?: string;
http2?: boolean;
- undici?: Undici.Pool.Options;
+ undici?: boolean | Undici.Pool.Options;
cacheURLs?: number;
requests?: {
http?: Http.Agent,
https?: Https.Agent
};
keepAliveMsecs?: number;
maxSockets?: number;
rejectUnauthorized?: boolean;
queryString?: QueryStringModule;
}Are there any plans to support proxying of websockets? Or perhaps this is already possible and just needs documentation?
We are currently using node-http-proxy to proxy http and ws requests. We would like to refactor to use fast-proxy instead, but it does not appear that it's possible to proxy websockets with fast-proxy.
No response
4.14.1
2.1.0
16.x
Linux
NA
When the request encounters and error and that error is passed to the callback it is possible to get into a scenario where the response attempts to write after it has already been sent because res.sent is undefined. This results in an uncaught exception [ERR_STREAM_WRITE_AFTER_END]: write after end to be thrown if a response has already been sent.
Looking at the type of res argument (http.ServerResponse) there isn't an attribute called sent so it seems like this we will always get into this conditional here .
Unsure of how to replicate this.
A check to see if all writeable data has been flushed in order to get into this conditional. Perhaps checking response.writeableFinished
Hello,
would it be possible to add key and cert in the method agentOptions please ?
The use case is if the backend use TLS auth.
Thanks
In
Line 78 in 238019d
async function is used in an errorback. Doing so exposes us at a risk of memory leaks and DoS attacks in case an unplanned error (bug) occurs that would be extremely hard to track.Any good reason to keep package-lock.json checked in here? Shall we remove and ignore it?
How can I test this package locally? It includes lots of great integration tests, but the documentation not include anything about it.
Only reference to testing plugins is in .github/workflows/ci.yml which refers to fastify/workflows/.github/workflows/plugins-ci.yml@v3, but for a newbie it would be super hard to trace and contribute to this package (and test it locally)
Currently, if an error is thrown when the proxied request is made, the error is handled by fast-proxy. It would be nice for an optional callback to be passed so that the error, req, and res can be exposed to the user and handled as they see fit.
No response
function handleErrors (err, req, res) {
If(res.writableFinished === false) {
if(err.code === 'SOMETHING') {
// some logic here
}
}
}
proxy(req, res, source, { onError: handleErrors })
This is not an ESM, so we need to have 'use strict'
2.0.0
12.20
macOS
10.15
When I try to use fast-proxy with undici as client on a next.js API route to proxy browser requests i get an 500 error invalid connection header
pages/v1/[...args].js (route to proxy)import { NextApiRequest, NextApiResponse } from 'next';
const { proxy } = require('fast-proxy')({
base: process.env.EXTERNAL_API_URL,
undici: {
connections: 100,
pipelining: 0,
},
});
export default function api(req: NextApiRequest, res: NextApiResponse) {
return runMiddleware(req, res, () => proxy(req, res, req.url, {}));
}
export const config = {
api: {
bodyParser: false,
},
};
//@ts-expect-error
function runMiddleware(req, res, fn) {
return new Promise((resolve, reject) => {
//@ts-expect-error
fn(req, res, (result) => {
if (result instanceof Error) {
return reject(result);
}
return resolve(result);
});
});
}When I perform a proxy request with undici activated, it should not return 500
when someone use fast-proxy like:
const base = 'http://somewhere.com'
const { proxy, close } = require('fast-proxy')({
base
})and give the req.url like "//10.10.10.10/", the result of getReqUrl method is:
URL {
href: 'http://10.10.10.10/',
origin: 'http://10.10.10.10',
protocol: 'http:',
username: '',
password: '',
host: '10.10.10.10',
hostname: '10.10.10.10',
port: '',
pathname: '/',
search: '',
searchParams: URLSearchParams {},
hash: '' }but the result we expected is:
URL {
href: 'http://somewhere.com/',
origin: 'http://somewhere.com',
protocol: 'http:',
username: '',
password: '',
host: 'somewhere.com',
hostname: 'somewhere.com',
port: '',
pathname: '/',
search: '',
searchParams: URLSearchParams {},
hash: '' }it can be used to make SSRF attack.
so i suggest to pollyfill the method getResUrl:
when we give new URL instance, we need to exclude some source that begin with '//', to make it like '/'.
After the 2.1.0 release, undici dependency has been updated 4a6ca91
It solved a blocking issue happening with the latest version available on npm #49 (which I also encountered)
Isn't that significant enough to update the version and the npm package?
When using onResponse the remote http statusCode is not being used and therefore all status are 200
in opts.test.js extend the service and add an additional route that returns different status code then 200.
add a test that calls this route and expect to get the different status
line 113 in index.js
onResponse(req, res, stream)the onResponse should get the response object
onResponse(req, res, stream, response)
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.