Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Fastify version

?

Plugin version

No response

Node.js version

14.17.6

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

Description

Proxy crashes server when proxying multiple similar requests.

 err: {
      "type": "TypeError",
      "message": "Cannot read property 'upgrade' of undefined",
      "stack":
          TypeError: Cannot read property 'upgrade' of undefined
              at Parser.2 (/[email protected]/node_modules/undici/lib/core/client.js:546:17)
    }

There is a bug fixed in undici, which seem fixed in later versions. nodejs/undici#756

Confirmed on my machine that the error does not occur on undici=4.9.5

Steps to Reproduce

nodejs/undici#756 has a comment on how to reproduce:

nodejs/undici#756 (comment)

Expected Behavior

No response

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Fastify version

4.14.1

Plugin version

2.1.0

Node.js version

16.x

Operating system

Linux

Operating system version (i.e. 20.04, 11.3, 10)

NA

Description

When the request encounters and error and that error is passed to the callback it is possible to get into a scenario where the response attempts to write after it has already been sent because res.sent is undefined. This results in an uncaught exception [ERR_STREAM_WRITE_AFTER_END]: write after end to be thrown if a response has already been sent.

Looking at the type of res argument (http.ServerResponse) there isn't an attribute called sent so it seems like this we will always get into this conditional here .

Steps to Reproduce

Unsure of how to replicate this.

Expected Behavior

A check to see if all writeable data has been flushed in order to get into this conditional. Perhaps checking response.writeableFinished

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the issue has not already been raised

Issue

After the 2.1.0 release, undici dependency has been updated 4a6ca91
It solved a blocking issue happening with the latest version available on npm #49 (which I also encountered)

Isn't that significant enough to update the version and the npm package?

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Fastify version

3.24.1

Plugin version

2.1.0

Node.js version

16.8.0

Operating system

Windows

Operating system version (i.e. 20.04, 11.3, 10)

Windows 10 21H1

Description

If you set a base option to the Proxy function, the header "Location" will not be rewrite because of the Undici Pool not updated.

At the start of the plugin, we provide a base option, this option will provide the base URL for the Undici Pool creation under the "buildRequest" function in request.js

So if you have "http://google.com" as your default base option and under the Proxy function you have "http://127.0.0.1:3001" the location header will be "https://google.com/3001", so the request will "fail".

Steps to Reproduce

const { proxy } = require('../index')({ base: 'https://google.com', undici: { connections: 100, pipelining: 10 } }) const service = require('restana')() service.all('/*', (req, res) => proxy(req, res, req.url, { base: 'http://127.0.0.1:3001', rewriteRequestHeaders (req, headers) { delete headers.connection return headers } })) service.start(3000)

Expected Behavior

The request should go trough with the location header set by the base option under the Proxy function.

Any good reason to keep package-lock.json checked in here? Shall we remove and ignore it?

🚀 Feature Proposal

Currently for POST actions fast-proxy support plain/text and application/json.

Motivation

Adding support for additional application/x-www-form-urlencoded

Example

curl -i -H "content-type: application/x-www-form-urlencoded" -X POST -d 'a=1' http://localhost:4000/api/v1/service

Handling the body is likewise:

          if (req.body instanceof Stream) {
            body = req.body
          } else if (typeof req.body === 'string') {
            body = req.body
            populateHeaders(headers, body, 'text/plain')
          } else {
            body = JSON.stringify(req.body)
            populateHeaders(headers, body, 'application/x-www-form-urlencoded')
          }

It should be:

const qs = require('querystring')

          if (req.body instanceof Stream) {
            body = req.body
          } else if (typeof req.body === 'string') {
            body = req.body
            populateHeaders(headers, body, 'text/plain')
          } else if (headers['content-type'] === 'application/x-www-form-urlencoded') {
            const qs = require('querystring');
            body = qs.stringify(req.body)
            populateHeaders(headers, body, 'application/x-www-form-urlencoded')
          } else {
            body = JSON.stringify(req.body)
            populateHeaders(headers, body, 'application/x-www-form-urlencoded')
          }

🐛 Bug Report

undici timeout err.code=UND_ERR_REQUEST_TIMEOUT is not handled properly, therefore instead of getting error 504 I get error 500

We need to add in index.js line 88 the undici timeout error code as well:

} else if (err.code === 'ECONNRESET' || err.code === 'UND_ERR_REQUEST_TIMEOUT' ) {

To Reproduce

Steps to reproduce the behavior:
in undici set the time out to a low number:

    undiciOptions: {
      connections: 4000,
      pipelining: 20,
      proxyCacheSize: 50,
      proxyCacheTtl: 1000 * 60 * 60 * 24,
      requestTimeout: 1000
    }

in test server open a route with sleep:
for example:

  .get('/api/v1/my-route/timeout', async (req, res) => {
    await sleep(3000);
    res.end();
  })

Expected behavior

Should handle timeout code from undici

Note: I think the following error codes should also be added: ESOCKETTIMEDOUT and ETIMEDOUT

Your Environment

• node version: 12
• fast-proxy version: 1.6.3
• os: Mac

I want to proxy on a per - route basis. I tried but keep getting an error in kubernetes containers and is behind an nginx ingress.

You have already researched for similar issues?

// issue ref: https://github.com/fastify/fast-proxy/issues/42
function buildURL (source, reqBase) {
  const dest = new URL(source, reqBase)

  // if base is specified, source url should not override it
  if (reqBase) {
    if (!reqBase.endsWith('/') && dest.href.length > reqBase.length) {
      reqBase = reqBase + '/'
    }

    if (!dest.href.startsWith(reqBase)) {
      throw new Error('source must be a relative path string')
    }
  }

  return dest
}

What are you trying to achieve, or the steps to reproduce?

this works:

 upstream: 'https://example.com',
  prefix: '/example',
  rewritePrefix: 'https://example.com/',
  http2: false,
  type: 'JWT',
  beforeHandler: [Function: beforeHandler] // auth works this way
}

this does not work, when I put it into kubernetes, but seems to work when I directly access on my local machine instance:

 upstream: 'https://examplek8-service',
  prefix: '/example',
  rewritePrefix: 'http://examplek8-service:1111/health/,
  http2: false,
  type: 'JWT',
  beforeHandler: [Function: beforeHandler] // auth works this way
}

What was the result you received?

error I get is

{"statusCode":500,"error":"Internal Server Error","message":"source must be a relative path string"}

What did you expect?

{success:true} // health endpoint on kubernetes.

Context

• node version: 12
• fastify version: >=0.37.0
• os: Mac, Windows: mac, alpine container
• any other relevant information: I'd like a way to turn this security feature off in a config , or find some way to proxy .I am coming from express and koa to fastly, maybe there is a simple request-get-as-proxy command I'm missing?

Please read this entire template before posting any issue. If you ignore these instructions
and post an issue here that does not follow the instructions, your issue might be closed,
locked, and assigned the missing discussion label.
^^ affirmative.

🐛 Bug Report

When using onResponse the remote http statusCode is not being used and therefore all status are 200

To Reproduce

in opts.test.js extend the service and add an additional route that returns different status code then 200.
add a test that calls this route and expect to get the different status

line 113 in index.js

onResponse(req, res, stream)

Expected behavior

the onResponse should get the response object

onResponse(req, res, stream, response)

Your Environment

• node version: 12
• fastify version: latest
• os: Mac

🐛 Bug Report

I reran the benchmarks, my results are way lower than yours.

To Reproduce

Rerun the benchmarks, with the following bugfix:

-router.on(['GET', 'POST', 'PUT', 'PATCH', 'OPTIONS', 'DELETE'], '/service/*', (req, res) => {
+router.get('/service/*', (req, res) => {

wrk -t8 -c50 -d20s http://127.0.0.1:8080/service/hi

Expected behavior

Somewhat similar benchmarks.

My benchmarks:

fast-proxy-undici/0http: Requests/sec 10259.59 (HTTP pipelining = 10)
fast-proxy/0http: Requests/sec 6773.80
fast-proxy/restana: Requests/sec 6460.21
fast-proxy-undici/0http: Requests/sec 9448.67 (HTTP pipelining = 1)
fastify-reply-from: Requests/sec 5635.55
http-proxy: Requests/sec 3105.40

As you can see I'm getting max 3.3x performance gain instead of your 46.6x.
Without the above mentioned bugfix, the first test clocks in at 39305.96 Requests/sec (12x faster than http-proxy). Even then it is WAY slower compared to your benchmarks.

I don't know what is exactly going on, but I think it's fair to say that your benchmarks are wrong and misleading.

Your Environment

• node version: 12
• os: Linux

If I get the chance to PR this I will, but did want to mention the use case and see if there is perhaps something obvious I'm missing. Also wanted to add a workaround which may or may not work (just started writing some basic tests).

Presently you can only specify a single base URL, which is required using undici or http2. I see that for undici that is a constraint in using a Pool. Perhaps then instead of a shared pool there could be a per base pool instantiated.

As a workaround I've decached the module and initialized it for each target.

const proxyAgents = {}
const getProxy = (base) => {
  if (proxyAgents[base]) return proxyAgents[base].proxy
  delete require.cache[require.resolve('fast-proxy')]
  proxyAgents[base] = require('fast-proxy')(Object.assign({ base }, proxyOpts))
  return proxyAgents[base].proxy
}

In

As this module is now maintained by the fastify organisation members. We should remove the .github folder including the FUNDING.yml configuration.

I believe there is some overlap here between this library and https://github.com/nxtedition/node-http2-proxy and just wanted to give a ping and see if there is any potential to combine efforts?

I'm looking to spend some extra time with it or something corresponding at some point in order to support undici.

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure it has not already been reported

fast-proxy version

2.0.0

Node.js version

12.20

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

10.15

Description

When I try to use fast-proxy with undici as client on a next.js API route to proxy browser requests i get an 500 error invalid connection header

Steps to Reproduce

• create next.js app
• create pages/v1/[...args].js (route to proxy)
• add the proxy code

import { NextApiRequest, NextApiResponse } from 'next';
const { proxy } = require('fast-proxy')({
  base: process.env.EXTERNAL_API_URL,
  undici: {
     connections: 100,
     pipelining: 0,
   },
});

export default function api(req: NextApiRequest, res: NextApiResponse) {
  return runMiddleware(req, res, () => proxy(req, res, req.url, {}));
}

export const config = {
  api: {
    bodyParser: false,
  },
};

//@ts-expect-error
function runMiddleware(req, res, fn) {
  return new Promise((resolve, reject) => {
    //@ts-expect-error
    fn(req, res, (result) => {
      if (result instanceof Error) {
        return reject(result);
      }

      return resolve(result);
    });
  });
}

• when performing a request, it get's captured and passed to the proxy, the proxy returns 500, this doesn't happen if I remove the undici options.

Expected Behavior

When I perform a proxy request with undici activated, it should not return 500

I am not 100% sure what correct behavior should be, but the host on the header of a proxied request looses its port.

Proxy server:

const { proxy } = require('fast-proxy')({
    base: 'http://localhost:9600'
})
const gateway = require('restana')()

gateway.all('/', (req, res) => {
    proxy(req, res, req.url, {})
})

gateway.start(8080)

Target server:

const server = require('restana')();
server.get('/', (req, res) => {
    console.log(req.headers.host);
    res.send('Hello from simple http service');
});
server.start(9600);

When the proxy server is requested and proxied to the target server, the target server will print just localhost. I expect it to print localhost:8080.

Is current behavior correct or should the port be forwarded too?

It would also be nice to have a way to override the upstream http headers from the proxy to the target.

I'm happy to provide a PR to fix the missing port or add a solution to be able to set http headers on the upstream request.

when someone use fast-proxy like:

const base = 'http://somewhere.com'
const { proxy, close } = require('fast-proxy')({
      base
})

and give the req.url like "//10.10.10.10/", the result of getReqUrl method is:

URL {
  href: 'http://10.10.10.10/',
  origin: 'http://10.10.10.10',
  protocol: 'http:',
  username: '',
  password: '',
  host: '10.10.10.10',
  hostname: '10.10.10.10',
  port: '',
  pathname: '/',
  search: '',
  searchParams: URLSearchParams {},
  hash: '' }

but the result we expected is:

URL {
  href: 'http://somewhere.com/',
  origin: 'http://somewhere.com',
  protocol: 'http:',
  username: '',
  password: '',
  host: 'somewhere.com',
  hostname: 'somewhere.com',
  port: '',
  pathname: '/',
  search: '',
  searchParams: URLSearchParams {},
  hash: '' }

it can be used to make SSRF attack.
so i suggest to pollyfill the method getResUrl:

when we give new URL instance, we need to exclude some source that begin with '//', to make it like '/'.

I think we should be archiving this module, because it was born out of the will of @jkyberneees and myself to merge it and fastify-reply-from, creating a generic module for proxying in node.js.
However, such goals never materialized.

I would like to transfer this module back to @jkyberneees or archive this module if he is not interested in maintaining it anymore.

For some strange reasons I cannot add other people to do releases on this module, and I have no time to do them.

🚨 You need to enable Continuous Integration on Greenkeeper branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet.
We recommend using:

• CircleCI
• Travis CI
• Buildkite
• CodeShip
• Azure Pipelines
• TeamCity
• Buddy
• AppVeyor
  But Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please click the 'fix repo' button on account.greenkeeper.io.

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the feature has not already been requested

🚀 Feature Proposal

Are there any plans to support proxying of websockets? Or perhaps this is already possible and just needs documentation?

Motivation

We are currently using node-http-proxy to proxy http and ws requests. We would like to refactor to use fast-proxy instead, but it does not appear that it's possible to proxy websockets with fast-proxy.

Example

No response

🐛 Bug Report

Server crash when aborting wrk when using fast-proxy with 0http with low server

To Reproduce

I have the following code:
(Happens also without undici)

    const { proxy } = fastProxy({
      base: 'http://localhost:4000',
      undiciOptions: {
        connections: 4000,
        pipelining: 20,
        requestTimeout: 5000,
      },
    });
    const { router, server } = cero({
      server: low(),
    });

    router.all('/*', (req, res) => proxy(req, res, req.url, {}));

    server.listen(3000, (socket) => {
      if (socket) {
        console.log('HTTP server ready!');
      }
    });

With an additional restana server listening on port 4000
Then run
wrk -t8 -c50 -d20s http://localhost:3000/api/v1/test
hit CTRL-C to stop wrk.

Result:
process abort on with the following error:

Invalid access of discarded (invalid, deleted) uWS.HttpResponse/SSLHttpResponse.
There was an uncaught exception. I'm now going to commit suicide; farewell all.

When I'm using 0http without low it doesn’t happen

Expected behavior

The process should not abort if I stop wrk

Your Environment

• node version: 12
• fast-prxy version: latest
• os: Mac

🚀 Feature Proposal

add an optional flag not to use caching

Motivation

In my application 99% of the url are dynamically routes. i.e. /users/:userId/orders/:orderId and also use query params

therefore the cache is not required.

Example

const { proxy, close } = fastProxy({ base: null, useCache: false });

Thanks,

Yohay

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the issue has not already been raised

Issue

A thing to be aware of when undici is used.
In a CORS situation, preflight requests from Safari lead to 500 error.

Indeed, Safari sends an OPTIONS request with a content-length: 0 header, which is not well handled by undici in that situation.

It can be fixed by removing that content-length header, if present (Chrome doesn't send it), before calling the proxy function:

if(req.method === 'OPTIONS' && req.headers.hasOwnProperty('content-length')){
  delete req.headers['content-length'];
}
proxy(...

Hello,

would it be possible to add key and cert in the method agentOptions please ?

The use case is if the backend use TLS auth.

Thanks

This is not an ESM, so we need to have 'use strict'

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the feature has not already been requested

🚀 Feature Proposal

Currently, if an error is thrown when the proxied request is made, the error is handled by fast-proxy. It would be nice for an optional callback to be passed so that the error, req, and res can be exposed to the user and handled as they see fit.

Motivation

No response

Example

function handleErrors (err, req, res) {
   If(res.writableFinished === false) {
      if(err.code ===  'SOMETHING') {
         // some logic here
     }
   }
}
proxy(req, res, source, { onError: handleErrors })

Before you submit an issue we recommend you drop into the Gitter community or Fastify Help and ask any questions you have or mention any problems you've had getting started with Fastify.

Please read this entire template before posting any issue. If you ignore these instructions
and post an issue here that does not follow the instructions, your issue might be closed,
locked, and assigned the missing discussion label.

🚀 Feature Proposal

Enable changing the base on each request will enable dynamic routing without the need to preconfigure proxies for different targets

Example

proxy(req, res, req.url, {base: route})

code change:

change:
url = new URL(source, base)
to
url = new URL(source, opts.base)

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the issue has not already been raised

Issue

How can I test this package locally? It includes lots of great integration tests, but the documentation not include anything about it.

Only reference to testing plugins is in .github/workflows/ci.yml which refers to fastify/workflows/.github/workflows/plugins-ci.yml@v3, but for a newbie it would be super hard to trace and contribute to this package (and test it locally)

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the issue has not already been raised

Issue

We would like to use fast-proxy in our koa application.
Is there a proxy call option that is compatible with kos?

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the issue has not already been raised

Issue

The type for the option undici can only be satisfied by undefined or Undici.Pool.Options, in contrast of what the documentation says:

Set to true to use undici instead of require('http').
And what the code does:

fast-proxy/lib/request.js

Line 13 in a44be16

const isUndici = !!opts.undici

fast-proxy/lib/request.js

Lines 41 to 43 in a44be16

	if (typeof opts.undici !== 'object') {
	opts.undici = {}
	}

My proposal is to update the type as such:

export interface FastProxyOptions {
  base?: string;
  http2?: boolean;
- undici?: Undici.Pool.Options;
+ undici?: boolean | Undici.Pool.Options;
  cacheURLs?: number;
  requests?: {
    http?: Http.Agent,
    https?: Https.Agent
  };
  keepAliveMsecs?: number;
  maxSockets?: number;
  rejectUnauthorized?: boolean;
  queryString?: QueryStringModule;
}

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Fastify version

2.1.0

Plugin version

No response

Node.js version

16.x

Operating system

Linux

Operating system version (i.e. 20.04, 11.3, 10)

N/A

Description

In #42 a scenario was given where a user might be able to perform an SSRF attack by using a specially crafted URL with two leading forward slashes. The implemented fix has the side effect of stripping all double slashes from the entire request URL.

For example, given the following URL:

https://chicken.com/duck?redirect_uri=//quack-a-doodle-doo.com

The proxy will then strip the additional / off of the URL there.

The slash in a query parameter does not need to be url encoded.

The characters slash ("/") and question mark ("?") may represent data within the query component.

If I am understanding the vulnerability originally reported in #42 , it only exists if the // is at the beginning of the URL.

What I would like to propose is to only convert // if they are at the beginning of the URL.

Steps to Reproduce

Pass a request to fast-proxy with double slashes in the URL, such as /duck?redirect_uri=//quack-a-doodle-doo.com.

Expected Behavior

No response