Giter Club home page Giter Club logo

hoxy's Introduction

Hoxy

Hoxy is an HTTP traffic-sniffing and manipulation tool for JavaScript programmers.

Full Documentation

http://greim.github.io/hoxy/

Example

var hoxy = require('hoxy');
var proxy = hoxy.createServer().listen(8080);
proxy.intercept({

  // intercept during the response phase
  phase: 'response',

  // only intercept html pages
  mimeType: 'text/html',

  // expose the response body as a cheerio object
  // (cheerio is a jQuery clone)
  as: '$'
}, function(req, resp) {

  resp.$('title').text('Unicorns!');
  // all page titles will now say "Unicorns!"
});

Version 3.0

Hoxy has released version 3.0. This release simplifies the API and better supports ES6. Notable changes:

  • A done callback is no longer passed as the third arg to interceptors. Interceptor arity is, accordingly, no longer a switch for async behavior. Rather, it solely depends on the return type of the interceptor (i.e. promises or iterators over promises).
  • The third argument to interceptors is now the cycle object, === to this. This was based on a suggestion from @nerdbeere, with a view toward supporting arrow functions, in which this is lexical.
  • The CLI has been completely removed from the project. The reasoning is that, by simplifying the project, I can more easily maintain it. If there's a need, it can be brought back as a separate npm module. Perhaps somebody else can take that on.
  • Undocumented hoxy.forever() function goes away.

Release notes:

  • 3.3.1 Merged PR #109. Thanks @nerdbeere.
  • 3.3.0 Ability to filter by status.
  • 3.2.2 Merged PR #95. Thanks devjerry.
  • 3.2.1 Fixed test failures regarding content-length and transfer-encoding headers.
  • 3.2.0 Better error handling. Added query getter/setter to request.
  • 3.1.3 Merged PR #62. Thanks jonsharratt.
  • 3.1.2 Make Proxy#log() chainable.
  • 3.1.1 Prevent EADDRNOTAVAIL on Windows when using certAuthority.
  • 3.1.0 Filtering options now accept functions.
  • 3.0.3 Fixed Cycle#serve() breakage on Windows.
  • 3.0.2 Fix for a Windows EADDRNOTAVAIL error.
  • 3.0.1 Fixed bug where as intercepts weren't catching async errors properly.
  • 3.0.0 Simplify the API and better support ES6.
  • 2.3.1 Back-ported 3.0.1 async as intercept fix.
  • 2.3.0 Added getter and setter for proxy-level slow options.
  • 2.2.6 Added eslint npm script. Thanks @nerdbeere.
  • 2.2.5 Fixed a bug where .buffer was always undefined. Thanks @Timwi.
  • 2.2.4 Added babel optional runtime transformer.
  • 2.2.3 Fixed broken reference to lodash-node in CLI.
  • 2.2.2 Updated hoxy version in CLI.
  • 2.2.1 Fixed error in npmignore.
  • 2.2.0 Added proxy-level throttling.
  • 2.1.1 Ditched babel require hook and instead use compile/prepublish.
  • 2.1.0 Ability to run reversy proxy as an HTTPS server. Thanks @snoj.
  • 2.0.0 Direct HTTPS proxying and improved async support in interceptors. Thanks @snoj, @Phoenixmatrix, @sholladay and others for helping with the HTTPS stuff!
  • 1.2.4 Improved cheerio markup serialization. Thanks Seth Holladay.
  • 1.2.3 Test command now npm test instead of mocha. Proxy close() method now passes args through to server close. Thanks Seth Holladay.
  • 1.2.2 Fixed errors and test failures occurring on io.js.
  • 1.2.1 Make listen() accept same args as native server.listen() instead of just port. Thanks Seth Holladay.
  • 1.2.0 Send content-length whenever reasonably possible. (minor version bump since minor alteration to existing behavior)
  • 1.1.5 Default protocol to 'http:' if not found because I'm a genius.
  • 1.1.4 Default protocol to 'http' if not found.
  • 1.1.3 Don't munge content-length headers unless necessary.
  • 1.1.2 Burned a version number because I suck at npm.
  • 1.1.1 Added SSL support for requests (protocol: https).
  • 1.1.0 Added CLI functionality to scaffold new proxy projects.
  • 1.0.5 Fixed static conditional get fail, flexible contentType matching, ability to set fullUrl.
  • 1.0.4 npm distribution no longer contains test directory.
  • 1.0.3 Fixed issue #21 causing breakage on windows, due to use of unix domain sockets.
  • 1.0.2 Added tee() method to requests and responses, and accompanying tests.
  • 1.0.1 Fixed bug with URL pattern matching, added filtering tests.
  • 1.0.0 Initial release of Hoxy 1.0 rewrite.

Regarding Development of this Tool

I began writing code for this in 2010 while I worked at Cisco—and later at various startups—to solve my own needs for advanced testing and debugging production websites. The most common—although by no means only—use case for me was swapping in my own JavaScript onto web pages, in order to do rapid debugging without actually pushing console.logs to a staging server or (god forbid!) production. I could just edit a file on my local disk and reload my browser.

This enabled me to act quickly when hard-to-debug production issues arose, which translated into a competitive advantage for both myself and my company. Over time, however, I've used it less frequently, for a few reasons:

  1. Devtools. Built-in debuggers got wayyy more powerful. Starting in ~2014, major browsers shipped with source-map-enabled debuggers, which allow setting breakpoints and stepping through code in original, non-uglified, non-bundled form. That was key, and alleviated 80% of what I need Hoxy for.
  2. HTTPS everywhere. HTTPS became widely used in production during this same period. Hoxy supports HTTPS proxying, but involves extra setup, which adds friction to an already high-friction tool. Seriously, Hoxy makes it easier, but MITM-hacking your own HTTP traffic isn't exactly the definition of ease and simplicity.
  3. Evergreen browsers. Happily, the amount of time I spend on cross-browser differences has gone down as of mid 2018. Along those lines, a secondary use for Hoxy was injecting code into older, single-digit versions of IE, which had zero devtools capabilities. Nowadays I simply don't bother with those browsers.
  4. Functional programming. React (which I started using in 2014) is an enabling technology for functional programming. FP in turn makes it easier to test, debug, and reason about a piece of code in isolation. This even further alleviates the need to use Hoxy to see how something behaves in prod, since I can just write unit tests, reason about the code in my head, etc.

All of that said, this is still a valuable tool in my toolbox, and I use it when the need arises. I'm not putting much time and attention into bug fixes and feature improvements these days, but PRs are always welcome!

hoxy's People

Contributors

bradgearon avatar devjerry avatar facetcounter avatar floby avatar greim avatar nerdbeere avatar sholladay avatar timwi avatar tobli avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hoxy's Issues

no https support

Hoxy currently doesn't support https proxying. In order to be useful, it would need to be able to see content passing through UNencrypted. I'm not sure if there's a proxying scheme supported by browsers that makes this possible, and even if there was, I'm not sure I'd want to do it. On the off chance there's a proxying scheme supported by browsers that allows just HTTP headers through as clear text, but not the content bodies, that might be an acceptable compromise. Need to do more research.

proxyReq.socket is undefined

I updated to the latest node to see if it would help but "socket" is nowhere to be found.

TypeError: Cannot call method 'on' of undefined
    at EventEmitter.<anonymous> (hoxy.js:155:21)

Windows, cycle.serve does nothing

Hello,

Unfortunately, I can't figure out what is wrong with my code.
I don't see the content of package.json when I open http://www.google.com/script.js
But, It works good on MacOSx!

Windows 8.1
node v0.12.7
hoxy 3.0.0

var hoxy = require('hoxy');
var proxy = new hoxy.Proxy();
var port = 3080;

proxy.intercept({
    phase: 'request',
    fullUrl: 'http://www.google.com/script.js'
}, function (req, resp, cycle) {
    resp.statusCode = 200;
    console.log(__dirname + '\\package.json');
    return cycle.serve(__dirname + '\\package.json');

});

proxy.listen(port, 'localhost', function () {
    console.log('The proxy is listening on port ' + port + '.';);
});

Pass cycle to intercept callback instead of binding the callback to it

Disclaimer: This is more like a suggestion I'd like to discuss.

Currently the callback that a user of hoxy passes to the intercept method is bound to an instance of cycle.

With es6 I'd like to define my callback using fat arrow functions to bind my current scope to the callback.
In order to achieve this at the moment, my code looks something like this:

proxy.intercept('request', function(req, res, done) {
  that._onInterceptRequest(req, res, this, done);
});

My suggestion would be to pass the cycle to the callback (like I do) instead of binding it to the callback.
While I agree that this changes the API, maybe the rewrite might be a good opportunity to do that.

Error reporting for missing mime type

In many cases the stack trace for an exception is clear. It would be nice to have a gernal error module where exceptions can be trapped and reported as errors in the log. I would like to work on this.

uncaught exception: Cannot call method 'indexOf' of undefined
TypeError: Cannot call method 'indexOf' of undefined
at isText (/Users/rinaldo/Documents/workspace/Maxwell/DirectPrint/src/main/proxy/hoxy/plugins/ghost-server.js:141:15)
at getContentType (/Users/rinaldo/Documents/workspace/Maxwell/DirectPrint/src/main/proxy/hoxy/plugins/ghost-server.js:149:5)
at /Users/rinaldo/Documents/workspace/Maxwell/DirectPrint/src/main/proxy/hoxy/plugins/ghost-server.js:106:28
at [object Object]. (fs.js:107:5)
at [object Object].emit (events.js:61:17)
at afterRead (fs.js:878:12)
at wrapper (fs.js:245:17)

Rinaldo

Use ThrottleGroup to throttle proxy throughput

Me again with another feature request. :)
I'd like to implement a throttling feature in james (which is basically a frontend for hoxy) as mentioned here. But currently request.slow and response.slow only throttle per request.
What I would need is throttling on proxy level, something like proxy.slowUpstream(rate) and proxy.slowDownstream.

After looking into the source I found out that stream-throttle is used in hoxy which has a feature called ThrottleGroup. This might make it easy to implement slowing on proxy level.

Don't use chunked transfer encoding

The XMPP server we use (ejabberd) does not accept clients posting data with chunked transfer encoding. The browser does not do this, but hoxy does so this client request:

POST http://chat.cego.dk/http-bind/ HTTP/1.1
Host: chat.cego.dk
Proxy-Connection: keep-alive
Content-Length: 210
Cache-Control: no-cache
Pragma: no-cache
Origin: http://localhost
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
Content-Type: text/plain;charset=UTF-8
Accept: */*
Referer: http://localhost/chat.html
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,da;q=0.6,de;q=0.4,nl;q=0.2,nb;q=0.2

<body rid='2808090377' xmlns='http://jabber.org/protocol/httpbind' to='komogvind.dk' xml:lang='en' wait='60' hold='1' content='text/xml; charset=utf-8' ver='1.6' xmpp:version='1.0' xmlns:xmpp='urn:xmpp:xbosh'/>

...is rewritten by hoxy to this:

POST /http-bind/ HTTP/1.1
host: chat.cego.dk
cache-control: no-cache
pragma: no-cache
origin: http://localhost
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
content-type: text/plain;charset=UTF-8
accept: */*
referer: http://localhost/chat.html
accept-language: en-US,en;q=0.8,da;q=0.6,de;q=0.4,nl;q=0.2,nb;q=0.2
Connection: keep-alive
Transfer-Encoding: chunked

d2
<body rid='2808090377' xmlns='http://jabber.org/protocol/httpbind' to='komogvind.dk' xml:lang='en' wait='60' hold='1' content='text/xml; charset=utf-8' ver='1.6' xmpp:version='1.0' xmlns:xmpp='urn:xmpp:xbosh'/>
0

...and that made the XMPP server respond with:

HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=utf-8
Content-Length: 336

<?xml version='1.0'?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"/></head><body><h1>400 Bad Request</h1></body></html>

hoxy probably should't use chunked encoding unless forwarding very large payloads (maybe not even then).

Using hoxy with express and vhost

I'm trying to set a subdomain to a reverse proxy doing this:

var hoxy = require('hoxy');

var proxy = new hoxy.Proxy({reverse: 'http://google.com/'});

proxy.intercept({
    phase: 'response',
    mimeType: 'text/html',
    as: '$'
}, function(req, resp) {
    resp.$('title').text('proxy');
});
var express = require('express'),
    vhost = require('vhost'),
    app = express();

app.use(vhost('domain.com', function (req, res) {
    proxy.emit('request', req, res);
}));

Any aproach of how can I do it? As proxy.listen() simply passes its arguments to Node's server.listen() I tried to follow this example: https://www.npmjs.com/package/vhost#using-with-any-generic-request-handler

Any help is well appreciated.

SSL support

Hi greim

May you please provide �some examples for SSL request of proxy.intercept
I still can't listen SSL page

Thanks

.listen() should respect optional args for the http module

Most modules that rely upon http, such as connect and express, allow more arguments to .listen() than just the port to listen on. such as a callback to be run when the server is truly listening.

API docs: http://nodejs.org/api/http.html#http_server_listen_port_hostname_backlog_callback

For example, I am used to doing things like...

server.listen(port, function () {
    console.log('My-awesome-server is on port ' + port + '.');
});

It would be easier to get started with hoxy and make everyone's code more portable if it passed these arguments through - if and when it makes sense.

Currently, starting at line 147 in lib/proxy.js, there is this...

listen: function (port) {
    this._server.listen(port);
    ...
}

Any problem with changing that to....

listen: function (port) {
    this._server.listen.apply(this._server, arguments);
    ...
}

?

Source IP Address of request not accessible.

Its unclear to me how to access the source IP address of the request. The request object doesn't contain a connection key, might this require more introspection or extension of the code in proxy.js to pass it along?

Use ES6 in hoxy, particularly generators.

PROS:

Reduces dependence on callbacks. Writing code in a synchronous/suspended style would make it easier to reason about and refactor.

CONS:

Limits hoxy use to newer node platforms, namely iojs or node 0.11 / --harmony.

Cons might be mitigated by use of transpilers. Also by hoxy being a dev-focused tool rather than a production one, thus not locked to mission-critical systems and more under control of devs.

If/when iojs becomes stable/popular and/or Joyent releases node 0.12 with native generator support the cons will be minimized in my view.

Hoxy and tape don't like each other?

I'm working on a PR for PouchDB which required me to simulate latency between a local client and server. Hoxy seemed like a great choice. But we are using tape to do our perf testing and if I so much as 'require('hoxy')' (without ever actually calling it or doing anything with it) then when the tape tests finish things just hang. I have no clue why Hoxy and Tape don't like each other. I thought I would at least drop a note about it.

If you want to replicate then:

  1. git clone https://github.com/pouchdb/pouchdb.git
  2. cd pouchdb
  3. sudo npm install <-- I run into permission issues if I don't use sudo but that could just because of where I'm installing
  4. PERF=true npm test <--- This is just to convince yourself that the tests actually work
  5. npm install hoxy
  6. Edit tests/performance/perf.basics.js to include:
  var hoxy = require('hoxy');
  // the next line is just to keep jshint happy
  console.log(hoxy);
  1. PERF=true npm test

Notice what happens. You see 'Tests complete!' and then the command line just hangs, never returning.

EADDRNOTAVAIL when using SSL

I'm getting the following error trying to proxy through ssl on windows:

events.js:85
      throw er; // Unhandled 'error' event
            ^
Error: connect EADDRNOTAVAIL
    at exports._errnoException (util.js:746:11)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1010:19)

This is my code, http traffic works but whenever I try a https site it error's out.

var port = 46769;
var proxy = hoxy.createServer({
    certAuthority: {
        key: fs.readFileSync('f:/ssl/key.pem'),
        cert: fs.readFileSync('f:/ssl/cert.pem')
    }
}).listen(port);

proxy.intercept({
    phase: 'request'
}, function(req, resp, cycle) {
    console.log(123);
});
console.log('tunnel through', port);

Windows Error: listen EACCES

using the example code...

var hoxy = require('hoxy');
var proxy = new hoxy.Proxy().listen(8000);
proxy.intercept('request', function(req, resp){
console.log('request made to: ' + req.fullUrl());
});

Windows 7 - node 10.x I get this error...

Error: listen EACCES
at errnoException (net.js:901:11)
at Server._listen2 (net.js:1020:19)
at listen (net.js:1061:10)
at Server.listen (net.js:1121:5)
at Object.statics (C:\dvl\JS\proxy\hoxy\node_modules\hoxy\lib\cycle.js:264:10)
at C:\dvl\JS\proxy\hoxy\node_modules\hoxy\node_modules\await\await.js:399:47
at Object. (C:\dvl\JS\proxy\hoxy\node_modules\hoxy\node_modules\await\await.js:229:28)
at Array.forEach (native)
at Object.Promise.keep (C:\dvl\JS\proxy\hoxy\node_modules\hoxy\node_modules\await\await.js:227:14)
at C:\dvl\JS\proxy\hoxy\node_modules\hoxy\lib\cycle.js:254:9

PS: port 8000 is free

thanks!

Promise dependency seams to fail

node HttpsProxyAgent.js

/home/aemonge/user/studies/mongoose/node_modules/hoxy/node_modules/co/index.js:50
  return new Promise(function(resolve, reject) {
             ^
ReferenceError: Promise is not defined
    at Proxy.co (/home/aemonge/user/studies/mongoose/node_modules/hoxy/node_modules/co/index.js:50:14)
    at Server.<anonymous> (/home/aemonge/user/studies/mongoose/node_modules/hoxy/lib/proxy.js:244:23)
    at Server.emit (events.js:98:17)
    at HTTPParser.parser.onIncoming (http.js:2109:12)
    at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:122:23)
    at Socket.socket.ondata (http.js:1967:22)
    at TCP.onread (net.js:528:27)

more robust mime-type support for ghost-server?

Current toy implementation works well for common formats like HTML/CSS/JS, but not things like PDF or XLS (ad infinitum). Since it works by file extension how would we ghost-serve something called for example foo.do or bar.app? Content-inspection? Pass a default mime-type?

response: if $file eq 'foo.do', @ghost-server('/path/to/dir','application/xml')

A workaround is to add a subsequent response-phase rule to manually set the mime-type.

response: if $file eq 'foo.do', @ghost-server('/path/to/dir')
response: if $file eq 'foo.do', $mime-type.set-to('application/xml')

Rename "url" APIs to "path".

Having url and fullUrl is an odd concept to me. It seems like a wart in Hoxy's otherwise excellent semantics. I find myself explaining it in code comments.

Would you be open to renaming the usages of "url" to "path" where that is what it really means?

As a stretch goal, "fullUrl" could then become "url".

Add option to set cipher suite(s) supported

It would be great to be able to control the choice of ciphers when proxying.

I'm particularly interested in the proxy-client part of the journey, so that the proxy could negotiate a strong cipher (e.g. the tls library default list) with the origin website but I could ten force a non-Diffie Hellman cipher to be used between the proxy and the client. That would mean I could tcpdump the traffic and decrypt it, which isn't possible with DH key exchanges.

If controlling the ciphersuite on the proxy-client leg independently of the server-proxy leg is hard to implement I would settle for being able to set one overall ciphersuite setting to override the tls defaults.

Many thanks!

Release 3.0

Major version bump for the following breaking changes:

1) Interceptor done callbacks will be replaced by the cycle object.

// old signature
proxy.intercept('request', function(req, resp, done) {
  assert(typeof done === 'function');
  doSomethingAsyncThatTakesCallback(function(err) {
    done(err);
  });
});
// new signature
proxy.intercept('request', function(req, resp, cycle) {
  assert(typeof cycle !== 'function');
  assert(this === cycle);
  return doSomethingAsyncThatReturnsPromise();
});

The reason for this is that it simplifies the internals a bit, which makes the project incrementally less bug-prone and easier to maintain. But also, it allows writing interceptors as arrow functions, which otherwise wouldn't have access to the cycle object. (@nerdbeere)

2) CLI goes away

Not sure how many people used it beyond myself. I often forgot to update it as the project evolved, and by removing it I can focus on the core. Someone else is welcome to spin up a separate CLI project based on Hoxy, if they want.

3) hoxy.forever() goes away

This is an undocumented function that will go away.

socket error not handled

Not sure what the reason for it is but I sometimes get the error { [Error: socket hang up] code: 'ECONNRESET' } but I cannot simply reproduce it.
I have a trace of when it happens, in this case I'm running NODE_DEBUG=net so it logs a lot, see here:
https://gist.github.com/editedredx/94e3ed4fdfadf8cf6418

Running windows and the request at the top is a https POST Firefox makes automatically every X minutes. This often seems to be the culprit however it looks like some other requests can also trigger it.

Fails for WebSocket CONNECT tunnels

When a client application establishes a WebSocket connection through a proxy server, it first sends a HTTP CONNECT request to the proxy, then it sends a GET with an Upgrade header (for ws://).

It currently looks like Hoxy breaks this flow, as it assumes that any incoming CONNECT will be followed by a TLS ClientHello message for a HTTPS request.

Hoxy's implementation of cheerio is incompatible with XHTML

Took some digging to figure out why some specific pages were not rendering correctly for me. Turns out it is due to how cheerio is implemented for as:'$' intercepts. The pages were using XML-flavored markup and hoxy always instructs cheerio to render HTML, causing syntax errors which are only detected client-side.

Test case:
Spin up hoxy with an empty intercept like this (does not need to edit the response):

proxy.intercept(
    {
        phase    : 'response',   // run before we send anything back to the client
        as       : '$'           // ask for a cheerio object, to manipulate DOM
    },
    function (req, resp) {
    }
);

Then visit a page with XML, such as:

Responsible code:
Line 26 in lib/dom-reader.js:

DomReader.prototype.toString = function(){
  return this._$.html();  // line 26
};

Changing that to...

  return this._$.xml();

...fixes the problem.

But of course this has negative side-effects for non-XML pages. Seems like something that needs some thoughtful design and automated tests. The MIME type listed in HTTP headers often is not correct, so I'm not sure at the moment how to make this reliable without a lot of overhead. It could parse and render an exact copy with both strategies internally and then detect which one caused the least amount of change... ;)

Issue with large responses (perhaps chunked encoding)

The following script:

var hoxy = require('hoxy'),
    proxy = new hoxy.Proxy();

proxy.intercept({
        phase : 'response',
        fullUrl : 'http://:host/game.js',
        as : 'string'
    }, function(req, rsp) {
        console.log('Intercepting');
    });

proxy.listen(8080);

...fails when I generate a 500.000 byte game.js file on my webserver and download it through the proxy:

$ python -c 'open("../WWW/game.js", "w").write("A" * 500000)'
$ http_proxy=localhost:8080 wget  http://192.168.192.249/game.js
--2014-06-04 15:43:39--  http://192.168.192.249/game.js
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: unspecified [application/javascript]
Saving to: ‘game.js’

    [<=>

wget never finishes, it just stands there. I do receive some of the file thou, trying multiple times I get this:

$ ls -l game.js*
-rw-r--r-- 1 robert robert 43488 Jun  4 15:43 game.js
-rw-r--r-- 1 robert robert 43488 Jun  4 15:45 game.js.1
-rw-r--r-- 1 robert robert 43488 Jun  4 15:45 game.js.2
-rw-r--r-- 1 robert robert 43488 Jun  4 15:45 game.js.3

However, if I change the 'fullUrl' in the intercept so that the response isn't interceptet, the file is downloaded fine.

How to get request json on phase: 'response' ? (to responde conditionally something based on the sent data)

Hi, I'm struggling on how to get the request body:

proxy.intercept({
  phase: 'response'
}, function(req, resp) {
  var requestHash = {
    url: req.url,
    method: req.method,
    headers: req.headers,
    body: req.json // <- wont work, eithe req.body
  };
  console.log('======================== REQ');
  console.log(req);
  console.log('======================== REQ HASH');
  console.log(requestHash);
  resp.json = {
    happy: 'to be with you'
  };
});

Enable basic proxy auth

Our proxy uses basic auth. So I had to insert this line to enable. I'm not sure if this is the best place but works for me:

@@ -189,6 +189,7 @@ HTTP.createServer(function handleRequest(request, response) {
                                }

                                // this method makes node re-use client objects if needed
+                if(pEnvProxy.auth) reqInfo.headers['Proxy-Authorization'] = 'Basic ' + new Buffer(pEnvProxy.auth).toString(
                                var proxyReq = HTTP.request({
                                        method: reqInfo.method,

what do you think?

how to make hoxy server listen to itself

For some reasons i also want to mock hoxy server for some functions, that it returns some response to the client when client requests on xyz port. as my proxy server listening at xyz port.

Now when i request on a 9000 port (), i get :

request GET : 'http://localhost:9000'

ERROR: Error: invalid protocol: null
and then

ERROR: Error: connect ECONNREFUSED
How to get rid of these errors??

EDIT : Here is a few lines:

var proxy = new hoxy.Proxy({ port : 3000 }).listen(9000);
proxy._server.on('request', function(){
console.log('Request Recieved');
});
proxy.intercept({ phase : 'request' }, function(){
//interception logic
});
I am not able to figure out what is the issue??

Accept connections over HTTPS.

This feature request is geared towards being able to run an HTTPS proxy server, as opposed to being able to connect through to HTTPS target pages on the other side, as is the goal of #4.

The Problem

  • Presently, all client connections to hoxy are only possible over plain HTTP.
  • In reverse mode, target pages that assume or demand HTTPS in their implementation may break completely because an HTTPS reverse proxy endpoint is not available.
  • In forward mode, target pages that assume or demand HTTPS in their implementation will generally load their resources in a way that avoids proxying completely, because typically the OS routes HTTPS requests through an entirely different proxy configuration (via the $HTTPS_PROXY environment variable, for example, instead of $HTTP_PROXY) and in many cases this is out of the control of the user of hoxy's API. Setting both configurations to the same proxy will work in some scenarios when #4 lands, but will break when the environment assumes (reasonably) that the proxy itself speaks HTTPS.

The Solution

  • Provide an API to create a hoxy instance that will respond to HTTPS connections. This could be exposed through configuration on the existing options object given to Proxy or via a new class, as is common with other libraries.
  • Ideally, all combinations of client and target protocols work by the time this is complete.
Client Protocol Target Protocol Proxy Direction Works Today
HTTP HTTP Forward ✔️
HTTP HTTPS Forward ❌ (soon)
HTTPS HTTP Forward
HTTPS HTTPS Forward
HTTP HTTP Reverse ✔️
HTTP HTTPS Reverse ✔️
HTTPS HTTP Reverse
HTTPS HTTPS Reverse

Non-modifying intercept modifies the headers

While #27 solved the issue of removing the Content-Length header and using Transfer-Encoding: chunked for as: 'string' intercepts, the same issue still applies to other as intercepts.

While it is possible to use 'string' and parse the content manually, it would be much better if hoxy automatically handled changes to (the length of) the content without using chunked mode or at least provided a way to get a read-only copy of the parsed body which kept the original request intact.

(In my case, the original request is specified as being HTTP 1.0, so no server can expect it to use Transfer-Encoding: chunked which was only introduced in HTTP 1.1.)

Unable to pass other data instead of reverse-proxy data.

I have something like:

var proxy = hoxy.createServer({reverse: 'http://website.com'}).listen(8080);

proxy.intercept({
    phase: 'response',
    url: /.*\.pdf/,
    as: 'buffer'
}, function(req, resp, cycle) {
    download(req.url, function (file, headers) {
        resp.headers = headers;
        resp.buffer = file;
    });
});

and a function which downloads the file and replace the headers and the buffer with the downloaded data. And to acces the proxy I have a http.request passing the request data to hoxy. But it sends the original reversed-webpage insted of the downloaded data with the download function.

I've been trying things for hours a couldn't find a way to do it. Please help 😃

Unable to have multiple actions

Given a request I want to perform a redirect to a new url with some query params. I am able to get hoxy to perform the redirect but I am unable to insert the values of any of the request values into the redirect reponse.

response: if $url contains 'x.aspx', $response-headers['location'].set-to("http://y:9123/
aurl.html?a=b&c=d") $response-headers['ids'].set-to("$cookies['NAME]")

Passthru for long polling

It would be nice to have a passthru option. I am intercepting traffic to my site that uses the StropheJS XMPP library for chat and it is failing. Probably the browser expects to receive response headers (with chunked encoding) quickly but Hoxy doesn't send headers until entire response is received....which never happens.

Hoxy breaks statcounter.com

I haven't yet been able to figure out why this happens, so help would be appreciated.

Spinning up and using a barebones proxy (as well as my real-world app), this site does not render correctly.
http://gs.statcounter.com/

Reproducible code:

'use strict';
const hoxy = require('hoxy'),
      proxy = hoxy.createServer();
proxy.listen(8000);

Steps:

  1. Launch a server with the above code.
  2. Configure your browser/OS to use that forward proxy.
  3. In your browser, visit: http://gs.statcounter.com/
  4. Toggle your settings for using the proxy on and off and reload the page in between, to witness the difference in rendering.

Current behavior:
The page displays an error message:

Sorry, could not understand url: #browser-ww-monthly-201410-201510

Expected behavior:
The site appears identical whether visited through the proxy or not.

Pass variables gained from route-pattern to request object

It would be nice to be able to use the route variables mentioned in the url/fullUrl interception parameters.
Something in the line of this:

proxy.intercept({
  phase: 'request',
  method: 'GET',
  url: '/dir/:page'
}, function(req, resp, cycle) {
  console.log(req.route.namedParams.page);
});

You are un-contactable.

I wanted to thank you for Hoxy, I use it every single day. Please let me donate towards to the project, somehow. Paypal etc? Your work has saved me a great deal of time over the last 6 months.

Also, i cant find a blog/twitter etc of yours to thank you or contact you other than creating this 'Issue'

Consider this issue resolved when you have
A. A public contact method attached to your github account! (i recommend twitter)
B. allowed me to donate!

Many thanks

create gui for config, rules syntax is wonky

Okay. More than one smart person has made the argument that hoxy's custom rule syntax isn't exactly an attractive feature.

I largely feel the same way, having resisted only because I wanted something concise and human-readable, whereas XML or JSON would be bulky and more difficult to code by hand. It's a rock and a hard place, but I feel ever so slightly more inclined to keep the current rule syntax, at least in the short term, since conciseness and human-readability were some of my initial design goals.

Mid- to long-term, a possible solution would be to build a GUI rule configuration control panel (web based) that connects to a service running alongside hoxy, which can obviate the need for a rules syntax and hopefully be much more user friendly.

gzip not supported

Hoxy strips out the accept-encoding header so theoretically this shouldn't be an issue. However google appears to gzip responses anyway. This is only an issue when you try to read/write the response body, but then it's a very bad issue.

unrecognized phases fail silently

A rule starting with "reguest: " for example will not trigger an error and may result in wasted time while the user wonders why the rule isn't firing.

Intercept req/resp bodies as streams.

This would allow people to manipulate source streams directly, as in:

proxy.intercept({
  phase: 'response',
  as: 'stream'
}, function(req, res) {
  resp.stream = resp.stream.pipe(myTransformStream);
});

https traffic is not proxied (in windows)

After the error fix from #56 I'm still not able to catch https traffic, firefox is responding with a Secure Connection Failed error. Http traffic is working as expected. The code is still the same:

var port = 46769;
var proxy = hoxy.createServer({
    certAuthority: {
        key: fs.readFileSync('f:/ssl/key.pem'),
        cert: fs.readFileSync('f:/ssl/cert.pem')
    }
}).listen(port);

proxy.intercept({
    phase: 'request'
}, function(req, resp, cycle) {
    console.log(123);
});
console.log('tunnel through', port);

No console logging happens when requesting through https and the error in firefox happens as a timeout. (I've tried it in Chrome with no difference in behaviour)

schermafbeelding_082115_083519_am

In non debug mode exceptions should return 500

I noticed that the client connection doesn't appear to be closed when there is an exception. I think it should return 500 for server error in that case. In debug mode it is useful to have the the console display the error and perhaps not return a 500 to the user. This could be a Denail of Service attack by sending bogus urls and consuming all the connections on the server.

Rinaldo

reverse proxy for multiple domain?

Hello! This is very useful tool for testing. for my case, I want it not only intercept one domain in reverse proxy mode but only multiple domain. Hope you consider it. thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.