Comments (9)
From what I can tell it uses the global agent because it does not specify the agent
attribute: https://github.com/kriskowal/q-io/blob/master/http.js#L278
I'm fine with this, but my guess is others may want to control the agent on a request by request basis, so this issue could probably turn into a feature request.
from q-io.
in a similar vane, a knox client provides a similar interface to node's http client but adds an additional layer of logic to make it easier to work with amazon's s3. i've been wrapping that client to use q-io/reader and q-io/writer on the requests and responses and i'd like to see something within q-io/http that would take care of that for me. it's essentially the logic that already exists in q-io/http.request to map the _request
interaction to something based on promises - i just want to have the chance to use something other than node's http (or https) as the source of that request.
EDIT: in case it isn't clear, the reason i said that my request is in a similar vane is because i expect that a utility to turn an instance of node's http.ClientRequest into something based on promises would satisfy @doapp-ryanp's needs too. maybe something along the lines of this API
var req = require('q-io/http').ClientRequest(require('http').request(options));
from q-io.
sorry... one more thought - i guess a more complete API would expose q-io/http.IncomingMessage as well.
from q-io.
Q-IO/HTTP request now accepts an "agent" property on the "request". Does that address this socket pooling issue adequately @doapp-ryanp?
@neonstalwart, Can you link a reference or describe what you mean by IncomingMessage? Is this distinct from http.ClientResponse or http.ServerResponse?
Please reopen attn me. Trying to wrangle my task queue.
from q-io.
@kriskowal yep that does the trick. Thanks!
from q-io.
@kriskowal it's been a while since i had my head in the code that caused me to make that comment so i'm kind of fuzzy on what i had in mind but i believe what i was suggesting was a utility for turning http://nodejs.org/api/http.html#http_http_incomingmessage into something based on promises and it's quite possible that q-io/http.ClientResponse is actually already this but now i don't recall if i just didn't see it previously or if there was some reason i thought a distinct IncomingMessage would be useful.
however, the "issue" i was trying to have addressed is that q-io/http.request is hardcoded to use node's http (or https) module as the way to generate _request
. in addition to that, the mapping of the options passed to node's http.request are also hard-coded (which was the original issue raised here).
if you were to refactor q-io/http.request so that it generated _request
(as it does now) but then used some utilities that were exported from q-io/http to make the request and response promise-based then we would have the building blocks to be able to use another client with the same API as node's http.request to generate a request - knox is one example of such a client.
var req = require('q-io/http').ClientRequest(knoxClient.request(options));
Q.when(req, function (response) {
// response would be promise-based stream
response.forEach( ... );
});
if you took on this idea, you would have something like the following in q-io/http (consider this code just to be an outline of the idea).
exports.request = function (request) {
return Q.when(request, function (request) {
request = exports.normalizeRequest(request);
var ssl = request.ssl;
var http = ssl ? HTTPS : HTTP;
var headers = request.headers || {};
headers.host = headers.host || request.host;
var _request = exports.ClientRequest(http.request({
host: request.host,
port: request.port || (ssl ? 443 : 80),
path: request.path || "/",
method: request.method || "GET",
headers: headers,
agent: request.agent
}), request.charset);
return Q.when(request.body.forEach(_request.write), _request.close).then(function () {
return _request;
});
});
};
exports.ClientRequest = function (_request, charset) {
return Q.when(_request, function (_request) {
var deferred = Q.defer();
_request.on('response', function (_response) {
deferred.resolve(exports.ClientResponse(_response, request.charset));
_response.on('error', deferred.reject); // ?! this is going to reject an already resolved deferred
});
_request.on('error', deferred.reject);
return deferred.promise;
});
};
i've probably messed up the logic in that code but hopefully it's good enough to give an outline of what i mean.
as far as how any of this relates to the original issue, code that was previously using q-io/http.request when it didn't pass along the agent could have been changed like so
// didn't work
var request = require('q-io/http').request;
var req = request({
host: ' ... ',
port: ' ... ',
agent: anAgent
});
// while waiting on q-io to add support for agent could be refactored to
var ClientRequest = require('q-io/http').ClientRequest;
var req = ClientRequest(require('http').request({
host: ' ... ',
port: ' ... ',
agent: anAgent
}));
from q-io.
Ah, thanks @neonstalwart. I think you’ll find that all mostly works, and you can use the NodeReader(nodeReadable)
and NodeWriter(nodeWritable)
constructors to create Q-IO streams from Node.js streams. These are in q-io/reader and q-io/writer, but will be in q-io/node/reader and q-io/node/writer in v2 unless something changes.
from q-io.
yes, those are what i used and it worked except for one small thing that didn't work as advertised - writer is missing the node
property that exists in reader and is also documented as existing so i had to add it myself.
var writer = new Writer(stream);
return writer.then(function (writer) {
// XXX: This is missing from the q-io writer even though the docs talk about it
writer.node = stream;
return writer;
});
from q-io.
@neonstalwart Thanks, I’ve added writer.node in my working copy. Sorry for missing that.
from q-io.
Related Issues (20)
- Reader().map() fails to limit properly the number of parallel instances of a function. HOT 3
- proxy requests HOT 2
- join() doesn't support network shares HOT 4
- not able to do x-www-unencoded body HOT 5
- Crash on reading large objects HOT 1
- problems by copying / moving data-files with special characters within path-variable HOT 7
- HTTP.read resolves with buffer instead of string HOT 1
- q-io downloading or writing too much data in version 1.13 HOT 14
- Mock: fs.copy() is buggy HOT 1
- removeTree() thinks it fails with ENOTEMPTY, but works on second try HOT 2
- `write()` resolves before content is written HOT 5
- Bump version of collections to avoid conflict with ES6 HOT 3
- filetree compatible with jstree
- `fs.copyTree` does not copy a large amount of files HOT 1
- is v2 up to date? HOT 3
- [v2] Last written file is always empty HOT 1
- Drop usage of collections, it breaks globals HOT 15
- Fixing a Prototype Override Protection Bypass Vulnerability in qs HOT 3
- Connect Timeout setting HOT 1
- Need 'qs' version updated to latest version to resolve Security Vulnerabilities HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from q-io.