aspnet / proxy Goto Github PK
View Code? Open in Web Editor NEW[Archived] Proxy library for ASP.NET Core. Project moved to https://github.com/aspnet/AspLabs
License: Apache License 2.0
[Archived] Proxy library for ASP.NET Core. Project moved to https://github.com/aspnet/AspLabs
License: Apache License 2.0
I'm working on some reverse proxy middleware for my day job and wanted to get an understanding of where this project is at—and where it is going—to see if there is potential to get involved.
After looking through the samples/unit tests/issues, I have a few questions about the goal of the project and its current status.
#20 from April 2016:
This implements a reverse proxy for forwarding incoming requests to back-end services
From the samples and unit tests, it looks like this middleware would be configured for an edge server that would pass requests back to one port on one host identified by the ProxyOptions
class.
If this is the case, how would one hook multiple back-end services into the edge server?
For example. given services A, B, and C running on different hosts—and for that matter listening on different ports—how would I use the current project to hook them into a single endpoint
I can see there are a handful of active contributors and a pair of lengthy PRs in progress. But the last Nuget package went out 10 months ago.
Does the project team see this repo as moving toward a production version, or is this for experimentation? If it is moving toward production, is there a target for a 1.0 package release? Or a list of features/issues that need to be built/fixed before then?
When using the proxy from within a controller action method the original request payload is empty when the proxy request is call if the method has a property binding, please allow the proxy to accept a lamda or a callback where the request payload/parameter collection maybe used/modified prior to forwarding the request to the destination url
The ProxyWebSocketsSmokeTest is flaky. This needs to be investigated and re-enabled.
Microsoft.AspNetCore.Proxy.Samples.csproj references nonexisting packages (1.2.0-*)
What do you think about adding the ability to add additional headers to the proxy request. For example, Authorization: Basic SomeBase64String
I was thinking of adding Headers as an IDictionary<string, string> property on the ProxyOptions.
Then in the ProxyMiddleware adding this on line 84:
// Add additional headers
foreach(var header in _options.Headers){
if (!requestMessage.Headers.TryAddWithoutValidation(header.Key, header.Value) && requestMessage.Content != null)
{
requestMessage.Content?.Headers.TryAddWithoutValidation(header.Key, header.Value);
}
}
I seems the HTTP the spec suggests removing the content-length header when a transfer-encoding header is set.
If a message is received with both a Transfer-Encoding and a
Content-Length header field, the Transfer-Encoding overrides the
Content-Length. Such a message might indicate an attempt to
perform request smuggling (Section 9.5) or response splitting
(Section 9.4) and ought to be handled as an error. A sender MUST
remove the received Content-Length field prior to forwarding such
a message downstream.
https://tools.ietf.org/html/rfc7230#section-3.3.3
It would be great, if I could provide a callback that will make decision where to route request. Without this I need to fork entire middleware (because my routing rules change during runtime).
If that's acceptable I would be happy to make a pull request.
It would be great to be able to utilize SignalR behind a Kestrel-hosted reverse proxy.
Thinking out loud...
HttpClient
does not do that.HttpClientHandler
with AllowAutoRedirect = false
is passed in the options, the redirects seem to go through, however the 301
responses reach the client unaltered and the proxy is bypassed on the next request.This is more of a support request than an issue maybe. If this is the wrong place for this question please direct me somewhere else.
When I run the samples it forwards websockets to an Angular Cli running ng serve just fine. When I try to use it with my newly created ASP.NET Core 2.0 that I created with VS 2017 15.3 it doesn't work. The forwarding works perfectly but it has trouble upgrading the connection to websockets.
I have the UseWebSockets in place:
app.UseWebSockets().RunProxy(new ProxyOptions
{
Scheme = scheme,
Host = host,
Port = port
});
And it proxies the HTTP stuff just fine, so I think I'm close.
Is there more documentation about how to use this?
Add the possibility to intercept the proxy response to eg filter out data, etc.
This is obsolete now that CoreCLR has a functioning HttpClient.
We have two Webservices. The first APP1 serves the wwwroot, a WEBAPI api/app1/.. and the proxy to the second Webservice (old ASP.NET) api/app2/..., with MapWhen (httpContext.Request.Path.Value.StartsWith == /api/app2/).
Everything works fine without IIS.
In IIS we use https://server/APP1 as base URI.
Requests for the second Webservice get a 400 HTTP error, received from the second Webservice.
After removing Request.PathBase from URI in ProxyMiddleware.cs everything worked.
Am i doing something wrong?
From @AceCoderLaura on March 31, 2017 3:56
When using a proxy to redirect calls from ASP to my WCF service, timeouts occur. I suspect that the proxy's HttpClient is using the default request timeout. There needs to be a way to set the timeout.
app.MapWhen(c => IsWCFCall(c), builder =>
{
builder.RunProxy(new ProxyOptions { Scheme = "http", Host = "localhost", Port = XivicConfiguration.WCFPort.ToString() });
});
Apologies if this is the wrong repository.
Copied from original issue: dotnet/aspnetcore#1981
I cannot find this package through Nuget. The only available version I see is 0.2.0 from Nov 15, 2016. I am looking at the Microsoft.AspNetCore.Proxy package. Is this project under a different name? Can spmeone help me on locating the most recent version? Thank you.
OPTIONS https://xyz.co.uk/front/api/Eze/GetEcPrt?_=1528800677059 401 (Unauthorized)
Proxy is not forwarding cookies due to a regression in HTTP client.
This is related to dotnet/corefx#16464 .
Current workaround would be to reference specific System.Net.Http 4.3.0.
I have created a regression test for which I will create a pull request. However I would prefer to have my PRs merged earlier.
Please consider adding some middleware logging/metrics to determine perform and/or any issues with routing to services
Hello, this issue tracker is no longer used. Please log new issues here: https://github.com/aspnet/Home/issues
For information about this change, see aspnet/Announcements#283
Is this package to allow proxying of incoming or outgoing requests from the application
I am looking for a core compatible client proxy as WebProxy only exists in desktop.
Thanks
I have discovered following issue in the Service Fabric Reverse Proxy.
Please see: microsoft/service-fabric-issues#210 for details.
However there is a possible performance improvement in ASP.NET Core Proxy to utilize the full buffer in case its underlying http server provides data slowly.
I have created an alternative CopyAsync implementation which uses buffer cyclically and more actively drains the source stream. It performs much better than original CopyAsync in scenario where source stream provides data in small chunks e.g. 4KiB and destination stream has Write with non-zero latency.
In that case CopyAsync has its throughput artificially decreased to: chunksize / latency. For example 4KiB / 50ms = 640kbps.
Below is the code of my proposed GreedyCopyToAsync based on the original Stream.CopyToAsync:
public static async Task GreedyCopyToAsync(this Stream source, Stream destination, int bufferSize, CancellationToken cancellationToken)
{
var buffer = ArrayPool<byte>.Shared.Rent(bufferSize);
long readPos = 0;
long readReservedTo = 0;
long writePos = 0;
int bytesRead = -1;
int bytesWritten = -1;
var currentReadTask = Task.CompletedTask;
var currentWriteTask = Task.CompletedTask;
try
{
while (true)
{
var writeCompleted = currentWriteTask.IsCompleted;
var readCompleted = currentReadTask.IsCompleted;
if (bytesWritten > 0 && writeCompleted)
{
writePos += bytesWritten;
bytesWritten = -1;
}
if (bytesRead > 0 && readCompleted)
{
readPos += bytesRead;
readReservedTo = readPos;
bytesRead = -1;
}
if (readCompleted)
{
if (bytesRead == 0) //End of stream reached
{
currentReadTask = currentWriteTask;
}
else
{
var readBuffBytesAvailable = buffer.Length - (int)(readPos - writePos);
if (readBuffBytesAvailable > 0)
{
var buffPos = (int)(readPos % buffer.Length);
var toRead = Math.Min(buffer.Length - buffPos, readBuffBytesAvailable);
currentReadTask = source.ReadAsync(buffer, buffPos, toRead, cancellationToken).ContinueWith(res => bytesRead = res.Result);
readReservedTo = readPos + toRead;
}
else
{
currentReadTask = currentWriteTask;
}
}
}
if (writeCompleted)
{
var writeBuffBytesAvailable = (int)(readPos - writePos);
if (writeBuffBytesAvailable > 0)
{
var buffPos = (int)(writePos % buffer.Length);
var toWrite = Math.Min(buffer.Length - buffPos, writeBuffBytesAvailable);
currentWriteTask = destination.WriteAsync(buffer, buffPos, toWrite, cancellationToken);
bytesWritten = toWrite;
}
else if (bytesRead == 0) //End of stream reached
{
return;
}
else
{
currentWriteTask = currentReadTask;
}
}
await Task.WhenAny(currentReadTask, currentWriteTask).ConfigureAwait(false);
}
}
finally
{
Array.Clear(buffer, 0, readPos > bufferSize ? bufferSize : (int)readPos); // clear only the most we used
ArrayPool<byte>.Shared.Return(buffer, clearArray: false);
}
}
Imagine you want to have the proxy done in such a way that
example1.com/my/method
is proxied to example2.com/some/folder/my/method
. The current implementation does not support this case as the resource path is always taken from the original request.
This could be achieved by having additional parameter in ProxyOptions
and adding this parameter's value to the proxied request URL.
This allows you to restore the values of the request with the override header middleware
Hello I'm using proxy on the azure VM connected to internal network via VPN.
It works all right most of the time however periodically I experience TCP packet loss (with tcp retransmission) when sending POST request. This leads to TaskCancelledException (operation timed out) in the following line:
HttpResponseMessage responseMessage = await _httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, context.RequestAborted);
I set HttpClient Timeout period to 15 minutes (which is more than necessary) but it seems when I start getting packet send problems following with (RST, ACK), connection stuck forever and terminates on timeout.
Sometimes I have the same Timeout error after 15-20 seconds no matter that I set ReceiveHeadersTimeout and SendTimeout to 900 seconds.
I never see the same problem when I run proxy on local network. With VPN, about 2-5 POST requests of 100 are failed. I'm not sure if I will have the same problem on full .net 4.5.
use case: sometimes a developer may wish to perform custom actions prior to proxying request to other services; these custom action may be generic for all proxy request or bespoke for a single type of request; by allowing the ProxyRequest extension to HttpContext to return an HttpResponseMessage to a controller that controller can them manipulate the response accordingly and the reply to the original request.
Please consider allowing an MVC controller calling the ProxyRequest to accept an HttpRequestMessage and/or returning a HttpMessageResponse and exposing the CreateProxyHttpRequest, SendProxyHttpRequest and CopyProxyHttpResponse methods to allow more customisations
Started performance loading this library in the last few days and I ran into an issue where I could only get it to perform around 3.3 requests per second.
Had to add System.Net.ServicePointManager.DefaultConnectionLimit = 1000; to my app startup to fix this issue.
I am wondering how much work, and how complicated it would be to add websocket support.
If anyone has some resources worth reading up on that can explain whats required to do so is also of help.
I can modify host, scheme or port via ProxyOptions. Can I modify also url?
I want url http://localhost:5000/Api/Controller/Id to be proxied to http://localhost:5001/Controller/Id.
Currently I have:
app.Map(@"/Api", builder => builder.RunProxy(new ProxyOptions
{
Scheme = "http",
Host = "localhost",
Port = "5001"
}));
The proxy should be able to send the underlying request to different hosts based on the request context.
For example
Incoming request: https://mysite.com/api1/
Outgoing request : http://myapi1.local/
Incoming request: https://mysite.com/api2/
Outgoing request : http://myapi2.local/
We could imagine a IProxyRouter. Default implementation is a DirectProxyRouter (the current implementation). Other implementation could be a rule-based router.
This will allow to have a basic reverse proxy.
Please provide more samples on how to use the Proxy middleware; dynamic routing and/or selective routing of URIs to backend services etc.
nginx
, or perhaps a closer match - golang's httputil.ReverseProxy
?It looks like NETStandard.Library is a global dependency in project.json, this leads to .NET Core dependencies even if targeting full framework.
Needs logic to handle other types of requests
PUT ,DELETE , HEAD etc
I have spent a good chunk of time chasing down an issue where a long running request through the proxy deadlocked other short requests until it finished. I finally narrowed it down to the fact that it was using the same MessageHandler for every request, and HttpClientHandler only seems to support one request at a time (I am on an older version of Core fwiw).
It may be best to make this option a function so that the user can create a new instance of the MessageHandler for each request to avoid deadlocks.
Think that I'm creating an application;
And does nothing (does not forward) for other URLs (example: /api/service3).
Thus, we can forward some URLs but also use MVC framework to provide other services in our application.
So, we need to a configuration option: a mapping or URLs
Opening as requested in #41
Currently due to lack of WebException support in netstandard13 in case of connect error, a generic 400 error is returned instead of forwarding original response.
In full .NET Framework following code could be used to extract the original response
var res = (ex.InnerException as WebException)?.Response;
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.