Giter Club home page Giter Club logo

proxy's Introduction

ASP.NET Core Proxy [Archived]

This GitHub project has been archived. Ongoing development on this prototype can be found in https://github.com/aspnet/AspLabs.

This repo hosts a proxy library for ASP.NET Core.

This project is part of ASP.NET Core. You can find samples, documentation and getting started instructions for ASP.NET Core at the AspNetCore repo.

proxy's People

Contributors

ajaybhargavb avatar analogrelay avatar aspnetci avatar brennanconroy avatar davidfowl avatar dougbu avatar eilon avatar hishamco avatar javiercn avatar jaymzzh avatar jkotalik avatar juntaoluo avatar kichalla avatar mikaelm12 avatar mkosieradzki avatar muratg avatar natemcmaster avatar ntaylormullen avatar pakrym avatar pranavkm avatar ryanbrandenburg avatar tratcher avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

proxy's Issues

Allow providing dynamic routing callback

It would be great, if I could provide a callback that will make decision where to route request. Without this I need to fork entire middleware (because my routing rules change during runtime).

If that's acceptable I would be happy to make a pull request.

Samples

Please provide more samples on how to use the Proxy middleware; dynamic routing and/or selective routing of URIs to backend services etc.

Redirect behavior

Thinking out loud...

  1. The default way for a reverse proxy to handle redirects should be to let them go back to the client. By default HttpClient does not do that.
  2. While passing redirect responses back to the client, the proxy should be cleaver enough to detect which URLs point to the upstream server behind the proxy and change them to point to the proxy instead (otherwise the client will bypass the proxy when following the redirect).
  3. If a HttpClientHandler with AllowAutoRedirect = false is passed in the options, the redirects seem to go through, however the 301 responses reach the client unaltered and the proxy is bypassed on the next request.

Suggestion: SharedProxyOptions.MessageHandler should be a Func that returns the MessageHandler

I have spent a good chunk of time chasing down an issue where a long running request through the proxy deadlocked other short requests until it finished. I finally narrowed it down to the fact that it was using the same MessageHandler for every request, and HttpClientHandler only seems to support one request at a time (I am on an older version of Core fwiw).

It may be best to make this option a function so that the user can create a new instance of the MessageHandler for each request to avoid deadlocks.

Proxy is not forwarding cookies

Proxy is not forwarding cookies due to a regression in HTTP client.
This is related to dotnet/corefx#16464 .

Current workaround would be to reference specific System.Net.Http 4.3.0.

I have created a regression test for which I will create a pull request. However I would prefer to have my PRs merged earlier.

Support for proxying to path in a (virtual) folder

Imagine you want to have the proxy done in such a way that
example1.com/my/method is proxied to example2.com/some/folder/my/method. The current implementation does not support this case as the resource path is always taken from the original request.

This could be achieved by having additional parameter in ProxyOptions and adding this parameter's value to the proxied request URL.

Strip Content-Length header when Transfer-Encoding header is set on request and response

I seems the HTTP the spec suggests removing the content-length header when a transfer-encoding header is set.

If a message is received with both a Transfer-Encoding and a
Content-Length header field, the Transfer-Encoding overrides the
Content-Length. Such a message might indicate an attempt to
perform request smuggling (Section 9.5) or response splitting
(Section 9.4) and ought to be handled as an error. A sender MUST
remove the received Content-Length field prior to forwarding such
a message downstream.
https://tools.ietf.org/html/rfc7230#section-3.3.3

aspnet/KestrelHttpServer#1163 (comment)

Proxy Callback/Lamda

When using the proxy from within a controller action method the original request payload is empty when the proxy request is call if the method has a property binding, please allow the proxy to accept a lamda or a callback where the request payload/parameter collection maybe used/modified prior to forwarding the request to the destination url

Behind IIS had to remove Request.PathBase from URI in ProxyMiddleware.cs

We have two Webservices. The first APP1 serves the wwwroot, a WEBAPI api/app1/.. and the proxy to the second Webservice (old ASP.NET) api/app2/..., with MapWhen (httpContext.Request.Path.Value.StartsWith == /api/app2/).

Everything works fine without IIS.

In IIS we use https://server/APP1 as base URI.
Requests for the second Webservice get a 400 HTTP error, received from the second Webservice.

After removing Request.PathBase from URI in ProxyMiddleware.cs everything worked.

Am i doing something wrong?

Load Testing

Started performance loading this library in the last few days and I ran into an issue where I could only get it to perform around 3.3 requests per second.

Had to add System.Net.ServicePointManager.DefaultConnectionLimit = 1000; to my app startup to fix this issue.

WebSocket connect error is not forwarded

Opening as requested in #41

Currently due to lack of WebException support in netstandard13 in case of connect error, a generic 400 error is returned instead of forwarding original response.

In full .NET Framework following code could be used to extract the original response

var res = (ex.InnerException as WebException)?.Response;

What is this Package For

Is this package to allow proxying of incoming or outgoing requests from the application
I am looking for a core compatible client proxy as WebProxy only exists in desktop.

Thanks

RunProxy Uses Default Timeout

From @AceCoderLaura on March 31, 2017 3:56

When using a proxy to redirect calls from ASP to my WCF service, timeouts occur. I suspect that the proxy's HttpClient is using the default request timeout. There needs to be a way to set the timeout.

app.MapWhen(c => IsWCFCall(c), builder =>
{
    builder.RunProxy(new ProxyOptions { Scheme = "http", Host = "localhost", Port = XivicConfiguration.WCFPort.ToString() });
});

Apologies if this is the wrong repository.

Copied from original issue: dotnet/aspnetcore#1981

Q: Questions - name, performance?

  1. Name - wouldn't ReverseProxy be a better fitting name?
  2. Performance - what should be expected, compared to say nginx, or perhaps a closer match - golang's httputil.ReverseProxy?
  3. Limitations?

Enable dynamic routing

The proxy should be able to send the underlying request to different hosts based on the request context.

For example
Incoming request: https://mysite.com/api1/
Outgoing request : http://myapi1.local/

Incoming request: https://mysite.com/api2/
Outgoing request : http://myapi2.local/

We could imagine a IProxyRouter. Default implementation is a DirectProxyRouter (the current implementation). Other implementation could be a rule-based router.
This will allow to have a basic reverse proxy.

NETStandard as global dependency

It looks like NETStandard.Library is a global dependency in project.json, this leads to .NET Core dependencies even if targeting full framework.

Adding additional Headers?

What do you think about adding the ability to add additional headers to the proxy request. For example, Authorization: Basic SomeBase64String

I was thinking of adding Headers as an IDictionary<string, string> property on the ProxyOptions.

Then in the ProxyMiddleware adding this on line 84:
// Add additional headers
foreach(var header in _options.Headers){
if (!requestMessage.Headers.TryAddWithoutValidation(header.Key, header.Value) && requestMessage.Content != null)
{
requestMessage.Content?.Headers.TryAddWithoutValidation(header.Key, header.Value);
}
}

MVC Controller Action ProxyRequest

use case: sometimes a developer may wish to perform custom actions prior to proxying request to other services; these custom action may be generic for all proxy request or bespoke for a single type of request; by allowing the ProxyRequest extension to HttpContext to return an HttpResponseMessage to a controller that controller can them manipulate the response accordingly and the reply to the original request.

Please consider allowing an MVC controller calling the ProxyRequest to accept an HttpRequestMessage and/or returning a HttpMessageResponse and exposing the CreateProxyHttpRequest, SendProxyHttpRequest and CopyProxyHttpResponse methods to allow more customisations

Connection Timeout on packets loss

Hello I'm using proxy on the azure VM connected to internal network via VPN.
It works all right most of the time however periodically I experience TCP packet loss (with tcp retransmission) when sending POST request. This leads to TaskCancelledException (operation timed out) in the following line:

HttpResponseMessage responseMessage = await _httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, context.RequestAborted);

I set HttpClient Timeout period to 15 minutes (which is more than necessary) but it seems when I start getting packet send problems following with (RST, ACK), connection stuck forever and terminates on timeout.
Sometimes I have the same Timeout error after 15-20 seconds no matter that I set ReceiveHeadersTimeout and SendTimeout to 900 seconds.

I never see the same problem when I run proxy on local network. With VPN, about 2-5 POST requests of 100 are failed. I'm not sure if I will have the same problem on full .net 4.5.

Install through Nuget

I cannot find this package through Nuget. The only available version I see is 0.2.0 from Nov 15, 2016. I am looking at the Microsoft.AspNetCore.Proxy package. Is this project under a different name? Can spmeone help me on locating the most recent version? Thank you.

Possible performance improvement when dealing with HTTP Servers with small buffers

I have discovered following issue in the Service Fabric Reverse Proxy.

Please see: microsoft/service-fabric-issues#210 for details.

However there is a possible performance improvement in ASP.NET Core Proxy to utilize the full buffer in case its underlying http server provides data slowly.

I have created an alternative CopyAsync implementation which uses buffer cyclically and more actively drains the source stream. It performs much better than original CopyAsync in scenario where source stream provides data in small chunks e.g. 4KiB and destination stream has Write with non-zero latency.

In that case CopyAsync has its throughput artificially decreased to: chunksize / latency. For example 4KiB / 50ms = 640kbps.

Below is the code of my proposed GreedyCopyToAsync based on the original Stream.CopyToAsync:

        public static async Task GreedyCopyToAsync(this Stream source, Stream destination, int bufferSize, CancellationToken cancellationToken)
        {
            var buffer = ArrayPool<byte>.Shared.Rent(bufferSize);

            long readPos = 0;
            long readReservedTo = 0;
            long writePos = 0;
            int bytesRead = -1;
            int bytesWritten = -1;

            var currentReadTask = Task.CompletedTask;
            var currentWriteTask = Task.CompletedTask;

            try
            {
                while (true)
                {
                    var writeCompleted = currentWriteTask.IsCompleted;
                    var readCompleted = currentReadTask.IsCompleted;

                    if (bytesWritten > 0 && writeCompleted)
                    {
                        writePos += bytesWritten;
                        bytesWritten = -1;
                    }

                    if (bytesRead > 0 && readCompleted)
                    {
                        readPos += bytesRead;
                        readReservedTo = readPos;
                        bytesRead = -1;
                    }

                    if (readCompleted)
                    {
                        if (bytesRead == 0) //End of stream reached
                        {
                            currentReadTask = currentWriteTask;
                        }
                        else
                        {
                            var readBuffBytesAvailable = buffer.Length - (int)(readPos - writePos);
                            if (readBuffBytesAvailable > 0)
                            {
                                var buffPos = (int)(readPos % buffer.Length);
                                var toRead = Math.Min(buffer.Length - buffPos, readBuffBytesAvailable);
                                currentReadTask = source.ReadAsync(buffer, buffPos, toRead, cancellationToken).ContinueWith(res => bytesRead = res.Result);
                                readReservedTo = readPos + toRead;
                            }
                            else
                            {
                                currentReadTask = currentWriteTask;
                            }
                        }
                    }

                    if (writeCompleted)
                    {
                        var writeBuffBytesAvailable = (int)(readPos - writePos);

                        if (writeBuffBytesAvailable > 0)
                        {
                            var buffPos = (int)(writePos % buffer.Length);
                            var toWrite = Math.Min(buffer.Length - buffPos, writeBuffBytesAvailable);
                            currentWriteTask = destination.WriteAsync(buffer, buffPos, toWrite, cancellationToken);
                            bytesWritten = toWrite;
                        }
                        else if (bytesRead == 0) //End of stream reached
                        {
                            return;
                        }
                        else
                        {
                            currentWriteTask = currentReadTask;
                        }
                    }

                    await Task.WhenAny(currentReadTask, currentWriteTask).ConfigureAwait(false);
                }
            }
            finally
            {
                Array.Clear(buffer, 0, readPos > bufferSize ? bufferSize : (int)readPos); // clear only the most we used
                ArrayPool<byte>.Shared.Return(buffer, clearArray: false);
            }
        }

metrics

Please consider adding some middleware logging/metrics to determine perform and/or any issues with routing to services

Should websockets work on release versions of ASP.NET Core and .NET Core?

This is more of a support request than an issue maybe. If this is the wrong place for this question please direct me somewhere else.

When I run the samples it forwards websockets to an Angular Cli running ng serve just fine. When I try to use it with my newly created ASP.NET Core 2.0 that I created with VS 2017 15.3 it doesn't work. The forwarding works perfectly but it has trouble upgrading the connection to websockets.

I have the UseWebSockets in place:

            app.UseWebSockets().RunProxy(new ProxyOptions
            {
                Scheme = scheme,
                Host = host,
                Port = port
            });

And it proxies the HTTP stuff just fine, so I think I'm close.

Is there more documentation about how to use this?

Here's the result of the request:
image

General info, project goals, status

I'm working on some reverse proxy middleware for my day job and wanted to get an understanding of where this project is at—and where it is going—to see if there is potential to get involved.

After looking through the samples/unit tests/issues, I have a few questions about the goal of the project and its current status.

Project goals

#20 from April 2016:

This implements a reverse proxy for forwarding incoming requests to back-end services

From the samples and unit tests, it looks like this middleware would be configured for an edge server that would pass requests back to one port on one host identified by the ProxyOptions class.

If this is the case, how would one hook multiple back-end services into the edge server?

For example. given services A, B, and C running on different hosts—and for that matter listening on different ports—how would I use the current project to hook them into a single endpoint

Project status

I can see there are a handful of active contributors and a pair of lengthy PRs in progress. But the last Nuget package went out 10 months ago.

Does the project team see this repo as moving toward a production version, or is this for experimentation? If it is moving toward production, is there a target for a 1.0 package release? Or a list of features/issues that need to be built/fixed before then?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.