Giter Club home page Giter Club logo

aws-signature-version-4's Introduction

Well, hello there!

I'm FantasticFiasco.
My work is, as my name, sometimes debatable...

I've had the fortune of making my personal interest in software development into a career.

I believe in open source and am in awe of the responsibility developers take on their code created in the public space. I can't imagine where we'd been if it wasn't for open source. Because of this I'd like to do my part.

I'm a Jack of all trades and master of none which reflects on my contributions in the open source community. The majority of my work has been done in the .NET ecosystem, but there are contributions in JavaScript/TypeScript as well. A couple of my projects have been successful, with millions of downloads and hundreds of GitHub stars. A few of them are showcased in reference applications created by Microsoft.

Please get in touch with me if you have any questions regarding my work, either by opening an issue on one of my projects, or by contacting me on LinkedIn.



aws-signature-version-4's People

Contributors

daniel-rck avatar dependabot[bot] avatar fantasticfiasco avatar github-actions[bot] avatar mungojam avatar renovate-bot avatar renovate[bot] avatar timovzl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

aws-signature-version-4's Issues

Overloads of GetStringAsync

Is your feature request related to a problem? Please describe.
The GetStringAsync() extension on HttpClient is exactly what I need, but it's not currently supported with this (super) package

Describe the solution you'd like
I'd like it to work just like the other overloads, taking the AWS credentials as a parameter

Describe alternatives you've considered
I can call GetAsync() instead but then I need to work a bit harder with the response status codes etc.

Support Amazon S3

This package does not support Amazon S3 (Simple Storage Service). The decision for excluding S3 is based on the fact that it is a unique service when it comes to Signature Version 4, with exceptions to the rules of the signing algorithm.

Please give a thumbs up for this feature if you are requiring support for S3, and with enough votes, it will be implemented.

X-Amz-Content-SHA256 should be present when querying Amazon OpenSearch Serverless

Describe the bug

Update: Issue described below with the VPCE is due to the VPCE mutating a signed header. X-Amz-Content-SHA256 header is still a relevant issue but not the one described below.

Only reproducible when querying an Amazon OpenSearchServerless collection with a private network policy (accessibly only via VPCE). I am unable to reproduce on a collection with a public network policy.

Method: GET, RequestUri: 'https://host-id.us-west-2.aoss.amazonaws.com/_cat/indices', Version: 1.1, Content: System.Net.Http.StringContent, Headers:

{

  Accept: */*
  accountid: 855676708012
  User-Agent: curl/8.4.0
  x-forwarded-for: 15.248.7.84
  x-forwarded-port: 443
  x-forwarded-proto: https
  X-Amz-Date: 20240422T182608Z
  x-amz-security-token:  <redacted>
  Host:host-id.us-west-2.aoss.amazonaws.com
  Authorization: AWS4-HMAC-SHA256 Credential=ASIA4OOSOMSWA3GBWWEE/20240422/us-west-2/aoss/aws4_request, SignedHeaders=accept;accountid;host;user-agent;x-amz-date;x-amz-security-token;x-forwarded-for;x-forwarded-port;x-forwarded-proto, Signature=ec37d19eaf6227efe60c26a8690242c2126ef5449ed29a3755105e939d93004b
  Content-Length: 0
  Content-Type: application/json; charset=utf-8

} with  made by arn:aws:sts::855676708012:assumed-role/InsightsStack-EtlStorageOpenSearchServerlessApiProx-UMqk35Yx0VO4/InsightsStack-EtlStorageOpenSearchServerlessApiPro-p6lr3QP2XtH5 failed with StatusCode: 403, ReasonPhrase: 'Forbidden', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:

{

  X-Request-ID: d8d04efc-0f9e-97d6-bf07-bc655fd42594
  Date: Mon, 22 Apr 2024 18:26:08 GMT
  x-aoss-response-hint: X01:gw-helper-deny
  Server: aoss-amazon
  Content-Type: application/json
  Content-Length: 121
} Forbidden is not successful with '{"status":403,"request-id":"d8d04efc-0f9e-97d6-bf07-bc655fd42594","error":{"reason":"403 Forbidden","type":"Forbidden"}}

--

Based on the documentation,

The following requirements apply when signing requests to OpenSearch Serverless collections when you construct HTTP requests with another clients.
You must specify the service name as aoss.
The x-amz-content-sha256 header is required for all AWS Signature Version 4 requests. It provides a hash of the request payload. If there's a request payload, set the value to its Secure Hash Algorithm (SHA) cryptographic hash (SHA256). If there's no request payload, set the value to e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, which is the hash of an empty string.

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-clients.html#serverless-signing

Expected Behavior

X-Amz-Content-SHA256 should be present when service identifier is 'aoss'

Current Behavior

X-Amz-Content-SHA256 is not present when querying Amazon OpenSearch Serverless

Reproduction Steps

Infra
Client -> APIG -> Lambda -> VPCE -> AOSS (private)

Via the APIGateway Service Console:

  1. Create an API Gateway with a Lambda proxy integration.

Via the Lambda Service Console:

  1. Create a Lambda function

Via the OpenSearch Service Console:

  1. Create an Amazon OpenSearchServerless (AOSS) collection. When creating...
  2. Configure the collection with a "private" network policy.
  3. Create VPCE to access the private instance. Ensure the VPCE and Lambda are in the same subnet.
    1. Add an Inbound / Outbound rule for the Lambda SG
    2. Add an Inbound / Outbound rule for the AOSS SG
  4. Configure the collection with a data access policy. Add the Lambda execution role to the policy.

Sample Lambda Code

using System.Net;
using System.Net.Security;
using System.Security.Authentication;
using System.Text;
using System.Text.Json;

using Amazon.Lambda.APIGatewayEvents;
using Amazon.Lambda.Core;
using Amazon.Lambda.Serialization.SystemTextJson;
using Amazon.Runtime;

using AWS.Lambda.Powertools.Logging;

using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;

using EnvironmentVariables = Insights.Shared.Variables.EnvironmentVariables;
using HttpMethod = System.Net.Http.HttpMethod;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(DefaultLambdaJsonSerializer))]

namespace Functions;

//You can simplify to just query aoss directly in the lambda. This function is a generic proxy.
public class Function
{
    private const string ServiceKey = "awsService";
    private const string ServiceUriKey = "awsUri";
    private static readonly string Region = "us-west-2";
    /// <summary>
    /// Filter out query parameters used to configure the proxy request as these may cause the receiving service to fail. e.g. OpenSearch returns BadRequest when there are excess parameters
    /// </summary>
    private static readonly IReadOnlySet<string> ReservedQueryParameterKeys = new HashSet<string>([ServiceKey, ServiceUriKey]);
    /// <summary>
    /// Headers which should not be forwarded.
    /// See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Connection
    /// See https://www.rfc-editor.org/rfc/rfc7230#section-6.1
    /// </summary>
    private static readonly IReadOnlySet<string> HopByHopHeaders = new HashSet<string>([
        //sigv4
        "authorization",
        //standard
        "host",
        "connection",
        "keep-alive",
        "proxy-authenticate",
        "proxy-authorization",
        "te",
        "trailer",
        "transfer-encoding",
        "upgrade"
    ]);

    private static readonly TimeSpan Timeout = TimeSpan.FromSeconds(30);
    private static HttpClient CreateHttpClient()
    {
        var handler = new SocketsHttpHandler
        {
            PooledConnectionLifetime = Timeout * 2,
            ConnectTimeout = Timeout,
            SslOptions = new SslClientAuthenticationOptions
            {
                EnabledSslProtocols = SslProtocols.None
            }
        };
        return new HttpClient(handler)
        {
            Timeout = Timeout
        };
    }

    public HttpClient HttpClient { get; init; } = CreateHttpClient();
    public ILogger Logger { get; init; } = AWS.Lambda.Powertools.Logging.Logger.Create<Function>();
    public Func<AWSCredentials> CredentialProvider { get; init; } = FallbackCredentialsFactory.GetCredentials;
    //Absurd levels of indirection to mock things out
    public Func<HttpClient, HttpRequestMessage, string, string, AWSCredentials, Task<HttpResponseMessage>> SignAndSendAsyncFunc { get; init; } = SignAndSendAsync;

    [Logging(LogEvent = true)]
    public async Task<APIGatewayHttpApiV2ProxyResponse> HandleRequestAsync(APIGatewayHttpApiV2ProxyRequest request, ILambdaContext context)
    {
        try
        {
            using HttpResponseMessage response = await this.SendRequestAsync(request);
            string content = await response.Content.ReadAsStringAsync();
            if (response.IsSuccessStatusCode == false)
            {
                string requestContent = await (response.RequestMessage?.Content?.ReadAsStringAsync() ?? Task.FromResult(string.Empty));
                this.Logger.LogError("'{Request}' with '{RequestContent}' failed with '{Response}' '{Content}'", response.RequestMessage, requestContent, response, content);
            }
            return new APIGatewayHttpApiV2ProxyResponse
            {
                StatusCode = (int) response.StatusCode,
                Headers = response.Headers.ToDictionary(kvp => kvp.Key, kvp => string.Join(",", kvp.Value)),
                Body = content,
                IsBase64Encoded = false
            };
        }
        catch (HttpRequestException e)
        {
            this.Logger.LogError(e, "Failed to proxy request to AWS service");
            return new APIGatewayHttpApiV2ProxyResponse
            {
                StatusCode = (int) (e.StatusCode ?? HttpStatusCode.InternalServerError),
                Headers = new Dictionary<string, string>
                {
                    { "Content-Type", "application/json; charset=utf-8" }
                },
                Body = JsonSerializer.Serialize(new ErrorResponse(e.StatusCode ?? HttpStatusCode.InternalServerError, e.Message, request.RequestContext.RequestId), new JsonSerializerOptions().ConfigureDefaults()),
                IsBase64Encoded = false
            };
        }
        catch (Exception e)
        {
            this.Logger.LogError(e, "Failed to proxy request to AWS service");
            return new APIGatewayHttpApiV2ProxyResponse
            {
                StatusCode = StatusCodes.Status400BadRequest,
                Headers = new Dictionary<string, string>
                {
                    { "Content-Type", "application/json; charset=utf-8" }
                },
                Body = JsonSerializer.Serialize(new ErrorResponse(HttpStatusCode.BadRequest, e.Message, request.RequestContext.RequestId), new JsonSerializerOptions().ConfigureDefaults()),
                IsBase64Encoded = false
            };
        }
    }

    private async Task<HttpResponseMessage> SendRequestAsync(APIGatewayHttpApiV2ProxyRequest apiRequest)
    {
        var headers = apiRequest.Headers
            .Select(kvp => new KeyValuePair<string, string>(kvp.Key.ToLowerInvariant(), kvp.Value))
            //Remove 'x-amz' and 'authorization' SigV4 headers as these will cause the signing to fail
            .Where(kvp => kvp.Key.StartsWith("x-amz") == false)
            .Where(kvp => HopByHopHeaders.Contains(kvp.Key) == false)
            .GroupBy(kvp => kvp.Key)
            .ToDictionary(group => group.Key, group => string.Join(',', group.Select(kvp => kvp.Value)));

        if (string.IsNullOrWhiteSpace(apiRequest.RequestContext.Http.Method))
        {
            throw new InvalidOperationException("APIGatewayHttpApiV2ProxyRequest.RequestContext.Http.Method is empty / not set");
        }
        var method = HttpMethod.Parse(apiRequest.RequestContext.Http.Method);

        bool isContentTypeRequired = HttpMethod.Put.Equals(method) || HttpMethod.Post.Equals(method) || HttpMethod.Patch.Equals(method);
        if (headers.TryGetValue("content-type", out string? contentType) == false || string.IsNullOrWhiteSpace(contentType))
        {
            if (isContentTypeRequired)
            {
                throw new InvalidOperationException($"APIGatewayProxyRequest.Header 'content-type' is empty / not set and required for method '{method}'");
            }
        }
        if (apiRequest.PathParameters.TryGetValue("proxy", out string? path) == false || string.IsNullOrWhiteSpace(path))
        {
            throw new InvalidOperationException("APIGatewayHttpApiV2ProxyRequest.PathParameters 'proxy' is empty / not set. Cannot forward request");
        }

        if (apiRequest.QueryStringParameters.TryGetValue(ServiceKey, out string? service) == false || string.IsNullOrWhiteSpace(service))
        {
            throw new InvalidOperationException(
                $"APIGatewayHttpApiV2ProxyRequest.QueryStringParameters '{ServiceKey}' is empty / not set. Cannot forward request. Expect service authorization token. " +
                $"e.g. 'aoss'. See https://docs.aws.amazon.com/service-authorization/latest/reference/reference_policies_actions-resources-contextkeys.html");
        }

        if (apiRequest.QueryStringParameters.TryGetValue(ServiceUriKey, out string? baseUri) == false || string.IsNullOrWhiteSpace(baseUri))
        {
            throw new InvalidOperationException(
                $"APIGatewayHttpApiV2ProxyRequest.QueryStringParameters '{ServiceUriKey}' is empty / not set. Cannot forward request. Expect absolute Uri. e.g. 'https://domain-id.us-west-2.aoss.amazonaws.com'");
        }

        var parameterList = apiRequest.QueryStringParameters
            .Where(kvp => ReservedQueryParameterKeys.Contains(kvp.Key) == false)
            .Select(kvp => $"{kvp.Key}={kvp.Value}")
            .ToList();
        string parameters = parameterList.Count == 0 ? string.Empty : "?" + string.Join("&", parameterList);
        //We need to add '/' because the proxy path parameter does not include it
        var requestUri = new Uri($"{baseUri}/{path}{parameters}", UriKind.Absolute);
        string body = (apiRequest.IsBase64Encoded ? Encoding.UTF8.GetString(Convert.FromBase64String(apiRequest.Body)) : apiRequest.Body) ?? string.Empty;
        var request = new HttpRequestMessage(method, requestUri)
        {
            Content = new StringContent(body)
        };

        foreach (KeyValuePair<string, string> header in headers)
        {
            //Content headers must go into the content headers else it throws
            if (header.Key.StartsWith("content"))
            {
                _ = request.Content.Headers.Remove(header.Key);
                _ = request.Content.Headers.TryAddWithoutValidation(header.Key, header.Value);
            }
            else
            {
                //Remove request defaults because .NET hates humanity
                _ = request.Headers.Remove(header.Key);
                _ = request.Headers.TryAddWithoutValidation(header.Key, header.Value);
            }
        }
        this.Logger.LogInformation("Sending {HttpRequestMessage} to {Service} at {Region}", request, service, Region);
        return await this.SignAndSendAsyncFunc.Invoke(this.HttpClient, request, Region, service, this.CredentialProvider.Invoke());
    }

    private static async Task<HttpResponseMessage> SignAndSendAsync(HttpClient client, HttpRequestMessage request, string region, string service, AWSCredentials credentials) =>
        await client.SendAsync(request, region, service, credentials);

    private sealed record ErrorResponse(HttpStatusCode StatusCode, string Message, string RequestId);
}

Possible Solution

Update https://github.com/FantasticFiasco/aws-signature-version-4/blob/master/src/Private/Signer.cs#L138

Additional Information/Context

Context

I am attempting to proxy requests through API Gateway to our private AOSS collection via VPCE.

Infra
Client -> APIG -> Lambda -> VPCE -> AOSS

Security

  1. APIG
    1. IAM Authorizer
  2. Lambda
    1. Execution Role
      1. Allows all aoss operations
    2. Verified it is in the same VPC / Subnets as VPCE
  3. VPCE
    1. Security Group
      1. Allows Task Ingress / Egress
      2. Allows Lambda Ingress / Egress
      3. Allows AOSS Ingress / Egress
  4. AOSS
    1. Network Policy - insightsstack-network-3cd87f61b1
      1. Allows VPCE
    2. DataAccess Policy - insightsstack-data-access-0c0bf1
      1. Allows ECS Task Role
      2. Allows Lambda Task Role

AWS .NET SDK and/or Package version used

3.7.303.38

Targeted .NET Platform

.NET 8

Operating System and version

MacOS 13.6.1 (22G313), Amazon Linux

Default Request Headers are double added on Android

Describe the bug
A HttpClient with client.DefaultRequestHeaders.Add("x-api-key", myApiKey); is signed correctly, but the client itself adds the header twice into the request:

"Headers": [
{
  "Key": "x-api-key",
  "Value": [
    "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  ]
},
{
  "Key": "X-Amz-Date",
  "Value": [
...
}

To Reproduce
Steps to reproduce the behavior:
Try this code:

var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", accessToken);

// do a random request
var response = await client.GetAsync(requestUri, cancellationToken, regionName, serviceName, credentials);

You can see the request object before sending here.

Compare to the request object that is return in response.RequestMessage.

Expected behavior
The Request Header should contain the header "x-api-key" only once. Like this:

"Headers": [
{
  "Key": "x-api-key",
  "Value": [
    "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  ]
},
{
  "Key": "X-Amz-Date",
  "Value": [
...
}

Environment (please complete the following information):

  • AWS API Gateway]
  • OS: Android 8.0
  • .NET version: Mono.Android

Additional context
I have tested an .NET Core 2.1 Console application where everything runs fine.

How To Mock SendAsyncExtensions.SendAsync()?

Since SendAsyncExtensions.SendAsync() is an extension method, it cannot be mocked with Moq. What is the recommendation for mocking this method in unit tests?

I tried to mock the underlying method that is being called, but it looks like it never gets invoked when calling this extension method.

_mockHttpClient .Setup(x => x.SendAsync(It.IsAny<HttpRequestMessage>(), It.IsAny<CancellationToken>()))

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Warning

These dependencies are deprecated:

Datasource Name Replacement PR?
npm vm2 Unavailable

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

  • chore(deps): update aws sdk (.net) (AWSSDK.Core, AWSSDK.S3, AWSSDK.SecurityToken)
  • fix(deps): update dependency aws-cdk-lib to v2.148.0
  • fix(deps): update dependency constructs to v10.3.0

Pending Status Checks

These updates await pending status checks. To force their creation now, click the checkbox below.

  • chore(deps): update dependency @types/node to v20.14.11
  • chore(deps): update dependency prettier to v3.3.3
  • chore(deps): update eslint packages (js) to v7.16.1 (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile
infrastructure/.devcontainer/Dockerfile
  • mcr.microsoft.com/devcontainers/javascript-node 1-20-bullseye
github-actions
.github/workflows/ci-cd.yml
  • actions/checkout v4
  • codecov/codecov-action v4
.github/workflows/greetings.yml
  • actions/first-interaction v1
.github/workflows/infrastructure.yml
  • actions/checkout v4
  • actions/setup-node v4
.github/workflows/update-copyright-years-in-license-file.yml
  • actions/checkout v4
  • FantasticFiasco/action-update-license-year v3
npm
infrastructure/package.json
  • aws-cdk-lib 2.145.0
  • constructs 10.1.175
  • source-map-support 0.5.21
  • @types/aws-lambda 8.10.141
  • @types/node 20.14.10
  • @typescript-eslint/eslint-plugin 7.16.0
  • @typescript-eslint/parser 7.16.0
  • eslint 8.57.0
  • prettier 3.3.2
  • rimraf 5.0.9
  • typescript 5.5.3
  • ansi-regex >=5.0.1
  • pac-resolver >=5.0.0
  • vm2 >=3.9.4
scripts/package.json
  • @octokit/rest 20.1.1
nuget
src/AwsSignatureVersion4.csproj
  • Microsoft.SourceLink.GitHub 8.0.0
  • AWSSDK.Core 3.7.304.22
test/AwsSignatureVersion4.Test.csproj
  • xunit.runner.visualstudio 2.8.2
  • xunit 2.9.0
  • Shouldly 4.2.1
  • Microsoft.NET.Test.Sdk 17.10.0
  • Microsoft.Extensions.Http.Polly 8.0.7
  • Microsoft.Extensions.Http 8.0.0
  • Microsoft.Extensions.DependencyInjection 8.0.0
  • coverlet.collector 6.0.2
  • AWSSDK.SecurityToken 3.7.300.112
  • AWSSDK.S3 3.7.309.11

  • Check this box to trigger a request for Renovate to run again on this repository

PostAsync issues

When using PostAsync we are getting the following error

"Please specify data to sign. (Parameter 'data')"

This is our code

var credentials = new ImmutableCredentials("xxxxxxxxxxx", "xxxxxxxxxxxxxxxxx", null);
var client = new HttpClient();
var content = JsonConvert.SerializeObject(<Object>);
var stringContent = new StringContent(content, Encoding.UTF8, "application/json");

var response = await client.PostAsync(url, stringContent,
    "eu-west-2",
    "",
    credentials);

I'm assuming we are doing something wrong but unable to see where, grateful for any help. Thanks,

Deadlock when calling the code from an asp.net 4.8 web forms code behind

Describe the bug
Deadlock when calling the code from an asp.net 4.8 web forms code behind which is a synchronous code.

To Reproduce
Steps to reproduce the behavior:

  1. Create an asp.net web forms project
  2. Put following code in page_load event of a default.aspx page:
  3.  var content = new StringContent("..."
             , Encoding.UTF8, "application/json");
         var request = new HttpRequestMessage
         {
             Method = HttpMethod.Post,
             RequestUri = new Uri("..."),
             Content = content,
         };
         var credentials = new ImmutableCredentials("...", "...", null);
         var client = new HttpClient();
         var response = client.SendAsync(
           request: request,
           regionName: "...",
           serviceName: "execute-api",
           credentials: credentials).ConfigureAwait(false).GetAwaiter().GetResult();
    
  4. Code hangs

Expected behavior
Method should not hang

Environment (please complete the following information):

  • AWS service [AWS API Gateway]
  • OS: [Windows 10]
  • .NET version: [4.8]

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

File: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Invalid regExp for packageRules[0].packagePatterns: ^@aws-cdk,^aws-cdk``

HttpClient.Send (synchronous API) is not intercepted

Describe the bug

The package correctly intercepts HttpClient.SendAsync() (and all the extension methods that invoke it, such as GetAsync() or PostAsync()), adding the Authorization header and such.

However, the synchronous API is not properly intercepted. HttpClient.Send() does not cause the necessary headers to be added.

Presumably, this is because this API was added in .NET 5. A new target would be needed.

To Reproduce

In .NET 5, create an HttpClient configured with this package's message handler, which normally signs any request made. Construct an HttpRequestMessage and send it using HttpClient.Send(). After sending, confirm the absence of headers on the HttpRequestMessage. Compare this to HttpClient.SendAsync(), which causes the headers to be added as expected.

Expected behavior

On send, the necessary headers are expected to be added to the HttpRequestMessage, such as the Authorization header.

Environment

.NET 5.

Additional context

It would probably make sense to include .NET 5 as an additional target platform, so that the synchrous API can be intercepted overridden.

There are significant use cases for the HttpClient's synchronous API. The original proposal contains extensive explanations and discussions about it. It was finally added in .NET 5.

Stephen Toub explains:

The key is [sync-over-async] ends up requiring many more threads in order to sustain throughput. And it can take a long time for the thread pool to ramp up to the necessary level, while in the interim the system can essentially be deadlocked. That means either the app-level dev needs to explicitly set a high min thread count to force the ramp up, or we need to make the pool way more aggressive at thread injection, which harms other cases.

My own experience confirms that thread pool exhaustion happens extremely quickly with sync-over-async - much more so than one might expect. This limits the number of simultaneous operations to a fraction of what is achievable with a purely synchronous API.

In a .NET 5 preview blog, Stephen Toub further summarizes the addition of a synchronous API:

While HttpClient was designed for asynchronous usage, we have found situations where developers are unable to utilize asynchrony, such as when implementing an interface method that’s only synchronous, or being called from a native operation that requires a response synchronously, yet the need to download data is ubiquitous. In these cases, forcing the developer to perform “sync over async” (meaning performing an asynchronous operation and then blocking waiting for it to complete) performs and scales worse than if a synchronous operation were used in the first place. As such, .NET 5 sees limited new synchronous surface area added to HttpClient and its supporting types. dotnet/runtime does itself have use for this in a few places. For example, on Linux when the X509Certificates support needs to download a certificate as part of chain building, it is generally on a code path that needs to be synchronous all the way back to an OpenSSL callback; previously this would use HttpClient.GetByteArrayAsync and then block waiting for it to complete, but that was shown to cause noticeable scalability problems for some users…

Struggling to make API Gateway call

Just trying to write a simple console app to call API gateway using IAM authentication pulled from the IAM role associated with the EC2 instance the code is running on.

But I can't even get the code to compile, I must be missing something obvious but I have read the examples and cannot see what I am doing wrong. It's like .NET doesn't see the HttpClient() extensions your library provides.

I looked through the test folder and also Google'd for examples but came up empty.

We are using .NET Core 2.1 at present, but soon to be .NET Core 3.1

using System;
using System.Net.Http;
using System.Threading.Tasks;
using Amazon.Runtime;
using AwsSignatureVersion4;

namespace console
{
    class Program
    {
        static void Main(string[] args)
        {
          MainAsync().GetAwaiter().GetResult();
        }

      private static async Task MainAsync()
      {
        var credentials = new InstanceProfileAWSCredentials();
        var client = new HttpClient();
        client.DefaultRequestHeaders.Add("x-apigw-api-id","#####");
        var response = await client.GetAsync(
        "https://#####.execute-api.ap-southeast-2.vpce.amazonaws.com/Prod/api/values",
        regionName: "ap-southeast-2",
        serviceName: "execute-api",
        credentials: credentials);
      }
    }
}

NOTE: I removed the specifics of our environment.

Build error is

Program.cs(23,9): error CS1739: The best overload for 'GetAsync' does not have a parameter named 'regionName' [/home/ubuntu/code/console/console.csproj]

Any assistance would be greatly appreciated.

Support Unity game engine

Discussed in #474

Originally posted by AUEzzat August 19, 2021
Have anyone tried using it with unity?
I have an issue in il2cpp related to System.Web dll

Error: called non-existent method System.Collections.Specialized.NameValueCollection System.Web.HttpUtility::ParseQueryString(System.String)

Add PatchAsync overload

Is your feature request related to a problem? Please describe.
One of our APIs uses PATCH which isn't directly supported by this library.

Describe the solution you'd like
PATCH overload that would be very similar to the POST overload that already exists.

Describe alternatives you've considered
I'll use your SendAsync overload for now.

Additional context
I'll hopefully contribute more to this library in future, but lots going on at the moment. Thanks again for making it!

Don't push coverage on pull requests

Integration tests are not run on pull requests, thus it always looks like pull requests will lower the coverage.

We can run coverage on pull requests if we wan't, but we don't have to push the result.

Example Request: use with retrieving credentials from running instance

Hi! I'm being tasked with providing some examples for showing signature requests in multiple languages for our dev teams since I'm the one who suggested we use signed API-GATEWAY requests for a new project :)

This led me to find this package as .NET AWS SDK does not apparently have its own.

In our use case (and I imagine many others) the credentials used to sign the request will be coming from the running role of ec2/eks/ecs/lambda etc.

I've been looking for an example on .NET of how to retrieve the running instance session credentials but not having much luck.

If you could provide an example on how to sign using this package and the IAM ROLE provided credentials that would help me (and I assume many ohters!) out quite a bit.

Thanks!

Binary content does not POST. 408 error returned.

Passing a binary content will return 408 from the endpoint after a long wait.

To Reproduce

The following code will return a 408 from the service

var credentials = new ImmutableCredentials("AKIA2WNEXAMPLE", "rObND507EXAMPLE", null);

var client = new HttpClient();

byte[] fileBytes = File.ReadAllBytes(@"C:\File.pdf");
MemoryStream destination = new MemoryStream(fileBytes);
destination.Seek(0, SeekOrigin.Begin);

HttpContent content = new StreamContent(destination);

content.Headers.Add("x-api-key", "apikey");

var response = await client.PostAsync(
  "https://uat-customamazonservice.domain.net/v2/extract?inpFormat=PDF",
  content: content,
  regionName: "us-east-1",
  serviceName: "execute-api",
  credentials: credentials);

Checking web debugging tool (e.g. fiddler) you can see the content of the request is empty.

Expected behavior

The request should be sent and processed.

Desktop (please complete the following information):

Windows 11 .net 6 console client.

Additional context

If you send base64 encoded StringContent rather than StreamContent, the request is successful.

The fix is to reset the stream back to the beginning. I suggest doing this when the content hash is created by adding the following at line 30 in ContentHashcs.

contentStream.Seek(0, System.IO.SeekOrigin.Begin);

Working sample for simple GET request

New to github so sorry if this is the wrong place to ask. I am pretty confused with AWS singing and I have to talk to one of their new APIs (no SDK available) without any previous experience. Do you have a working example showing a full simple get request using your library to sign it? I am struggling getting off the ground with their docs and no experts to ask for help. Hoping for a little nudge.

Could not load file or assembly 'AwsSignatureVersion4, Version=4.0.0.0

I am making API calls to Lambda function url with HttpClient in an IntegrationTestSuite. When run independently, it works fine but when I call the IntegrationTestSuite.dll from another project, I get the below exception
System.IO.FileNotFoundException: 'Could not load file or assembly 'AwsSignatureVersion4, Version=4.0.0.0, Culture=neutral, PublicKeyToken=9830731f15ae0355'. The system cannot find the file specified.'

CancellationToken parameters must come last

According to CA1068 the cancellation token must come last. This would be applicable to both private/internal and public methods.

Since this is a breaking change, we will have to wait with fixing this issue.

Usage of IHttpClientFactory won't work in scenario where HttpClient is reused over longer time than passed token is valid

Describe the bug

In one of the projects that I am working on, I have used the approach described in https://github.com/FantasticFiasco/aws-signature-version-4#integration-with-ihttpclientfactory. Unfortunately, I believe it won't work in cases where for example I am passing a temporary token valid for a short amount of time, and the HttpClient is going to be reused after the token was expired.

To Reproduce

  1. Register services with AwsSignatureHandlerSettings that get token valid for a limited amount of time.
  2. Reuse HttpClient after the credentials passed to AwsSignatureHandlerSettings expired.

Expected behavior

It would be great to have some sort of refresh mechanism where we could update credentials passed to AwsSignatureHandlerSettings after a certain amount of time.

Actual behavior

Calls using expired token are responded with The security token included in the request is expired

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.