Giter Club home page Giter Club logo

aws-xray-sdk-dotnet's Introduction

Build Status

📣 OpenTelemetry .NET with AWS X-Ray

AWS X-Ray supports using OpenTelemetry .NET and the AWS Distro for OpenTelemetry (ADOT) Collector to instrument your application and send trace data to X-Ray. The OpenTelemetry SDKs are an industry-wide standard for tracing instrumentation. They provide more instrumentations and have a larger community for support, but may not have complete feature parity with the X-Ray SDKs. See choosing between the ADOT and X-Ray SDKs for more help with choosing between the two.

If you want additional features when tracing your .NET applications, please open an issue on the OpenTelemetry .NET Instrumentation repository.

AWS X-Ray SDK for .NET and .NET Core

Screenshot of the AWS X-Ray console

Installing

The AWS X-Ray SDK for .NET and .NET Core (.netstandard 2.0 and above) is in the form of Nuget packages. You can install the packages from Nuget gallery or from Visual Studio editor. Search AWSXRayRecorder* to see various middlewares available.

Getting Help

Use the following community resources for getting help with the SDK. We use the GitHub issues for tracking bugs and feature requests.

Opening Issues

If you encounter a bug with the AWS X-Ray SDK for .NET/.NET Core, we want to hear about it. Before opening a new issue, search the existing issues to see if others are also experiencing the issue. Include platform (.NET/ .NET Core). In addition, include the repro case when appropriate.

The GitHub issues are intended for bug reports and feature requests. For help and questions about using the AWS X-Ray SDK for .NET and .NET Core, use the resources listed in the Getting Help section. Keeping the list of open issues lean helps us respond in a timely manner.

Documentation

The developer guide provides in-depth guidance about using the AWS X-Ray service. Following API reference documentation provides guidance for using the SDK and module-level documentation.

Quick Start

  1. Configuration
  2. ASP.NET Core Framework
  3. ASP.NET Framework
  4. Trace AWS SDK request
  5. Trace out-going HTTP requests
  6. Trace Query to SQL Server
  7. Trace SQL Query through Entity Framework
  8. Multithreaded Execution
  9. Trace custom methods
  10. Creating custom Segment/Subsegment
  11. Adding metadata/annotations
  12. AWS Lambda support (.NET Core)
  13. ASP.NET Core on AWS Lambda
  14. Logging
  15. Enabling X-Ray on Elastic Beanstalk
  16. Enabling X-Ray on AWS Lambda

Configuration

.NET

You can configure X-Ray in the appsettings of your App.config or Web.config file.

<configuration>
  <appSettings>
    <add key="DisableXRayTracing" value="false"/>
    <add key="AWSXRayPlugins" value="EC2Plugin, ECSPlugin, ElasticBeanstalkPlugin"/>
    <add key="SamplingRuleManifest" value="JSONs\DefaultSamplingRules.json"/>
    <add key="AwsServiceHandlerManifest" value="JSONs\AWSRequestInfo.json"/>
    <add key="UseRuntimeErrors" value="false"/>
    <add key="CollectSqlQueries" value="false"/>
  </appSettings>
</configuration>

.NET Core

Following are the steps to configure your .NET Core project with X-Ray.

a) In appsettings.json file, configure items under XRay key

{
  "XRay": {
    "DisableXRayTracing": "false",
    "SamplingRuleManifest": "SamplingRules.json",
    "AWSXRayPlugins": "EC2Plugin, ECSPlugin, ElasticBeanstalkPlugin",
    "AwsServiceHandlerManifest": "JSONs\AWSRequestInfo.json",
    "UseRuntimeErrors":"false",
    "CollectSqlQueries":"false"
  }
}

b) Register IConfiguration instance with X-Ray:

using Amazon.XRay.Recorder.Core;
AWSXRayRecorder.InitializeInstance(configuration); // pass IConfiguration object that reads appsettings.json file

Note:

  1. You should configure this before initialization of AWSXRayRecorder instance and using any AWS X-Ray methods.
  2. If you manually need to configure IConfiguration object refer: Link
  3. For more information on configuration, please refer : Link

Programmatic Configuration (.NET and .NET Core)

Alternatively, you can also set up the AWSXRayRecorder instance programmatically by using the AWSXRayRecorderBuilder class instead of a configuration file. For initializing an AWSXRayRecorder instance with default configurations, simply do the following.

using Amazon.XRay.Recorder.Core;

AWSXRayRecorder recorder = new AWSXRayRecorderBuilder().Build();
AWSXRayRecorder.InitializeInstance(recorder: recorder);

The following code initializes an AWSXRayRecorder instance with a custom IStreamingStrategy and a custom ISamplingStrategy.

using Amazon.XRay.Recorder.Core;

AWSXRayRecorder recorder = new AWSXRayRecorderBuilder().WithStreamingStrategy(new CustomStreamingStrategy()).WithSamplingStrategy(CustomSamplingStrategy()).Build();
AWSXRayRecorder.InitializeInstance(recorder: recorder);

Note:

  1. CustomStreamingStrategy and CustomSamplingStrategy must implement IStreamingStrategy and ISamplingStrategy before being used to build the recorder.
  2. recorder must be instantiated using AWSXRayRecorder.InitializeInstance(recorder: recorder) before being used in the program.

How to Use

Incoming Requests

ASP.NET Core Framework (.NET Core) : Nuget

You can instrument X-Ray for your ASP.NET Core App in the Configure() method of Startup.cs file of your project.
Note :

  1. For .Net Core 2.1 and above, use app.UseXRay() middleware before any other middleware to trace incoming requests. For .Net Core 2.0 place the app.UseXRay() middleware after the app.UseExceptionHandler("/Error") in order to catch exceptions. You would be able to see any runtime exception with its stack trace, however, the status code might show 200 due to a known limitation of the ExceptionHandler middleware in .Net Core 2.0.
  2. You need to install AWSXRayRecorder.Handlers.AspNetCore nuget package. This package adds extension methods to the IApplicationBuilder to make it easy to register AWS X-Ray to the ASP.NET Core HTTP pipeline.

A) With default configuration:

  • For .Net Core 2.1 and above:
using Microsoft.AspNetCore.Builder;

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UseXRay("SampleApp"); // name of the app
    app.UseExceptionHandler("/Error");
    app.UseStaticFiles(); // rest of the middlewares
    app.UseMVC();
}
  • For .Net Core 2.0:
using Microsoft.AspNetCore.Builder;

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UseExceptionHandler("/Error");
    app.UseXRay("SampleApp"); // name of the app
    app.UseStaticFiles(); // rest of the middlewares
    app.UseMVC();
}

B) With custom X-Ray configuration

using Microsoft.AspNetCore.Builder;

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UseExceptionHandler("/Error");
    app.UseXRay("SampleApp",configuration); // IConfiguration object is not required if you have used "AWSXRayRecorder.InitializeInstance(configuration)" method
    app.UseStaticFiles(); // rest of the middlewares
    app.UseMVC();	
}

Instead of name you can also pass SegmentNamingStrategy in the above two ways. Please refer: Link

ASP.NET Framework (.NET) : Nuget

HTTP Message handler for ASP.NET framework
Register your application with X-Ray in the Init() method of Global.asax file

using Amazon.XRay.Recorder.Handlers.AspNet;

public class MvcApplication : System.Web.HttpApplication
{
     public override void Init()
     {
        base.Init();
        AWSXRayASPNET.RegisterXRay(this, "ASPNETTest"); // default name of the web app
     }
}

At the start of each Http request, a segment is created and stored in the context (Key : AWSXRayASPNET.XRayEntity) of HttpApplication instance. If users write their custom error handler for ASP.NET framework, they can access segment for the current request by following way :

<%@ Import Namespace="Amazon.XRay.Recorder.Handlers.AspNet" %>
<%@ Import Namespace="Amazon.XRay.Recorder.Core.Internal.Entities" %>
<script runat="server">
  protected void Page_Load(object sender, EventArgs e)
  {
     var context = System.Web.HttpContext.Current.ApplicationInstance.Context;
     var segment = (Segment) context.Items[AWSXRayASPNET.XRayEntity]; // get segment from the context
     segment.AddMetadata("Error","404");
  }
</script>

Trace AWS SDK request (.NET and .NET Core) : Nuget

using Amazon.XRay.Recorder.Handlers.AwsSdk;

AWSSDKHandler.RegisterXRayForAllServices(); //place this before any instantiation of AmazonServiceClient
AmazonDynamoDBClient client = new AmazonDynamoDBClient(RegionEndpoint.USWest2); // AmazonDynamoDBClient is automatically registered with X-Ray

Methods of AWSSDKHandler class:

AWSSDKHandler.RegisterXRayForAllServices(); // all instances of AmazonServiceClient created after this line are registered

AWSSDKHandler.RegisterXRay<IAmazonDynamoDB>(); // Registers specific type of AmazonServiceClient : All instances of IAmazonDynamoDB created after this line are registered

AWSSDKHandler.RegisterXRayManifest(String path); // To configure custom AWS Service Manifest file. This is optional, if you have followed "Configuration" section

Trace out-going HTTP requests (.NET and .NET Core) : Nuget

Using System.Net.HttpWebRequest

Synchronous request

An extension method GetResponseTraced() is provided to trace GetResponse() in System.Net.HttpWebRequest class. If you want to trace the out-going HTTP request, call the GetResponseTraced() instead of GetResponse(). The extension method will generate a trace subsegment, inject the trace header to the out-going HTTP request header and collect trace information.

using Amazon.XRay.Recorder.Handlers.System.Net;

HttpWebRequest request = (HttpWebRequest) WebRequest.Create(URL); // enter desired url

// Any other configuration to the request

request.GetResponseTraced();

for query parameter stripped http requests in trace

using Amazon.XRay.Recorder.Handlers.System.Net;

HttpWebRequest request = (HttpWebRequest) WebRequest.Create(URL); // enter desired url

// Any other configuration to the request

request.GetResponseTraced(true);

Asynchronous request

An extension method GetAsyncResponseTraced() is provided to trace GetResponseAsync() in System.Net.HttpWebRequest class. If you want to trace the out-going HTTP request, call the GetAsyncResponseTraced() instead of GetResponseAsync(). The extension method will generate a trace subsegment, inject the trace header to the out-going HTTP request header and collect trace information.

using Amazon.XRay.Recorder.Handlers.System.Net;

HttpWebRequest request = (HttpWebRequest) WebRequest.Create(URL); // enter desired url

// Any other configuration to the request

request.GetAsyncResponseTraced();

for query parameter stripped http requests in trace

using Amazon.XRay.Recorder.Handlers.System.Net;

HttpWebRequest request = (HttpWebRequest) WebRequest.Create(URL); // enter desired url

// Any other configuration to the request

request.GetAsyncResponseTraced(true);

Using System.Net.HttpClient

A handler derived from DelegatingHandler is provided to trace the HttpMessageHandler.SendAsync method

using Amazon.XRay.Recorder.Handlers.System.Net;

var httpClient = new HttpClient(new HttpClientXRayTracingHandler(new HttpClientHandler()));

// Any other configuration to the client

httpClient.GetAsync(URL);

If you want to santize the Http request tracing then define the Tracing Handler as -

using Amazon.XRay.Recorder.Handlers.System.Net;

var httpClient = new HttpClient(new HttpClientXRaySanitizedTracingHandler(new HttpClientHandler()));

// Any other configuration to the client

httpClient.GetAsync(URL);

Using System.Net.Http.HttpClientFactory (.Net Core 2.1 and above)

The Amazon.XRay.Recorder.Handlers.System.Net package includes a delegate that can be used to trace outbound requests without the need to specifically wrap outbound requests from that class.

Register the HttpClientXRayTracingHandler as a middleware for your http client.

services.AddHttpClient<IFooClient, FooClient>()
        .AddHttpMessageHandler<HttpClientXRayTracingHandler>();

or

services.AddHttpClient("foo")
        .ConfigurePrimaryHttpMessageHandler(() =>
        {
            return new HttpClientXRayTracingHandler(new HttpClientHandler());
        });

And to get sanitized http requests in tracing

services.AddHttpClient<IFooClient, FooClient>()
        .AddHttpMessageHandler<HttpClientXRaySanitizedTracingHandler>();

or

services.AddHttpClient("foo")
        .ConfigurePrimaryHttpMessageHandler(() =>
        {
            return new HttpClientXRaySanitizedTracingHandler(new HttpClientHandler());
        });

Use the above client factory to create clients with outgoing requests traced.

var client = _clientFactory.CreateClient("foo");
var request = new HttpRequestMessage(HttpMethod.Get, "https://www.foobar.com");
var response = await client.SendAsync(request);

Trace Query to SQL Server (.NET and .NET Core) : Nuget

The SDK provides a wrapper class for System.Data.SqlClient.SqlCommand. The wrapper class can be used interchangable with SqlCommand class. By replacing instance of SqlCommand to TraceableSqlCommand, synchronized/asynchronized method will automatically generate subsegment for the SQL query.

Following examples illustrate the use of TraceableSqlCommand to automatically trace SQL Server queries using Synchronous/Asynchronous methods:

Synchronous query

using Amazon.XRay.Recorder.Handlers.SqlServer;

using (var connection = new SqlConnection("fake connection string"))
using (var command = new TraceableSqlCommand("SELECT * FROM products", connection))
{
    command.ExecuteNonQuery();
}

Asynchronous query

using Amazon.XRay.Recorder.Handlers.SqlServer;

using (var connection = new SqlConnection(ConnectionString))
{
    var command = new TraceableSqlCommand("SELECT * FROM Products FOR XML AUTO, ELEMENTS", connection);
    command.Connection.Open();
    await command.ExecuteXmlReaderAsync();
}

Capture SQL Query text in the traced SQL calls to SQL Server

You can also opt in to capture the SqlCommand.CommandText as part of the subsegment created for your SQL query. The collected SqlCommand.CommandText will appear as sanitized_query in the subsegment JSON. By default, this feature is disabled due to security reasons. If you want to enable this feature, it can be done in two ways. First, by setting the CollectSqlQueries to true in the global configuration for your application as follows:

For .Net (In appsettings of your App.config or Web.config file)
<configuration>
  <appSettings>
    <add key="CollectSqlQueries" value="true">
  </appSettings>
</configuration>
For .Net Core (In appsettings.json file, configure items under XRay key)
{
  "XRay": {
    "CollectSqlQueries":"true"
  }
}

This will enable X-Ray to collect all the sql queries made to SQL Server by your application.

Secondly, you can set the collectSqlQueries parameter in the TraceableSqlCommand instance as true to collect the SQL query text for SQL Server query calls made using this instance. If you set this parameter as false, it will disable the CollectSqlQuery feature for this TraceableSqlCommand instance.

using Amazon.XRay.Recorder.Handlers.SqlServer;

using (var connection = new SqlConnection("fake connection string"))
using (var command = new TraceableSqlCommand("SELECT * FROM products", connection, collectSqlQueries: true))
{
    command.ExecuteNonQuery();
}

NOTE:

  1. You should not enable either of these properties if you are including sensitive information as clear text in your SQL queries.
  2. Parameterized values will appear in their tokenized form and will not be expanded.
  3. The value of collectSqlQueries in the TraceableSqlCommand instance overrides the value set in the global configuration using the CollectSqlQueries property.

Trace SQL Query through Entity Framework (.NET and .NET Core) : Nuget

Setup

.NET Core

AWS XRay SDK for .NET Core provides interceptor for tracing SQL query through Entity Framework Core (>=3.0).

For how to start with Entity Framework Core in an ASP.NET Core web app, please take reference to Link

NOTE:

  • You need to install AWSXRayRecorder.Handlers.EntityFramework nuget package. This package adds extension methods to the DbContextOptionsBuilder to make it easy to register AWS X-Ray interceptor.
  • Not all database provider support Entity Framework Core 3.0 and above, please make sure that you are using the Nuget package with a compatible version (EF Core >= 3.0).

Known Limitation (as of 12-03-2020): If you're using another DbCommandInterceptor implementation along with the AddXRayInterceptor in the DbContext, it may not work as expected and you may see a "EntityNotAvailableException" from the XRay EFCore interceptor. This is due to AsyncLocal not being able to maintain context across the ReaderExecutingAsync and ReaderExecutedAsync methods. Ref here for more details on the issue.

In order to trace SQL query, you can register your DbContext with AddXRayInterceptor() accordingly in the ConfigureServices method in startup.cs file.

For instance, when dealing with MySql server using Nuget: Pomelo.EntityFrameworkCore.MySql (V 3.1.1).

using Microsoft.EntityFrameworkCore;

public void ConfigureServices(IServiceCollection services)
{ 
    services.AddDbContext<your_DbContext>(options => options.UseMySql(your_connectionString).AddXRayInterceptor());
}

Alternatively, you can register AddXRayInterceptor() in the Onconfiguring method in your DbContext class. Below we are using Nuget: Microsoft.EntityFrameworkCore.Sqlite (V 3.1.2)

using Microsoft.EntityFrameworkCore;

public class your_DbContext : DbContext 
{
	protected override void OnConfiguring(DbContextOptionsBuilder options)
    => options.UseSqlite(your_connectionString).AddXRayInterceptor();
}

The connection string can be either hard coded or configured from appsettings.json file.

.NET

AWS XRay SDK for .NET provides interceptor for tracing SQL query through Entity Framework 6 (>= 6.2.0).

For how to start with Entity Framework 6 in an ASP.NET web app, please take reference to link.

For instrumentation, you will need to install AWSXRayRecorder.Handlers.EntityFramework nuget package and call AWSXRayEntityFramework6.AddXRayInterceptor() in your code. Make sure to call it only once to avoid duplicate tracing.

For instance, you can call AddXRayInterceptor() in the Application_Start method of Global.asax file.

using Amazon.XRay.Recorder.Handlers.EntityFramework;

protected void Application_Start()
{
    AWSXRayEntityFramework6.AddXRayInterceptor();
}

Or you can call it in the DbConfiguration class if there is one in your application to configure execution policy.

using Amazon.XRay.Recorder.Handlers.EntityFramework;

public class YourDbConfiguration : DbConfiguration
{
    public YourDbConfiguration()
    {
        AWSXRayEntityFramework6.AddXRayInterceptor();
    }
}

Capture SQL Query text in the traced SQL calls to SQL Server

You can also opt in to capture the DbCommand.CommandText as part of the subsegment created for your SQL query. The collected DbCommand.CommandText will appear as sanitized_query in the subsegment JSON. By default, this feature is disabled due to security reasons.

.NET Core

If you want to enable this feature, it can be done in two ways. First, by setting the CollectSqlQueries to true in the appsettings.json file as follows:

{
  "XRay": {
    "CollectSqlQueries":"true"
  }
}

Secondly, you can set the collectSqlQueries parameter in the AddXRayInterceptor() as true to collect the SQL query text. If you set this parameter as false, it will disable the collectSqlQueries feature for this AddXRayInterceptor(). Opting in AddXRayInterceptor() has the highest execution priority, which will override the configuration item in appsettings.json mentioned above.

using Microsoft.EntityFrameworkCore;

public void ConfigureServices(IServiceCollection services)
{
    services.AddDbContext<your_DbContext>(options => options.UseMySql(your_connectionString).AddXRayInterceptor(true));
}

Or

using Microsoft.EntityFrameworkCore;

public class your_DbContext : DbContext 
{
	protected override void OnConfiguring(DbContextOptionsBuilder options)
    => options.UseSqlite(your_connectionString).AddXRayInterceptor(true);
}
.NET

You can enable tracing SQL query text for EF 6 interceptor in the Web.config file.

<configuration>
  <appSettings>
    <add key="CollectSqlQueries" value="true"/>
  </appSettings>
</configuration>

You can also pass true to AddXRayInterceptor() to collect SQL query text, otherwise pass false to disable. Opting in AddXRayInterceptor() has the highest execution priority, which will override the configuration item in Web.config mentioned above.

using Amazon.XRay.Recorder.Handlers.EntityFramework;

AWSXRayEntityFramework6.AddXRayInterceptor(true);

Multithreaded Execution (.NET and .NET Core) : Nuget

In multithreaded execution, X-Ray context from current to its child thread is automatically set.

using Amazon.XRay.Recorder.Core;

private static void TestMultiThreaded()
{
    int numThreads = 3;
    AWSXRayRecorder.Instance.BeginSegment("MainThread");
    Thread[] t= new Thread[numThreads];
 
    for(int i = 0; i < numThreads; i++)
    {
    	t[i] = new Thread(()=>MakeHttpRequest(i)); 
        t[i].Start();
    }
    for (int i = 0; i < numThreads; i++)
    {
        t[i].Join();
    }

    AWSXRayRecorder.Instance.EndSegment();
}

private static void MakeHttpRequest(int i)
{
    AWSXRayRecorder.Instance.TraceMethodAsync("Thread "+i, CreateRequestAsync<HttpResponseMessage>).Wait();
}

private static async Task<HttpResponseMessage> CreateRequestAsync <TResult>()
{
    var request = new HttpClient();
    var result = await request.GetAsync(URL); // Enter desired url
    return result;
}

Note:

  1. Context used to save traces in .NET : CallContext
  2. Context used to save traces in .NET Core : AsyncLocal

Trace custom methods (.NET and .NET Core)

It may be useful to further decorate portions of an application for which performance is critical. Generating subsegments around these hot spots will help in understanding their impact on application performance.

Synchronous method

using Amazon.XRay.Recorder.Core;

AWSXRayRecorder.Instance.TraceMethod("custom method", () => DoSomething(arg1, arg2, arg3));

Asynchronous method

using Amazon.XRay.Recorder.Core;

var response = await AWSXRayRecorder.Instance.TraceMethodAsync("AddProduct", () => AddProduct<Document>(product));

private async Task<Document> AddProduct <TResult>(Product product)
{
    var document = new Document();
    document["Id"] = product.Id;
    document["Name"] = product.Name;
    document["Price"] = product.Price;
    return await LazyTable.Value.PutItemAsync(document);
}

Creating custom Segment/Subsegment (.NET and .NET Core)

Segment

using Amazon.XRay.Recorder.Core;

AWSXRayRecorder.Instance.BeginSegment("segment name"); // generates `TraceId` for you
try
{
    DoSometing();
    // can create custom subsegments
}
catch (Exception e)
{
    AWSXRayRecorder.Instance.AddException(e);
}
finally
{
    AWSXRayRecorder.Instance.EndSegment();
}

If you want to pass custom TraceId:

using Amazon.XRay.Recorder.Core;

String traceId = TraceId.NewId(); // This function is present in : Amazon.XRay.Recorder.Core.Internal.Entities
AWSXRayRecorder.Instance.BeginSegment("segment name",traceId); // custom traceId used while creating segment
try
{
    DoSometing();
    // can create custom subsegments
}
catch (Exception e)
{
    AWSXRayRecorder.Instance.AddException(e);
}
finally
{
    AWSXRayRecorder.Instance.EndSegment();
}

Subsegment

Note: This should only be used after BeginSegment() method.

using Amazon.XRay.Recorder.Core;

AWSXRayRecorder.Instance.BeginSubsegment("subsegment name");
try
{
    DoSometing();
}
catch (Exception e)
{
    AWSXRayRecorder.Instance.AddException(e);
}
finally
{
    AWSXRayRecorder.Instance.EndSubsegment();
}

Adding metadata/annotations (.NET and .NET Core)

using Amazon.XRay.Recorder.Core;
AWSXRayRecorder.Instance.AddAnnotation("mykey", "my value");
AWSXRayRecorder.Instance.AddMetadata("my key", "my value");

AWS Lambda support (.NET Core)

The AWS Lambda execution environment by default creates a Segment before each Lambda function invocation and sends it to the X-Ray service. AWS X-Ray .NET/Core SDK will make sure there will be a FacadeSegment inside the lambda context so that you can instrument your application successfully through subsegments only. Subsegments generated inside a Lambda function are attached to this FacadeSegment and only subsegments are streamed by the SDK. In addition to the custom subsegments, the middlewares would generate subsegments for outgoing HTTP calls, SQL queries, and AWS SDK calls within the lambda function the same way they do in a normal application.

Note: You can only create and close Subsegment inside a lambda function. Segment cannot be created inside the lambda function. All operations on Segment will throw an UnsupportedOperationException exception.

public string FunctionHandler(string input, ILambdaContext context)
{
    AWSXRayRecorder recorder = new AWSXRayRecorder();
    recorder.BeginSubsegment("UpperCase");
    recorder.BeginSubsegment("Inner 1");
    String result = input?.ToUpper();
    recorder.EndSubsegment();
    recorder.BeginSubsegment("Inner 2");
    recorder.EndSubsegment();
    recorder.EndSubsegment();
    return result;
}

Oversampling Mitigation

Oversampling mitigation allows you to ignore a parent segment/subsegment's sampled flag and instead sets the subsegment's sampled flag to false. This ensures that downstream calls are not sampled and this subsegment is not emitted.

using Amazon.Lambda.Core;
using Amazon.Lambda.SQSEvents;
using Amazon.XRay.Recorder.Core;
using Amazon.SQS;
using Amazon.SQS.Model;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]

namespace MyFunction;

public class Function
{
    public string HandleSQSEvent(SQSEvent sqsEvent, ILambdaContext context)
    {
        AWSXRayRecorder.Instance.BeginSubsegmentWithoutSampling("Processing Event");

        var client = new AmazonSQSClient();

        var request = new ListQueuesRequest();

        var response = client.ListQueuesAsync(request);

        foreach (var url in response.Result.QueueUrls)
        {
            Console.WriteLine(url);
        }

        AWSXRayRecorder.Instance.EndSubsegment();

        return "Success";
    }
}

The code below demonstrates overriding the sampled flag based on the SQS messages sent to Lambda.

using Amazon.Lambda.Core;
using Amazon.Lambda.SQSEvents;
using Amazon.XRay.Recorder.Core;
using Amazon.XRay.Recorder.Core.Lambda;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]

namespace MyFunction;

public class Function
{
    public string HandleSQSEvent(SQSEvent sqsEvent, ILambdaContext context)
    {

        foreach (SQSEvent.SQSMessage sqsMessage in sqsEvent.Records)
        {
            if (SQSMessageHelper.IsSampled(sqsMessage))
            {
                AWSXRayRecorder.Instance.BeginSubsegment("Processing Message");
            }
            else
            {
                AWSXRayRecorder.Instance.BeginSubsegmentWithoutSampling("Processing Message");
            }


            // Do my processing work here
            Console.WriteLine("Doing processing work");

            // End my subsegment
            AWSXRayRecorder.Instance.EndSubsegment();
        }

        return "Success";
    }
}

ASP.NET Core on AWS Lambda (.NET Core)

We support instrumenting ASP.NET Core web app on Lambda. Please follow the steps of ASP.NET Core instrumentation.

Logging (.NET)

The AWS X-Ray .NET SDK share the same logging mechanism as AWS .NET SDK. If the application had already configured logging for AWS .NET SDK, it should just work for AWS X-Ray .NET SDK. The recommended way to configure an application is to use the element in the project’s App.config or Web.config file.

<configuration>
  <configSections>
    <section name="aws" type="Amazon.AWSSection, AWSSDK.Core"/>
  </configSections>
  <aws>
    <logging logTo="Log4Net"/>
  </aws>
</configuration>

Other ways to configure logging is to edit the element in the App.config or Web.config file, and set property values in the AWSConfig class. Refer to the following page for more details and example : Link

Logging (.NET Core)

The AWS X-Ray .NET SDK share the same logging mechanism as AWS .NET SDK. To configure logging for .NET Core application, pass one of these options to the AWSXRayRecorder.RegisterLogger method.

Following is the way to configure log4net with X-Ray SDK:

using Amazon;
using Amazon.XRay.Recorder.Core;

class Program
{
    static Program()
    {
         AWSXRayRecorder.RegisterLogger(LoggingOptions.Log4Net); // Log4Net instance should already be configured before this point
    }
}

log4net.config example:

<?xml version="1.0" encoding="utf-8" ?>
<log4net>
  <appender name="FileAppender" type="log4net.Appender.FileAppender,log4net">
    <file value="c:\logs\sdk-log.txt" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %level %logger - %message%newline" />
    </layout>
  </appender>
  <logger name="Amazon">
    <level value="DEBUG" />
    <appender-ref ref="FileAppender" />
  </logger>
</log4net>

Note: For log4net configuration, refer : Link

License

The AWS X-Ray SDK for .NET and .NET Core is licensed under the Apache 2.0 License. See LICENSE and NOTICE.txt for more information.

aws-xray-sdk-dotnet's People

Contributors

atshaw43 avatar bhautikpip avatar dependabot[bot] avatar djluck avatar fodonnel avatar hyandell avatar jaredcnance avatar jaybyrrd avatar jj22ee avatar jon-armen avatar ljacobsson avatar lupengamzn avatar madsciencist avatar marcomagdy avatar martincostello avatar matthewdouglas avatar mpetito avatar mrlarssonjr avatar nathanielrn avatar nedjs avatar normj avatar ovalba avatar rversaw avatar samunro avatar sdolier avatar sergedomk avatar srprash avatar stevejgordon avatar wangzlei avatar willarmiros avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-xray-sdk-dotnet's Issues

Entity doesn't exist in AsyncLocal exception

Hi,

I'm currently working with ASP.NET Core 2.1 and AWSXRayRecorder 2.5.0.
I think that something is missing (at least a better exception) about using aws services outside a http request.

Here is a quick sample to reproduce the issue:

        public Startup(IConfiguration configuration)
        {
            AWSXRayRecorder.InitializeInstance(configuration);
            AWSSDKHandler.RegisterXRayForAllServices();
        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            app.UseXRay("sample");
            Test().GetAwaiter().GetResult();
        }

        private static async Task Test()
        {
            var s3 = new AmazonS3Client(new AnonymousAWSCredentials(), RegionEndpoint.USEast1);
            await s3.GetObjectAsync("testBucket", "testKey");
        }

When using RegisterXRayForAllServices(), from my point of view, something is missing when using a service without adding BeginSegment() and EndSegment() around it.
If you're initializing a library or anything else which is calling a aws service, it will throw an exception.

I didn't test yet but we probably have the same issue with a BackgroundService if we miss adding a new Segment.

What's your opinion about it?

Exception: "cannot add subsegment to a completed segment."

We are receiving an Amazon.XRay.Recorder.Core.Exceptions.EntityNotAvailableException with the message "Cannot add subsegment to a completed segment."

What we are using
.net core 2.0 MVC
installed

  • AWSSDK.Core - 3.3.23
  • AWSSDK.S3 - 3.3.18
  • AWSXRayRecorder - 2.2.0-beta
  • AWSXRayRecorder.Core - 2.2.0-beta
  • AWSXRayRecorder.Handlers.AwsSdk - 2.2.0-beta

We have a NuGet package with our business logic code. Within our API controller we call our business logic code within a Task. It appears that this error is related to our spinning up a new thread via Task.Run(...) The reason why we say this is when we remove the Task.Run(...) from the code, we no longer receive the AWSXRay exception "cannot add subsegment to a completed segment."

We really want to use AWSXRay because we see the value in it - we have a UI API that uses AWSXRay as well as some AWS Lambdas. We have to have spin up a thread to run our business logic because it requires time to crunch. (If it didn't take so long, we would be using Lambdas.)

So, what steps/code do we have to do to resolve this exception?

Code snippets:
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
Amazon.XRay.Recorder.Handlers.AwsSdk.AWSSDKHandler.RegisterXRayForAllServices();

	...
	
	services.AddMvc();
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
	...
	app.UseXRay("Phoenix.Api.Analysis");
	app.UseMvc();
}

}

[Route("v1/api/analysis")]
public class AnalysisController : Controller
{
[HttpPost]
public IActionResult Create([FromBody] AnalysisModel analysisModel)
{
try
{
...
var analysisEngineContainer = new AnalysisEngineContainer(...);
Task.Run(() => analysisEngineContainer.StartAnalyze(...));
}
catch (Exception ex)
{
...
return StatusCode(500);
}
}
}

Caught exception:
"{"ClassName":"System.AggregateException","Message":"One or more errors occurred.","Data":{},"InnerException":{"ClassName":"Amazon.XRay.Recorder.Core.Exceptions.EntityNotAvailableException","Message":"Cannot add subsegment to a completed segment.","Data":{},"InnerException":null,"HelpURL":null,"StackTraceString":" at Amazon.XRay.Recorder.Core.Internal.Entities.Entity.AddSubsegment(Subsegment subsegment)\r\n at Amazon.XRay.Recorder.Core.AWSXRayRecorder.AddSubsegment(Subsegment subsegment)\r\n at Amazon.XRay.Recorder.Core.AWSXRayRecorder.BeginSubsegment(String name)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at Amazon.XRay.Recorder.Core.AWSXRayRecorder.BeginSubsegment(String name)\r\n at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.ProcessBeginRequest(IExecutionContext executionContext)\r\n at Amazon.XRay.Recorder.Handlers.Aws
Sdk.Internal.XRayPipelineHandler.d__271.MoveNext()\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__91.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Amazon.Runtime.Internal.ErrorCallbackHandler.d__51.MoveNext() in E:\\\\JenkinsWorkspaces\\\\v3-trebuchet-release\\\\AWSDotNetPublic\\\\sdk\\\\src\\\\Core\\\\Amazon.Runtime\\\\Pipeline\\\\Handlers\\\\ErrorCallbackHandler.cs:line 58\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n at Syste m.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n at Amazon.Runtime.Internal.MetricsHandler.<InvokeAsync>d__11.MoveNext()","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":"AWSXRayRecorder.Core","WatsonBuckets":null},"HelpURL":null,"StackTraceString":" at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)\\r\\n at Phoenix.Business.Analysis.Engine.AnalysisBuilder.AnalysisNotification.SnsAnalysisNotificationService.Notify(Int64 analysisId, Int32 modelId, String message) in C:\\\\TeamCity\\\\buildAgent\\\\work\\\\Phoenix.Business.Analysis.Engine.AnalysisBuilder\\\\1.0.106\\\\AnalysisBuilder\\\\AnalysisNotification\\\\SnsAnalysisNotificationService.cs:line 30\\r\\n at Phoenix.Business.Analysis.Engine.AnalysisBuilder.SingleAnalysisEngineContainer.Analyze(Int64 analysisId, IEnumerable1
analysisEngineInputs, WarehouseInput resultsWarehouseInput, AnalysisRecordingType analysisRecordingType, Nullable1 parentAnalysisId) in C:\\\\TeamCity\\\\buildAgent\\\\work\\\\Phoenix.Business.Analysis.Engine.AnalysisBuilder\\\\1.0.106\\\\AnalysisBuilder\\\\SingleAnalysisEngineContainer.cs:line 59\\r\\n at Phoenix.Business.Analysis.Engine.AnalysisBuilder.AnalysisEngineContainer.StartAnalyze(ImpactAnalysis impactAnalysis, ISingleAnalysisEngineContainer singleAnalysisEngineContainer, Int64 analysisId, List1 analysisEngineInputs, WarehouseInput warehouseInput, List1 mrioRegionModelIds, AnalysisRecordingType analysisRecordingType) in C:\\\\TeamCity\\\\buildAgent\\\\work\\\\Phoenix.Business.Analysis.Engine.AnalysisBuilder\\\\1.0.106\\\\AnalysisBuilder\\\\AnalysisEngineContainer.cs:line 125\\r\\n at Phoenix.Api.Analysis.AnalysisApi.Controllers.v1.AnalysisController.<>c__DisplayClass7_1.<Create>b__0() in C:\\\\Projects\\\\Bitbucket\\\\Phoenix\\\\Api\\\\Analysis\\\\AnalysisApi\\\\Controllers\\\\v1\\\\AnalysisC ontroller.cs:line 230\\r\\n at System.Threading.Tasks.Task1.InnerInvoke()\r\n at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":"System.Private.CoreLib","WatsonBuckets":null,"InnerExceptions":[{"ClassName":"Amazon.XRay.Recorder.Core.Exceptions.EntityNotAvailableException","Message":"Cannot add subsegment to a completed segment.","Data":{},"InnerException":null,"HelpURL":null,"StackTraceString":" at Amazon.XRay.Recorder.Core.Internal.Entities.Entity.AddSubsegment(Subsegment subsegment)\r\n at Amazon.XRay.Recorder.Core.AWSXRayRecorder.AddSubsegment(Subsegment subsegment)\r\n at Amazon.XRay.Recorder.Core.AWSXRayRecorder.BeginSubsegment(String name)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.
Throw()\r\n at Amazon.XRay.Recorder.Core.AWSXRayRecorder.BeginSubsegment(String name)\r\n at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.ProcessBeginRequest(IExecutionContext executionContext)\r\n at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.d__271.MoveNext()\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__91.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Amazon.Runtime.Internal.ErrorCallbackHandler.d__51.MoveNext() in E:\\\\JenkinsWorkspaces\\\\v3-trebuchet-release\\\\AWSDotNetPublic\\\\sdk\\\\src\\\\Core\\\\Amazon.Runtime\\\\Pipeline\\\\Handlers\\\\ErrorCallbackHandler.cs:line 58\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n at Amazon.Runtime.Internal.MetricsHandler.<InvokeAsync>d__11.MoveNext()","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":"AWSXRayRecorder.Core","WatsonBuckets":null}]}"

UdpSegmentEmitter throws Message too long

Sometimes happens that the buffer to sent to xray deamon exceed the maximum udp packet size.

As result it throws a
Failed to send package through socket. --> System.Net.Sockets.SocketException (0x80004005): Message too long

HybridContext Implementation for ASP.NET Framework

Problem

For X-Ray .NET SDK version <= 2.3.1-beta, ASP.NET XRay middleware usedTraceContext that implemented CallContext as an underlying data structure. This CallContext stored Segment/Subsegment created for the given lifecycle of incoming HTTP request. Value (Segment/Subsegment) in CallContext for POST request with certain payload is lost resulting inEntityNotAvailableException. This behavior is unpredictable.

Reference : issue#26

Analysis

  1. After some research, it became apparent that, it is not guaranteed the same thread serves incoming request for ASP.NET framework.
  2. For the above described issue, ASP.NET framework makes a thread switch while processing POST request. This thread switch results in CallContext value to be lost in the controller that handles POST request. The CallContext value is not propagated due to ASP.NET thread pooling.
  3. As a result, Segment stored in CallContext at the start of incoming request is not available in the controller, that results in exception for creating Subsegment.

Research

  1. HttpContext object is per incoming HTTP request and data stored in HttpContext.Items is persisted across components of ASP.NET.
  2. As per official documentation, this HttpContext object is only available in the thread controlled by ASP.NET. Usage in background threads could lead to undefined behavior.
  3. This object is ready for garbage collection when the HttpRequest is completed. Its usage after the request completes could lead to undefined behavior, such as a NullReferenceException.

Solution

Instead of using only CallContext , TraceContext now uses HybridContext : CallContext for all CRUD operations on Entity (Segment/Subsegment) and HttpContext.Items to store the Segment started by ASP.NET X-Ray middleware.

At any point if the CallContext doesn't have value, Segment reference is copied from HttpContext to CallContext. This results in restoring already created Segment-Subsegment hierarchy and new Subsegment will be attached to the current Segment.

The implementation is available here and is released in version 2.4.0-beta Changelog.

ASP.NET X-Ray middleware by default now uses HybridContextContainer.

Limitation

For the POST request, if async task is directly started in the controller, without using X-Ray wrapper TraceMethodAsync, the application may experience EntityNotAvailableException. The HttpContext has the segment before starting async task, however, CallContext doesn't (due to above issue) and the HttpContext may not be propagated to async task.

Workaround

InjectEntityInTraceContext method should be explicitly called before spawning async task. This method ensures, Segment reference is copied from HttpContext to CallContext and CallContext get propagated to the following async task.

We would appreciate if you have any feedback/suggestions.

Thanks,
Yogi

Don't set MaxStackFrameSize to 50 in a const variable like you do now for Exception handling

The max stack frame size is cutting off the interesting bits of my Exception. If you could make that value at least configurable somehow or make it a public static value that I could set on application start that would allow me to capture more Exception details than I'm getting now. The value is currently hardcoded to 50. I haven't found any XRay docs stating 50 is a limit for the stack frame limit.

https://github.com/aws/aws-xray-sdk-dotnet/blob/191a7f085cbb8eb3e2a2121754d057db1f095e05/sdk/src/Core/Internal/Entities/Cause.cs

Error when pulling secret using custom Manifest

Just pulled in 2.6.0 and got error attempting to pull secret from Secrets Manager.

I haven't researched this extensively yet, but it appears to be directly related to custom Manifest changes added in #63.

Given that I was responsible for this change, I feel I must apologize for not catching it.

Received the following error regardless of whether the custom manifest was added as a path or stream.

AmazonXRayClient 20|2019-04-22T09:10:01.942Z|ERROR|An exception of type HttpErrorResponseException was handled in ErrorHandler. --> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\HttpHandler\_mobile\HttpRequestMessageFactory.cs:line 539
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\HttpHandler\HttpHandler.cs:line 175
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
AmazonXRayClient 21|2019-04-22T09:10:02.041Z|ERROR|AmazonSecretsManagerException making request GetSecretValueRequest to https://secretsmanager.us-east-1.amazonaws.com/. Attempt 1. --> Amazon.SecretsManager.AmazonSecretsManagerException: Missing Authentication Token ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\HttpHandler\_mobile\HttpRequestMessageFactory.cs:line 539
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\HttpHandler\HttpHandler.cs:line 175
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleException(IExecutionContext executionContext, HttpErrorResponseException exception) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\ErrorHandler\HttpErrorResponseExceptionHandler.cs:line 60
   at Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext executionContext, Exception exception) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\ErrorHandler\ErrorHandler.cs:line 212
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\ErrorHandler\ErrorHandler.cs:line 104
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\EndpointDiscoveryHandler.cs:line 79
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CredentialsRetriever.cs:line 98
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)

Copying the default manifest and using that as a custom manifest still resulted in the error. Removing the loading of the custom manifest resulted in no error.

I got the error regardless of whether or not I had XRay tracing enabled.

MonoAndroid support

I'd like to have MonoAndroid 9 support in order to use the SDK in my Xamarin App.

Could not install package 'AWSXRayRecorder.Core 1.1.2'. You are trying to install this package into a project that targets 'MonoAndroid,Version=v9.0', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.

Can't target asp.net core on .net framework

According to this issue #36 , it is not possible to target asp.net core on .net framework. Since this is the recommended stack for new .net framework development, I think it should be supported.

Support for DB calls via Entity Framework Core

Do you have any plans to add an Entity Framework Core handler? At the moment it feels like a big piece of the puzzle is missing for most of my APIs.

I have been testing the SDK / X-Ray for the first time today. I have just managed to hack together a basic piece of code based on what your SQL handler does. However, I'm using the DiagnosticListener to grab the EF core diagnostic events needed to start and stop a subsegment. It's actually working but I have a few concerns about how it would work if things are happening concurrently.

I've just forked the code so that I can look deeper into how things work. My main question is what happens if code executes concurrently, how do you ensure the correct subsegments are being closed off. It feels like there needs to be a principle of controlling which subsegment you are done with.

Ignoring for now what I'm doing with EF, if two SDK calls are made concurrently. The first one started takes 2 seconds and the second takes 5. How would the tracing know which is which when the EndSubSegment is called as they return?

As an aside, do you have a public roadmap for the SDK you would be able to share in terms of planned features, handles etc?

Make internal AWSXRayRecorder constructors public

Currently there is a overload of AWSXRayRecorder.InitializeInstance which takes 2 parameters

  • configuration: IConfiguration
  • recorder: AWSXRayRecorder

This seems well suited to supporting custom configurations (such as supplying your own implementation or wrapper around the UDP segment emitter).

But unfortunately the useful constructor overloads of AWSXRayRecorder in this instance are marked internal so you can't inject your own implementation of a segment emitter.

To work around this I need to do something like:

var constructors = typeof(AWSXRayRecorder).GetConstructors(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);
var constructor = constructors.First(x => {
	var parameters = x.GetParameters();
	return parameters.Length == 2 && parameters[0].ParameterType == typeof(ISegmentEmitter) && parameters[1].ParameterType == typeof(XRayOptions);
});
var xrayOptions = Configuration.GetXRayOptions();
var emitter = appOptions.LogAwsXRaySegments ? (ISegmentEmitter)new LoggingUdpSegmentEmitter() : new UdpSegmentEmitter();
var recorder = (AWSXRayRecorder)constructor.Invoke(new object[] {emitter, xrayOptions});
AWSXRayRecorder.InitializeInstance(Configuration, recorder);

Which is more friction then I would like - it would be great if some of those internal constructors could be made public.

Entity doesn't exist in AsyncLocal exception when X-Ray tracing disabled

Hi there,

When calling GetAsyncResponseTraced, I get the following exception:

Entity doesn't exist in AsyncLocal\n at Amazon.XRay.Recorder.Core.Internal.Context.AsyncLocalContextContainer.GetEntity() at Amazon.XRay.Recorder.Handlers.System.Net.Utils.RequestUtil.ProcessRequest(Uri uri, String method, Action`1 addHeaderAction) at Amazon.XRay.Recorder.Handlers.System.Net.Utils.RequestUtil.ProcessRequest(WebRequest request) at Amazon.XRay.Recorder.Handlers.System.Net.HttpWebRequestTracingExtension.GetAsyncResponseTraced(WebRequest request)

It seems that RequestUtil.ProcessRequest is attempting to access AWSXRayRecorder.Instance.TraceContext.GetEntity even though I have disabled x-ray tracing via my configuration (debugging my application confirms x-ray tracing is disabled). This seems like a bug to me- I wouldn't expect these instrumented methods to do anything if tracing was disabled.

ASP.NET Core 2.0 not recording correct status code for request in XRay

Problem: When a request throws an exception and returns a 500 type response to the user, the ASP.NET XRay middleware is telling XRay that the whole request was a 200 even though one of the subsegments has an error. This makes searching for problems more difficult in XRay. Notice the search result says there were no faults or errors but there should have been one faulting request.

Issue 2 but maybe related: It seems like there are two segments in the XRay trace instead of just one. My understanding it there should be just one but I could be confused on that. The documentation you guys wrote says not to define new Segments if the app is running in a Lambda because a segment is already generated for us.

image

image

You can see how the requests all look good when they really aren't.

image

Environment
ASP.NET Core 2.0 application
Running in AWS Lambda

First segment in XRay dashboard show wrong client_ip

When I used XRay SDK in non WAF environment, it showed the correct client_ip in XRay tracing dashboard, but when it tracing in the production environment in WAF integrate, its display the WAF IP in client_ip instead of true client IP, in raw data of tracing, the first segment display WAF IP. but the subsegment displays the correct client IP.

aws x ray entity does not exist in async local

While executing all tests at a time aws x-ray performing weird.
Its throwing exception as "Entity does not exist in async local". I'm not sure but it can be due to missing traceId at AWSXRayRecorder initialization, check the code below

`public static AWSXRayRecorder XrayRecorderInstance
{
get
{
if (xrayRecorder != null)
{
return xrayRecorder;
}
else
{
lock (threadLock)
{
if (xrayRecorder == null)
{
xrayRecorder = new AWSXRayRecorder
{
ContextMissingStrategy = Amazon.XRay.Recorder.Core.Strategies.ContextMissingStrategy.LOG_ERROR

                        };
                        Amazon.XRay.Recorder.Handlers.AwsSdk.AWSSDKHandler.RegisterXRayForAllServices();
                        return xrayRecorder;
                    }
                    else
                    {
                        return xrayRecorder;
                    }
                }
            }
        }
    }`

do I need to add segment here?How? Please help me to resolve this issue.

Max allowed object depth reached while trying to export from type System.String

Hi,
We've enabled X-Ray in lambda constructor using this line AWSSDKHandler.RegisterXRayForAllServices()
When we tried to run the code we got the following exception

One or more errors occurred. (Max allowed object depth reached while trying to export from type System.String)

Our code looking something like this:
var table = Table.LoadTable(dynamoDbClient, "tableName");
var res = table.Scan(filter);
var list = await res.GetRemainingAsync();

Full exception is bellow:

2018-03-01 13:15:17: One or more errors occurred. (Max allowed object depth reached while trying to export from type System.String): AggregateException
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at lambda_method(Closure , Stream , Stream , LambdaContextInternal )

at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at Amazon.XRay.Recorder.Core.Internal.Emitters.JsonSegmentMarshaller.WriteEntityFields(Entity entity, JsonWriter writer)
at Amazon.XRay.Recorder.Core.Internal.Emitters.JsonSegmentMarshaller.EntityExporter(Entity entity, JsonWriter writer)
at ThirdParty.LitJson.JsonMapper.WriteValue(Object obj, JsonWriter writer, Boolean writer_is_private, Int32 depth)
at ThirdParty.LitJson.JsonMapper.ToJson(Object obj)
at Amazon.XRay.Recorder.Core.Internal.Emitters.JsonSegmentMarshaller.Marshall(Entity segment)
at Amazon.XRay.Recorder.Core.Internal.Emitters.UdpSegmentEmitter.Send(Entity segment)
at Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl.StreamSubsegments(Entity entity)
at Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl.StreamSubsegments(Entity entity)
at Amazon.XRay.Recorder.Core.AWSXRayRecorder.EndFacadeSegment()
at Amazon.XRay.Recorder.Core.AWSXRayRecorder.EndSubsegment()
at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.ProcessEndRequest(IExecutionContext executionContext)
at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.MetricsHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.RuntimePipeline.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.AmazonServiceClient.Invoke[TRequest,TResponse](TRequest request, IMarshaller2 marshaller, ResponseUnmarshaller unmarshaller) at Amazon.DynamoDBv2.DocumentModel.Search.GetNextSetHelper(Boolean isAsync) at Amazon.DynamoDBv2.DocumentModel.Search.GetRemainingHelper(Boolean isAsync) at Amazon.Runtime.Internal.AsyncRunner.<>c__DisplayClass1_21.b__1()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Amazon.Runtime.Internal.AsyncRunner.<>c__DisplayClass1_0`1.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at ProviderMetadataAPI.Data.ProviderRepository.d__6.MoveNext() in D:\Source\Repos\content-store-provider-metadata-api\ProviderMetadataAPI.Data\ProviderRepository.cs:line 36

NullReferenceException in PopulateRequestAttributes

In ASP.NET Core (2.2, but earlier versions also apply), you can get a NullReferenceException for requests that do not contain the host header:

Object reference not set to an instance of an object.

 at Microsoft.AspNetCore.Http.Extensions.UriHelper.GetDisplayUrl(HttpRequest request)
   at Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.PopulateRequestAttributes(HttpRequest request, Dictionary`2 requestAttributes)
   at Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.ProcessHTTPRequest(HttpContext context)
   at Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.Invoke(HttpContext context)

Related issue: dotnet/aspnetcore#2718

This can be solved by implementing your own GetDisplayUrl variant, which does the null check. You can basically just copy the one which will be used in .NET Core 3.0: dotnet/aspnetcore@187e89f

Cannot provide my own sampling strategy

I am trying to write my own sampling strategy, however, the AWSXRayRecorder.Instance has the LocalizedSamplingStrategy instead of mine. From looking at the code, it seems like the InitializeInstance method I am using builds a new AWSXRayRecorder based on instance I am providing, but the sampling strategy does not carry over.

AWSXRayRecorder.InitializeInstance(Configuration, BuildAWSXRayRecorder());

private static AWSXRayRecorder BuildAWSXRayRecorder()
        {
            var builder = new AWSXRayRecorderBuilder()
                .WithSamplingStrategy(new AWSXRaySamplingStrategy());

            var result = builder.Build();

            return result;
        }

AwsXrayRecoroder trying to modify a non-concurrent collection without exclusive access?

Hi there I can't really provide code snippets for the project, but the context is that we are simply trying to call AWSXrayRecorder.Instance.BeginSubsegment("some name"); inside of a web api and it throws the following exception:

Error Message:
 System.InvalidOperationException : Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.
Stack Trace:
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.set_Item(TKey key, TValue value)
   at Amazon.XRay.Recorder.Core.Internal.Emitters.JsonSegmentMarshaller..ctor()
   at Amazon.XRay.Recorder.Core.AWSXRayRecorder.get_Instance()
   at <redacted>
   at <redacted>
Error Message:
 System.InvalidOperationException : Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.
Stack Trace:
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.set_Item(TKey key, TValue value)
   at Amazon.XRay.Recorder.Core.Internal.Emitters.JsonSegmentMarshaller..ctor()
   at Amazon.XRay.Recorder.Core.AWSXRayRecorder.get_Instance()
   at <redacted>
   at <redacted>
   at <redacted>

This is happening from inside a dependency injected class, so it would follow the following pattern (where everything is scoped as far as DI goes)
Controller --> Provider --> ProviderWithXraySubsegment

We do see the API tracing the segments started by making the Http Request just fine, but when we add subsegments, it seems to fail with this kind of an error.

This will impact whether or not we use X-Ray at our business. We have also tried using the TraceMethod functions instead of creating subsegments.

MissingMethodException: Method not found: 'System.String Amazon.XRay.Recorder.Core.Strategies.SegmentNamingStrategy.GetSegmentName(Microsoft.AspNetCore.Http.HttpRequest)'.

Hi,

I am are trying to integrate XRay into a ASP.NET Core 2.1 Application using full .NET Targeting (.NET Framework 4.6.1) running in Elastic Beanstalk. The reason for full ..NET targeting is integration with legacy services using assemblies only supporting older .net versions.

When registering XRAY using the .NET Core method Link the following exception occurs:

MissingMethodException: Method not found: 'System.String Amazon.XRay.Recorder.Core.Strategies.SegmentNamingStrategy.GetSegmentName(Microsoft.AspNetCore.Http.HttpRequest)'.
Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.ProcessHTTPRequest(HttpContext context)
Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

It believe a similar issue was raised in issue #36 which has been closed but I don't see any comments on how work around this issue?

The XRayPipelineHandler will throw a null reference exception if the client does not have a RegionEndpoint assigned.

The XRayPipelineHandler will throw a null reference exception if the client does not have a RegionEndpoint assigned.

https://github.com/aws/aws-xray-sdk-dotnet/blob/master/sdk/src/Handlers/AwsSdk/Internal/XRayPipelineHandler.cs#L300

subsegment.Aws["region"] = client.RegionEndpoint.SystemName;

If for some reason you have an AWS service where you are explicitly setting the clients ServiceURL then the region endpoint will be null - in these cases this will cause the XRay SDK to blow up.

A good example of this is if your developing against local dev copies of AWS services (e.g. when employing local stack for local development).

It appears the AWS sub segment does not have anywhere to put the service URL for a request - but it does at least allow the region to be omitted if unavailable, so it wuold be great to change the code to be:

if (client.RegionEndpoint != null) {
    subsegment.Aws["region"] = client.RegionEndpoint.SystemName;
}

Support tracing of HTTP requests using HttpClient

Feature request.

Currently the SDK only supports out-going HTTP requests using the HttpWebRequest client. It would be useful having tracing capabilities for those using System.Net.HttpClient.

For now I've created a custom MessageHandler for this, but would be great to have this or something similar as part of the SDK.

Edit: I just noticed that this has already been requested in #3.

I will look at adding that as a pull request

Entity doesn't exist in CallContext

The logical call context used to store the segment in ASP.NET / .NET Framework seems to be lost depending on the size of the request made to the application. This results in the following exception:

Amazon.XRay.Recorder.Core.Exceptions.EntityNotAvailableException: Entity doesn't exist in CallContext

throw new EntityNotAvailableException("Entity doesn't exist in CallContext");

After tracking this down it is easily reproduced by simply posting a form with data of a sufficient length to either an MVC or Web API controller. See https://github.com/mpetito/xray-mvc-test for an example repo. I've tested this in .NET Framework 4.5 and 4.6.1.

I can only speculate at the root cause. I suspect the length of the request determines whether the request body is read in its entirety before BeginRequest is invoked (where the call context is initialized). If the remainder of the request is read after BeginRequest then the call context is lost.

The only workaround I've come up with is to write a global filter (both for MVC and API) which extracts the current entity set by ProcessHTTPRequest from HttpContext.Items and re-assigns it using SetEntity before the action begins executing.

JsonSegmentMarshaller ctor throws System.IndexOutOfRangeException

Hi, I'm getting this issue running xunit integration tests in parallel

image

after a quick google search seems like the exception is being thrown because Dictionary is not thread safe on add.

looking at the code seems like the crashing one is in this call

https://github.com/aws/aws-xray-sdk-dotnet/blob/master/sdk/src/Core/ThirdParty/LitJson/JsonMapper.cs#L914

and if we swap the dictionaries used on JsonMapper for ConcurrentDictionary it may resolve the issue, or just adding another lock to avoid this particular case.

https://github.com/aws/aws-xray-sdk-dotnet/blob/master/sdk/src/Core/ThirdParty/LitJson/JsonMapper.cs#L134

let me know if you would take a pull request,

Regards.

Path in tests assume Windows path separators

Found private static String _path = @"JSONs\AWSRequestInfo.json"; in ./sdk/test/UnitTests/AWSSdkTracingHandlerTests.cs.

This seems to prevent me to run the tests on macOS. When I run the tests I get:

Amazon.XRay.Recorder.UnitTests.AWSSdkTracingHandlerTests.TestDynamoDbClient

Test method Amazon.XRay.Recorder.UnitTests.AWSSdkTracingHandlerTests.TestDynamoDbClient threw exception: 
System.IO.FileNotFoundException: Could not find file '/Users/jesperlarsson/RiderProjects/aws-xray-sdk-dotnet/sdk/test/UnitTests/bin/Debug/netcoreapp2.0/JSONs\AWSRequestInfo.json'.
    at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access)
   at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler..ctor(String path) in /Users/jesperlarsson/RiderProjects/aws-xray-sdk-dotnet/sdk/src/Handlers/AwsSdk/Internal/XRayPipelineHandler.cs:line 85
   at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineCustomizer.Customize(Type serviceClientType, RuntimePipeline pipeline) in /Users/jesperlarsson/RiderProjects/aws-xray-sdk-dotnet/sdk/src/Handlers/AwsSdk/Internal/XRayPipelineHandler.cs:line 651
   at Amazon.Runtime.Internal.RuntimePipelineCustomizerRegistry.ApplyCustomizations(Type type, RuntimePipeline pipeline) in E:\JenkinsWorkspaces\v3-trebuchet-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Internal\RuntimePipelineCustomizerRegistry.cs:line 108
   at Amazon.XRay.Recorder.UnitTests.AWSSdkTracingHandlerTests.TestDynamoDbClient() in /Users/jesperlarsson/RiderProjects/aws-xray-sdk-dotnet/sdk/test/UnitTests/AWSSdkTracingHandlerTests.cs:line 268

Support for asp.net core targeting full framework

I'm working on an asp.net project that is targeting the full .net framework. I get the following error when attempting to use the asp.net core XRay library:

System.MissingMethodException: Method not found: 'Boolean Amazon.XRay.Recorder.Core.AWSXRayRecorder.IsLambda()'.
   at Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.ProcessHTTPRequest(HttpContext context)
   at Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.<Invoke>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.<Invoke>d__7.MoveNext()

I haven't dug in to try and find where the issue is but I have created a simple example that illustrates the issue here: https://github.com/blebaron/AWSXRayExample

I created the example by using the VS 2017 asp.net core api new project template and then modifying it to target the net461

AspNet Handler Exception without User Agent

There's a NullReferenceException when using the ASP.NET handler if the inbound HTTP request does not have a User-Agent header specified. This is a simple fix if you need a PR.

requestAttributes["user_agent"] = request.UserAgent.ToString();

Exception information: 
    Exception type: NullReferenceException 
    Exception message: Object reference not set to an instance of an object.
   at Amazon.XRay.Recorder.Handlers.AspNet.AWSXRayASPNET.ProcessRequestAttributes(HttpRequest request, Dictionary`2 requestAttributes)
   at Amazon.XRay.Recorder.Handlers.AspNet.AWSXRayASPNET.ProcessHTTPRequest(Object sender, EventArgs e)
   at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

AWS_XRAY_CONTEXT_MISSING should honor UseRuntimeErrors

It seems odd to me that I can set UseRuntimeErrors=false in my config file, but I also have to set AWS_XRAY_CONTEXT_MISSING=LOG_ERROR in the environment to fully prevent runtime errors. It's quite common for an application to perform database operations on startup, outside of a request scope. Currently, this would cause the app to crash on start.

Publish Update

Can you please publish a 2.2.1 update to the nuget packages? I need the changes from #22 in order to work with the AWS IOT Sdks. I've confirmed the latest changes address my issue by grabbing/building the latest code, and injecting the updated nuget locally.

Best approach for local testing of handlers

Hi!

I'm working on a XRay handler for WCF (both on service and client side) (https://github.com/Kralizek/xray-wcf)

I've got the basics going and now it's about polishing. Currently I've been deploying my sample project to am EC2 instance but it gets annoying after a while. And it makes difficult to debug.

Is there any better approach to allow local development/testing?

Question: Lambda "Enable active tracing"

Hi,

You can enable/disable active tracing at a Lambda function level, using the console. Is there a way to find out what is the option that has been set during Lambda execution? I have some additional Xray tracing code, which I do not want to run when this option is disabled in the console as then the default segment is not created by the Lambda function.

If I look at the IsTracingDisabled Function, it always returns false, even if this option is disabled in the console. Looks like this picks the value from the app settings and not the console settings.

Is there a way to check what is the setting in the console/lambda function level?

Allow AWS_XRAY_CONTEXT_MISSING to be configured via IConfiguration

I've just started trying out this library today. One thing I found annoying with the default for AWS_XRAY_CONTEXT_MISSING to cause a runtime error. I'd like to deploy this onto existing production apps to see if it proves useful, but it's slightly riskier since if I miss any places where SDK calls, for example, are made outside of a normal ASP.NET Core request flow, it throws.

I realise I can set the AWS_XRAY_CONTEXT_MISSING environment variable to control this, but that adds work to deployments and if missed could cause the app to fail, purely over tracing needs. I'd much prefer a silent failure as the default here.

An alternative (and I've not dug into the code deeply yet) is that it would be nice to control this setting from the settings available in appSettings.json so I can set the default to logging only which I know will be deployed with the app. We can then override with an environment variable if we prefer.

Missing AWSXRayRecorder.Handlers.System.Net;

Hi,
I am using XRay API mentioned in this Repo, for logging Lambda Function that i built on ASPNetCore 2.0 and used httpclient to call another endpoint to get JSON result, so I wanted to follow the instruction to log the request as mentioned in Trace out-going HTTP requests (.NET and .NET Core) and noticed the following:

  1. using AWSXRayRecorder.Handlers.System.Net; -- Is Missing although I have added all packages exactly as mentioned in the beginning of article, " Search AWSXRayRecorder* to see various middlewares available.... etc."
  2. HttpClientTracingHandler in : the code line below is Missing, do I need to write this function?
    var httpClient = new HttpClient(new HttpClientTracingHandler(new HttpClientHandler()));

It's good to mention that I was able to use Tracing Asynchronous method and Creating Segment and Subsegment to the LambdaFunction.

Thanks
Basel

Exception thrown when hosting ASP.NET Core MVC application in lambda using Amazon.Lambda.AspNetCoreServer

I have a ASP.NET Core MVC application hosted inside of the lambda using the Amazon.Lambda.AspNetCoreServer

When I try to initialise the X-Ray I have the following exception when the app is starting up inside the lambda

Unhandled Exception: System.AggregateException: One or more errors occurred. (Unable to cast object of type 'Amazon.XRay.Recorder.Core.Internal.Entities.FacadeSegment' to type 'Amazon.XRay.Recorder.Core.Internal.Entities.Subsegment'.) ---> System.InvalidCastException: Unable to cast object of type 'Amazon.XRay.Recorder.Core.Internal.Entities.FacadeSegment' to type 'Amazon.XRay.Recorder.Core.Internal.Entities.Subsegment'.
  at Amazon.XRay.Recorder.Core.AWSXRayRecorder.ProcessSubsegmentInLambdaContext(String name)
  at Amazon.XRay.Recorder.Core.AWSXRayRecorder.BeginSubsegment(String name)
  at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.ProcessBeginRequest(IExecutionContext executionContext)
  at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.<InvokeAsync>d__27`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  at Amazon.Runtime.Internal.ErrorCallbackHandler.<InvokeAsync>d__5`1.MoveNext() in E:\JenkinsWorkspaces\v3-trebuchet-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\ErrorCallbackHandler.cs:line 58

The exception seems to originate from the following method, where there is an assumption that if there and entity it for sure is a subsegment in lambda...

private void ProcessSubsegmentInLambdaContext(string name)
    {
        if (!TraceContext.IsEntityPresent()) //No facade segment available and first subsegment of a subsegment branch needs to be added
        {
            AddFacadeSegment(name);
            AddSubsegmentInLambdaContext(new Subsegment(name));
        }
        else //continuation of subsegment branch
        {
            var parentSubsegment = (Subsegment)TraceContext.GetEntity();
            var environmentRootTraceId = TraceHeader.FromString(GetTraceVariablesFromEnvironment()).RootTraceId;
            if ((null != environmentRootTraceId) && !environmentRootTraceId.Equals(parentSubsegment.RootSegment.TraceId)) //If true, customer has leaked subsegments across invocation
            {
                TraceContext.ClearEntity(); //reset TraceContext
                BeginSubsegment(name); //This adds Facade segment with updated environment variables
            }
            else
            {
                AddSubsegmentInLambdaContext(new Subsegment(name));
            }

        }
    }

however in my case it is simply not because this method gets called earlier

    private void BeforeRequestEventHandler(object sender, RequestEventArgs e)
    {
        var args = e as WebServiceRequestEventArgs;
        if (args == null)
        {
            _logger.DebugFormat("Failed to handle BeforeRequestEvent, because e can't be converted to WebServiceRequestEventArgs.");
            return;
        }

        if (TraceHeader.TryParse(TraceContext.GetEntity(), out TraceHeader traceHeader))
        {
            args.Headers[TraceHeader.HeaderKey] = traceHeader.ToString();
        }
        else
        {
            _logger.DebugFormat("Failed to inject trace header to AWS SDK request as the segment can't be converted to TraceHeader.");
        }
        _recorder.BeginSubsegment(RemoveAmazonPrefixFromServiceName(args.ServiceName));
        _recorder.SetNamespace("aws");
    }

and the call to TraceContext.GetEntity() in case of running in lambda initialises the entity to FacadeSegment

Can you let me know if I am maybe using these tools incorrectly or is it some sort of a bug?

SQL Errors Not Recorded Correctly

When an error occurs it is displayed as a different service named "remote" even though it is actually the same database. I believe this is because the SQL information is not collected until after the call is executed. From what I can tell, there is no reason this can't be collected before the call. Additionally, we may be able to collect more information from the SQL Exception if we catch it.

recorder.SetNamespace("remote");
var ret = method();
CollectSqlInformation();

Example stack trace:

SqlException: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.at System.Data.SqlClient.SqlConnection.OnError
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning
at System.Data.SqlClient.TdsParser.TryRun
at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData
at System.Data.SqlClient.SqlDataReader.get_MetaData
at System.Data.SqlClient.SqlCommand.FinishExecuteReader
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds
at System.Data.SqlClient.SqlCommand.ExecuteReader
at Amazon.XRay.Recorder.Handlers.SqlServer.TraceableSqlCommand.Intercept

image

Error when using IHostedService

I have an IHostedService that uses IAmazonS3 to retrieve data every cron defined time. It's works well until I set AWSSDKHandler.RegisterXRayForAllServices();.

If I try to use AWSSDKHandler.RegisterXRayForAllServices(); to use XRay to trace my S3 call I get the following error:

Amazon.XRay.Recorder.Core.Exceptions.EntityNotAvailableException: Entity doesn't exist in AsyncLocal    
at Amazon.XRay.Recorder.Core.Internal.Utils.TraceContext.GetEntity()    
at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.ProcessBeginRequest(IExecutionContext executionContext)    
at Amazon.XRay.Recorder.Handlers.AwsSdk.Internal.XRayPipelineHandler.<InvokeAsync>d__27`1.MoveNext() 
--- End of stack trace from previous location where exception was thrown ---    
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext() 
--- End of stack trace from previous location where exception was thrown ---    
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
at Amazon.S3.Internal.AmazonS3ExceptionHandler.<InvokeAsync>d__1`1.MoveNext() in E:\JenkinsWorkspaces\v3-trebuchet-release\AWSDotNetPublic\sdk\src\Services\S3\Custom\Internal\AmazonS3ExceptionHandler.cs:line 62 
--- End of stack trace from previous location where exception was thrown ---    
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
at Amazon.Runtime.Internal.ErrorCallbackHandler.<InvokeAsync>d__5`1.MoveNext() in E:\JenkinsWorkspaces\v3-trebuchet-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\ErrorCallbackHandler.cs:line 58 --- 
End of stack trace from previous location

Optimize dependencies for RegisterXRayForAllServices()

I added these two lines to my .csproj file so I could invoke AWSSDKHandler.RegisterXRayForAllServices():

    <PackageReference Include="AWSXRayRecorder.Core" Version="2.5.0"/>
    <PackageReference Include="AWSXRayRecorder.Handlers.AwsSdk" Version="2.5.0"/>

It resulted in a surprising amount of dependencies being pulled in

  Installing runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing Microsoft.AspNetCore.Http.Features 2.0.0.
  Installing runtime.native.System.Security.Cryptography.OpenSsl 4.3.2.
  Installing Microsoft.AspNetCore.Http.Abstractions 2.0.0.
  Installing Microsoft.Net.Http.Headers 2.0.0.
  Installing Microsoft.Extensions.Options 2.0.0.
  Installing Microsoft.AspNetCore.WebUtilities 2.0.0.
  Installing AWSXRayRecorder.Core 2.5.0.
  Installing AWSXRayRecorder.Handlers.AwsSdk 2.5.0.
  Installing AWSSDK.XRay 3.3.3.
  Installing AWSSDK.Core 3.3.25.1.
  Installing Microsoft.Extensions.Configuration 2.0.0.
  Installing Microsoft.AspNetCore.Http 2.0.0.
  Installing System.Net.Http 4.3.3.
  Installing Microsoft.Extensions.Configuration.Abstractions 2.0.0.

Maybe the ASP.NET Core dependencies could be omitted by moving them to a specialized package?

Signature of TraceMethodAsync shoud probably be async Task instead of async void

I was trying to use the async method TraceMethodAsync and noticed it has a return type of void. Generally with C# methods that are async that don't return anything should have a return type of Task, not void unless they're event handlers. Your code might actually be handling it okay but it would be better to just return Task since that is the C# best practice.

https://haacked.com/archive/2014/11/11/async-void-methods/

https://msdn.microsoft.com/en-us/magazine/jj991977.aspx

Async void methods have different error-handling semantics. When an exception is thrown out of an async Task or async Task method, that exception is captured and placed on the Task object. With async void methods, there is no Task object, so any exceptions thrown out of an async void method will be raised directly on the SynchronizationContext that was active when the async void method started. Figure 2 illustrates that exceptions thrown from async void methods can’t be caught naturally.

Lambda x-ray trace missing request information

When capturing X-Ray trace information for my ASP.NET core 2.1 WebAPI application, request information is missing (e.g. url, user agent, etc):

image

Looking at the raw JSON of a trace, it seems there are two subsegments, one for the request and one for the response (sensitive information has been scrubbed):

{
    "Duration": 0.814,
    "Id": "1-5c6b564b-40eca8a7451d03810d3da705",
    "Segments": [
        {
            "Document": {
                "id": "23dbc54b6e683eaa",
                "name": "[OMITTED]",
                "start_time": 1550538315.511,
                "end_time": 1550538316.315,
                "parent_id": "059037fcdf2fb610",
                "aws": {
                    "function_arn": "[OMITTED]",
                    "resource_names": [
                        "[OMITTED]"
                    ],
                    "account_id": "[OMITTED]"
                },
                "trace_id": "1-5c6b564b-40eca8a7451d03810d3da705",
                "origin": "AWS::Lambda::Function",
                "subsegments": [
                    {
                        "id": "98262d52ae5e1959",
                        "name": "[OMITTED]",
                        "start_time": 1550538315.511743,
                        "end_time": 1550538316.315486,
                        "http": {
                            "request": {
                                "url": "[OMITTED]",
                                "method": "GET",
                                "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36",
                                "client_ip": "[OMITTED]",
                                "x_forwarded_for": true
                            },
                            "response": {
                                "status": 0
                            }
                        }
                    }
                ]
            },
            "Id": "23dbc54b6e683eaa"
        },
        {
            "Document": {
                "id": "059037fcdf2fb610",
                "name": "[OMITTED]",
                "start_time": 1550538315.504,
                "end_time": 1550538316.318,
                "http": {
                    "response": {
                        "status": 200
                    }
                },
                "aws": {
                    "request_id": "e9520324-cd8f-40c5-a5c9-849adc02163f"
                },
                "trace_id": "1-5c6b564b-40eca8a7451d03810d3da705",
                "origin": "AWS::Lambda",
                "resource_arn": "[OMITTED]"
            },
            "Id": "059037fcdf2fb610"
        }
    ]
}

Does this look valid? It seems that AWS x-ray is having issues understanding this format.

AWSXRayASPNET not found

Hi,

I'm trying to implement X-Ray into my asp.net application. But, I'm sticking on the initial step by register X-Ray on Global.aspx.cs

AWSXRayASPNET.RegisterXRay(this, "MyApp");

I have already download the package from nuget. But, the application always alert the "does not exist" message. How can I resolve this?

Thanks

XRay throws exceptions when running integration tests which create additional Hosts

I can run any of my integration tests (based on nUnit) individually and they succeed. But when I run them all I receive the following error on further tests after a new Host is built by one of the tests:

Amazon.XRay.Recorder.Core.Exceptions.EntityNotAvailableException : Entity doesn't exist in AsyncLocal

I'm assuming this is because XRay data is stored statically and assumes only one Host will be created.

Any suggestions to work around this?

Outgoing HTTP request tracing

Hiya,

It would be good if the outgoing HTTP tracing was extended to also include HTTP requests issued through System.Net.Http.HttpClient. This is just anecdotal, but it seems there is an industry-wide shift towards this new API.

Ideally, the SDK might provide a DelegatingHandler subclass that we can inject into the HTTP client for a most-invisible installation of the SDK across our code base. Developers are reluctant to change how they interact with the HttpClient (e.g. no SendAsyncTraced())

Thoughts?

Add the ability to add additional attributes to the http segment

It would be useful to be able to add additional attributes to the http segment that we might want to pull from the request, such as specific cookie values, claims from a header token (or other header information), etc. This would have an impact, I assume, on the filtering within the XRay UI to be able to filter by those additional attributes.

Add option to register XRay manifest from embedded resource

Version: 2.5.0 (issue still exists on master)

I'm attempting to create a NuGet package for internal use that takes care of some of the XRay setup we are adding to a number of applications/services.

One of the requirements is adding some Secrets Manager calls to the whitelist. In fact, it was the primary reason for creating an internal library. That, and a DB profiler that worked on our PostgreSQL DB connections. Unfortunately, contentFiles are not pulled into projects that don't use project.json (https://blog.nuget.org/20160126/nuget-contentFiles-demystified.html - See section : Supported Project Types). In our case, this is almost all of them.

As, the SDK offers no option to pull in a manifest other than from file, I'm going to have to resort to taking my embedded manifest document from my NuGet package and streaming it to disk so I can then feed the path to the RegisterXRayManifest() method.

Ideally I would be able to register the embedded resource directly similar to how the DefaultAWSWhitelist.json resource is handled in InitWithDefaultAWSWhitelist() method in Handlers/AwsSdk/Internal/XRayPipelineHandler.cs or via a more generic approach that allows for passing a stream to RegisterXRayManifest().

Is it possible to add SNS to the default whitelist ?

It would help me to use automatic permission discovery tool,

Here is the list I had in mind.

"SimpleNotificationService": {
"operations": {
"AddPermission": {
"request_parameters": [
"TopicArn"
]
},
"ConfirmSubscription": {
"request_parameters": [
"TopicArn"
]
},
"CreatePlatformApplication": {
"request_parameters": [
"Name",
"Platform"
]
},
"CreatePlatformEndpoint": {
"request_parameters": [
"PlatformApplicationArn"
]
},
"CreateTopic": {
"request_parameters": [
"Name"
]
},
"DeletePlatformApplication": {
"request_parameters": [
"PlatformApplicationArn"
]
},
"DeleteTopic": {
"request_parameters": [
"TopicArn"
]
},
"GetPlatformApplicationAttributes": {
"request_parameters": [
"PlatformApplicationArn"
]
},
"GetSubscriptionAttributes": {
"request_parameters": [
"SubscriptionArn"
]
},
"GetTopicAttributes": {
"request_parameters": [
"TopicArn"
]
},
"ListEndpointsByPlatformApplication": {
"request_parameters": [
"PlatformApplicationArn"
]
},
"ListSubscriptionsByTopic": {
"request_parameters": [
"TopicArn"
]
},
"Publish": {
"request_parameters": [
"TopicArn",
"TargetArn"
]
},
"RemovePermission": {
"request_parameters": [
"TopicArn"
]
},
"SetPlatformApplicationAttributes": {
"request_parameters": [
"PlatformApplicationArn"
]
},
"SetSubscriptionAttributes": {
"request_parameters": [
"SubscriptionArn"
]
},
"SetTopicAttributes": {
"request_parameters": [
"TopicArn"
]
},
"Subscribe": {
"request_parameters": [
"TopicArn",
"Protocol",
"Endpoint"
]
},
"Unsubscribe": {
"request_parameters": [
"SubscriptionArn"
]
}
}
}
}

Custom start and end date for Segment and Subsegment

Hi,

I'm currently facing a little issue trying to profile Elasticache Redis commands.
I'm using StackExchange.Redis client and the profiling system included is not really friendly with the AWSXRayRecorder design.

All what I can have is a list of commands elapsed time.
I tried to set the start and end times for a subsegment but AWSXRayRecorder.Instance.EndSubsegment(); is overriding the end time with utcnow (which is fair).
For now I just added the elapsed time in the subsegment metadata but it will be complicate to investigate later in XRay console.

recorder.BeginSubsegment("Redis Commands");
recorder.SetNamespace("Database");
recorder.AddMetadata(nameof(command.Command), command.Command);
recorder.AddMetadata(nameof(command.ElapsedTime), command.ElapsedTime);
AWSXRayRecorder.Instance.EndSubsegment();

So my first question is: Do you think that customizing start and end time would be a solution at some point?
As StackExchange.Redis does not provide any callback where I could call a BeginSubsegment and EndSubsegment...

Second question: Do you have any other recommendation to profile Redis?

Regards

Entity not received by daemon if total size exceeds 64KB

STEPS TO REPRODUCE THE ISSUE

  1. BeginSegment
  2. BeginSubsegment
  3. AddMetadata (optional, but it makes the reproduction easier if large metadata is added, for example 32KB)
  4. EndSubsegment
  5. Repeat steps 2 to 4 until total segment size is greater than 64KB (for example, 2 subsegments with 32KB of metadata)
  6. EndSegment

According to documentation (https://docs.aws.amazon.com/xray/latest/devguide/xray-api-segmentdocuments.html#api-segmentdocuments-subsegments), a segment document can be up to 64 KB, but subsegments can be sent separately to avoid exceeding the maximum segment document size. This scenario should work. However, you can see that the daemon does not receive the segment.
I found in the code that ProcessEndSegment method sends the entire segment to the daemon (on step 6), and that single document exceeds 64KB. The subsegments here should be sent separately.

As a workaround I changed the constant MaxSubsegmentSize from 100 to 1. That forces each subsegment to send immediately after calling EndSubsegment, as ShouldStreamSubsegments returns true. This works as expected for this case, but it is not optimal if the subsegments are small enough to fit into one single document.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.