Application Insights Profiler sample and documentation

License: MIT License

applicationinsights-profiler-aspnetcore's Introduction

Application Insights Profiler for ASP.NET Core

Announcement

Upgrade to 2.7.0. Reduced the telemetry noise. Check out the details in the release notes.
Running inside Azure Fucntion? Follow this example.

Previous announcements

Worker service support is in beta testing now.
- Check out the example for package info and the instructions.

Description

This is the project home page for Microsoft Application Insights Profiler for ASP.NET Core. The NuGet packages can be found here.

Get Started

⚠️ These are steps for the ASP.NET Core applications. For the Worker Service, refer to this example.

Create an application

dotnet new webapi

Add NuGet packages

dotnet add package Microsoft.ApplicationInsights.AspNetCore
dotnet add package Microsoft.ApplicationInsights.Profiler.AspNetCore

Notice: .NET Core 2.2 is out of support, it is recommended to migrate your project to .NET 6.x. Refer to .NET Core page for details. If you have to stay on .NET Core 2.2 for now, please this specific version of Microsoft.ApplicationInsights.AspNetCore v2.14 alone with the profiler package.

Tips: Find official migration documentation here.

Enable Application Insights Profiler

Open Startup.cs

using System.Diagnostics;
...
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    ...
    // Adding the following lines to enable application insights and profiler.
    services.AddApplicationInsightsTelemetry();
    services.AddServiceProfiler();
}

Add a bottleneck

To make it real, make use the following code to add some delay in the WeatherForecastController.cs to simulate the bottleneck:

using System.Threading;
...
private static void SimulateDelay()
{
    // Delay for 200ms to 5s to simulate a bottleneck.
    Thread.Sleep((new Random()).Next(200, 5000));
}

And call it from the controller methods:

[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
    SimulateDelay();
    var rng = new Random();
    return Enumerable.Range(1, 5).Select(index => new WeatherForecast
    ...
}

Setup the connection string for debugging

In appsettings.Development.json, add the following configuration:

{
    ...
    "ApplicationInsights": {
        "ConnectionString": "replace-with-your-connection-string"
    }
    ...
}

Alternatively, use a connection string like in QuickStart3.1 example.
Run the WebAPI, generate traffic for profiling

To run your application locally:

dotnet run

At the beginning of the application, OneTimeSchedulingPolicy will immediately kick in, profiling for 2 minutes. Hit the endpoint in a browser during the profiling session in your browser:

https://localhost:5001/weatherforecast

Analyze the trace from Azure Portal

Get a coffee and wait for a couple of minutes - the backend needs some time, usually a 2 minutes, to ingest the data.

Then, open the application insights resource in Azure Portal, go to the performance blade, and use the button of Configure Profiler. There, you are going to see all profiling sessions:

Tip: Click on the trace to open the trace analyzer.

Configurations for the Profiler - describes how to customize various settings of the profiler.
Dockerize an application with Profiler
Profiler Sessions - describes when the profiler starts, stops and what is traced.
Trace Analysis - introduce the trace analysis, get a basic understanding of AWAIT TIME, Blocked time, Unmanaged Async, etc.
Diagnosing a WebAPI experiencing intermittent high CPU using the Application Insights Profiler.
The call tree filter.

Supported Versions

To find out the proper version of the Profiler to use, please refer to Support Matrix.

Examples

References

Profile ASP.NET Core Azure Linux web apps with Application Insights Profiler

CAUTION

This is a documentation/sample repository. The LICENSE covers the content in this repository but does NOT cover the use of the product of Microsoft.ApplicationInsights.Profiler.AspNetCore. Please reference EULA-prerelease.md for any prerelease product and EULA-GA.md for any non-prerelease product.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

applicationinsights-profiler-aspnetcore's People

Contributors

Stargazers

Watchers

applicationinsights-profiler-aspnetcore's Issues

AppIdNotFoundException on windows, but profiling seems to work.

When looking at the diagnostic logs stream on an App Service Plan I see a AppIdNotFoundException , which I'd guess is the app insights instrumentation key?

I'm providing the key via ARM to be an app setting / environment variable with the name:
ApplicationInsights:InstrumentationKey

I did read that __ should be used rather than :. Due to colon not working on all platforms, however the .net core settings provider does support ":" normally.

Interestingly the profiles do appear to be coming through anyway. I was surprised to see the following extension installed which made me wonder if the old profiler has become active somehow
"Application Insights extension for Azure App Service": "2.6.5"

I purposefully removed the extension from ARM and deleted the resource group to clear everything (it's a development environment) as advised that it's unlikely to be compatible: #35

2019-02-11T11:35:26  Welcome, you are now connected to log-streaming service.
2019-02-11T11:36:21  PID[6036] Information Unpacking resources...
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Executing Application Insights Profiler agent version 2.4.60711.2 with arguments:
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Detected an Azure App Service sandbox.
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Detected an Azure App Service sandbox.
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Minimum version enabled on machine:
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Instrumentation key:
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Application Insights Profiler endpoint: https://agent.azureserviceprofiler.net
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Application Insights endpoint: default
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Runtime Environment: Antares
2019-02-11T11:36:21  PID[6036] Information 02-11 11:36:21 Information: Support files: D:\local\Temp\jobs\continuous\ApplicationInsightsProfiler2\naj33tg5.cjh\spsupport
2019-02-11T11:36:22  PID[6036] Information 02-11 11:36:21 Information: Providing usage data (to opt out set the AppSettings ApplicationInsightsProvideUsageTelemetryData with the value "false"): True
2019-02-11T11:36:22  PID[6036] Information 02-11 11:36:21 Information: Engine mode: Immediate
2019-02-11T11:36:22  PID[6036] Information 02-11 11:36:22 Error: Unexpected exception in agent main process. Details: Microsoft.ServiceProfiler.Utilities.AppIdNotFoundException: Unable to find AppId for iKey:
2019-02-11T11:36:22  PID[6036] Information    at Microsoft.ServiceProfiler.Utilities.AppInsightsProfileFetcher.<FetchProfileAsync>d__3.MoveNext()
2019-02-11T11:36:22  PID[6036] Information --- End of stack trace from previous location where exception was thrown ---
2019-02-11T11:36:22  PID[6036] Information    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2019-02-11T11:36:22  PID[6036] Information    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2019-02-11T11:36:22  PID[6036] Information    at Microsoft.ServiceProfiler.DiagnosticsAgent.Program.GetApplicationInsightsAppId(String instrumentationKey, String appInsightsEndpoint)
2019-02-11T11:36:22  PID[6036] Information    at Microsoft.ServiceProfiler.DiagnosticsAgent.Program.Main(String[] cmdArgs)
2019-02-11T11:37:23  PID[6036] Information Unpacking resources...
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Executing Application Insights Profiler agent version 2.4.60711.2 with arguments:
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Detected an Azure App Service sandbox.
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Detected an Azure App Service sandbox.
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Minimum version enabled on machine:
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Instrumentation key:
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Application Insights Profiler endpoint: https://agent.azureserviceprofiler.net
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Application Insights endpoint: default
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Runtime Environment: Antares
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Support files: D:\local\Temp\jobs\continuous\ApplicationInsightsProfiler2\naj33tg5.cjh\spsupport
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Providing usage data (to opt out set the AppSettings ApplicationInsightsProvideUsageTelemetryData with the value "false"): True
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:23 Information: Engine mode: Immediate
2019-02-11T11:37:24  PID[6036] Information 02-11 11:37:24 Error: Unexpected exception in agent main process. Details: Microsoft.ServiceProfiler.Utilities.AppIdNotFoundException: Unable to find AppId for iKey:
2019-02-11T11:37:24  PID[6036] Information    at Microsoft.ServiceProfiler.Utilities.AppInsightsProfileFetcher.<FetchProfileAsync>d__3.MoveNext()
2019-02-11T11:37:24  PID[6036] Information --- End of stack trace from previous location where exception was thrown ---
2019-02-11T11:37:24  PID[6036] Information    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2019-02-11T11:37:24  PID[6036] Information    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2019-02-11T11:37:24  PID[6036] Information    at Microsoft.ServiceProfiler.DiagnosticsAgent.Program.GetApplicationInsightsAppId(String instrumentationKey, String appInsightsEndpoint)
2019-02-11T11:37:24  PID[6036] Information    at Microsoft.ServiceProfiler.DiagnosticsAgent.Program.Main(String[] cmdArgs)

Failed adding start activity: /#1/1/4/

Hi
After first start profiler has been able to collect some traces,
But next runs were failed with such errors:

{"Timestamp":"2018-08-09T12:05:01.3892611+00:00","Level":"Information","MessageTemplate":"Service Profiler session started.","Properties":{"SourceContext":"ServiceProfiler.EventPipe.Client.Schedules.TraceSchedule","ConversationId":"(none)","ProfilingContext":"(none)"}}

{"Timestamp":"2018-08-09T12:05:29.3812704+00:00","Level":"Warning","MessageTemplate":"Failed adding start activity: /#1/1/4/","Properties":{"SourceContext":"ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener","ConversationId":"(none)","ProfilingContext":"(none)"}}

{"Timestamp":"2018-08-09T12:05:30.0249730+00:00","Level":"Information","MessageTemplate":"There is no matched start activity found for this activity: id /#1503500717/. This could happen for the first few activities.","Properties":{"SourceContext":"ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener","ConversationId":"(none)","ProfilingContext":"(none)"}}

What does it means?

Provide an example with CLR 2.1

There are small tweaks needed for the profiler to run with CLR 2.1. Considering update the examples or providing new examples.

Support more configurations

Duration, Interval, InitialDelay
Update document for how to use it.

Could not get instance for TraceUploaderProxy when using StructureMap as DI container

I followed the steps as described in the ReadMe. But it seems that when trying to instantiate classes from Application insights profiler (using services.AddServiceProfiler()), StructureMap throws an exception when trying to instantiate TraceUploaderProxy.
Bellow the exception I get:

Unable to create a build plan for concrete type Func<String, String, Task<Int32>> new Func<String, String, Task<Int32>>(Object, IntPtr method) Object = **Default** IntPtr method = Required primitive dependency is not explicitly defined 1.) Attempting to create a BuildPlan for Instance of Func<String, String, Task<Int32>> -- Func<String, String, Task<Int32>> 2.) new TraceUploaderProxy(*Default of IUploaderLocator*, *Default of IProfilerFrontendClient*, *Default of IFileWrapper*, *Default of IOutOfProcCaller*, *Default of IServiceProfilerContext*, *Default of ILogger<TraceUploaderProxy>*, *Default of IOptions<UserConfiguration>*, *Default of Func<String, String, Task<Int32>>*, *Default of Func<String>*) 3.) ServiceProfiler.EventPipe.Client.TraceUploader.TraceUploaderProxy 4.) Instance of ServiceProfiler.EventPipe.Client.TraceUploader.ITraceUploader (ServiceProfiler.EventPipe.Client.TraceUploader.TraceUploaderProxy) 5.) new ServiceProfilerProvider(*Default of TraceConfiguration*, *Default of IServiceProfilerContext*, *Default of ITraceControl*, *Default of ITraceSessionListenerFactory*, *Default of ITelemetryTrackerFactory*, *Default of ICustomEventsTracker*, *Default of ITraceUploader*, *Default of ITraceValidatorFactory*, *Default of IMetadataWriter*, *Default of IOptions<UserConfiguration>*, *Default of IAppInsightsSinks*, *Default of ILogger<ServiceProfilerProvider>*) 6.) ServiceProfiler.EventPipe.Client.ServiceProfilerProvider 7.) Instance of ServiceProfiler.EventPipe.Client.IServiceProfilerProvider (ServiceProfiler.EventPipe.Client.ServiceProfilerProvider) 8.) new OrchestratorEventPipe(*Default of IServiceProfilerProvider*, *Default of IOptions<UserConfiguration>*, *Default of IEnumerable<SchedulingPolicy>*, *Default of IDelaySource*, *Default of ILogger<OrchestratorEventPipe>*) 9.) Microsoft.ServiceProfiler.EventPipe.Orchestration.OrchestratorEventPipe 10.) Instance of Microsoft.ServiceProfiler.Orchestration.IOrchestrator (Microsoft.ServiceProfiler.EventPipe.Orchestration.OrchestratorEventPipe) 11.) new ServiceProfilerStartupFilter(*Default of IOrchestrator*, *Default of IOptions<UserConfiguration>*, *Default of ICompatibilityUtility*, *Default of ILogger<ServiceProfilerStartupFilter>*) 12.) ServiceProfiler.EventPipe.AspNetCore.ServiceProfilerStartupFilter 13.) Instance of Microsoft.AspNetCore.Hosting.IStartupFilter (ServiceProfiler.EventPipe.AspNetCore.ServiceProfilerStartupFilter) 14.) All registered children for IEnumerable<IStartupFilter> 15.) Instance of IEnumerable<IStartupFilter> 16.) new GenericWebHostService(*Default of IOptions<GenericWebHostServiceOptions>*, *Default of IServer*, *Default of ILoggerFactory*, *Default of DiagnosticListener*, *Default of IHttpContextFactory*, *Default of IApplicationBuilderFactory*, *Default of IEnumerable<IStartupFilter>*, *Default of IConfiguration*, *Default of IWebHostEnvironment*) 17.) Microsoft.AspNetCore.Hosting.GenericWebHostService 18.) Instance of Microsoft.Extensions.Hosting.IHostedService (Microsoft.AspNetCore.Hosting.GenericWebHostService) 19.) All registered children for IEnumerable<IHostedService> 20.) Instance of IEnumerable<IHostedService> 21.) Container.GetInstance(IEnumerable<IHostedService>)

I wanted to force StructureMap to use the default constructor if it exists, but unfortunately I can't, source code isn't available in the repo.

Bucketing sample caught

In a busy website, it doesn't make sense to gather every single requests if they are similar. It probably is better to bucketing the requests and recording activities in buckets of operations and duration.

Zipped trace file is left on the disk post uploading

The leftover will add up over time. Have to try to clean it up.

There's double enablement

It is reported that set the hosting startup environment at the same time call services.EnableServiceProfiler() leads to double enablement.
Needs to reproduce the issue and fix.

Worker Service applications (non-HTTP applications) support

Hi,

Are there any plans to support worker service applications much like it was done with the Microsoft.ApplicationInsights.WorkerService package?

With an invalid AppInsight instrumentation key I got "No stamp id is fetched. Abort the upload process." error message

It would be great to have a better error message to indicate that the key is invalid.

Profiler analysis for Windows App Services reports blob not found

For apps hosted in Azure App Service, the trace analysis failed with the following message:

The blob "url_to_the_blob.etl.zip" doesn't exist.

Update EventPipe Default configuration to try enable both provider name and guid for CLR 2.1

When CLR 2.1 arrives, the enablement requires provider name than guid. Try to enable both for backward compatibility. Expect either name or guid will be hit but not both.

Update the document to state clearly that 'Enable Profiler' button is not needed in AppInsights-Profiler-AspNetCore for Linux

@CawaMS As we discussed offline, could you please update the document, find a proper place to state this clear? Thanks!

Profiler on AppService/Linux fails compatability test

Deploying a .NET Core 2.1 Web API to App Service/Linux causes failure during profiler start. Logs show:

"Compatibility test failed. Reason: Runtime version 4.6.26628.5 is smaller than the minimum version of 4.6.26919.2"

I can work around the issue by setting the App Setting ServiceProfiler_IsSkipCompatabilityTest to true.

For Windows as well as linux?

Hi,

In the home readme it states: "App Services Linux profiler"

However deeper in the documentation: "https://github.com/Microsoft/ApplicationInsights-Profiler-AspNetCore/blob/master/examples/EnableServiceProfilerInVSCLR2_2_Win/README.md"

It also indicates Windows.

For Application Insights Profiler 1.1.4-beta1 and above, profiling on Windows is supported as an experimental feature for ASP.NET Core 2.2 and above applications.

I've never had any luck enabling the profiler via the traditional way in the portal (https://docs.microsoft.com/en-us/azure/azure-monitor/app/profiler) - a separate issue for another thread..... So hoping to give this nuget package a shot?

Hosting Startup is not working in dotnet core 3.0 Azure App Service (System.MissingMethodException: Method not found)

Hi,
i'm using the Microsoft.ApplicationInsights.Profiler.AspNetCore 2.0.0-beta1 in azure App Service with configured runtime netcore 3.0 and the following environment variable
ASPNETCORE_HOSTINGSTARTUPASSEMBLIES=Microsoft.ApplicationInsights.Profiler.AspNetCore
It fails with this exception :

2019-10-25T15:09:06.749908649Z Unhandled exception. System.MissingMethodException: Method not found: 'Microsoft.AspNetCore.Hosting.IHostingEnvironment Microsoft.AspNetCore.Hosting.WebHostBuilderContext.get_HostingEnvironment()'.
2019-10-25T15:09:06.749966649Z at Microsoft.AspNetCore.Hosting.ServiceProfilerInjection.<>c.b__0_0(WebHostBuilderContext hostingContext, IConfigurationBuilder config)
2019-10-25T15:09:06.749973349Z at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
2019-10-25T15:09:06.749978149Z at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
2019-10-25T15:09:06.756355094Z at MyApp.Program.Main(String[] args) in C:\BuildAgent\work\bfbade9ea212e645\MyApp\Program.cs:line 14

Everything works fine if i disable it (by removing the env variable above).
Thanks

What about Mac support?

I was attempting to try this out on my .net core 2.2 asp.net application. I am currently running it on a Mac for development and then pushing it to Azure app services running on linux. When testing on the Mac, the program crashes before it fully starts up. I see the charts show what versions work with linux and what versions work with Windows. What is the recommendation for Mac?

No customer events released error ASP.NET Core 3

I tried to follow the instructions for an ASP.NET Core 3.0 App, and I get the following error

fail: ServiceProfiler.EventPipe.Client.ServiceProfilerProvider[0]
Expected trace validation error.
ServiceProfiler.EventPipe.Client.TraceValidators.ValidateFailedException: [ActivityListValidator] No sample activity matches the trace.
at ServiceProfiler.EventPipe.Client.TraceValidators.ActivityListValidator.ValidateImp(IEnumerable1 samples) at ServiceProfiler.EventPipe.Client.TraceValidators.TraceValidatorBase.Validate(IEnumerable1 samples)
at ServiceProfiler.EventPipe.Client.TraceValidators.TraceValidatorBase.Validate(IEnumerable`1 samples)
at ServiceProfiler.EventPipe.Client.ServiceProfilerProvider.OnTraceStopped(Object sender, TraceStopEventArgs e)
warn: ServiceProfiler.EventPipe.Client.ServiceProfilerProvider[0]
Trace validation failed. No customer events released.

Is there an additional step other than turn off the compat test?

ConvertTraceToEtlxValidator shall follow trace's path to avoid permission issue

The ConvertTraceToEtlxValidator is using the current folder as the working folder, where it might not have writing permission.
Following the trace file folder is more reasonable.

Trace Upload failure, in some situation, is not caught

When upload fails in some case (different stamp id fetched) for example, the uploader is still returning 0, causing the events being sent without proper trace file uploaded.

This is due to a bug in the uploader that the exception is swallowed.

There should be a fix in the next release.

Add configurations to allow not deleting the local trace and/or bypass upload

Do not delete local traces
Bypass the upload

How to open downloaded .netperf file?

Is it any way to view downloaded .netperf files using some desktop tools like perfview or others?

Is ApplicationInsights Profiler open source?

That repository contains examples and documentation, but not the ApplicationInsights Profiler source code. Is ApplicationInsights profiler open source?

Consider adding Microsoft.ApplicationInsights.AspNetCore as a dependency in 2.1-based package

In ASP.NET Core 2.1, the default project template does not pull in the Microsoft.ApplicationInsights.AspNetCore so I had to add it in order for the newer 2.1-based profiler package to work. It is better to specify it as a dependency so end users don't need to include it by themselves.

Adding profiler to dotCore 2.1 on linux (AKS) causes huge memory usage.

We have added the profiler to our project. We are using dotNet Core on linux (Ubuntu) within kubernetes. Without Application Insights profiler, our application sits between 200m and 512m. When we add the profiler. We see it immediately jump up to 1.2 GB and then it keeps climbing settling in around 7.5 GB to 8 GB per pod. We have used both workstation and server GC. Workstation tends to increase memory slower.

Investigating adding trace info into the zipped file

Having the trace location info in the zipped etl file will simplify the troubleshooting and might open other scenarios. Note this down just so that the idea doesn't get lost.

Instructions for EnableServiceProfilerInVS sample are a bit outdated with respect to Docker tooling

Starting with Visual Studion 15.8 enabling Docker does not automatically add a docker-compose.yml file to the project. You need to do an extra step of right-clicking the project and doing Add | Container Orchestrator Support:

Understand Visual Studio ASP.NET Core 2.0 template and provide example to enable Service Profiler

Updating Dockerfile in ASP.NET Core 2.0 project dockerized by Visual Studio doesn't seem work. Investigate why and provide another example.

Service Profiler backend can't handle CLR 2.1 (rc) trace

When the trace for CLR 2.1 is submitted to the Service Profiler backend, the trance analysis failed.

ASP.NET Core 3.0 requires Microsoft.ApplicationInsights.AspNetCore 2.8+ but there's no hard dependency

There was a bug in MS.AI.AspNetCore before 2.8 crashing the app. Refer to discussion in #52
It would be great if there could be different dependencies for different .NET Core runtime for the best of backward compatibility as well as correct dependencies.

For .NET Core 2.x, the requirement should keep 2.3+ while for Core 3.0, the requirement should be 2.8+.

The initial trace session is not stable

Needs to investigate this and come up with a mitigation.

If 404 is hit first, there is a big chance there's no trace;
Sometimes, there just has no activity caught in EventPipe;
Check out the investigation details in #1 .
No matter what, when this happens, it is a really bad experience. Need to address this ASAP.

.NET Core 3.0 app running in Azure Web Apps keeps the trace for only 24 hours

This is a regression in 2.0.0-beta2. It is introduced by the inconsistency for the machine names between the metadata and the blob in the backend.

The fix is going to be released in 2.0.0-beta3.

Support configuration of buffer size

Introduce the configuration model to AI Profiler
Allow configuration for buffer size
Update the document for how to use it

StackOverflowException from ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener when LogLevel trace is used

Repro

AddServiceProfiler() with the ApplicationInsightsLogger and min loglevel of trace. The TraceSessionListener tries to log, which seems to trigger the TraceSessionListener again, resulting in a stack overflow.

What is printed to console

trce: ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener[0]
      OnRichPayloadEventWritten - EventName: Message, Keywords: 2, OpCode: Info
trce: ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener[0]
      OnRichPayloadEventWritten - EventName: Message, Keywords: 2, OpCode: Info
.....
..... Keeps printing this until it crashes

Stack Trace (captured from WinDgb)

... Keeps going like this in a loop for a long time
...
000000BE9517AE40 00007ffe50579d9e Microsoft.Extensions.Logging.Logger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging/src/Logger.cs @ 50]
000000BE9517AF68 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517af68] 
000000BE9517AFC0 00007ffe505795a7 Microsoft.Extensions.Logging.Logger`1[[System.__Canon, System.Private.CoreLib]].Microsoft.Extensions.Logging.ILogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging.Abstractions/src/LoggerOfT.cs @ 45]
000000BE9517B088 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517b088] 
000000BE9517B0E0 00007ffe505792a2 Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[]) [/_/src/Logging/Logging.Abstractions/src/LoggerExtensions.cs @ 397]
000000BE9517B150 00007ffe512513aa ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener.OnRichPayloadEventWritten(System.Diagnostics.Tracing.EventWrittenEventArgs)
000000BE9517B3F0 00007ffea9253bb1 System.Diagnostics.Tracing.EventSource.DispatchToAllListeners(Int32, System.Guid*, System.Diagnostics.Tracing.EventWrittenEventArgs) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\EventSource.cs @ 2076]
000000BE9517B450 00007ffea9259160 System.Diagnostics.Tracing.EventSource.WriteImpl(System.String, System.Diagnostics.Tracing.EventSourceOptions ByRef, System.Object, System.Guid*, System.Guid*, System.Diagnostics.Tracing.TraceLoggingEventTypes) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 702]
000000BE9517B700 00007ffea9257eb1 System.Diagnostics.Tracing.EventSource.Write[[System.__Canon, System.Private.CoreLib]](System.String, System.Diagnostics.Tracing.EventSourceOptions, System.__Canon) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 219]
000000BE9517B760 00007ffe512506f2 Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.WriteEvent[[System.__Canon, System.Private.CoreLib]](System.String, System.String, System.Collections.Generic.IDictionary`2<System.String,System.String>, System.__Canon, Int64, System.Diagnostics.Tracing.EventKeywords)
000000BE9517B830 00007ffe50bb163c Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.Process(Microsoft.ApplicationInsights.Channel.ITelemetry)
000000BE9517B8B0 00007ffe50bae235 Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [E:\A\_work\419\s\src\ILogger\ApplicationInsightsLogger.cs @ 103]
000000BE9517B9B8 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517b9b8] 
000000BE9517BA10 00007ffe50579d9e Microsoft.Extensions.Logging.Logger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging/src/Logger.cs @ 50]
000000BE9517BB38 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517bb38] 
000000BE9517BB90 00007ffe505795a7 Microsoft.Extensions.Logging.Logger`1[[System.__Canon, System.Private.CoreLib]].Microsoft.Extensions.Logging.ILogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging.Abstractions/src/LoggerOfT.cs @ 45]
000000BE9517BC58 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517bc58] 
000000BE9517BCB0 00007ffe505792a2 Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[]) [/_/src/Logging/Logging.Abstractions/src/LoggerExtensions.cs @ 397]
000000BE9517BD20 00007ffe512513aa ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener.OnRichPayloadEventWritten(System.Diagnostics.Tracing.EventWrittenEventArgs)
000000BE9517BFC0 00007ffea9253bb1 System.Diagnostics.Tracing.EventSource.DispatchToAllListeners(Int32, System.Guid*, System.Diagnostics.Tracing.EventWrittenEventArgs) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\EventSource.cs @ 2076]
000000BE9517C020 00007ffea9259160 System.Diagnostics.Tracing.EventSource.WriteImpl(System.String, System.Diagnostics.Tracing.EventSourceOptions ByRef, System.Object, System.Guid*, System.Guid*, System.Diagnostics.Tracing.TraceLoggingEventTypes) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 702]
000000BE9517C2D0 00007ffea9257eb1 System.Diagnostics.Tracing.EventSource.Write[[System.__Canon, System.Private.CoreLib]](System.String, System.Diagnostics.Tracing.EventSourceOptions, System.__Canon) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 219]
000000BE9517C330 00007ffe512506f2 Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.WriteEvent[[System.__Canon, System.Private.CoreLib]](System.String, System.String, System.Collections.Generic.IDictionary`2<System.String,System.String>, System.__Canon, Int64, System.Diagnostics.Tracing.EventKeywords)
000000BE9517C400 00007ffe50bb163c Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.Process(Microsoft.ApplicationInsights.Channel.ITelemetry)
000000BE9517C480 00007ffe50bae235 Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [E:\A\_work\419\s\src\ILogger\ApplicationInsightsLogger.cs @ 103]
000000BE9517C588 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517c588] 
000000BE9517C5E0 00007ffe50579d9e Microsoft.Extensions.Logging.Logger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging/src/Logger.cs @ 50]
000000BE9517C708 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517c708] 
000000BE9517C760 00007ffe505795a7 Microsoft.Extensions.Logging.Logger`1[[System.__Canon, System.Private.CoreLib]].Microsoft.Extensions.Logging.ILogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging.Abstractions/src/LoggerOfT.cs @ 45]
000000BE9517C828 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517c828] 
000000BE9517C880 00007ffe505792a2 Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[]) [/_/src/Logging/Logging.Abstractions/src/LoggerExtensions.cs @ 397]
000000BE9517C8F0 00007ffe512513aa ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener.OnRichPayloadEventWritten(System.Diagnostics.Tracing.EventWrittenEventArgs)
000000BE9517CB90 00007ffea9253bb1 System.Diagnostics.Tracing.EventSource.DispatchToAllListeners(Int32, System.Guid*, System.Diagnostics.Tracing.EventWrittenEventArgs) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\EventSource.cs @ 2076]
000000BE9517CBF0 00007ffea9259160 System.Diagnostics.Tracing.EventSource.WriteImpl(System.String, System.Diagnostics.Tracing.EventSourceOptions ByRef, System.Object, System.Guid*, System.Guid*, System.Diagnostics.Tracing.TraceLoggingEventTypes) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 702]
000000BE9517CEA0 00007ffea9257eb1 System.Diagnostics.Tracing.EventSource.Write[[System.__Canon, System.Private.CoreLib]](System.String, System.Diagnostics.Tracing.EventSourceOptions, System.__Canon) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 219]
000000BE9517CF00 00007ffe512506f2 Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.WriteEvent[[System.__Canon, System.Private.CoreLib]](System.String, System.String, System.Collections.Generic.IDictionary`2<System.String,System.String>, System.__Canon, Int64, System.Diagnostics.Tracing.EventKeywords)
000000BE9517CFD0 00007ffe50bb163c Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.Process(Microsoft.ApplicationInsights.Channel.ITelemetry)
000000BE9517D050 00007ffe50bae235 Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [E:\A\_work\419\s\src\ILogger\ApplicationInsightsLogger.cs @ 103]
000000BE9517D158 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517d158] 
000000BE9517D1B0 00007ffe50579d9e Microsoft.Extensions.Logging.Logger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging/src/Logger.cs @ 50]
000000BE9517D2D8 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517d2d8] 
000000BE9517D330 00007ffe505795a7 Microsoft.Extensions.Logging.Logger`1[[System.__Canon, System.Private.CoreLib]].Microsoft.Extensions.Logging.ILogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging.Abstractions/src/LoggerOfT.cs @ 45]
000000BE9517D3F8 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517d3f8] 
000000BE9517D450 00007ffe505792a2 Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[]) [/_/src/Logging/Logging.Abstractions/src/LoggerExtensions.cs @ 397]
000000BE9517D4C0 00007ffe512513aa ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener.OnRichPayloadEventWritten(System.Diagnostics.Tracing.EventWrittenEventArgs)
000000BE9517D760 00007ffea9253bb1 System.Diagnostics.Tracing.EventSource.DispatchToAllListeners(Int32, System.Guid*, System.Diagnostics.Tracing.EventWrittenEventArgs) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\EventSource.cs @ 2076]
000000BE9517D7C0 00007ffea9259160 System.Diagnostics.Tracing.EventSource.WriteImpl(System.String, System.Diagnostics.Tracing.EventSourceOptions ByRef, System.Object, System.Guid*, System.Guid*, System.Diagnostics.Tracing.TraceLoggingEventTypes) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 702]
000000BE9517DA70 00007ffea9257eb1 System.Diagnostics.Tracing.EventSource.Write[[System.__Canon, System.Private.CoreLib]](System.String, System.Diagnostics.Tracing.EventSourceOptions, System.__Canon) [E:\A\_work\644\s\src\mscorlib\shared\System\Diagnostics\Tracing\TraceLogging\TraceLoggingEventSource.cs @ 219]
000000BE9517DAD0 00007ffe512506f2 Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.WriteEvent[[System.__Canon, System.Private.CoreLib]](System.String, System.String, System.Collections.Generic.IDictionary`2<System.String,System.String>, System.__Canon, Int64, System.Diagnostics.Tracing.EventKeywords)
000000BE9517DBA0 00007ffe50bb163c Microsoft.ApplicationInsights.Extensibility.Implementation.RichPayloadEventSource.Process(Microsoft.ApplicationInsights.Channel.ITelemetry)
000000BE9517DC20 00007ffe50bc9984 Microsoft.ApplicationInsights.AspNetCore.Logging.ApplicationInsightsLogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>)
000000BE9517DD38 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517dd38] 
000000BE9517DD90 00007ffe50579d9e Microsoft.Extensions.Logging.Logger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging/src/Logger.cs @ 50]
000000BE9517DEB8 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517deb8] 
000000BE9517DF10 00007ffe505795a7 Microsoft.Extensions.Logging.Logger`1[[System.__Canon, System.Private.CoreLib]].Microsoft.Extensions.Logging.ILogger.Log[[System.__Canon, System.Private.CoreLib]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.__Canon, System.Exception, System.Func`3<System.__Canon,System.Exception,System.String>) [/_/src/Logging/Logging.Abstractions/src/LoggerOfT.cs @ 45]
000000BE9517DFD8 00007ffeafb1b1d6 [StubHelperFrame: 000000be9517dfd8] 
000000BE9517E030 00007ffe505792a2 Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[]) [/_/src/Logging/Logging.Abstractions/src/LoggerExtensions.cs @ 397]
000000BE9517E0A0 00007ffe5123dd05 ServiceProfiler.EventPipe.Client.ServiceProfilerProvider.StartServiceProfilerAsync(Microsoft.ServiceProfiler.Orchestration.IProfilerSource)
000000BE9517E190 00007ffe5123c8c2 Microsoft.ServiceProfiler.EventPipe.Orchestration.OrchestratorEventPipe+d__2.MoveNext() [E:\A\_work\366\s\src\ServiceProfiler.EventPipe\ServiceProfiler.EventPipe.Orchestration\OrchestratorEventPipe.cs @ 82]
000000BE9517E250 00007ffe5123b569 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.ServiceProfiler.EventPipe.Orchestration.OrchestratorEventPipe+d__2, Microsoft.ServiceProfiler.EventPipe.Orchestration]](d__2 ByRef) [E:\A\_work\644\s\src\mscorlib\src\System\Runtime\CompilerServices\AsyncMethodBuilder.cs @ 958]
000000BE9517E2C0 00007ffe5123a585 Microsoft.ServiceProfiler.EventPipe.Orchestration.OrchestratorEventPipe.StartProfilingAsync(Microsoft.ServiceProfiler.Orchestration.SchedulingPolicy, System.Threading.CancellationToken)
000000BE9517E330 00007ffe5123753b Microsoft.ServiceProfiler.Orchestration.SchedulingPolicy+d__1.MoveNext()
000000BE9517E490 00007ffe51233169 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.ServiceProfiler.Orchestration.SchedulingPolicy+d__1, Microsoft.ServiceProfiler.Orchestration]](d__1 ByRef) [E:\A\_work\644\s\src\mscorlib\src\System\Runtime\CompilerServices\AsyncMethodBuilder.cs @ 958]
000000BE9517E500 00007ffe51232eb3 Microsoft.ServiceProfiler.Orchestration.SchedulingPolicy.StartPolicyAsync(System.Threading.CancellationToken)
000000BE9517E5A0 00007ffe50fc66af Microsoft.ServiceProfiler.EventPipe.Orchestration.OrchestratorEventPipe+d__1.MoveNext() [E:\A\_work\366\s\src\ServiceProfiler.EventPipe\ServiceProfiler.EventPipe.Orchestration\OrchestratorEventPipe.cs @ 60]
000000BE9517E670 00007ffea9170af9 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [E:\A\_work\644\s\src\mscorlib\shared\System\Threading\ExecutionContext.cs @ 167]
000000BE9517E6F0 00007ffe51231857 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Microsoft.ServiceProfiler.EventPipe.Orchestration.OrchestratorEventPipe+d__1, Microsoft.ServiceProfiler.EventPipe.Orchestration]].MoveNext() [E:\A\_work\644\s\src\mscorlib\src\System\Runtime\CompilerServices\AsyncMethodBuilder.cs @ 568]
000000BE9517E750 00007ffea925c954 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [E:\A\_work\644\s\src\mscorlib\src\System\Threading\Tasks\TaskContinuation.cs @ 798]
000000BE9517E7B0 00007ffea9225d63 System.Threading.Tasks.Task.RunContinuations(System.Object) [E:\A\_work\644\s\src\mscorlib\src\System\Threading\Tasks\Task.cs @ 3258]
000000BE9517E880 00007ffea928bbbd System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib]].TrySetResult(System.Threading.Tasks.VoidTaskResult) [E:\A\_work\644\s\src\mscorlib\src\System\Threading\Tasks\future.cs @ 424]
000000BE9517E8C0 00007ffea9266bfc System.Threading.Tasks.Task+DelayPromise.Complete() [E:\A\_work\644\s\src\mscorlib\src\System\Threading\Tasks\Task.cs @ 5453]
000000BE9517E900 00007ffea9170af9 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [E:\A\_work\644\s\src\mscorlib\shared\System\Threading\ExecutionContext.cs @ 167]
000000BE9517E980 00007ffea926731c System.Threading.TimerQueueTimer.CallCallback() [E:\A\_work\644\s\src\mscorlib\src\System\Threading\Timer.cs @ 610]
000000BE9517E9E0 00007ffea926710a System.Threading.TimerQueueTimer.Fire() [E:\A\_work\644\s\src\mscorlib\src\System\Threading\Timer.cs @ 569]
000000BE9517EA50 00007ffea925ea32 System.Threading.ThreadPoolWorkQueue.Dispatch() [E:\A\_work\644\s\src\mscorlib\src\System\Threading\ThreadPool.cs @ 588]
000000BE9517EED0 00007ffeafb1a673 [DebuggerU2MCatchHandlerFrame: 000000be9517eed0]

Workaround

services.Configure<LoggerFilterOptions>(opts =>
 {
     opts.AddFilter("ServiceProfiler.EventPipe.Client.EventListeners.TraceSessionListener", LogLevel.Debug);
});

Caught trace is less than expected for nested requests

When there's chain of requests (one calling into another request), the caught trace is way less than expected.
This is caused by the algorithm in the request event matching. The fix will be fixed in 2.0.0-beta4.

Release Uploader as binaries on github

Do NOT over-reporting activity not match issue

When activity mismatch happens, it will be there until the start of the next session.

There is no matched start activity found for this activity...
There is no matched start activity found for this activity...

We don't need the report of the issue multiple times per second.

profiler is not cleaning up netperf files

We are seeing our /tmp directory increasing in size due to the netperf files; upto 3.8GB. It looks like the profiler is not cleaning up these files after upload.

According to the docs it looks like expected behavior is for these to be deleted after upload.

This is what the current /tmp looks like.

3.8G ./tmp/

drwxr-xr-x 4 root root 4096 Apr 13 10:04 0f5q0aim.m21
drwxr-xr-x 4 root root 4096 Apr 13 23:06 0v2bc13i.3ej
drwxr-xr-x 4 root root 4096 Apr 13 05:03 0y4kwp0h.2fm
drwxr-xr-x 4 root root 4096 Apr 15 15:12 11zy22hc.ldk
drwxr-xr-x 4 root root 4096 Apr 15 10:11 1xe4gp4p.ydd
drwxr-xr-x 4 root root 4096 Apr 12 22:02 2arqpmha.3jb
drwxr-xr-x 4 root root 4096 Apr 15 02:10 2zipqmio.z2q
drwxr-xr-x 4 root root 4096 Apr 13 03:03 35kscfzn.r2z
drwxr-xr-x 4 root root 4096 Apr 15 16:12 3i0vpefx.zxp
drwxr-xr-x 4 root root 4096 Apr 12 19:02 4d2acvr5.ldb
drwxr-xr-x 4 root root 4096 Apr 12 21:02 4fn2nl13.tx1
drwxr-xr-x 4 root root 4096 Apr 13 08:04 4qi5rk5u.cqy
drwxr-xr-x 4 root root 4096 Apr 15 08:11 4rlsctm1.nkq
drwxr-xr-x 4 root root 4096 Apr 14 02:06 5mhqgmci.cr2
drwxr-xr-x 4 root root 4096 Apr 12 23:02 5mjdvl25.pju
drwxr-xr-x 4 root root 4096 Apr 13 12:04 5plcall3.qyg
drwxr-xr-x 4 root root 4096 Apr 14 19:09 a3bstomk.2yv
drwxr-xr-x 4 root root 4096 Apr 12 16:01 a4nefdbe.twk
drwxr-xr-x 4 root root 4096 Apr 15 11:11 arlebobb.bzo
drwxr-xr-x 4 root root 4096 Apr 15 09:11 b5svqnpp.4xn
drwxr-xr-x 4 root root 4096 Apr 15 04:10 bs4nar3f.40l
drwxr-xr-x 4 root root 4096 Apr 14 23:09 cap0kzpu.y41
drwxr-xr-x 4 root root 4096 Apr 14 06:07 cfl3nmff.xoj
drwxr-xr-x 4 root root 4096 Apr 15 05:10 civu3vnq.q1t
prwx------ 1 root root 0 Apr 12 13:59 clr-debug-pipe-1-684823511-in
prwx------ 1 root root 0 Apr 12 13:59 clr-debug-pipe-1-684823511-out
drwxr-xr-x 4 root root 4096 Apr 14 17:09 d5ssnagd.3wc
drwxr-xr-x 4 root root 4096 Apr 13 18:05 dkhr2ywv.q4q
drwxr-xr-x 4 root root 4096 Apr 15 14:12 duv3vvtv.tdw
drwxr-xr-x 4 root root 4096 Apr 13 04:03 eyt4ui2h.poo
drwxr-xr-x 4 root root 4096 Apr 14 21:09 fwoypt0e.w03
drwxr-xr-x 4 root root 4096 Apr 14 16:08 gemz0fre.ehu
drwxr-xr-x 4 root root 4096 Apr 14 15:08 ghepdpsd.51g
drwxr-xr-x 4 root root 4096 Apr 12 15:01 giskwfxc.zea
drwxr-xr-x 4 root root 4096 Apr 14 08:07 hbzsctic.i41
drwxr-xr-x 4 root root 4096 Apr 13 11:04 hdhg1c3z.qlw
drwxr-xr-x 4 root root 4096 Apr 13 15:05 hgkuuwv5.wm3
drwxr-xr-x 4 root root 4096 Apr 14 20:09 housrtei.smt
drwxr-xr-x 4 root root 4096 Apr 13 21:06 hw5kf0rf.snr
drwxr-xr-x 4 root root 4096 Apr 12 17:01 icbhpjwh.qrb
drwxr-xr-x 4 root root 4096 Apr 12 18:02 icibf1b2.qvo
drwxr-xr-x 4 root root 4096 Apr 14 03:07 jcgi0xwl.rme
drwxr-xr-x 4 root root 4096 Apr 15 13:12 jmuwxmvf.rjb
drwxr-xr-x 4 root root 4096 Apr 14 07:07 jnctro4r.qvc
drwxr-xr-x 4 root root 4096 Apr 13 14:05 jtzvsese.hrx
drwxr-xr-x 4 root root 4096 Apr 13 06:03 jyyszxdi.4al
drwxr-xr-x 4 root root 4096 Apr 14 12:08 k4vgr5wk.l34
drwxr-xr-x 4 root root 4096 Apr 14 18:09 kmlcgbpb.x4c
drwxr-xr-x 4 root root 4096 Apr 13 19:05 kvu4buul.q1k
drwxr-xr-x 4 root root 4096 Apr 13 16:05 l4i1rqy2.gsj
drwxr-xr-x 4 root root 4096 Apr 13 13:04 maugkj3e.4pn
drwxr-xr-x 4 root root 4096 Apr 13 17:05 ndzhnn51.kr4
drwxr-xr-x 4 root root 4096 Apr 14 22:09 nv0gp2w2.c02
drwxr-xr-x 4 root root 4096 Apr 15 01:10 nx0upoqe.nyx
drwxr-xr-x 4 root root 4096 Apr 13 02:03 nz0dgb1r.oqh
drwxr-xr-x 4 root root 4096 Apr 15 03:10 nzgb4s4p.ied
drwxr-xr-x 4 root root 4096 Apr 14 00:06 olqbnj2x.xei
drwxr-xr-x 4 root root 4096 Apr 14 09:07 pgewy3ze.yhl
drwxr-xr-x 4 root root 4096 Apr 12 14:01 q5nupwna.kgx
drwxr-xr-x 4 root root 4096 Apr 13 22:06 qn2cxfog.5ph
drwxr-xr-x 4 root root 4096 Apr 13 20:05 qw1m0cfv.pvn
drwxr-xr-x 4 root root 4096 Apr 15 12:11 ryzf0kgp.hvc
drwxr-xr-x 4 root root 4096 Apr 15 07:11 s3a0n3dh.fu0
drwxr-xr-x 4 root root 4096 Apr 15 06:10 s3sjyvpo.ok4
drwxr-xr-x 4 root root 4096 Apr 15 17:12 sgse4xjj.tlj
drwxr-xr-x 4 root root 4096 Apr 13 07:04 sp3og1qo.s1m
drwxr-xr-x 4 root root 4096 Apr 13 00:02 stjwctcj.h1p
drwxr-xr-x 4 root root 4096 Apr 14 13:08 ui1eafxp.irs
drwxr-xr-x 4 root root 4096 Apr 14 14:08 v0idmuvv.xjf
drwxr-xr-x 4 root root 4096 Apr 14 04:07 w0ilwlbj.eza
drwxr-xr-x 4 root root 4096 Apr 14 11:08 w3q4wsrd.4u0
drwxr-xr-x 4 root root 4096 Apr 13 09:04 wzvmb2r3.beg
drwxr-xr-x 4 root root 4096 Apr 13 01:03 x2aqlqje.cjp
drwxr-xr-x 4 root root 4096 Apr 14 01:06 x2i0jspo.opu
drwxr-xr-x 4 root root 4096 Apr 14 10:08 y0e0ca4h.3vq
drwxr-xr-x 4 root root 4096 Apr 14 05:07 yhfvz4vr.df5
drwxr-xr-x 4 root root 4096 Apr 12 20:02 z1hqtkw2.3wv
drwxr-xr-x 4 root root 4096 Apr 15 00:10 zkxwrkrg.k5q

Version compatibility test fails when running in .NET Core 3.0

In .NET Core 3.0, the returned value for the .NET Core version will return the correct value, which breaks the existing compatibility test.
Work around:
By pass the compatibility test:

  ...
  "ServiceProfiler": {
    ...
    "IsSkipCompatibilityTest": false
  },

Details:
Refer to What's new in .NET Core 3.0 - improved net core version APIs.

Profiler not enabled in Azure

As simple as this seems, I can't get this to work in my existing ASP.NET Core application. The application already insights enabled and a publish profile to Azure, and Insights events and exceptions are shown smoothly.
So instead of the three-step instructions here, I did this:

Added nuget reference to the project. Since it's a VS project I didn't use the Dockerfile for this.
The InstrumentationKey is already existing in the appsettings.json, so I didn't add that either in the Dockerfile.
Don't know if it's needed, but I added the ASPNETCORE_HOSTINGSTARTUPASSEMBLIES to the Dockerfile.
Published the project to Azure.

Now when I log in to Application Insights, the profiler is not enabled. When I click on the Enable Profiler button, I can see that a "Profiler Agent" is not installed. I get the option to install the extension (Application Insights Site Extension), but that ends with the exception of:

"Kudu.Contracts.SiteExtensions.InvalidEndpointException: Invalid remote feed url: https://www.nuget.org/api/v2/ at Kudu.Core.SiteExtensions.FeedExtensions...."

Severely degraded performance while profiling a Linux app

Enabling the Application Insights profiler for an ASP.NET Core app on Azure App Service results in response times slowing down to over 30 seconds (during profiling) where they would normally be in the milliseconds. This is reproducable by following the guide at https://docs.microsoft.com/en-us/azure/application-insights/app-insights-profiler-aspnetcore-linux to set up a ASP.NET app and placing it on a B1 app service plan.

Idle memory use increases by 300-400MB after adding the profiler, and every time the profiler activates memory use spikes up by about 600MB and response times slow down significantly. Since the response times become so slow it's not feasible to actually use the profiler at all.

Here's a chart from application insights displaying the memory use of an example app. The profiler was added at the very beginning of the chart, where memory use jumps up from ~50MB to nearly 700MB. It then keeps spiking for each profiling session it starts. If you try to load the web apps pages during a profiling session you'll encounter response times of over 30 seconds.

Is this kind of resource use expected?

AI Profiler not working well with .Net Core 3.0 and AKS

When using AI Profiler together with DEV-Spaces and AKS, I receive this error in my logs.

It was not possible to find any compatible framework version
The specified framework 'Microsoft.NETCore.App', version '2.0.0' was not found.
  - The following frameworks were found:
      3.0.0 at [/usr/share/dotnet/shared/Microsoft.NETCore.App]

You can resolve the problem by installing the specified framework and/or SDK.

The .NET Core frameworks can be found at:
  - https://aka.ms/dotnet-download
�[41m�[30mfail�[39m�[22m�[49m: ServiceProfiler.EventPipe.Client.TraceUploader.TraceUploaderProxy[0]
      Trace upload failed.

I am creating my docker container using this build file:

FROM mcr.microsoft.com/dotnet/core/sdk:3.0
ARG BUILD_CONFIGURATION=Debug
ENV ASPNETCORE_ENVIRONMENT=Development
ENV DOTNET_USE_POLLING_FILE_WATCHER=true
ENV ASPNETCORE_URLS=http://+:80
EXPOSE 80

WORKDIR /src
COPY ["src/Test.Service/Test.Service.csproj", "src/Test.Service/"]

RUN dotnet restore "src/Test.Service/Test.Service.csproj"
COPY . .
WORKDIR "/src/src/Test.Service"
RUN dotnet build --no-restore "Test.Service.csproj" -c $BUILD_CONFIGURATION

RUN echo "exec dotnet run --no-build --no-launch-profile -c $BUILD_CONFIGURATION --" > /entrypoint.sh

ENTRYPOINT ["/bin/bash", "/entrypoint.sh"]

I find it strange that it says that it has found .Net Core 3.0 but it needs .Net Core 2.0.
I am using the latest beta-2 package.

Any idea on how to solve this?

Update packages in examples to address the vulnerability issues

Package of Microsoft.AspNetCore.All needs to be updated to the latest to address the vulnerability concerns.

Trace analysis result of netperf on Windows is noisy

Because of the path format differences, the grouping rules for netperf file from Linux does not apply to traces from windows platform.

503 after enabling Profiler

Hi!
I have a small .net core app. This app is hosted as a docker container in Kubernetes. When I enable Application Insights Profiller then all request to this application falls in 503 error (Bad Gateway)
I tryed to enable profiling using both methods "ENV ASPNETCORE_HOSTINGSTARTUPASSEMBLIES Microsoft.ApplicationInsights.Profiler.AspNetCore" and "services.AddServiceProfiler();"
But the error still remained. How can I investigate this problem?

I see in application logs only:
{"Timestamp":"2018-05-03T12:08:38.1261884+00:00","Level":"Information","MessageTemplate":"Service Profiler session started.","SourceContext":"ServiceProfiler.EventPipe.Client.Schedules.TraceSchedule","ConversationId":"(none)","ProfilingContext":"(none)"}

"Profile now" from portal should work for dockerized applications

Following the tutorials I've been able to run AI profiling on a dockerized asp.net core application (asp.net core running on linux docker locally at the moment).

I see the server registered in AI; live metrics streaming in the "live metrics stream" and requests/error/dependencies showing up as they should. For the profiler I see that it profiles two minutes at startup and then idle for 58 minutes before another profiling session. This all works as expected.

But the profile now button doesn't seem to work. And this is what could make it a killer feature in my book. We've got several applications/micro services talking together and they have the AI instance in common. So if we could say profile now and gather metrics/debug data across them that would be golden.

If it's already supported and I've missed some configuration, please let me know and I'll kill this issue. I looked around but couldn't seem to find anything pinpointing this use case.

Linux Profiler traces can't be seen from the Azure Portal

There's a known bug in the latest deployed backend that leading to trace analysis failures.
While the issue is fixed, please expect there to be a period before we finish the test and get the fix deployed.

To workaround the issue:

Download the trace from the analysis page using the 'Download' button on the right upper corner.
Rename the trace file from *.etl.zip to *.netperf.zip.
Open it using the latest perfview: https://github.com/Microsoft/perfview

Feel free to open issue for any specific problem you encountered.

microsoft / applicationinsights-profiler-aspnetcore Goto Github PK