-
Notifications
You must be signed in to change notification settings - Fork 6.1k
[Diagnostics] Updating docs and adding performance investigation guidance #55913
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
e8e2044
fb7afea
44d240a
61dc851
a4c7e3b
f5c4756
1307431
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,15 +1,13 @@ | ||||||
| --- | ||||||
| title: Debug high CPU usage - .NET Core | ||||||
| description: A tutorial that walks you through debugging high CPU usage in .NET Core. | ||||||
| title: Debug high CPU usage - .NET | ||||||
| description: A tutorial that walks you through debugging high CPU usage in .NET. | ||||||
| ms.topic: tutorial | ||||||
| ms.date: 03/19/2026 | ||||||
| ms.date: 09/08/2026 | ||||||
| --- | ||||||
|
|
||||||
| # Debug high CPU usage in .NET Core | ||||||
| # Debug high CPU usage in .NET | ||||||
|
|
||||||
| **This article applies to: ✔️** .NET Core 3.1 SDK and later versions | ||||||
|
|
||||||
| In this tutorial, you'll learn how to debug an excessive CPU usage scenario. Using the provided example [ASP.NET Core web app](/samples/dotnet/samples/diagnostic-scenarios) source code repository, you can cause a deadlock intentionally. The endpoint will stop responding and experience thread accumulation. You'll learn how you can use various tools to diagnose this scenario with several key pieces of diagnostics data. | ||||||
| In this tutorial, you'll learn how to debug an excessive CPU usage scenario. Using the provided example [ASP.NET Core web app](/samples/dotnet/samples/diagnostic-scenarios), you can intentionally run CPU-intensive work and use metrics and platform-appropriate profiling tools to identify the expensive code. | ||||||
|
|
||||||
| In this tutorial, you will: | ||||||
|
|
||||||
|
|
@@ -25,9 +23,9 @@ In this tutorial, you will: | |||||
|
|
||||||
| The tutorial uses: | ||||||
|
|
||||||
| - [.NET Core 3.1 SDK](https://dotnet.microsoft.com/download/dotnet) or a later version. | ||||||
| - A supported [.NET SDK](https://dotnet.microsoft.com/download/dotnet). | ||||||
| - [Sample debug target](/samples/dotnet/samples/diagnostic-scenarios) to trigger the scenario. | ||||||
| - [dotnet-trace](dotnet-trace.md) to list processes and generate a profile. | ||||||
| - [dotnet-trace](dotnet-trace.md) to collect CPU profiles and runtime traces. | ||||||
| - [dotnet-counters](dotnet-counters.md) to monitor cpu usage. | ||||||
|
|
||||||
| ## CPU counters | ||||||
|
|
@@ -171,15 +169,17 @@ Throughout the duration of the request, the CPU usage will hover around the incr | |||||
|
|
||||||
| At this point, you can safely say the CPU is running higher than you expect. Identifying the effects of a problem is key to finding the cause. We will use the effect of high CPU consumption in addition to diagnostic tools to find the cause of the problem. | ||||||
|
|
||||||
| ## Analyze High CPU with Profiler | ||||||
| ## Analyze high CPU with a profiler | ||||||
|
|
||||||
| When analyzing an app with high CPU usage, use a profiler to understand what the code is doing. `dotnet-trace collect` works on all operating systems, but safe-point bias and managed-only callstacks limit it to more general information than a kernel-aware profiler like ETW for Windows or `perf` for Linux. Depending on your operating system and .NET version, improved profiling capabilities might be available—see the platform-specific tabs that follow for detailed guidance. | ||||||
| When analyzing an app with high CPU usage, use a profiler to understand what the code is doing. `dotnet-trace collect` works on all operating systems, but safe-point bias and managed-only call stacks limit it to more general information than kernel-aware profiling through ETW on Windows or `perf_events` on Linux. Depending on your operating system and .NET version, improved profiling capabilities might be available. See the platform-specific tabs that follow for detailed guidance. | ||||||
|
|
||||||
| ### [Linux](#tab/linux) | ||||||
|
|
||||||
| Prefer `dotnet-trace collect-linux` for the .NET-oriented Linux workflow. Use OneCollect `record-trace` when you need its lower-level scripting, filtering, or output controls, and use `perf` directly only when you need `perf.data`, perf-native analysis, or hardware performance counters. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since this is a tutorial I think its better to offer fewer options when possible. I'd suggest we only describe dotnet-trace collect-linux for .NET 10+ and perf as an alternative covering scenarios where collect-linux isn't available. Currently the user experience for record-trace doesn't feel mature enough that its worth complicating the tutorial to add it (the project README describes it as pre-release with no clear distribution or usage documentation) |
||||||
|
|
||||||
| #### Use `dotnet-trace collect-linux` (.NET 10+) | ||||||
|
|
||||||
| On .NET 10 and later, [`dotnet-trace collect-linux`](dotnet-trace.md#dotnet-trace-collect-linux) is the recommended profiling approach on Linux. It combines EventPipe with OS-level perf_events to produce a single unified trace that includes both managed and native callstacks, all without requiring a process restart. This requires root permissions and Linux kernel 6.4+ with `CONFIG_USER_EVENTS=y`. See [collect-linux prerequisites](dotnet-trace.md#prerequisites) for full requirements. | ||||||
| On .NET 10+, [`dotnet-trace collect-linux`](dotnet-trace.md#dotnet-trace-collect-linux) is the recommended Linux workflow. It retains .NET runtime and application event collection while adding kernel CPU samples, native call stacks, and selected Linux events through `perf_events`, all without requiring a process restart. This requires root permissions and Linux kernel 6.4+ with `CONFIG_USER_EVENTS=y`. See [collect-linux prerequisites](dotnet-trace.md#prerequisites) for full requirements. | ||||||
|
|
||||||
| Ensure the [sample debug target](/samples/dotnet/samples/diagnostic-scenarios) is configured to target .NET 10 or later, then run it and exercise the high CPU endpoint (`https://localhost:5001/api/diagscenario/highcpu/60000`) again. While it's running within the 1-minute request, run `dotnet-trace collect-linux` to capture a machine-wide trace: | ||||||
|
|
||||||
|
|
@@ -191,11 +191,28 @@ Let it run for about 20-30 seconds, then press <kbd>Ctrl+C</kbd> or <kbd>Enter</ | |||||
|
|
||||||
| Open the `.nettrace` with [`PerfView`](https://github.com/microsoft/perfview/blob/main/documentation/Downloading.md) and use the **CPU Stacks** view to identify the methods consuming the most CPU time. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is PerfView still a better experience than Visual Studio here? If it is thats fine but hopefully we'd be working with the VS team to resolve whatever issues make us recommend PerfView instead of VS. |
||||||
|
|
||||||
| For information about resolving native runtime symbols in the trace, see [Get symbols for native runtime frames](dotnet-trace.md#get-symbols-for-native-runtime-frames). | ||||||
| PerfView and TraceEvent 3.2.1 or later can resolve .NET native and R2R symbols at analysis time. In PerfView, select unresolved module frames and choose **Lookup Symbols**. For other native libraries, configure a local symbol path. For more information, see [Get symbols for native runtime frames](dotnet-trace.md#get-symbols-for-native-runtime-frames). | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Tutorial users will probably be using GUI tools, not writing custom tooling using TraceEvent. |
||||||
|
|
||||||
| For a broader workflow that covers CPU, blocking, GC, exceptions, I/O, and startup, see [Investigate Linux performance with `dotnet-trace collect-linux`](dotnet-trace-collect-linux-performance.md). | ||||||
|
|
||||||
| #### Use OneCollect `record-trace` | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggest we remove this section for brevity and clarity. |
||||||
|
|
||||||
| OneCollect's [`record-trace`](https://github.com/microsoft/one-collect/tree/main/record-trace) tool provides lower-level control over event selection, process and CPU filtering, scripts, and output format. See the [OneCollect build instructions](https://github.com/microsoft/one-collect/blob/main/CONTRIBUTING.md#building-the-project) to obtain the tool. | ||||||
|
|
||||||
| This CPU profiling workflow doesn't require .NET 10 or `user_events`. After making the executable available on `PATH`, exercise the high CPU endpoint again, and while it's running, capture a 30-second machine-wide CPU profile: | ||||||
|
|
||||||
| ```bash | ||||||
| sudo record-trace \ | ||||||
| --on-cpu \ | ||||||
| --duration 30 \ | ||||||
| --out highcpu.nettrace | ||||||
| ``` | ||||||
|
|
||||||
| This example uses the default NetTrace output so that you can open `highcpu.nettrace` in PerfView and inspect **CPU Stacks**. `record-trace` can also write PerfView XML with `--format perfview-xml`, display samples while recording with `--live`, and use Rhai scripts for more detailed event configuration. | ||||||
|
|
||||||
| #### Use `perf` | ||||||
|
|
||||||
| The `perf` tool can also be used to generate .NET Core app profiles. Exit the previous instance of the [sample debug target](/samples/dotnet/samples/diagnostic-scenarios). | ||||||
| Use `perf` directly when the investigation requires the Linux perf ecosystem, such as `perf.data`, `perf report`, `perf annotate`, established flame graph scripts, or hardware performance counters. The following steps demonstrate the standard `perf record` and `perf report` workflow. Exit the previous instance of the [sample debug target](/samples/dotnet/samples/diagnostic-scenarios). | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd reword this to also recognize that this is the default path when you want kernel level CPU sampling and dotnet-trace collect-linux isn't available. Many people might use this path even when they don't specifically care about perf ecosystem compat. |
||||||
|
|
||||||
| Set the `DOTNET_PerfMapEnabled` environment variable to cause the .NET app to create a `map` file in the `/tmp` directory. This `map` file is used by `perf` to map CPU addresses to JIT-generated functions by name. For more information, see [Export perf maps and jit dumps](../runtime-config/debugging-profiling.md#export-perf-maps-and-jit-dumps). | ||||||
|
|
||||||
|
|
@@ -245,18 +262,18 @@ Open the `nettrace` with [`PerfView`](https://github.com/microsoft/perfview/blob | |||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ## Analyzing High CPU Data with Visual Studio | ||||||
| ## Analyze high CPU data with Visual Studio | ||||||
|
|
||||||
| All \*.nettrace files can be analyzed in Visual Studio. To analyze a Linux \*.nettrace file in Visual Studio, transfer the \*.nettrace file, in addition to the other necessary documents, to a Windows machine, and then open the \*.nettrace file in Visual Studio. For more information, see [Analyze CPU Usage Data](/visualstudio/profiling/beginners-guide-to-performance-profiling?#step-2-analyze-cpu-usage-data). | ||||||
|
|
||||||
| ## See also | ||||||
|
|
||||||
| - [dotnet-trace](dotnet-trace.md) to list processes | ||||||
| - [dotnet-trace](dotnet-trace.md) to collect CPU profiles and runtime traces | ||||||
| - [dotnet-counters](dotnet-counters.md) to check managed memory usage | ||||||
| - [dotnet-dump](dotnet-dump.md) to collect and analyze a dump file | ||||||
| - [dotnet/diagnostics](https://github.com/dotnet/diagnostics/tree/main/documentation/tutorial) | ||||||
|
|
||||||
| ## Next steps | ||||||
|
|
||||||
| > [!div class="nextstepaction"] | ||||||
| > [Debug a deadlock in .NET Core](debug-deadlock.md) | ||||||
| > [Debug a deadlock in .NET](debug-deadlock.md) | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,13 +2,11 @@ | |
| title: Debug a memory leak tutorial | ||
| description: Learn how to debug a memory leak in .NET. | ||
| ms.topic: tutorial | ||
| ms.date: 11/13/2023 | ||
| ms.date: 09/08/2026 | ||
| --- | ||
|
|
||
| # Debug a memory leak in .NET | ||
|
|
||
| **This article applies to:** ✔️ .NET Core 3.1 SDK and later versions | ||
|
|
||
| Memory can leak when your app references objects that it no longer needs to perform the desired task. Referencing these objects prevents the garbage collector from reclaiming the memory used. That can result in performance degradation and an <xref:System.OutOfMemoryException> exception being thrown. | ||
|
|
||
| This tutorial demonstrates the tools to analyze a memory leak in a .NET app using the .NET diagnostics CLI tools. If you're on Windows, you may be able to [use Visual Studio's Memory Diagnostic tools](/visualstudio/profiling/memory-usage) to debug the memory leak. | ||
|
|
@@ -27,7 +25,7 @@ In this tutorial, you will: | |
|
|
||
| The tutorial uses: | ||
|
|
||
| - [.NET Core 3.1 SDK](https://dotnet.microsoft.com/download/dotnet) or a later version. | ||
| - A supported [.NET SDK](https://dotnet.microsoft.com/download/dotnet). | ||
| - [dotnet-counters](dotnet-counters.md) to check managed memory usage. | ||
| - [dotnet-dump](dotnet-dump.md) to collect and analyze a dump file (includes the [SOS debugging extension](sos-debugging-extension.md)). | ||
| - A [sample debug target](/samples/dotnet/samples/diagnostic-scenarios/) app to diagnose. | ||
|
|
@@ -141,6 +139,8 @@ Observe that the memory usage has grown to over 20 MB. | |
|
|
||
| By watching the memory usage, you can safely say that memory is growing or leaking. The next step is to collect the right data for memory analysis. | ||
|
|
||
| If you only need to compare managed heap composition or identify which object types are growing, start with [`dotnet-gcdump`](dotnet-gcdump.md), which collects less process state than a full dump. This tutorial uses `dotnet-dump` because the investigation continues from growing object types to the reference paths that keep those objects alive. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Given the reliability issues dotnet-gcdump has with large heaps on older runtime I hesitate to redirect people to it. It feels simpler to just ignore it here. |
||
|
|
||
| ### Generate memory dump | ||
|
|
||
| When analyzing possible memory leaks, you need access to the app's memory heap to analyze the memory contents. Looking at relationships between objects, you create theories as to why memory isn't being freed. A common diagnostic data source is a memory dump on Windows or the equivalent core dump on Linux. To generate a dump of a .NET application, you can use the [dotnet-dump](dotnet-dump.md) tool. | ||
|
|
@@ -266,7 +266,7 @@ You can also delete the dump file that was created. | |
|
|
||
| ## See also | ||
|
|
||
| - [dotnet-trace](dotnet-trace.md) to list processes | ||
| - [dotnet-trace](dotnet-trace.md) to collect runtime performance traces | ||
| - [dotnet-counters](dotnet-counters.md) to check managed memory usage | ||
| - [dotnet-dump](dotnet-dump.md) to collect and analyze a dump file | ||
| - [dotnet/diagnostics](https://github.com/dotnet/diagnostics/tree/main/documentation/tutorial) | ||
|
|
@@ -275,4 +275,4 @@ You can also delete the dump file that was created. | |
| ## Next steps | ||
|
|
||
| > [!div class="nextstepaction"] | ||
| > [Debug high CPU in .NET Core](debug-highcpu.md) | ||
| > [Debug high CPU in .NET](debug-highcpu.md) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A deadlock normally means two or more threads that block on each other indefinitely so anything that is intermittent isn't usually considered a deadlock. Without trying to be pedantic though I'd suggest not mentioning contention or the link here just to stay focused on a single symptom and its resolution. In the future if we suspected people were coming to this tutorial in error and really they had an intermittent latency issue rather than a deadlock then we might want to put some clarifying text/links in the intro to redirect them. Ideally we'd also redirect them to something that is more OS agnostic rather than just a linux specific example.