microsoft / dbgshell Goto Github PK

View Code? Open in Web Editor NEW

664.0 46.0 101.0 2.2 MB

A PowerShell front-end for the Windows debugger engine.

License: MIT License

C# 70.66% PowerShell 10.83% C++ 5.49% C 12.93% XSLT 0.06% JavaScript 0.03%

debugging debugger windbg debugger-extension

dbgshell's Introduction

DbgShell

A PowerShell front-end for the Windows debugger engine.

Ready to tab your way to glory? For a quicker intro, take a look at Getting Started.

Disclaimers

This project is not produced, endorsed, or monitored by the Windows debugger team. While the debugger team welcomes feedback about their API and front ends (windbg, kd, et al), they have no connection with this project. Do not file bugs or feedback to the debugger team concerning this project.
This is not a funded project: it has no official resources allocated to it, and is only worked on by volunteers. Do not take any production dependency on this project unless you are willing to support it completely yourself. Feel free to file Issues and submit Pull Requests, but understand that with the limited volunteer resources, it may be a while before your submissions are handled.
This is an experimental project: it is not fully baked, and you should expect breaking changes to be made often.

Corollary of above disclaimers: I would avoid attaching DbgShell to live targets of high value.

Binaries

https://aka.ms/dbgshell-latest

Motivation

Have you ever tried automating anything in the debugger? (cdb/ntsd/kd/windbg) How did that go for you?

The main impetus for DbgShell is that it's just waaaay too hard to automate anything in the debugger. There are facilities today to assist in automating the debugger, of course. But in my opinion they are not meeting people's needs.

Using the built-in scripting language is arcane, limited, difficult to get right, and difficult to get help with.
Writing a full-blown debugger extension DLL is very powerful, but it's a significant investment—way too expensive for solving quick, "one-off" problems as you debug random, real-world problems. Despite the cost, there are a large number of debugger extensions in existence. I think there should not be nearly so many; I think the only reason there are so many is because there aren't viable alternatives.
Existing attempts at providing a better interface (such as PowerDbg) are based on "scraping" and text parsing, which is hugely limiting (not to mention ideologically annoying) and thus are not able to fulfill the promise of a truly better interface (they are only marginally better, at best).
Existing attempts to provide an easier way to write a debugger extension are merely a stop-gap addressing the pain of developing a debugger extension; they don't really solve the larger problem. (for instance, two major shortcomings are: they are still too low-level (you have to deal with the dbgeng COM API), and there's no REPL)
The debugger team has recently introduce Javascript scripting. Javascript is a much better (and more well-defined) language than the old windbg scripting language, but I think that PowerShell has some advantages, the largest of which is that nobody really uses a Javascript shell--PowerShell is much better as a combined shell and scripting language.

The goal of the DbgShell project is to bring the goodness of the object-based PowerShell world to the debugging world. When you do 'dt' to dump an 'object', you should get an actual object. Scripting should be as easy as writing a PowerShell script.

The DbgShell project provides a PowerShell front-end for dbgeng.dll, including:

a managed "object model" (usable from C# if you wished), which is higher-level than the dbgeng COM API,
a PowerShell "navigation provider", which exposes aspects of a debugging target as a hierarchical namespace (so you can "cd" to a particular thread, type "dir" to see the stack, "cd" into a frame, do another "dir" to see locals/registers/etc.),
cmdlets for manipulating the target,
a custom PowerShell host which allows better control of the debugger CLI experience, as well as providing features not available in the standard powershell.exe host (namely, support for text colorization using ANSI escape codes (a la ISO/IEC 6429))

The custom host is still a command-line (conhost.exe-based) program (analogous to ntsd/cdb/kd), but it can be invoked from windbg (!DbgShell).

In addition to making automation much easier and more powerful, it will address other concerns as well, such as ease of use for people who don't have to use the debuggers so often. (one complaint I've heard is that "when I end up needing to use windbg, I spend all my time in the .CHM")

For seasoned windbg users, on the other hand, another goal is to make the transition as seamless as possible. So, for instance, the namespace provider is not the only way to access data; you can still use traditional commands like "~3 s", "k", etc.

What do you mean by "automation" and "scripting"?

I'm not only talking about the sort of thing where you open up a text editor and write some big script to do something complex—I'm also talking about being able to whip out relatively simple stuff directly on the command line. There are many situations where you would like to be able to use a little bit of logic, but nothing so big or re-usable that you would even want to save it. It should be easy to just whip off "one-liners" like "break on CreateFile if the file being opened is on the user's desktop and function Blah is on the stack."

Why PowerShell?

Let me be clear: it took me approximately 4 years to "warm up" to PowerShell. I feel it has sharp edges, aspects that are just plain difficult, and plenty of bugs, both in design and implementation. Sometimes it really irritates me. However, the benefits of PowerShell are compelling, and have convinced me that it's the best thing to use for this project:

It is both a scripting environment and a CLI environment. The fact that it has to do both leads to some negative things like a steeper learning curve, but in the end it is extremely handy, because you want to be able to both do stuff quickly in a command-line REPL, as well as write full-featured, robust scripts.
It is very discoverable—things like Get-Command, tab completion, the ability to expose hierarchical data like a filesystem, the facilities for providing and synthesizing help, are very good.
Tab completion. I know I mentioned it in the previous bullet, but it's awesome enough to get its very own bullet.
The object pipeline: the object-oriented nature of the PowerShell pipeline is so much more powerful and easy to use than the bad old days of string-parsing-based scripting that it's not even funny. Imagine doing "dt" to "dump" an "object", and actually getting an object. DbgShell does that.
People know it: I estimate that the number of people who know PowerShell and/or C# is at least a handful of orders of magnitude larger than the people who know windbg scripting techniques. That means more people will be able to easily "pick up" a PowerShell-based debugger; and it also means that when people need help, the pool of potential helpers is much larger (for scripting-related issues, anyway).
PowerShell is still a general-purpose shell: when using DbgShell, you have access to not just debugger commands, but you can "cd" over to the filesystem, registry, AD, etc.; you can execute Send-MailMessage, Get-WmiObject, Invoke-WebRequest, Invoke-RestMethod, run arbitrary programs, etc.

Current Status

DbgShell has been in "prototyping mode" for a long time. I have spent a lot of time figuring how something could or should be done, but not necessarily "finishing" everything. There are a huge number of TODOs in the current code. So although it has started to become actually useful, the project is still pretty green. However, it can definitely demonstrate enough to give you a good taste of what it should be like.

Below are some screenshots. It's important to note that nothing you see is dbgeng text output. Although some stuff in the output will look familiar, that is only because I have used PowerShell's formatting and output features to customize how certain objects are displayed—all the output you see actually corresponds to real, full .NET objects. For instance, those ModLoad messages each correspond to a MS.Dbg.ModuleLoadedEventArgs object, which has more properties than what get displayed when sent to Out-Default. There is no string parsing of anything from dbgeng whatsoever. (Well... almost. I've made a few compromises where there is no other way to get information. For instance, disassembly stuff, or parsing the symbolic name of an adjustor thunk function to find the offset.)

This is a sort of "hello world" scenario: attaching to an instance of cmd.exe. I first use the PowerShell built-in command Start-Process, then pipe the output to the DbgShell command Connect-Process, and then poke around the namespace:

Here I have attached to a test program, and looked at the stack, switched to a particular stack frame, dumped locals, inspected the value of a local std::map, and inspected some type information for a local enum value. Note the display of the enumeration value: not only does DbgShell handle looking up the symbolic name for single enumerands, but also when multiple enumerands are OR'ed together. You can't tell this from the screenshot, but there is tab completion for all of this stuff.

Notable Features

Color: support for text colorization using ANSI escape codes (a la ISO/IEC 6429)
Custom formatting engine: Don't like .ps1xml stuff? Me neither. In addition to standard table, list, and custom views, you can define "single-line" views which are very handy for customizing symbol value displays.
Custom symbol value conversion: For most variables, the default conversion and display are good. But sometimes, you'd like the debugger to do a little more work for you. The symbol value conversion feature allows, for instance, STL collection objects to be transformed into .NET collection objects that are much easier to deal with.
Derived type detection: For when your variable is an IFoo, but the actual object is a FooImpl.
Rich type information: exposed for your programmatic pleasure.
Q: Does it work in WinDbg? I will only use WinDbg. A: Yes--load up the DbgShellExt.dll extension DLL, and then run "!dbgshell" to pop open a DbgShell console.

Current Deficiencies

The biggest deficiency currently is that it does not support kernel mode well (if you are already in the proper context, you can display values, but you can't change context from within DbgShell, and the namespace is not wired up).
Although you can load and execute traditional debugger extensions in the usual way, there are still many windbg commands missing.
Remotes are not supported: the dbgeng API supports connecting to a remote debugger. Unfortunately, the symbol and type information exposed by the dbgeng API is critically insufficient for DbgShell's needs, so DbgShell uses the dbghelp API. Unfortunately, there is no such thing as remote dbghelp. We will need to work with the debugger team to solve this problem.

License

Licensed under the MIT License.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

See Contributing for more information about contributing to the project.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct.

For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

dbgshell's People

Contributors

Stargazers

Watchers

Forkers

johnjohnsp1 ajkhoury powercode 0xbadca7 jasonroth opedroso subzer0ms mr64bit homjxi0e kurtdegreeff drclean louwangzhiyuy bygreencn techlord-rce a-min3 heruix mikkelbu marc-aldorasi-imprivata jamesshew de8gman1990 jesusninoc ith4cker cxmlg shangadi aiday0503 fanzcsoft anarquias idkwim computerline1z explife0011 ryan-johnson2 rajivraj zhentar tralivali1234 kumarankamalanathan thenson81 damirainullin docouto natelipus0x anhkggfork metathinker liyang-love kevinmiles hjbfa amaroq-clearwater surfndez thlineric bitl8-byteshort imulmul jimshew lazerhawk modulexcite tools-env kwangsikjeon mdisec peterg75 mohitt joaquin6 bhaskers-blu-org2 taffywrinkle claudiusgonzo m4rm0k ch3rn0byl ricky-dutton zha0 invokethreatguy c0de3 tracywhodoesnot renenyffenegger rrg92 crackercat 5l1v3r1 arvindshmicrosoft wisemoth rhongwei neutronar test-mass-forker-org-1 binsys lyingsimon mvenditto blueinthedream newbienewbie highbor marmikreal bhardwajs

dbgshell's Issues

Get tests running in appveyor build

Formatting: Format-Table et al need to handle -Property

Repro Steps: (abstract)

Gather some objects which have view definitions registered with the alternate formatting engine.
Attempt to Format-Blah them, explicitly requesting certain properties ("$stuff | ft Name, DumpPath")

Expected results:
Something sensible. Ideally we can just generate an alt-formatting view definition on the fly and use that.

Actual results:
The already-registered view definition is used, ignoring the request for certain properties.

Concrete example:

$stuff = dir C:\temp\dumps\ -File | ForEach-DbgDumpFile {
   $thing = lm ntdll
   $thing | Format-List | Out-Null
   Write-Output $thing
}

# BUG: We get the default module table view here, not Name and DumpPath
$stuff | ft Name, DumpPath

Workarounds:
To get the desired properties with the alternate formatting engine, use the alternate formatting cmdlets directly (like "$stuff | fat Name, DumpPath").

To use the standard formatting engine, use the module-qualified name of the formatting cmdlets (like "$stuff | Microsoft.PowerShell.Utility\Format-Table Name, DumpPath").

Use ClrMd to synthesize a DbgModuleInfo for managed modules

Trying to access an element of a `Span<byte>` throws a VerificationException

Something is amiss with how PowerShell generates code to access a Span. (DbgShell currently uses Windows PowerShell 5.1 (latest Win10/.NET circa 1/1/2019).)

Repro steps:

ntna
$mem = dd $ip
$mem.GetMemory().Span
$mem.GetMemory().Span[ 0 ]

Expected result:
The last command should give you the first byte in the memory block.

Actual result:

Operation could destabilize the runtime.
At line:4 char:1
+ $mem.GetMemory().Span[ 0 ]
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], VerificationException
    + FullyQualifiedErrorId : System.Security.VerificationException

When searching for more info about this sort of exception, I found reports that indicate it is the kind of thing that can happen when the C# compiler accidentally generates unverifiable IL. In this case, it's not the C# compiler, of course; it's the PowerShell script interpreter/compiler or maybe the Dynamic stuff it uses.

Expanded error info:

AddedByShowError      : True
PSMessageDetails      :
Exception             : System.Security.VerificationException: Operation could destabilize the runtime.
                           at CallSite.Target(Closure , CallSite , Object , Int32 )
                           at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1
                        arg1)
                           at System.Management.Automation.Interpreter.DynamicInstruction`3.Run(InterpretedFrame frame)
                           at
                        System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame
                        frame)
TargetObject          :
CategoryInfo          : OperationStopped: (:) [], VerificationException
FullyQualifiedErrorId : System.Security.VerificationException
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 4
PipelineIterationInfo : {}

MyCommand             :
BoundParameters       : {}
UnboundArguments      : {}
ScriptLineNumber      : 4
OffsetInLine          : 1
HistoryId             : -1
ScriptName            :
Line                  : $mem.GetMemory().Span[ 0 ]

PositionMessage       : At line:4 char:1
                        + $mem.GetMemory().Span[ 0 ]
                        + ~~~~~~~~~~~~~~~~~~~~~~~~~~
PSScriptRoot          :
PSCommandPath         :
InvocationName        :
PipelineLength        : 0
PipelinePosition      : 0
ExpectingInput        : False
CommandOrigin         : Internal
DisplayScriptPosition :

00000000000000000000000000000000000000000000000000000000000000000000000000000000
ExceptionType : System.Security.VerificationException
Message        : Operation could destabilize the runtime.
Data           : {System.Management.Automation.Interpreter.InterpretedFrameInfo}
InnerException :
TargetSite     : System.Object CallSite.Target(System.Runtime.CompilerServices.Closure,
                 System.Runtime.CompilerServices.CallSite, System.Object, Int32)
StackTrace     :    at CallSite.Target(Closure , CallSite , Object , Int32 )
                    at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
                    at System.Management.Automation.Interpreter.DynamicInstruction`3.Run(InterpretedFrame frame)
                    at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame
                 frame)
HelpLink       :
Source         : Anonymously Hosted DynamicMethods Assembly
HResult        : -2146233075

Unhandled Exception: System.Exception: Could not load dbgeng.dll on Win7.

When running the latest binary release, on Windows 7 Pro SP1, both x86 and x64 version, I get the following error:

$ ./DbgShell.exe

Unhandled Exception: System.Exception: Could not load dbgeng.dll.
at MS.DbgShell.MainClass._NormalModeMainWrapper(String[] args) in C:\src\DbgShell\DbgShell\MainClass.cs:line 467
at MS.DbgShell.MainClass.Main(String[] args) in C:\src\DbgShell\DbgShell\MainClass.cs:line 130

So I look at the incriminating code:

              string pathToDbgEng = Path.Combine( rootDir, "Debugger", "dbgeng.dll" );

            IntPtr hDbgEng = NativeMethods.LoadLibraryEx( pathToDbgEng,
                                                          IntPtr.Zero,
                                                          LoadLibraryExFlags.LOAD_WITH_ALTERED_SEARCH_PATH );
            if( IntPtr.Zero == hDbgEng )
                throw new Exception( "Could not load dbgeng.dll." );

The file is indeed in the Debugger folder, yet the error persists.
I have tried to replace the file with the one from my WinDbg installation, but to no avail.

Is there something that I could have missed? Thanks.

Q: Is Shell the only option? What about a module?

The approach with DbgShell.exe is clear. But the console UX is somewhat poor. There are other hosts, VSCode + PowerShell, to start with, and some more, e.g. my own Far Manager + PowerShellFar (I am interested in using it specifically). Other hosts provide some powerful tools, UI, etc. For example, Tab completion in DbgShell is better than nothing but drop down Intellisense menus in other hosts are much better. Or, say, other hosts allow debugging of PowerShell scripts designed for debugging of something else. Etc., etc.

In other words, is DbgShell.exe the only option or the normal PowerShell module package possible, too?

Need to update ClrMd

The current snapshot of ClrMd code is badly out-of-date. Updating it will be a bit of a pain for two reasons:

There were a few small customizations made to the ClrMd code.
The current version of ClrMd code was taken from what seems to be a now-defunct branch (ClrObject).

AddressTransformation integer handling does the opposite of what it is supposed to

I had figured that the inconsistent handling between addresses with only decimal digits was just an unfortunate cost of PowerShell, until I saw the AddressTransformation... so I poked at it a bit more:

> Read-DbgMemory -Address 0x17409680
17409680  "󸇳."

> 0x17409680 | Read-DbgMemory
Read-DbgMemory : Could not access memory: 90108800
At line:1 char:14
+ 0x17409680 | Read-DbgMemory
+              ~~~~~~~~~~~~~~
    + CategoryInfo          : ReadError: (15301904384:UInt64) [Read-DbgMemory], DbgMemoryAccessException
    + FullyQualifiedErrorId : MemoryAccessFailure,MS.Dbg.Commands.ReadDbgMemoryCommand

> Read-DbgMemory -Address 17409680
Read-DbgMemory : Could not access memory: 0109a690
At line:1 char:1
+ Read-DbgMemory -Address 17409680
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ReadError: (17409680:UInt64) [Read-DbgMemory], DbgMemoryAccessException
    + FullyQualifiedErrorId : MemoryAccessFailure,MS.Dbg.Commands.ReadDbgMemoryCommand

Read-DbgDisassembly doesn't resolve Address from Piped Get-DbgSymbol

Get-DbgSymbol "MyClass::MyFunction" | Read-DbgDisassembly returns the error "Read-DbgDisassembly : Cannot process argument transformation on parameter 'Address'. Could not convert 'MS.Dbg.DbgPublicSymbol' to an address.". So I tried sticking a Select Address in there, but that didn't work either.

%{ $_.Address.ToString("X8") } got the job done but doesn't seem like the intended workflow.

captured stdout output is garbled

Running git --version directly returns correct result, but if I assign it to a variable then it will be garbled: $x = git --version; $x.

Missing files in DbgProvider project causing build errors

2>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(4406,5): error MSB3030: Could not copy the file "E:\Source\Repos\DbgShell\DbgProvider\Debugger.Converters.clr.ps1" because it was not found.
2>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(4406,5): error MSB3030: Could not copy the file "E:\Source\Repos\DbgShell\DbgProvider\Debugger.Converters.Cui.ps1" because it was not found.
2>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(4406,5): error MSB3030: Could not copy the file "E:\Source\Repos\DbgShell\DbgProvider\Debugger.Converters.dfsr.ps1" because it was not found.
2>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(4406,5): error MSB3030: Could not copy the file "E:\Source\Repos\DbgShell\DbgProvider\Debugger.Converters.tlx.ps1" because it was not found.
2>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(4406,5): error MSB3030: Could not copy the file "E:\Source\Repos\DbgShell\DbgProvider\Debugger.Converters.utl.ps1" because it was not found.
2>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(4406,5): error MSB3030: Could not copy the file "E:\Source\Repos\DbgShell\DbgProvider\Debugger.DebuggeeTypes.cui.psfmt" because it was not found.
2>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(4406,5): error MSB3030: Could not copy the file "E:\Source\Repos\DbgShell\DbgProvider\Debugger.DebuggeeTypes.dfsr.psfmt" because it was not found.

Should the files be added, or the project updated?

Skinny Null Terminated String Functions Don't Allow Partial Reads

The wide string functions work how I would expect, however the ascii/UTF8 string read functions simply throw exceptions with no partial results if the string is longer than the requested bytes.

Figure out how to get CI builds, host a binary release

Support kernel mode

The basic underpinnings for this are already in—for instance, the DbgEngContext field representing the process has been updated to be able hold a process ID or a EPROCESS address (ProcessIndexOrAddress). But there is a lot more to be done, like getting the namespace provider wired up.

Docs: need cookbook

Recipes:

Break on a function only when "foo" is on the callstack or argument is a particular value.
... add more to this list ...

Integrate capstone (or something) as the disassembler

https://github.com/aquynh/capstone: "Capstone is a disassembly framework with the target of becoming the ultimate disasm engine for binary analysis and reversing in the security community."

Rename Connect-Process to Attach-Process

Attach is more recognizable verb, I guess.

Get-ClrStack rename OsTid parameter to Tid

Class MS.Dbg.DbgUModeThreadInfo has a property uint32 Tid {get;}
Cmdlet Get-ClrStack has a parameter [-OsTid]

It would be great to have both names equal. It's better from naming conventions point of view and gives a better chance to use them in pipeline with bind by property name.

Add Output Decolorizer/No color option

I'm searching for... something in a haystack. I don't even know what exactly the something is yet. I've got a memory dump, and a million candidate addresses in a file; I want to read the value for each from the dump and analyze the results.

Get-Content Candidates.log | Read-DbgMemory -LengthInBytes 4 | Out-String -Stream | Out-File Values.txt gets me almost all of the way there - except that the output is crammed full of formatting control characters. An option to print things without formatting, or strip the control strings out, would be very helpful for this.

Memory Search

I've forked DbgShell and started putting together a basic memory search command (which hopefully I will be able to polish into a reasonable pull request before the ADHD decides otherwise for me). I wanted to share some thoughts and get some input.

I've forgone DbgEng's search granularity option so that I could implement search alignment independent of the search size, to allow things like searching for pointers to nearby addresses:

> Search-DbgMemory 04244c -SearchValueLengthInBytes 3 -SearchResultAlignment 4 -FromAddress 1 | % { $_.Address - 1 } | Read-DbgMemory -LengthInBytes 16
719b1c40  04244c8b 060441f7 b8000000 00000001
719d8df8  04244c13 042444dd c310c483 fe4356e9
71a1e99c  04244c8b 04c231d9 244c8b00 c221d904

On the one hand, awesome!, you're not going to be doing that in WinDbg... on the other hand, that's not a very straightforward approach and byte granularity means you're search for pointers in a 256 byte region or 64kB region, no in between... any thoughts on a better way to do it?

I'm taking the search value as a ulong, which means it caps out at 8 bytes... supporting strings seems easy enough, but I've no idea how to tackle 9+ byte non-string patterns. Are there any existing commands I can crib from?
My ultimate goal is to do something like this:

[Heap 007f0000 segment 52800000 (msvcrt!_crtheap)]
52808dec  04244c8b e808508d fff61480 cc0004c2
[<unknown>]
658d0160  04244c8d 04244489 8b4ceca1 24448965
[srvcli; "C:\Windows\System32\srvcli.dll"]
719b1c40  04244c8b 060441f7 b8000000 00000001
719d8df8  04244c13 042444dd c310c483 fe4356e9
71a1e99c  04244c8b 04c231d9 244c8b00 c221d904

Is there a way I can do grouping without accumulating?

Docs: need more cmdlet help content

A long time ago I started writing some manual MAML (in DbgProvider.dll-Help.xml), but manual MAML is not a lot of fun.

It seems like PlatyPS is the way to go here.

Note that for "about" topics, I don't think PlatyPS will do what I want--it expects .md input in a very particular format, and for "about" topics I'd like to be a little more free-form. But for cmdlet topics, PlatyPS might be just the thing.

Add new parameter set to Get-ClrStack

It would be great if Get-ClrStack will be able to accept instances of MS.Dbg.DbgThreadContainer as an input so it will be able to use in the pipeline.

PS Dbg:\NetDebugDemos_16124\Threads
> Get-ChildItem | Get-ClrStack

Need to document requirements

It's possible it's mentioned in one of the about_ help files, but it should be on the main readme.md as well.

PEB not included in process level namespace

I wanted to see the PEB for my process. Since the thread namespace includes a Teb, I figured there'd be a matching Peb in the process namespace, but alas, there is not. (I did eventually figure out that there is a $peb variable, at least)

Make enum for psuedo registers

Save my conscience from the magic string!
DbgPseudoRegisterInfo.GetDbgPsedoRegisterInfo(Debugger, "$peb").Value

Enhancement: dps but with recursive safe pointer following

Regular dps:

> dps 26ee7c30
26ee7c30  2368c010
26ee7c34  13234900
26ee7c38  5f080100 d2d1!CSRGBToLinearTranslator::s_shaders+400f0
26ee7c3c  26db1aa8
26ee7c40  26ee74a8
26ee7c44  26e80020

Awesome recursive dps:

26ee7c30  2368c010  -> 50afffa8 jscript9!Js::JavascriptLibrary::`vftable'
26ee7c34  13234900  -> 50b35650 jscript9!Js::GlobalObject::`vftable'
26ee7c38  5f080100 d2d1!CSRGBToLinearTranslator::s_shaders+400f0
26ee7c3c  26db1aa8  -> 50afa570 jscript9!ScriptEngine::`vftable'
26ee7c40  26ee74a8  -> 21dd7da0  -> 50afffa8 jscript9!Js::JavascriptLibrary::`vftable'
26ee7c44  26e80020  -> 26a0cda0  -> 50afffa8 jscript9!Js::JavascriptLibrary::`vftable'

Ah! It is so much more clear what I am looking at now! And with colorization it's even better!

Can't pipeline directory contents to Mount-DbgDumpFile

> Get-ChildItem C:\dumps -File | Mount-DbgDumpFile
Mount-DbgDumpFile : Cannot find path 'Dbg:\example.dmp' because it does not exist.
At line:1 char:32
+ Get-ChildItem C:\dumps -File | Mount-DbgDumpFile
+                                ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Dbg:\example.dmp:String) [Mount-DbgDumpFile], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,MS.Dbg.Commands.MountDbgDumpFileCommand

This would be to run an analysis script on all of the dump files in a folder.

Also helpful for that would be a way to automatically dismount dump files (or even actually dismount them at all) so you don't get target name conflicts if your script errors out and you have to restart it

Launch is slow

It took 5-6 seconds for every launch on my desktop. Would be appreciated if this could be improved.

AltPropertyColumn Displays null as $null rather than blank

My object has an int? property that is null when it doesn't apply/can't be calculated. The standard Format-Table doesn't print anything when it's null. The alternate table formatting displays it as "$null" which has the unfortunate effect of making the least important information the most visible, so I have to do a ScriptColumn with a null check to handle it.

Need some rails to prevent dbgeng-unsupported scenarios like mixed dump + live proc

DbgEng.dll has a concept of "context": what is the current "system", the current process within that system, the current thread within that process, and the current frame within that thread. (See the DbgEngContext class.) DbgShell exposes this via its namespace. It supports switching between contexts, such as between threads, or between multiple processes in a live debugging scenario.

However, some combinations of context just don't work together. For instance, if you are attached to a dump, and then try to launch a new, live process under the debugger, dbgeng context gets completely discombobulated.

DbgShell needs to add some guard rails so that you don't end up in situations that don't work.

Get-Help -ShowWindow gives an error

PS Dbg:\
> get-help Connect-Process -ShowWindow
get-help : Exception has been thrown by the target of an invocation.
At line:1 char:1
+ get-help Connect-Process -ShowWindow
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-Help], TargetInvocationException
    + FullyQualifiedErrorId : System.Reflection.TargetInvocationException,Microsoft.PowerShell.Commands.GetHelpCommand

PS Dbg:\
> get-help Get-ClrStack -ShowWindow
get-help : Exception has been thrown by the target of an invocation.
At line:1 char:1
+ get-help Get-ClrStack -ShowWindow
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-Help], TargetInvocationException
    + FullyQualifiedErrorId : System.Reflection.TargetInvocationException,Microsoft.PowerShell.Commands.GetHelpCommand

EnumerateLIST_ENTRY doesn't work

EnumerateLIST_ENTRY has two main troubles:

It yields PSObjects, not DbgValues.
It throws a NotImplementedException

The NotImplementedException can be resolved easily enough by skipping the dynamic entirely:

        var val = Debugger.GetValueForAddressAndType( curItemAddr, entryType, itemNamePrefix + idx.ToString(), false, false );
        yield return val;

        curListEntry = val.Properties[listEntryMemberPath].Value;
        curListEntry = curListEntry.Flink;

Additionally, the head = head.WrappingPSObject; at the start is a bit inconvenient, since it means one has to pass in the PSObject.BaseObject instead of the PSObject directly, and the is DbgPointerValue & is DbgValueError checks don't work, since the values are always PSObject.

DbgHelp is slow and other symbol "fun"

The purpose of this issue is to share some learnings (and some salt about DbgHelp) that I have gained in the past few days furthering ~~my descent into madness~~ my understanding of symbols; whether any of this should be considered an issue that prompts changes to DbgShell or an endorsement of any particular approach I leave to the readers' judgement.
SymbolSearchPerf.cs.txt

SymSearch results are a superset of SymEnumSymbols - there is no symbol that SymEnumSymbols will return that SymSearch can not also return.

"But wait!" you say, "I have done searches that get results from SymEnumSymbols but not SymSearch!" And indeed, you are correct. So I shall share the sad tale of a simple, unassuming symbol, known to its friends as mshtml!g_hIsolatedHeap.

> Get-DbgSymbol "mshtml!g_hIsolatedHeap" -UseSymEnum
      0x642b1ba0 g_hIsolatedHeap

As expected. And what about SymSearch?

> Get-DbgSymbol "mshtml!g_hIsolatedHeap" -UseSymSearch

Nothing. Have I lied to you? No! I have not! It is DbgHelp that lies! This symbol's proper name is not mshtml!g_hIsolatedHeap at all, but in truth is mshtml!_g_hIsolatedHeap!

> Get-DbgSymbol "mshtml!_g_hIsolatedHeap" -UseSymSearch
      0x642b1ba0 g_hIsolatedHeap

SymEnumSymbols succceeds where SymSearch does not because DbgHelp secretly inserts search globs to cover for the lies it tells! But at what cost, you ask? "No, I didn-" Yes, you did. But unfortunately, there are so many different ways that DbgHelp is heckin' slow, you can't unambiguously pin it to one single thing† - but one can at least quantify how slow the various ways of calling into it to ask about our friend mshtml!_g_hIsolatedHeap are!

Method	Mean	Error	Ratio
FindSymbol_Enum	1,751,811.693 us	77,915.3682 us	1.000
FindSymbol_Enum_ExplicitStar	1,225,950.550 us	22,614.1927 us	0.691
FindSymbol_Enum_NoType	1,167,999.680 us	17,032.5053 us	0.668
FindSymbol_Enum_Mangled_SymOpts	705,314.202 us	34,711.8878 us	0.421
FindSymbol_Enum_PublicsOnly_Mangled_SymOpts	670,834.033 us	16,164.5092 us	0.384
FindSymbol_Search_ForcedStarGlob	3,409,318.860 us	37,855.2651 us	1.949
FindSymbol_Search_SecretUncodumentedWildcard	3,017,108.117 us	156,676.2463 us	1.736
FindSymbol_Search_Mangled_SymOpts	368,730.864 us	6,861.2060 us	0.210

(spoiler alert: Benchmarkdotnet picked microseconds for this table because some of the rows I cut out actually need it)

There are few things you can tell from that table (should the names happen to make sense to you, or should you reference against the attached code). First is that with the default settings DbgHelp does try to match against your original search terms before searching again with the injected star glob. Next you might notice that SymSearch manages to take twice as long (unless you set symopts to search against still-"decorated" streams, a trival concession for our symbol with no decoration to be had). I'm not really sure how it does that. One might also notice that there are undocumented wildcards (that one might want to, you know, escape). In this case, I used #, which seems to mean "match the preceding character 0 or 1 times"‡, but there's also some handling for a [a-z] style syntax.

Now, if one were to examine DbgHelp more closely††, one might notice that the public apis seem to pretty much just be messy wrappers aound DIA functions. And there is an undocumented SymGetDiaSession, what might one get from that? Crashes, unless one also notices the SymFreeDiaString and surmises that DbgHelp's DIA is special and doesn't use SysAllocString to allocate "BSTR" values.

Method	Mean	Error	Ratio
findChildren_Mangled_DiaComInterop	2,816.546 us	45.3609 us	0.002
findChildren_Mangled_DiaCppCli	111.870 us	0.5413 us	0.000
findChildren_Unmangled_DiaCppCli	338,699.820 us	6,055.1215 us	0.194
findChildren_Mangled_DiaCppCli_BuiltinSearchGlob	146,313.182 us	7,739.2609 us	0.084

Wow, using .NET COM interop with an exact search string gets us an answer 100 times faster (as long as we don't look at any string properties...). C++/CLI our way around the string issue and we get a whopping 3,000 times faster. Even undecoration & search globs can't get us as slow as the DbgHelp APIs go. But wait! There's more... most of those 112 microsecond were just fetching the DiaSession!

Method	Mean	Error	Ratio
findChildren_Mangled_DiaCppCli_CachedSession	4.629 us	0.0850 us	0.000

But why stop there? If you profile it, it's painfully obvious that DbgHelp's DIA is hamstrung by some inefficient UTF8 -> UTF16 conversion logic... and the ~~disassembly~~ tarot cards will tell you that the version of DIA in msdia140 somewhat improves on that conversion, in addition to a statically linked towlower... so how would that fare?

Method	Mean	Error	Ratio
findChildren_Mangled_msdia140CppCli	3.573 us	0.1180 us	0.000
findChildren_Unmangled_msdia140CppCli	361,605.138 us	7,670.6590 us	0.206
findChildren_Mangled_msdia140CppCli_BuiltinSearchGlob	140,708.329 us	5,434.0552 us	0.080

Well, shoot. Those numbers are rather remarkably close to DbgHelp's... I profiled them to make sure I wasn't accidentally double testing. I'm starting to suspect there was a reason the DbgHelp DIA used LocalAlloc instead of SysAllocString... _{msdia140 will use LocalAlloc if you use a secret alternate GUID to get your IDiaDataSource but I think I should post this and go to sleep before I get lured into testing that}

( SymbolSearchPerf.cs.txt contains the benchmark code, which should compile & run against the head of my personal DbgShell repository)

† aside from the dynamically linked towlower call in DbgHelp!nextToken

‡ except in mshtml!_#_hIsolatedHeap where it means "get stuck in an infinite loop"

†† Not by diassembling it, of course, that would be against the terms of use!

Integrate with C#?

HI, I need to integrate with c# / Any known package? or this library can be integrated with c#