Analyze -v

There are two options added to the !process command that haven’t made it into the documentation yet.

The first option is the /h switch, which appears to allow for searching the process list for a process with too many open handles:

0: kd> !process /h 0n10000
Searching processes with HandleCount > 10000

The only problem with this is that it appears to stop on the first process that is above that limit. This prevents it from being a generically useful command as you’re probably looking for any processes with large handle counts. For that purpose, I’ve whipped up this WinDBG script that will search for any process with open handles above a specified amount:

.if (${/d:$arg2} != 1)
{
    .echo Performing scan for processes with handle count > ${$arg1};
    !for_each_process "$$>a<${$arg0} ${$arg1} 1";
}

.else
{
    r? @$t0 = ((nt!_eprocess *)${@#Process});
    aS handleCount "@@(@$t0->ObjectTable->HandleCount)";
    .block
    {
        .if (${handleCount} > ${$arg1})
        {
           .printf /on "Process %p has %d handles\n", @$t0, ${handleCount};
        }
    }
    ad handleCount;
}

Save that to a file called bighandle.wbs and execute the following command in WinDBG:

0: kd> $$>a<c:\\dumps\\bighandle.wbs 0n500
Performing scan for processes with handle count > 0n500
Process 848bfd40 has 5568 handles
Process 862e9530 has 579 handles
Process 86374d40 has 997 handles
Process 86334150 has 661 handles
Process 849eb4a0 has 973 handles 

Note that the double backslashes are important on the command line and it won’t work properly without them. This script could be enhanced to just grab the process with the greatest number of handles, if you want to give that a whirl and submit it pass it along to me (snoone at analyze-v dot com).

The other interesting undocumented feature of !process is bit 5 in the flags (0×20). This causes !process to include information about the process PEB in the output. This also requires that bit 4 (0×10) be present due to the fact that the PEB is process context specific. So, for example:

0: kd> !process 849eb4a0 31

(Remaining output left as an exercise to the reader.)

Every now and then I need to have the debugger break in when a user mode module loads. For example, I may want to set a process specific breakpoint but the process in question isn’t loaded yet. Normally this would be a giant pain, but with a few quick debugger commands it’s all too easy.

First thing we need to do is get the debugger notified of user module loads. Normally the OS doesn’t bother letting the kernel debugger know when a user module gets loaded, but this can be enabled via GFlags option Enable loading of kernel debugger symbols (a misnomer in this usage). Instead of running the GFlags application on the target machine, we can enable it dynamically in the debugger with the !gflags command:

0: kd> !gflag +ksl
Current NtGlobalFlag contents: 0x00040000
    ksl - Enable loading of kernel debugger symbols

Now that we have that done, we can use the sxe ld command to enable a breakpoint when the module of interest loads. Note that this could be a DLL, EXE, service, driver, whatever:

0: kd> sxe ld notepad.exe

Now I’ll run Notepad on the target machine and hopefully get a breakpoint (which indeed I did):

0: kd> g
nt!DbgLoadUserImageSymbols+0x30:
8281a182 int     3
0: kd> kc

nt!DbgLoadUserImageSymbols
nt!MiLoadUserSymbols
nt!MiMapViewOfImageSection
nt!MiMapViewOfSection
nt!MmInitializeProcessAddressSpace
nt!PspAllocateProcess
nt!NtCreateUserProcess
nt!KiFastCallEntry

You’ll note that we’re actually in the correct process context at this point:

0: kd> !process -1 0
PROCESS 84aaed40  SessionId: 1  Cid: 0000    Peb: 00000000  ParentCid: 0948
    DirBase: 7efc7600  ObjectTable: 961c2460  HandleCount:   0.
    Image: notepad.exe

So I can set a process specific breakpoint on a hot OS routine and start my analysis in the process that I’m interested in making the call that I’m interested in:

0: kd> bp /p @$proc ntfs!ntfscommoncreate
0: kd> g
Breakpoint 0 hit
Ntfs!NtfsCommonCreate:
889095fd push    94h
1: kd> kc

Ntfs!NtfsCommonCreate
Ntfs!NtfsCommonCreateCallout
nt!KiSwapKernelStackAndExit
nt!KiSwitchKernelStackAndCallout
nt!KeExpandKernelStackAndCalloutEx
Ntfs!NtfsCommonCreateOnNewStack
Ntfs!NtfsFsdCreate
nt!IofCallDriver
fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted
fltmgr!FltpCreate
nt!IofCallDriver
nt!IopParseDevice
nt!ObpLookupObjectName
nt!ObOpenObjectByName
nt!IopCreateFile
nt!NtOpenFile
nt!PfSnGetPrefetchInstructions
nt!PfSnBeginAppLaunch
nt!PfProcessCreateNotification
nt!PspUserThreadStartup
nt!KiThreadStartup

For whatever reason, my comments on the NT Debugging blog never get approved, so I figured I’d clarify something about a recent post and subsequent comment here.

If you’re ever debugging a similar issue yourself, the !chkimg extension can automate all the steps taken in the blog post by leveraging a properly configured symbol server. As it turns out, in addition to indexing symbol files on a symbol server you can also index image files. This is a handy feature that’s necessary when debugging mini-dump files, due to the fact that the original image files are required when debugging dumps of that type. !chkimg is aware of this, and will attempt to file the original image from your symbol server and download it to your local symbol store.

In order to facilitate debugging kernel mini-dumps, Microsoft indexes the O/S images on the MS symbol server. This means that if you want to verify any O/S image in your target machine, all you need to do is point your symbol search path to the MS symbol server and run !chkimg.

0: kd> !chkimg ntfs
SYMSRV:  Ntfs.sys from http://msdl.microsoft.com/download/symbols
DBGHELP: c:\websymbols\Ntfs.sys\4A5BBF4512f000\Ntfs.sys - OK
0 errors : ntfs

I often get dump files from customers zipped up with the appropriate PDBs for the images in the dump. I then have to extract the dump file to a folder, double click on the DMP file, and then add the path to the unzipped dump to my symbol search path so that I grab the appropriate PDBs (I wish the debugger engine would put the dump directory in the path automatically, but that’s another issue). That can lead to a lot of copy/pasting or typing, but luckily there’s a built in alias to the rescue: $CrashDumpPath.

Aliases are quite a nifty feature of WinDBG and I’m working on a post that will do them justice. In the meantime, just know that $CrashDumpPath is a built in alias that always expands out to the path of the crash dump file in any expression. For example, to quickly add the path of the crash dump file to my symbol search path, I can just call .sympath+ with the dump path alias:

0: kd> .sympath
Symbol search path is: <empty>
Expanded Symbol search path is: <empty>
0: kd> .sympath+ $CurrentDumpPath
Symbol search path is: C:\dumps\mpwhang
Expanded Symbol search path is: c:\dumps\mpwhang

This takes the variability out this step for me and also allows for easier scripting. Other built in aliases can be found on this page under the heading Automatic Aliases.

I’ve talked previously about thread specific breakpoints, which allow you to set a breakpoint that will only fire for a specific thread. Equally useful are process specific breakpoints, which will only fire for any thread within a given process.

To set a process specific breakpoint, you specify the /p switch to the bp command and supply a process object address:

bp /p 84996030 ntfs!ntfscommoncreate

The process address could, for example, be retrieved from the output of the !process 0 0 command or you can use the handy $proc pseudo register to specify the current process:

bp /p @$proc ntfs!ntfscommoncreate

Surprisingly, after almost a decade of using WinDBG I have never had to use a negative decimal value in a WinDBG expression. Until recently, that is…Because the MASM syntax was a bit trickier than expected, I thought I’d record it here for posterity.

When using the MASM evaluator (which is the default in WinDBG), hex values are the default and decimal numbers are indicated by using the 0n override.  Thus, for example, if I wanted to evaluate 123 in an expression I could use 0n123:

3: kd> ?0n123
Evaluate expression: 123 = 00000000`0000007b

However, if I want to use -123 there’s a bit of a catch. My natural inclination was to put the minus sign to the right of the 0n and the left of the 123, such as this: 0n-123. However, this yields an unexpected result:

3: kd> ?0n-123
Evaluate expression: -291 = ffffffff`fffffedd

That’s actually -0×123, not -123. To make matters even worse, the latest debugger even shows this syntax when displaying negative decimal values:

3: kd> dt nt!_mdl fffffa80077f1df0
   +0×000 Next             : (null)
   +0×008 Size             : 0n56
   +0x00a MdlFlags         : 0n-32701
   +0×010 Process          : 0xfffffa80`069dab30 _EPROCESS
   +0×018 MappedSystemVa   : 0xfffffa80`06797294 Void
   +0×020 StartVa          : 0×00000000`0404f000 Void
   +0×028 ByteCount        : 0x1c
   +0x02c ByteOffset       : 0x79c

However, the correct syntax is to put the minus sign to the left of the 0n:

3: kd> ?-0n123
Evaluate expression: -123 = ffffffff`ffffff85

For the C++ evaluator, things are much easier because the default radix is decimal. Thus, all you need to do is specify -123:

3: kd> ? @@c++(-123)
Evaluate expression: -123 = ffffffff`ffffff85

The trick now comes when you want to specify a negative hex number, which also requires the minus sign to be on the left of the modifier:

3: kd> ? @@c++(0x-123)
Evaluate expression: -123 = ffffffff`ffffff85
3: kd> ? @@c++(-0×123)
Evaluate expression: -291 = ffffffff`fffffedd

Been a bit of an unexpected  break for a while now, but hopefully back to regular posting…

You might never need to know this, but WinDBG will actually cache data read from the target. For example, this means that if you dd a memory location multiple times only the first dump of the memory will actually be read from target. Obviously this cache is invalidated when the target is resumed, so most of us won’t have an issue with this caching. However, if the memory you’re dumping is something like mapped device memory then this is an issue as the cache could be stale.

Enter the .cache command, which controls the size and state of the local cache (amongst other things, we’ve seen .cache before). Turning off the cache is as easy as executing .cache 0, which sets the size of the local cache to zero. This causes all of your reads to hit the target and ensure that you’re seeing the latest data. There are also the flushall, flushu, and flush parameters, which allow for flushing all or some of the cache from the local machine.