Archive for June, 2009

« Older Entries

My favorite bit in the system

Tuesday, June 30th, 2009

I don’t announce this in mixed company, but I do have a favorite bit in the system. My favorite bit by far is the DO_POWER_PAGABLE (dū pou‘ər păg‘a·ble) bit, which indicates whether or not your driver is pageable in the power path. It has nothing to do if whether or not your driver happens to be pagable.

Not only is DO_POWER_PAGABLE fun to say, it’s also fun to watch it grow as a spelling virus throughout the sample code, documentation, and news groups. For example, from the version of the CHM docs I have on my system at the moment:

pagablemem

Posted in Driver development, Non-technical | No Comments »

Where is my memory going?

Tuesday, June 30th, 2009

Lots of times you have a memory dump and need to get some information about the virtual and physical memory state. How much pool is in use? How much physical memory is in use? What about the system working set?

Turns out that there’s no one command that will tell you absolutely everything, you’ll need to bring together the output of several commands in order to get a clear picture of what’s going on.

!vm

Whenever I suspect I’m dealing with a system that is low on virtual memory, my go to command is !vm. This gives you an overview of the current page file usage, available physical pages, free system PTEs, the pools, process memory usage, etc:

vm

What’s also nice is that !vm knows about the failed allocation counters in the system, so, for example, if there have been any failed pool allocations !vm will let you know.

!sysptes

System PTEs are a popular resource in the system. They’re used for all kinds of things, a common one being kernel stacks. Also, a common question I get is, “why is MmGetSystemAddressForMdlSafe failing?” The most common answer is a lack of system PTEs, which we see in the !vm output but we can get a better picture of with !sysptes:

sysptes

We can see here the largest free block, which is going to indicate the largest free contiguous system PTE range there is in the system. This lets us know what the largest MDL we could current map with MmGetSystemAddressForMdlSafe is.

!memusage

!vm also gave us some info about physical memory in the system, but we can get a finer grained view with another command.

In Windows, the physical address space is broken up into PAGE_SIZE chunks. Each page is referenced by an index, which is called a page frame number (PFN). So, f you have a PFN, you can find the physical address by multiplying the PFN by PAGE_SIZE (4K on x86/x64, 8K on IA64).

All of the PFNs in the system are collectively referred to as the page frame database (PFD). Each page in the page frame database has a particular state associated with it. Aside from two of the states (active and transition), pages in the same state are linked together in a linked list. We can see the state of the pages in the PFD via the !memusage 8 command (!memusage with other flags provides even more information):

memusage

nt!MmSystemCacheWs

Finally, we can investigate the system working set, which is the memory in use from the file system cache, paged pool, and pageable driver/kernel sections. This one is a bit trickier to track down as it is a kernel data structure that we must explore with the dt command. First, we’ll find the address of nt!MmSystemCacheWs:

mmscws

Note the lack of type information here, which is standard for public PDBs such as the ones shipped for Microsoft components. Bummer, but some Googling lets us know that the data type of this structure is MMSUPPORT. Thus, we can inspect the structure and its fields:

mmsupport

The WorkingSetSize field is the interesting one for our discussion. This indicates the number of pages taken up by the system working set. On my x86 system this is (0x68fb * 4K), which is about 110MB.

Posted in WinDBG, Windows internals | 2 Comments »

Win7 may not save your crash dump

Monday, June 29th, 2009

This is a pretty big shocker to me. Based on your configuration, Win7 may not save a crash dump even though you’ve configured it to do so:

http://blogs.msdn.com/wer/archive/2009/02/09/kernel-dump-storage-and-clean-up-behavior-in-windows-7.aspx

Definitely going to have to keep that in mind when asking for crash dumps from the field. And the argument about drive space seems pretty specious, especially since I just picked up a 2.5″ 500GB external USB drive for $100.

Posted in Windows internals | No Comments »

Windows supports hot patching…

Sunday, June 28th, 2009

So why do I have to reboot when I get an IE update?

Posted in Non-technical | No Comments »

Need better screen capture software

Wednesday, June 24th, 2009

Just saw the blog from the first time with IE 7 and the screen grabs look pretty bad. I thought it would be easier than annotating listings from WinDBG, but maybe that decision needs to be revisited.

Posted in Non-technical | 2 Comments »

Piping command output with .foreach

Wednesday, June 24th, 2009

.foreach is a pretty nifty command, it allows you to take the output of one command and then feed it to another. For example, say I have a block of random data and that I think contains a pointer to an IRP:

98d7ee60

Using .foreach I can dump out the data with the dp command and run !irp on each pointer value:

.foreach (ptrVar { dp 98d7ee60}) { .echo Is ptrVar an IRP?; !irp ptrVar }

So, WinDBG will assign each resulting token from the dp command to the variable ptrVar. I can then use ptrVar as part of any expression that I like. In this case, I print out the pointer value so that I can find it later and run !irp on the address. Abbreviated output looks something like this:

myptrirp

And in this case we found an IRP at 0x862534f0 (which appears twice in the data block).

If you check out the docs, you’ll see that .foreach has lots of other features and options. Once nice addition is that it also supports piping data from a file on the host machine, so you can process output collected from elsewhere.

Posted in WinDBG | No Comments »

VirtualKD 2.0 released

Wednesday, June 24th, 2009

VirtualKD is an open source project that gives you a high speed debug connection between the host machine and guest VM:

http://virtualkd.sysprogs.org/

I haven’t been able to use this before because it didn’t support x64 hosts, but v2.0 was recently released with x64 support. After a day of using it, I’m absolutely hooked and can’t imagine debugging a VM without it. In fact, I’m not looking forward to my next live debug session using real machines.

Posted in WinDBG | No Comments »

Viewing cached files II (!finddata)

Monday, June 22nd, 2009

If you have a file object of a cached file, there’s an even better command to find the file data in the cache: !finddata. It takes a file object and an offset and shows you the data in the cache at that file offset.

Unfortunately, just ran it against the system I’m on right now and hit an error:

fd

What’s odd though is that I’m able to see the VACB fine:

vacb0622

So I’ll send a bug to the WinDBG folks using the feedback link available at the bottom of every documentation page.

Posted in WinDBG | No Comments »

MmGetSystemAddressForMdlSafe may need to be undone

Friday, June 19th, 2009

A thread in NTDEV yesterday and today brought to light an interesting misunderstanding:

http://www.osronline.com/showthread.cfm?link=158854

While it may not be obvious, MmGetSystemAddressForMdlSafe potentially has a side effect that must be undone. Unfortunately, the documentation makes no mention of this and so I can see why there is room for some confusion.

Turns out that this driver developer interface (DDI) is actually a macro, so we can see its inner workings:

#define MmGetSystemAddressForMdlSafe(MDL, PRIORITY)
     (((MDL)->MdlFlags & (MDL_MAPPED_TO_SYSTEM_VA |
                        MDL_SOURCE_IS_NONPAGED_POOL)) ?
                             ((MDL)->MappedSystemVa) :
                             (MmMapLockedPagesSpecifyCache((MDL),
                                                           KernelMode,
                                                           MmCached,
                                                           NULL,
                                                           FALSE,
                                                           (PRIORITY))))


The first thing we learn here is that this routine doesn’t do much if either the MDL_MAPPED_TO_SYSTEM_VA or MDL_SOURCE_IS_NONPAGED_POOL bits are set.

MDL_MAPPED_TO_SYSTEM_VA we’ll talk about in a moment, but MDL_SOURCE_IS_NONPAGED_POOL means that this MDL was built with MmBuildMdlForNonPagedPool. In that case, the MDL already has a non-paged system address for this MDL, so MmGetSystemAddressForMdlSafe can just be effectively a NOP and return the non-paged pool address passed to  MmBuildMdlForNonPagedPool.

If neither of these bits are set, then the macro makes a call to MmMapLockedPagesSpecifyCache to build a kernel mode virtual address to the MDL. This routine will map the MDL with a contiguous kernel virtual address range via system page table entries (PTEs). The base of the address range will be put in to the MappedSystemVa field of the MDL and the MDL_MAPPED_TO_SYSTEM_VA bit will be set in the MDL. Thus, the next time a call to MmGetSystemAddressForMdlSafe is made on the MDL the operation will be a NOP.

If you read the docs for MmMapLockedPagesSpecifyCache, you’ll see that this mapping needs to be torn down with a call to MmUnmapLockedPages. Thus, any MDL with the MDL_MAPPED_TO_SYSTEM_VA bit set must be unmapped before the MDL is freed.

However, what’s interesting is that you won’t actually find many drivers with calls to MmUnmapLockedPages with pointers returned from MmGetSystemAddressForMdlSafe. Why is that? Well, it’s actually a two part answer:

1) MmUnlockPages will automatically unmap the MDL for you if it sees that the MDL_MAPPED_TO_SYSTEM_VA bit is set:

unlockida1

2) The I/O manager calls MmUnlockPages on the MDL in Irp->MdlAddress as part of I/O completion

So, as driver devs we generally don’t have to worry about this as it’s done for us automatically behind the scenes.

Posted in Driver development | No Comments »

WinDBG supports tab completion

Friday, June 19th, 2009

In case you’ve never happened upon it, WinDBG does indeed support tab completion. Not only does it support it in the traditional sense, but it also supports wild cards. So you can do things such as:

bp nt!*purge<tab>

And keep hitting tab until you find the routine you wanted. Shift+Tab will even scroll backwards through the results if you were a little overeager and went too far.

One interesting thing that I’ve noticed is that you can also sometimes tab complete the module name, which is pretty handy. So if I’m setting a breakpoint in mydriver, I can just do:

bp myd<tab>

And get the right module.

But note that I did say, “sometimes”. The module tab completion appears to only work when the current instruction pointer maps to a module that you have full PDBs for. This makes no sense to me and I sent an inquiry about it to the WinDBG alias  last week, but haven’t heard anything back on that.


Posted in WinDBG | No Comments »

« Older Entries