Analyze -v

Deferred Procedure Call (DPCs) are callbacks to an arbitrary thread context at IRQL DISPATCH_LEVEL. There is a DPC queue per processor, and queueing a DPC performs two steps:

1) Inserts the DPC onto the DPC queue of the current processor.

2) Requests a DISPATCH_LEVEL software interrupt on the current processor.

Note that there are exceptions to both of those, though I’m not interested in talking about them at this moment

When the operating system is about to return to an IRQL < DISPATCH_LEVEL, the DISPATCH_LEVEL software interrupt is delivered to the processor. On XP, the ISR for this interrupt is hal!HalpDispatchInterrupt, which does some interrupt management work and calls nt!KiDispatchInterrupt. You can get a feel for how this works by setting a breakpoint on KiDispatchInterrupt and checking out a few call stacks, which should look like the following:

While KiInterruptDispatch serves a few different purposes, for our discussion all we care about is the beginning of the function shown here:

Note the call near the end of the listing to nt!KiRetireDpcList. This is the function that will sit in a loop dequeing DPCs from the current processor’s DPC queue and calling the callbacks. There’s some interesting code leading up to that call though, so let’s go line by line and figure out exactly what this code is doing.

nt!KiDispatchInterrupt:
mov     ebx,dword ptr fs:[1Ch]

This line is moving the contents of offset 0×1c from the far segment into EBX. In kernel mode, the base of the far segment is the base address of what is called the PCR for the current processor:

Thus, this code is grabbing whatever field is at offset 0×1c from the base of the PCR structure. Luckily we have the type information for the PCR, which is nt!_KPCR so we can easily see what is at that offset in the structure:

That is the SelfPcr field, which is just the flat address of the PCR (in this case that would be 0xffdff000). Let’s move on to the next fragment:

nt!KiDispatchInterrupt+0x7:
lea  eax,[ebx+980h]
cli
cmp  eax,dword ptr [eax]
je   nt!KiDispatchInterrupt+0x2f (805459df)

Here, we add 0×980 to the base address of the PCR and store the result in EAX. We then disable interrupts on the current processor and check to see if the contents of the pointer match the pointer address.

The CMP instruction will do a logical subtract of the two values and set the Z-Flag to one if the result is zero, which would mean that the two values are the same. The JE instruction will, “Jump if the Z-Flag Equals one”, so if the contents of the pointer match the address of the pointer then this code will jump over the code segment that calls KiRetireDpcList.

If you’ve never looked at much assembly that might seem a bit weird, so let’s see what’s add offset 0×980 from the PCR and see if we can figure out what this code is doing.

If you go to a full listing of the PCR structure, you’ll notice that the last offset given is 0×120 and that is the PrcbData field:

Thus, in order to figure out what’s at offset 0×980 from the base of the PCR we’ll need to go to offset 0×860 into the PRCB. We’ll find this by doing a dt nt!_kprcb and scanning the output:

Aha! That field is labeled as the DpcListHead (a.k.a. the DPC queue) and the type is a LIST_ENTRY, which is the standard type for a doubly linked list in the kernel.

LIST_ENTRY structures have two fields, a Flink field that points to the next entry and a Blink field that points to the previous entry. When a list is empty, the Flink field points back to the address of the head of the list. So our previous check above is testing the value of the Flink field against the address of the list head, in other words it is checking to see if the list is empty. If it is, the code avoids draining the DPC queue (which makes sense).

If the list is not empty, then the code sets up to call KiRetireDpcList:

nt!KiDispatchInterrupt+0x12:
push    ebp
push    dword ptr [ebx]
mov     dword ptr [ebx],0FFFFFFFFh
mov     edx,esp
mov     esp,dword ptr [ebx+988h]
push    edx
mov     ebp,eax
call    nt!KiRetireDpcList (80545e0e)

I’m going to save the first three instructions for another time if I ever get to talk about Structured Exception Handling (SEH). Right now it’s sufficient to set that the code there prevents kernel mode exceptions from being raised to user mode exception handlers.

The next two instructions are interesting though:

mov     edx,esp
mov     esp,dword ptr [ebx+988h]

Note that the code saves the current stack pointer and then overwrites ESP with a different pointer value from the PCR. We saw previously that the last offset in the PCR is 0×120, which is the beginning of the PRCB. So, whatever value is at offset 0×868 from the PRCB is what we put into the stack pointer register. If you scroll up to the previous graphic, you’ll see that field labeled as DpcStack:

   +0x868 DpcStack         : Ptr32 Void

Thus, each processor has its own DPC stack that is used when DPCs are executed. Shortly this is going to lead to an unexpected problem that this post will hopefully help you solve.

Lastly, the old stack pointer is pushed onto the stack and finally the call to KiRetireDpcList occurs. When it completes, the old stack is restored and all is right in the World.

However, there’s an interesting issue that can arise in your crash analysis. What if the system crashes inside a DPC? Due to the stack swap that occurs in KiRetireDpcList you’ll get this when you try to dump the call stack:

In other words, you’ll get a listing for the DPC stack and you won’t necessarily be able to see the actual kernel stack of the current thread. While in 99% of the cases the DPC stack will be the only stack that you care about, there’s that 1% where knowing the current thread stack will provide the insight necessary to solve the crash (in almost 10 years I’ve seen two). Luckily, it’s going to be relatively straightforward to get the stack back. Even more luckily, it’s mostly formulaic so even if you’re not sure why you can get it back you’ll still be able to

First thing you need is the old stack pointer, which is the first thing on the stack before the return address in the call to nt!KiRetireDpcList:

Then we’re going to dump this out with the dps command and find the return address to hal!HalpDispatchInterrupt that the nt!KiDispatchInterrupt will return to. We’ll also want the first thing on the stack after the return address:

In my case, I have 0xf715da0c and hal!HalpDispatchInterrupt+0xbb. Now all that’s left is to feed those two values into the special k syntax that allows you to specify your own EBP, ESP, and EIP overrides:

Note that there’s a cheater shortcut, I could have just done k = f715da00 f715da00 @eip in this case and gotten a slightly busted but still legible stack. The technique above gives a more attractive and correct stack in the end

Possibly we can cover why this command works in the future, but for now hopefully that’s enough of a guide for you to go experiment yourselves. Don’t forget that you can always play with this on a live system where you can verify your results by simply stepping out of nt!KiRetireDpcList.

Random Other Points

1) The DISPATCH_LEVEL software interrupt isn’t always requested, so the DPC isn’t always drained when returning to an IRQL < DISPATCH_LEVEL.

2) The Idle thread also checks the DPC queue and, if it isn’t empty, drains the queue by dequeueing entries and calling the callbacks. In this case, the DPCs execute on the Idle thread’s stack

3) It is possible to target a DPC to a processor other than the current processor