Debugging of an Application Crash

One of our common issues is troubleshooting application crashes (for example, the Print Spooler or a third-party application).  These crashes usually result in the infamous Dr. Watson error.  First, let's discuss terminology.  A crash is when something experiences a fault and has no choice but to exit.  In the case of an user mode process […]

One of our common issues is troubleshooting application crashes (for example, the Print Spooler or a third-party application).  These crashes usually result in the infamous Dr. Watson error. 

First, let's discuss terminology.  A crash is when something experiences a fault and has no choice but to exit.  In the case of an user mode process that generally means a Dr. Watson popup and application exit, and in the case of the kernel, a Bugcheck.  A crash can be caused by something as simple as a value being set to zero when a function is expecting a non-zero response, or trying to access a section in memory that has either become damaged or that belongs to another process.

Determining the cause of an application crash can be very simple, or extremely complicated - depending on the failure.  You can do something as simple as viewing the Dr. Watson log, or you can do a full debug.  Debugging is very much an art, and a complicated one at that, so we are only going to touch on the really basic concepts in this discussion.  First, let’s go the easy route.

When a process crashes, Dr. Watson will generally catch it and make a log entry as well as catch a dump of the faulting process. The log file is called Drwtsn32.log and it typically in the following directory: C:\Documents and Setting\All Users\Application Data\Microsoft\Dr Watson.  This location is configurable using the Dr Watson configuration tool (drwtsn32.exe)

This file may very well be several megabytes in size, since the log may be appended to each time Dr. Watson handles an exception.  The log is written top to bottom, so the last entry in the file is the most recent crash.  Each entry starts off telling what process crashed and the basic type of crash it was:

Application exception occurred:

App: C:\Program Files\Test\SampleApp\SMPL.exe (pid=2348)

When: 12/8/2006 @ 14:19:10.261

Exception number: c0000005 (access violation)

OK - so the first thing to understand is what this information is telling us.  It is fairly straightforward, but does contain very important basic data - the Name and Process ID (PID) of the application, the exception code (c0000005 in this case) and the exact time & date of the failure.

Under this will be quite a few pages of stack information, and unless you wrote the application probably won’t be too much help for you.  But what you can do that sometimes helps is to find the actual fault itself.  The easiest way to do this is to go to the bottom of the log and then search upward by doing a Find on the word ‘fault’. You should find an entry similar to this:

FAULT ->0040191c 8b4020 mov eax,[eax+0x20] ds:0023:00000020=????????

Below that will be the stack for the faulting thread, and may point to at least basically what is causing the issue.  For instance, if the stack referenced a third-party .DLL, it is very possibly the cause of the crash.  So, in the following example for instance, a module called CMGR caused a crash:

*----> Stack Back Trace <----*

WARNING: Stack unwind information not available. Following frames may be wrong.

ChildEBP RetAddr Args to Child

71ab4617 3d8151ec 71ac4028 71ab9448 f4840f56 CMGR+0x191c

8b55ff8b 00000000 00000000 00000000 00000000 0x3d8151ec

If looking at the Drwtsn32.log doesn’t help, we need to go a bit deeper, and actually debug the application fault.  So to do this, we will need to install the Debugging Tools for Windows.  Simply install the tools using the defaults and set your symbol path to the default listed on the web page.  Now comes our choice in how you wish to actually do the debug.  When an application crashes, it will generally be caught by Dr. Watson and create a dump in the same location as the Drwtsn32.log file.

This is the dump file we can open using the Debugging Tools for Windows. To do this, run Windbg.exe from the Debugging Tools folder. Click File – Open Crash Dump and browse to the .DMP file in question.  After you open the file, the initial output looks similar to this:

Microsoft (R) Windows Debugger  Version 6.7.0005.0
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [Z:\Dump\SVCHOST_2ndChance.dmp]
User Mini Dump File with Full Memory: Only application data is available

Comment: '2nd_chance_AccessViolation_exception_in_SVCHOST.EXE_running_on_<MACHINE_NAME>'
Symbol search path is: SRV*C:\SYMBOLS*http://msdl.microsoft.com/download/symbols

Executable search path is: 
Windows Server 2003 Version 3790 (Service Pack 1) UP Free x86 compatible
Product: Server, suite: TerminalServer SingleUserTS
Debug session time: Fri Apr 13 19:50:28.000 2007 (GMT-5)
System Uptime: 28 days 6:45:18.796
Process Uptime: 20 days 23:59:39.000
...................................................................................................................
Loading unloaded module list
................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1954.4708): Access violation - code c0000005 (first/second chance not available)
eax=80070002 ebx=00000073 ecx=00000007 edx=000000dd esi=00000002 edi=7ffffffe
eip=77bd4817 esp=0214f12c ebp=0214f59c iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
msvcrt!_woutput+0x6aa:
77bd4817 66833800        cmp     word ptr [eax],0         ds:0023:80070002=????
 
microsoft, windows, debug, debugging, application, crash, knowledgebase, application crash