Deck used in my session on debugging SQLServer crash dumps at SQL Bits VI.

1. SQL Server CrashDumpAnalysis A brief tour withWinDbg and otheruglytools Pablo Álvarez Doval Debugging & OptimizationTeam Lead pablod@plainconcepts.com

2. Who am I?

5. SessionObjectives Whatisthissessionabout? Whatisn’tthissessionabout?

7. Who are you?

8. Agenda Tools of theTrade Brief Windows ArchitectureRefresher SQL Server Post-mortem Debugging Handling SQL Server dumps Analyzing SQL Server dumps Debugging .NET Applicationswith SOS

9. Debugging Tools for Windows Free download: http://www.microsoft.com/whdc/devtools/debugging Updated several times a year Debuggers, extensions, tools and a great help file: windbg.exe, kd.exe, cdb.exe gflags.exe, tlist.exe, etc debugger.chm Can be installed via xcopy

10. Demo 0: … isitreally so ugly?

11. Thesaurus Just to keep with the forensics analogy: Corpse  Dump file Forensic Lab  WinDbg Forensic Scientist  You! Gray’s Anathomy  Windows Internals 5th Ed.  We are not going to get into details, but we will do a little refresher of some key concepts

12. Usermode vs. Kernelmode Windows on Windows wowexec.exe UNIX LSA Shell Lsass.exe Client/Server csrss.exe Notepad notepad.exe Virtual DOS Machine ntvdm.exe Win32 Interix User Mode Kernel Mode ExecutiveServices I/O IPC Memory Processes Security WM PNP GraphicsController Object Manager FS Device Drivers Microkernel Hardware AbstractionLayer (HAL)

13. Application, Processes and Threads An application is formed by one or more processes A process is an in-memory executable, which is made up of one or more threads and its resources A thread is the basic unit of execution and schedulingin the OS.

14. … isitreallyworthit?

16. Othergoodreasons…

17. Win32 Virtual MemoryAddressing (I) sqlsrv.exe Process n Process 1 Process 2 Thread 1 Thread 1 Thread 1 Thread 1 Thread2 Thread2 Thread2 Thread2 … : : : : 2 Gb Thread n Thread n Thread n Thread n 4Gb Kernel 2 Gb

18. Win32 Virtual MemoryAddressing(II)

19. Thread Call Stacks Shows part of the history of the function calls of the thread Each thread has its own Call Stack i.e: ntdll!KiFastSystemCallRet USER32!NtUserGetMessage+0xc notepad!WinMain+0xe5 notepad!WinMainCRTStartup+0x174 kernel32!BaseProcessStart+0x23

20. CallStacks (I) Eachthread of theprocess has itsowncallstack:

21. CallStacks (II) Eachframe has thefollowingstructure: Frame Parameters ReturnAddress Frame Pointer ExceptionHandler Local Variables Registros

22. Symbols Symbols make the call stack useful: Without Symbols: With Symbols: kernel32!+136aa kernel32!CreateFileW+0x35f

23. Symbol formats Current format: .PDB Old Format: .DBG Retail vs. Debug (Free vs. Checked) builds Private symbols vs. public symbols

24. Symbol Servers Uses the File System as a Symbol’s database: Organized by name and a unique identifier Folder structure: SymSrvile_name.pdbnique_number___ i.e: Symbolstdll.pdbB5EDCA52tdll.pdb Symbolstdll.pdb80FCC4F2tdll.pdb

25. Demo 1: Scheduler Non-Yielding

26. Scenario A customer’s SQL Server 2000 ishanging, showing 17883 errors in SQL Server’sErrorLog Whenthese errores ocurr, SQL Server automaticallytriggersthecreation of a dump … 2007-02-12 11:17:14.10 server Error: 17883, Severity: 1, State: 0 2007-02-12 11:17:14.10 server Process 59:0 (834) UMS Context 0x125ABD80 appears to be non-yielding on Scheduler 1. …

27. Demo 2: DBCC CHECKDB

28. Demo 3: ClusterResources

29. ManagedDebuggingwith .NET WinDbgis a nativedebugger In ordertodebug .NET codeweneedto use debuggerextensions: SOS.dll (untilframework .NET 3.5) CLR.dll (framework 4.0) Whyallthis? Isitworthit?

30. Demo 4: ManagedDebuggingwith SOS

31. Somecooltips… Didwereallygettothisslide in time?! Well.. enjoysome free tips!  Using SOS from VS.NET Memorydumpanalysisfrominside VS2010

32. Resources pablod@plainconcepts.com @Plain Concepts http://www.geeks.ms/blogs/palvarez http://www.geeks.ms/blogs/rcorral http://www.geeks.ms/blogs/luisguerrero @MSDN: http://blogs.msdn.com/tess/ Books: Microsoft Windows Internals, 5th Ed. [Mark E. Russinovich and David A. Solomon]Microsoft Press. Debugging Applications for Microsoft .NET and Microsoft Windows[John Robbins]Microsoft Press.

33. AnyQuestions? Thanks! 