Memory dumps for hang or performance issues, "manual" approach

  • Thread starter Thread starter Viorel-Alexandru
  • Start date Start date
V

Viorel-Alexandru

Guest
You should be using this method when the slow HTTP processing can be easily reproducible, when you can easily predict that a request processing would be slow.



An alternative to the approach below is to trigger a memory dump collection with a FREB rule, according to instructions in this article: Using FREB to generate a dump on a long running request - Microsoft Tech Community



Both ProcDump and DebugDiag can attach to a process and collect memory dumps from it. DebugDiag is more flexible, ProcDump has almost no footprint.







Using the command-line ProcDump




ProcDump does not require installation. Needs to be run from an administrative console. But ProcDump is not as easily configurable as DebugDiag…



  1. Download the tool and copy it on a disk folder, for example D:\Temp-Dumps\
    ProcDump - Windows Sysinternals
  2. Open an administrative console, from where to run commands.
    Then navigate to where the ProcDump tool was placed (D:\Temp-Dumps\).
  3. Find the process ID, the PID, of the IIS w3wp.exe worker process executing your application.
    From the administrative console, you can use AppCmd IIS tool to list processes for application pools:
    D:\Temp-Dumps> C:\Windows\System32\InetSrv\appcmd.exe list wp
  4. In the administrative console, prepare the following command to collect dumps (but don't hit enter yet).
    D:\Temp-Dumps> procdump.exe -accepteula -ma -n 5 -s 10 [PID]
    Replace [PID] with the actual Process ID integer number identified at the step 2.
    Please make sure that there is enough disk space on the drive where dumps are collected. Each process dump will take space in the disk approximately the same size the process uses in memory (column Commit Size in Task Manager). For example, if the process’ memory usage is ~1 GB, then the size of a dump file will be around 1 GB.

  5. Start reproducing the problem: issue a slow-running request from the client (browser) to the investigated IIS application.
    Or simply wait or make requests to the IIS/Asp.Net app until the slowness manifests.
    Then, as fast as possible…
  6. Hit enter on the above ProcDump.exe command (procdump.exe -accepteula -ma -n 5 -s 10 [PID])
  7. Please wait until all the dumps are written and then compress the files before uploading the dumps to the workspace.







Using the UI tool DebugDiag, Debug Diagnostics Collection




DebugDiag requires installation, but it is able to determine itself the whatever process instance - PID - happens to execute for an application pool at some point in time; even when that process may occasionally crash, hence restart with different PID.







#1.


Download and install the Debug Diagnostic tool on IIS machine:

Download Debug Diagnostic Tool v2 Update 2 from Official Microsoft Download Center v2.2 (if 32-bit system)
Download Debug Diagnostic Tool v2 Update 3.1 from Official Microsoft Download Center v2.3.1 (only supports 64-bit OS)







#2.


Open Debug Diagnostic Collection.
If a “rule wizard” appears, please click Cancel.






#3.


You may want to change the default location where the memory dumps are written.
By default, these are placed under C:\Program Files\DebugDiag\Logs\Misc
Change from Menu > Tools > Options And Settings, then modify Manual Userdump Save Folder:



large?v=v2&px=999.png





Please make sure that there is enough disk space on the drive where dumps are collected. Each process dump will take space in the disk approximately the same size the process uses in memory (column Commit Size in Task Manager). For example, if the w3wp.exe process memory usage is ~2 GB, then the size of each dump file will be around 2 GB.
Please do not choose a disk in network/UNC; choose a local disk.







#4.


Go to Processes tab. Sort the processes by Process Name and find the w3wp.exe process.
If there are more than one application pool running at the same time you may see more than one w3wp.exe.
In that case, you may differentiate by Process Identity or by checking the Web Application Pool Name column.


large?v=v2&px=999.png









#5.


Right click on the process and choose “Create Userdump Series…” and then…


large?v=v2&px=999.png









#6.


Collect the dump series with the following settings:

  1. Generate a Userdump every 10 seconds.
  2. Start the timer when writing the dump file completes.
  3. Stop after generating 5 userdumps.
  4. Collect Full Userdumps.
  5. As soon as you click Save & Close, you will see the following pop-up box and the dump collection will be started immediately:

large?v=v2&px=999.png



large?v=v2&px=999.png









#7.


Please wait until all the dumps are written on disk at the configured location.

Archive each dump file in its own ZIP and prepare to hand over to the support engineer; upload in a secure file transfer space.









Remember my article about how exceptions are handled and how to collect memory dumps to study them. We can double check if a crash occurred or not: read about w3wp.exe crashes.







Aside: Just in case you are wondering what I use to capture screenshots for illustrating my articles, check out this little ShareX application in Windows Store.

Continue reading...
 
Back
Top