Memory usage issues

[SteveHamblet]

I created a pretty small WinForms application to just delete files out of a directory that are older than X number of days. I had this application deleting around a million image files (around 5MB per file) out of a directory and I noticed that the memory usage for the application shown in Task Manger was around 540MB and later jumped up to 580MB while it was deleting the files. I didn't think much about it while it was processing the files because I figured that was the amount of memory that it needed to store the list of files that needed to be deleted, but that it would clear that up once the deletion processed was finished, but I left the application sitting in memory for a few hours after it completed it's task, but the memory the application was never released, it was just sitting at 580MB.

This caught my interest because I have a few server applications that handle a lot of files and they have issues where they eventually start throwing out of memory errors, so I thought that this little application might be an easy way to try to figure out where the memory usage issue is coming from.

I've tried a few different variations of this small application (changing from lists to arrays, clearing the Lists/Arrays and setting them null before exiting the main function, and tried using a queue object instead of the list/arrays), but it seems like most of the time, the memory used by this is not being released. Here is the code:


using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Windows.Forms;
using System.IO;

namespace DeleteStuff
{
public partial class Form1 : Form
{
private int fileCount = 0;
private int curFileNo = 0;
private bool stopDelete = false;

public Form1()
{
InitializeComponent();
}

private void Form1_Load(object sender, EventArgs e)
{

}

private void DeleteFilesFromDirectory(DirectoryInfo dir, int dayLimit, bool includeSubDirs)
{
DateTime cutoffDate = DateTime.Now.AddDays(-dayLimit);
List<FileInfo> filesToDelete = dir.GetFiles("*.*").Where(f => f.LastWriteTime <= cutoffDate).ToList();
fileCount += filesToDelete.Count;
label3.Text = "Files: " + curFileNo.ToString() + " / " + fileCount.ToString();
label3.Refresh();

foreach (FileInfo fi in filesToDelete)
{
if (stopDelete) return;
curFileNo++;
label3.Text = "Files: " + curFileNo.ToString() + " / " + fileCount.ToString();
label3.Refresh();
Log("Deleting " + fi.FullName + " Last Modified On " + fi.LastWriteTime);
try
{
fi.Delete();
}
catch (Exception ex)
{
Log("Failed to delete " + fi.FullName + ", Error: " + ex.Message);
}
}

if (includeSubDirs)
{
List<DirectoryInfo> subDirs = dir.GetDirectories().ToList();
foreach (DirectoryInfo di in subDirs)
{
if (stopDelete) return;
Log("Deleting old files from " + di.FullName);
DeleteFilesFromDirectory(di, dayLimit, includeSubDirs);
}
}
}

private void Log(string msg)
{
while (listBox1.Items.Count >= 5000)
{
listBox1.Items.RemoveAt(listBox1.Items.Count - 1);
}
listBox1.Items.Insert(0, "[" + DateTime.Now.ToString("HH:mm:ss.fff") + "] " + msg);
listBox1.Refresh();
Application.DoEvents();
}

private void button1_Click(object sender, EventArgs e)
{
if (button1.Text.Equals("Cancel"))
{
stopDelete = true;
return;
}
stopDelete = false;

DirectoryInfo dir = new DirectoryInfo(textBox1.Text);
if (!dir.Exists) return;
Log("Deleting old files from " + dir.FullName);
button1.Text = "Cancel";
fileCount = 0;
curFileNo = 0;
Cursor = Cursors.WaitCursor;
DeleteFilesFromDirectory(dir, Convert.ToInt32(textBox2.Text), checkBox1.Checked);
Cursor = Cursors.Default;
button1.Text = "Delete";
Log("Finished deleting old files");
}
}
}

The form itself is very basic... A textbox to enter the path in, a textbox for number of days to keep, a checkbox to include sub folders, a label to show how many files have been processed, a button to start/stop and a listbox to show what was deleted (up to a certain point). I have also tried not populating the list box.

I did try adding a GC.Collect() after the delete completes and that does seem to drop the memory usage back down, but that doesn't seem to help my server applications.

I'm just wondering if someone could help me understand why the memory usage isn't being cleared once this application finishes it's process. Any help or thoughts would be appreciated.

Continue reading...