Free Functional Testing Tool?

[mskeel]

Im currently working on a command line tool that I would like to automate tests for -- basically I need to run some functional/integration tests on the entire tool and I dont have a lot of money to spend.

The options Ive come up with so far:
-- Use NUnit to run the tests -- I would probably invoke the tool programmatically, capture output, and verify the results (both printed and functional)
-- Hand roll tests using a light weight scripting language such as Ruby or PowerShell.

Both of these cases really amount to the same thing -- in one I can start writing tests immediately, the other I have to do a little more work up front to figure out how it will all come together. The huge negative to both of these is that I will have to build extra functionality in to keep the tests easily adaptable to changes over time. They both feel kind of clunky to me and I would end up having to maintain not only the tests, but also the testing tool/scripts.

Does anyone know of a free/open source tool that does this kind of testing? I suppose it really doesnt need to be .Net specific as Ill be running the tool externally.

 

[mskeel]

I found what I was looking for. FIT is pretty much the de facto standard it seems and I must say, after playing with it for a few hours, it will blow your mind with its awesomeness. It is very cool.

 

[Malfunction]

never had the time nor the need (so far) to look at http://fitnesse.org/
hmm....fit...fitnesse ....? is it the same:

"Great software requires collaboration and communication. Fit is a tool for enhancing collaboration in software development."

"Great software requires collaboration and communication. FitNesse is a tool for enhancing collaboration in software development."

 

[mskeel]

As it turns out, FitNesse is a wiki front end for a heavily modified version of FIT which is kind of ironic because FIT was created by Ward Cunningham, the father of the Wiki. FitNesse eventually lead me to FIT. Sometimes its just a matter of finding the right vocabulary so you can go off and figure it out for yourself, which can be hard when youre first starting out in a new problem domain.

FIT has been pretty cool so far. All the tests are stored in human readable HTML which can make up your specification document. It makes it really easy for you or your client or a subject matter expert to modify, run, and understand the tests. You even get a green/red pass/fail report using the same template as the spec so you know exactly what is working and what isnt -- right now. All the test criteria are stored externally and there is a minimum amount of code necessary to put the hooks into your program. This addresses two of my biggest concerns with rolling my own solution.

For my project, I checked the FIT executable and library (for creating test fixtures) into version control so now other members of my team can easily run acceptance tests right out their sandbox without having to install and configure additional scripting languages or software that they may not have currently on their machines (powershell is still just a release candidate I believe).

Also, here is a good example of how FIT is used. Just ignore the little tool the author wrote. It sounds cool but it actually really sucks. Of particular interest in this article are the differences between acceptance testing and unit testing (the author