Writing a System Test Plan

Using a test plan to ensure your software really works.

"Extraordinary quality doesn't make good short-term economic sense."
- Peopleware, Tom Demarco

Imagine for a moment you’ve just finished a web application for a client. Swimming around within that application is 100 bugs. What proportion of those bugs would you say is acceptable for the client to find (e.g. 50, 30, 10)? The point is, those bugs are going to be found by someone, and its either going to be you or the client. You can think of it as a sliding scale, the more bugs the client finds, the more your creditability suffers and the more damage you do to the business relationship.

In the martial arts there’s a saying; “expect the best, but prepare for the worst”. The US Navy Seals have a slightly more gun-ho version which goes; “The more you sweat in training, the less you bleed in battle.” Notice how it says the ‘less’ you bleed in battle, implying that if you enter into combat, you are going to get bloody. Software development is like this too, if you’re going to create software, you are going to get bugs.

As a rough guide, you want to try catch at least 80% of all bugs. Ideally, you would go for 100%, a worthwhile but lofty goal. Your best bet for catching as many bugs as possible is to create a system test plan. Creating a decent system test plan is easier said then done. A good place to start is your functional specification (assuming you have one). In effect, your specification gets converted into the system test plan. What you are doing is checking to see if the things you said you were going to do have actually been done.

A test plan is nothing new, in fact, the structure I use was developed by Microsoft years ago. It’s a simple format, a test case has a title, some steps, and an expected result.



I write my test plans with the intent of having an independent QA tester run through it, so no prior knowledge of the system should be assumed. You want someone who has never used the system before because they will take a fresh approach. You’re counting on them to use it in a manner which was never intended (e.g. on the ‘My Profile’ page, they type in a really long name for the ‘Company’ field and crash the system because the database only has room for a 32 character string).



You should keep your test cases short, to the point, and self contained (i.e. they generally don’t link to other test cases). If you have a test case that goes over a page, consider breaking it into smaller units. The document itself should be self contained as well, that is; it shouldn’t require the QA tester to get up and ask programmers questions like “what’s the login password?”. Instruct your QA tester to log any bugs they find in your bug tracking system and also to write a proper description rather then just saying “Test Case 16 failed.” You may want to read my article on Logging Bugs Like a Pro for some suggestions on good bug logging practices.

Article reviewed by Alex Barclay.