Contents
Regression Testing - What is it?
When Should You Run Regression Tests?
How To Run A Regression Test
Automate Regression Testing
Regression Testing - What is it?
Regression testing is a "management by exception" technique.
Regression testing tests for changes. Typically a
regression test compares new output from a program against old output run
using the same data. Unless, the reason for changing the program is to
alter the output, the results should be identical.
Lets say the program was to add two numbers
together. In a previous test the results for adding 1 to 2 looked
as follows:
Number One: 1
Number Two: 2
Total:
3
Lets say we modified this program to be a 32 bit
program versus a 16 bit program. Lets also assume that a bug was
introduced where number one was added to itself instead of the second
number and the results look as follows:
Number One: 1
Number Two: 1
Total: 2
At first glance, it appears obvious that a simple
review of the output will catch the mistake. What if there are 20 pages of
computations? Would it be caught?
- Mistakes like this do happen.
- There is no guarantee that they will be caught by
the right person and at the lowest cost.
In the above example, a regression test would catch the problem.
Back to Contents
When Should You Run Regression Tests?
The more frequently the better!
- Anytime you add new features.
- Anytime you tweak the system to improve performance.
- Anytime you add a new component or third party
control to the system. The same holds true for updating an old control
or dll file.
- Regression testing can be used to easily see if different operating
systems or different versions of operating systems impact the way the
system computes the results.
- Whenever program maintenance is done to an existing system.
- At each new phase of a new system.
- Anytime you think someone could have inadvertently introduced (or
deleted) some code.
- It can be used to see if a bug in an old program has been cleared
up.
- Since this regression testing works with text
files, this software testing measure can also be used with systems
that generate Web pages.
- You are a contract programmer, and you want to
make sure that someone has not modified your work. Do you want to get
blamed for other programmers' bugs?
Back to Contents
How To Run A Regression Test
- Using the previous program version that you are confident that the
output is correct, run a test set of data to generate output in the
form of a text file. This is what we call the Control Output File.
- With the newer version of the program, run the same test set of data
that was used originally with the old version to generate output in
the form of a text file. This file is what we call the New Output
File.
- Compare the Control Output File to the New Output File. If the
Control Output File was created with a properly functioning version of
the program, both files should be the same. If they are not, you need
to determine why they are different.
- Repeat steps 1 through 3 with for all scenarios that you want to
test.
Back to Contents
Automate Regression Testing
If not automated, repeating the procedure to run the set of regression
tests for each change can be time consuming, boring and subject to error.
The trick is to automate the process. You can do things to automate
regression testing as follows:
- For each item you want to test for, you only need to create a
Control Output File once, unless the new version requires a
modification in the output.
- Design the program being tested to take a command line that includes
what the input file is, what the output file should be and something
to indicate that the program start and exit without a user's
intervention.
- Use a regression testing utility like Denver Tax Software's Match2
that can run from a script or batch file.
- Create a script or batch file that automatically runs the new
version of the program and the regression testing utility for each
test scenario.
- Once the upfront work has been done to create the scripts or batch
file, the regression testing can be scheduled to run at night, lunch
or coffee breaks.
- Check to see if an error log has been created.
Back to Contents
|
|
