Breaking Tests

I've got a relatively mature set of code that generates unit test data for a complex program we have. This code has been developed over the past 5 to 6 years. It creates about 15 postive test cases for different scenarios in the program. It also can also generate the same amount of negative test cases.

This year we added three more test cases. So the test code was modified to produce three more positive test cases worth of data. Things were looking good. I had a long standing action item to create data for a whole separate path that the program sometimes takes. This resulted in a new slew of tests.

At first I had some trouble generating the test data for the new scenarios. But I dug down and determined what I needed to change to get everything set up. I pretty much based the data creation on the existing routines to produce data for the existing scenarios. I was happy when the task was complete.

I decided to do a round of regression tests to make sure the original test scenarios still worked. Ooops. Looks like I broke half of them. Good thing I have an easy way to run the whole suite of tests for the original scenario. But now I got to big back in to ensure the new and old unit test scenarios are correctly served.