Changing Test Data

Our customer reported a bug in production. There were many theories as to why this problem was happening. I told people that these were all interesting theories. But these theories, like any opinions, were really not worth much. When I got some time, I dug into the problem. I traced the problem back to the line of code that caused the problem. In the end I determined some configuration data was missing. That resulted in the good code causing a bad transaction. I unit tested my fix by hacking together some data to meet the specific scenario where the error occurs.

Then the fix went to our internal test team. A tester called me up and asked me about the problem. I explained the problem from top to bottom. You would think that would be sufficient for a tester to run with the fix and test it out. However I got back a question that I seem to get a lot from our internal test team. How do I test that? In essence they wanted to know how you set up the data conditions that result in replication of the error.

Now I am sympathetic somewhat to the testers. They do not have the source code in front of them. The documentation for our system is merely average. So it might be unreasonable to assume they had all the tools they needed to figure things out. Therefore I outlined a set of steps that could be taken to replicate this problem.

Development had previously produced a tool which generated input files for our loads component. You could use the tool to generate input files which contain the data the make the problem happen. I recommended that the testers generate up the input files with the tool, run them through the loads program, and duplicate the problem.

I guess this was too much work. The testers tried to simulate all that work by taking data previously mocked and loaded, and performing updates on that data. This is not a bad decision. But you have to know what you are doing. Unfortunately they were unable to replicate the problem. My boss told me to get in there and help them out to get this fix shipped. Essentially I had to reverse engineer our loading code. Then I kept giving them all the data they needed to change manually in order to duplicate the problem.

This is not a good scene. I have things to do. If you want to add value as a tester, you got to do the hard work. Then you will come out smarter and be able to wing it one day. But not now guys.

Testing is Hard Work

Yesterday the test manager on my project came to visit me. She was planning the tests for some new changes I am working on. She asked me to explain some of the changes as they did not make much sense to her.

I tried to get into the details with her. She took some notes. Then she said she would get some people from her team to dig into it. Today I got a message from her. She said her team could not make heads or tails of the requirements, and asked me to help them.

Well I shot off an informative email on the current state of the application, and what the customer wanted instead. Then I got a response on how they could test that. I gave the test team a little more information. Then I got another response asking how they could set up data to do the tests.

I did not want to just provide them with my own unit test plan. Then they would not be adding any value to the work. In an ideal world, they would be able to figure out how to test according to the documented requirements. Unfortunately the requirements team does not know much more than the testers. So I continued to try to give more ideas until they felt like they had enough to go on. How do we get to that ideal world where there is independent validation of my work?