You need reliable test data to conduct independent verification. This is more complicated than it seems for some systems. The Software Maintenance blog demonstrates that sometimes even developers have trouble creating test data.Ideally you can run the system like it operates in a production environment. The closer to the real world you make it, the better the quality of your test data. However this is not always feasible. Sometimes you need to simulate the real world with generated test data. It is these circumstances where you really need to understand the system deeply. There is always the chance that you introduce differences in your data generation that skew test results.
One way to perform test data generation is to do the hard work once. Then you can save a copy of the data that you can use time and time again. There may be some circumstances where you will need to tweak the test data. However the heavy lifting will already be done. This is how we approach test data generation on my project. Some DBAs run processes each year to give us a new data set valid for that year. The results are passed to both the development and internal test teams. The work in generating this test data is not too high. The results are awesome.
