Automated Testing Depth

We are working on replacing an old and expensive ad hoc reporting system. The trouble is that there is a new database that holds all the information. The reports team tried to give the customer access to the current database. The data model was so different than the old one, the customer rejected that proposal.


So my team is coming in to create a bunch of tables and views to replicate the old schema. It is quite a chore as some of the data is tricky to produce. There is a dedicated test team making sure everything turns out okay. One of the testers has been tasked with automating the testing of our solution.


The tester is a Java programmer. She has written a program that will query our new database, match up the records with the old database, and alert us if there is any difference. That sounds like the best way to guarantee we get the values correct.


The big team lead stepped in to start peer reviewing our work on this system. He asked how we could be sure we were getting the values correct. I told him our Java programmer tester is checking it out. When pressed for details, the tester said she only compares 1 record per table. And if there are any NULL values involved, all bets are off. Oops. Need some more due diligence here.