More on Fuzzing

Development magazines are paying more attention to fuzzing. Fuzz testing is subjecting an application to random data to see what breaks. This is applicable to web apps. It can be part of a comprehensive security test. Fuzz testing let’s you try inputs that stimulate paths a tester might not normally think about, or have time to manually perform.

There are free fuzz test generators out there. Some are Peach Fuzzer, MiniFuzz, and Spike. There are also some commercial products to assist with fuzzing as well. But let’s get back to why fuzzing is a good idea. Normally you test positive aspects of requirements. However you should also be testing for things not covered by any requirement. This is where fuzz testing can come in real handy.

I am a developer. Right now I am not employing fuzz testing yet. However I hope the independent test team on my project is hot on the technique.

Pairwise Testing

Testers often need to verify systems with many related variables. This can result in a huge amount of test cases. Often the combinations can be out of control. However a technique called Pairwise Testing can reduce the amount of test cases while still covering all scenarios. The hard part of this is generating the minimal set of cases. This is a good time to use a program to do the hard work.

One pairwise test case generator is Pairwise Independent Combinatorial Testing (PICT). This product is based on a tool that Microsoft themselves use. There is a recent new tool called QICT which accomplishes the same task. The beauty of QICT is that the source code is provided free of charge.

Pairwise testing is good for many test scenarios. Some good examples are testing databases and configuration data. I know that I often have an onslaught of test cases that I think need to be executed to provide full coverage. I might just have to use a tool such as PICT to make my job easier. This can potentially reduce the test time necessary, and improve my test throughput.

.NET Runtimes

Microsoft is changing the way that new versions of the .NET framework impact existing applications. Here are the goals that Microsoft has. Installing new versions of the .NET framework should not break existing applications that are installed. The new version should also be backward compatible. Finally applications should run against the version of the .NET framework they were released under.

These new Microsoft goals are realized in the .NET framework version 4. This is a statement that must be tested thoroughly. Previously the new .NET frameworks would be used for any existing installed application. However now Microsoft claims the latest version will not break existing apps. I can envision the need of virtual machines to test .NET framework upgrades, and the impacts on applications which depend on prior versions of the framework.

Microsoft claims that the .NET framework version 4 is the most backward compatible version yet. My recommendation is to trust, but verify.