Shopping Cart Validation

Recently I had the task to ensure a mock shopping cart app on the web was working correctly. I decided to play with the darn thing to gain insight on all the things I needed to check. At first glance all seemed well. I added items to my cart. The total amount at the bottom seemed to be incrementing correctly. Then I found some disturbing behavior.

When I removed an item from my cart, it did not always accurately decrease my total price. It did work sometimes. But it did not work most of the time. There was also some weird errors when I removed the last item in the cart. I wanted to go further. I wanted to assist the developer in figuring out what was going wrong.

Initially the incorrect total price seemed random. Then I started calculating the amount it would decrement. Although it was not the amount of the item I removed from the cart, it would decrement by the sales price of some other items in the store. I found that the error would be predictable when I removed items from certain positions in my cart list. Those were the details that let the developer hone in on the exact problem. The cart was implemented in JavaScript. Multiple variables were used in the code. However some of them were supposed to be the same variable. The real lesson from this is that a tester can shine some great light into the nature of bugs. This can help get the bugs resolved faster.

TestComplete by Smart Bear Software

Smart Bear Software has announced the newest version of their TestComplete package. This has a number of tools for testing. I think it aims to allow people with limited technical skills to become productive in testing. However there are some hooks in there for advanced users as well.

There is a Test Visualizer that let's you set up tests by clicking around in the user interface. There is also another tool that let's you type in free form text of the test. It then reads your text and generates test scripts. That seems pretty amazing.

The tool also has a Data-Driven Loop Wizard which let's you import data from a file to create tests. Finally for the seasoned testers, there is the JavaBridge. It allows you to use an API to grab data from within a Java application. That is sweet. If I get into more Java development, that feature alone could be worth the cost. Pricing starts at a grand for the Standard version.

The Many Faces of Quality

I read an insightful article on the different quality professions out there. These include quality assurance, quality control, verification, and validation. These terms are often confused and used imprecisely. So what exactly do they refer to.

Quality Assurance deals with processes. That does not necessarily mean testing. In fact, QA can involve review of a design or requirements document. This is in line with verification, which checks that development was conducted correctly.

Quality control deals with the product of software development. This is achieved through the execution of tests. It is closely related to validation, which is the execution of code.

Visual Studio Ultimate

There are a number of tools in the Visual Studio Ultimate Edition that help the test mission. These tools are included in the price of Microsoft's MSDN subscription. Some of these tools include Test Manager and Lab Manager.

Test Manager is meant for non-techies to developer test cases. This helps eliminate non-technical testers clicking buttons manually.

Lab Manager gives you an environment to conduct tests. It works with Intellitrace and Hyper-V. Unfortunately Lab Manager does not support Silverlight applications. This tools does require a lot of hardware to run on.

Unit Test Worth

Just read a rant that unit tests are all but worthless. Is this true? Let's start at the beginning. Unit tests are supposed to validate independent pieces. They attack at the function level. Code with many dependencies do not work well with unit tests.

Unit tests may help find some small errors. However isn't that what debugging is for? I am speaking hypothetically here. You are supposed to be able to refactor your code and run your unit tests to check whether it still works. Unfortunately unit tests may be too brittle to still work after refactoring.

Developers are optimists. Therefore their unit tests may just choose the easy cases. That is not much of a return on investment. However you should use a tool like Jester. It rewrites your Java code to tweak the logic a bit. If you Jester your code, and the unit tests still past, your tests are not doing much.

Acceptance tests are better predictors of whether your code works. These can be employed after a refactoring to make sure the code still works. Some people still swear by unit tests. They have stories of doing 100% coverage. Then the eventual deployment had no problems at all. I think the jury is still out on unit test effectiveness.

Crowdsourcing

You have heard of outsourcing to lower costs to testing. But have you heard of crowdsourcing? This is a new spin where a company has independent testers all over the world. You company contracts with the umbrella company to get your tests executed. You only pay for defects found.

This method has the advantage that you do not need to set up the outsourced testers. A company has already done that for you. You also only have to pay for results you verify. A test cycle can run you as little as $5k. I just wonder what quality of tests you can get for such a small price. It could be worth a try.

Risk Based Testing

How do you prioritize testing? You can't do everything. So you should spend your time on tests that have the most bang for your buck. Risk based testing attempts to solve this problem.

There are two dimensions to risk. One is the technical dimension. This is where the developers use new technology. Another dimension is the business one. Where in the apps is there a high cost to the operations if there is failure?

You can use the two risk dimensions to prioritize testing. The areas that involve both a technical and business risk need the most testing resources. Other areas that have either one or the other dimensions of risk should be tested next. The low risk areas can be saved for last or skipped entirely. The result is that you spent your time wisely.