Problem:

Imagine you're a managing a project where the constraints like scope, time, cost, and quality have been already negotiated and approved. At some point before the project start, it turns out that a customer wants to change the scope so that a number of test combinations goes significantly up.

Example. A project is about developing a flight booking website. On the home page, there is a form with the following dropdown lists:

• From: country1, country2, country3
• To: country1, country2, country3

Now, the customer is surprised that the form doesn't contain a dropdown with Class options. I don't go into details why it could be missed, but from the customer's point of view the form should look like this:

• From: country1, country2, country3
• To: country1, country2, country3
• Class: coach, business, first

Adding that dropdown shouldn't add development efforts; however, it is going to increase a number of combinations to cover with tests. If initially we were thinking of nine combinations (3x3) to test, now the number of test cases equals to twenty seven (3x3x3, right?).

Apparently, we cannot test 27 test cases in a time supposed for 9. The customer doesn't want to pay for additional test efforts. And we cannot skip testing of the additional cases either. What can we do, then?

Solution:

Instead of testing all the combinations for all the variables, test all pairs of the variables. This approach is called pairwise testing, and it lets significantly reduce the number of tests.

In the above example, it's possible to cover all pairwise combinations in only 9 tests. Triple effort reduction. How does it work?

Studies have shown that either single input parameter or an interactions between pairs of parameters cause most software defects. Thus, fewer tests and almost the same coverage are the benefits of using this technique.

NB: Nevertheless, be aware of that there is no guarantee it won't miss an important combination that may reveal a bug. You should review the generated combinations and add any important or risky case that's missed.

How to:

There are several ways to generate all the pairs, including so called Orthogonal Arrays and numerous tools based on the AllPairs algorithm. The following tool called pairwise helped me once.

After installing, create yaml file with the lines as follows:

from: [country1, country2, country3]
to: [country1, country2, country3]
class: [coach, business, first]

Then run the tool in command line:

pairwise input.yaml

The output would be:

| class    | from     | to       |
| coach    | country1 | country1 |
| coach    | country2 | country3 |
| coach    | country3 | country2 |
| business | country1 | country3 |
| business | country2 | country2 |
| business | country3 | country1 |
| first    | country1 | country2 |
| first    | country2 | country1 |
| first    | country3 | country3 |

These are now the nine cases that can be used for testing purpose.

Conclusion:

The pairwise testing may help you out when the number of test combinations is very large. This reduces the test efforts while keeping the quality expectations at the high level. However, don't trust it blindly and include additionally the combinations known to be risky.