Black-box testing

Black-box testing (functional or behavioural testing) is a testing method in which test cases are derived from the specification of the software, without any knowledge of its internal code. The tester gives inputs and checks whether the outputs match the expected results.

 Input test data --> [  System under test  ] --> Output
                      (internal code hidden)        |
                                  Compare with expected

Errors it finds (Pressman)

• Incorrect or missing functions
• Interface errors
• Errors in data structures or external database access
• Behaviour or performance errors
• Initialisation and termination errors

Main techniques

1. Equivalence partitioning: divide the input domain into classes whose members are expected to behave the same; test one value from each valid and invalid class.
2. Boundary value analysis: test values at and next to the edges of each class, where errors are most common.
3. Cause-effect graphing / decision tables: test combinations of input conditions and their actions.
4. State transition testing: test valid and invalid transitions between system states.
5. Use-case (scenario) testing and error guessing based on experience.

Example

A login field accepts a password of 8 to 16 characters.

• Equivalence classes: fewer than 8 (invalid), 8–16 (valid), more than 16 (invalid).
• Boundary values: 7, 8, 16 and 17 characters. So test cases with 5, 7, 8, 12, 16, 17 and 20 characters check this rule without reading the code.

Advantages

• Tests from the user's point of view; no programming knowledge needed.
• Tester is independent of the developer, so there is less bias.
• Test cases can be designed as soon as the specification is ready.

Disadvantages

• Cannot show which parts of the code were executed; some paths may stay untested.
• Exhaustive testing of all inputs is impossible, so only a sample is tested.
• Test design is hard if the specification is unclear.