A thought that occurred to me recently while following procedures for testing using the boundary value testing method is, “What would happen if I tried values for inputs of an incorrect type rather than whatever values the particular testing method called for?” While simple test procedures such as normal boundary value testing clearly leave much to be desired, more comprehensive testing methods such as robust equivalence class testing seem to cover most of the potential sources of input errors, right? I think not. It was often difficult to fight my intuition and desire to choose test values that did not necessarily line up with the testing procedures covered in boundary value or equivalence class testing procedures. This desire to choose what to test and how to test it based simply on experience and intuition is whats known in the software quality assurance field as exploratory testing, and it is an extremely powerful method of testing when used properly and appropriately.
In a Let’s Talk About Tests podcast titled “Players gonna play play play play play,” from September 21, 2017, Gem Hills talks about the benefits of getting familiar and gaining a better understanding of a program simply by “playing about,” with the program itself. Hills argues that sometimes simply using the program and applying knowledge from previous experiences with similar programs or with the development team itself can often provide a software tester with potentially revealing test cases that would be missed if the tester was relying solely on a testing framework. In her post, Hills references Richard Bradshaw, perhaps better known as the “Friendly Tester.” Specifically, Hills talks about attending one of Bradshaw’s talks at the BBC in August where he spoke about how “having a play” with a program can become far more impactful by simply recording the results and tests, effectively turning this play into exploratory testing.
One of the reasons that Hills cites for playing with a program with no real agenda is that, “[She] is not confident that [she’s] tested everything but [she’s] not sure what [she’s] missing.” I found this point especially telling. This is exactly how I felt when I was writing test cases for the boundary value testing assignment, as if I was sure that I was missing something, but unsure as to exactly what it was that I was missing. It would therefore seem to make sense to me that I should begin playing around in an attempt to discover what it is that I am missing. Perhaps it is simply that I am lacking the appropriate testing method, and performing some exploratory testing provides insight into an effective method. It is possible, however, that a bug exists in the program that cannot be found by applying any number of testing procedures. In these cases, exploratory testing, or “playing about,” is the only effective testing tool.