Category Archives: Methodology

Context-Driven Testing

Context-driven testers choose their testing objectives, techniques, and deliverables (including test documentation) by looking first to the details of the specific situation, including the desires of the stakeholders who commissioned the testing. The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework.

Ultimately, context-driven testing is about doing the best we can with what we get. Rather than trying to apply “best practices,” we accept that very different practices (even different definitions of common testing terms) will work best under different circumstances.

Context-driven testing call it a set of values rather than a process or technique. It revolves around the fact that software users are human beings with diverse preferences, needs, abilities and limitations. A program that works well for one person in a given situation might prove inadequate or inappropriate for another person or situation. For example, a word processor with mathematical symbols and a set of tools for positioning and manipulating them might be ideal for a college professor writing a physics textbook but cumbersome and annoying for a novelist. Conversely, a simple text editor may be preferred by the novelist but be rejected by the professor. Context-driven testing revolves around the fact that there is no single “best solution” that applies to all cases. In addition, it takes into account the fact that complex software projects often evolve in unpredictable ways. Context-driven testing is based on the notion that a computer program should be treated as a solution. It follows that if a program does not resolve the problem or situation it is meant to address, then it cannot be considered a success.

Advantages of context-driven testing include enhanced user-friendliness of the end product, optimized functionality for intended users and adaptability of the product to changing markets and social values. The context-driven methodology does not necessarily work well in all situations. Other approaches might prove better for developers who are under the direct supervision and control of an autocratic “boss” who takes responsibility for the results of work done. Context-driven testing would likely prove superfluous in stable environments where conditions rarely or never change.


The Seven Basic Principles of the Context-Driven School

  1. The value of any practice depends on its context.
  2. There are good practices in context, but there are no best practices.
  3. People, working together, are the most important part of any project’s context.
  4. Projects unfold over time in ways that are often not predictable.
  5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
  6. Good software testing is a challenging intellectual process.
  7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

Illustrations of the Principles in Action

  • Testing groups exist to provide testing-related services. They do not run the development project; they serve the project.
  • Testing is done on behalf of stakeholders in the service of developing, qualifying, debugging, investigating, or selling a product. Entirely different testing strategies could be appropriate for these different objectives.
  • It is entirely proper for different test groups to have different missions. A core practice in the service of one mission might be irrelevant or counter-productive in the service of another.
  • Metrics that are not valid are dangerous.
  • The essential value of any test case lies in its ability to provide information (i.e. to reduce uncertainty).
  • All oracles are fallible. Even if the product appears to pass your test, it might well have failed it in ways that you (or the automated test program) were not monitoring.
  • Automated testing is not automatic manual testing: it’s nonsensical to talk about automated tests as if they were automated human testing.
  • Different types of defects will be revealed by different types of tests–tests should become more challenging or should focus on different risks as the program becomes more stable.
  • Test artifacts are worthwhile to the degree that they satisfy their stakeholders’ relevant requirements.

Context-driven techniques?

Context-driven testing is an approach, not a technique. Our task is to do the best testing we can under the circumstances–the more techniques we know, the more options we have available when considering how to cope with a new situation.

The set of techniques–or better put, the body of knowledge–that we need is not just a testing set. In this, we follow in Jerry Weinberg’s footsteps:  Start to finish, we see a software development project as a creative, complex human activity. To know how to serve the project well, we have to understand the project, its stakeholders, and their interests. Many of our core skills come from psychology, economics, ethnography, and the other socials sciences.

Taken from