Rapid Software Testing: Heuristics

As I mentioned before, Rapid Software Testing is a testing methodology defined by James Bach, Michael Bolton, and Paul Holland. This methodology focuses on the skillset and experience of the tester alongside with some rules and techniques to achieve a cost efficient quality assessment of a product.

This post will only focus on the Heuristics proposed by the methodology as a guideline for Exploratory Testing sessions although I’ve been using them as a source of inspiration for testing purposes. You can find the original document among other RST appendices. Let’s start the show!

I agree with RST approach that Testing is a context-driven activity, so you shouldn’t expect a cookbook explaining you step by step what you have to test. These represent “general techniques” which are generic enough to be relevant for a wide spectrum of contexts. Personally, these examples have helped me defining specific heuristics for a given product. These are the groups of test heuristics with some examples:

  • Functional testing. Determine the things the product can do, or this feature can offer. See that each function does what it’s supposed to do, and no what isn’t.
    • C’mon, you’re used to this testing! You don’t need help thinking about how to test functionality, do you? :p
  • Claims testing. Challenge every claim you find about the product (either explicit and implicit). Test the veracity and clarity of the claims.
    • Read the specification, is the product align with it?
    • Is the manual relevant to the current version of the product?
    • Are the descriptions, onboarding tips and help statements inside the product aligned with it?
  • Domain testing. Segment the data the product can process into tiers (invalid values, boundary values, best representatives, etc.). Consider combinations of data.
    • Is that numerical operation running the same logic with positive, negative or colossally long numbers?
    • Is the producing recognizing the language that String is using?
    • Does the product process differently Geo points from different countries?
  • User testing. Learn about user roles and categories, determining what are the differences in their needs and permissions.  Get real user data and involve the users (from different categories) in the test (or learn how to simulate one).
    • Is the “techie loner” persona  part of our user base?
    • Are we testing the admin side of the product?
  • Stress testing. Look for part of the product vulnerable to being overloaded, and generate challenging data to test its limits.
    • Extend the usage for a very long period of time with a steady load, or maybe try with peaks.
    • Test the limits on the input and output of our product.
    • Check concurrency is being handled correctly with some more data than the expected one.
  • Risk testing. Imagine a problem, then look for it. Focus on the kind of problems matters most. How would you detect problems? Your experience and knowledge about the product and the team will shine here!
    • Distributed applications struggle with concurrence. Exploit it!
    • Transactions have to deal with consistency. Exploit it!
    • This team has been failing to deliver cross-browser solutions, focus on it!
  • Flow testing. Perform multiple activities connected, don’t reset the system between user paths and vary timing and sequencing.
    • Run overlapped actions in a parallel way.
    • Verify that the product transits correctly between states, even on the non-happy sequences.
  • Scenario testing. Tell a compelling story. Design tests that involve meaningful and complex interactions with the product.
    • Reflect a scenario simulating someone who matters doing something that matters.
    • Think about the different applications of the product and how the scenarios would differ.
  • Automatic checking. Check a million different facts, hopefully getting coverage in the process and using oracles. Tools that empower the tester are always welcomed!
    • Automatic change detectors. Catch those regressions!
    • Automatic data generators. Help the tester with the easily automatable parts of the job.
    • Check permutations and combinations easily.
    • Tools that support the tester.

In my experience, this is an amazing starting point when you face a new product. These heuristic rapidly evolve into context-customized during your first exploratory testing session, serving as a baseline for the next ones.

I have to admit that the main reason to write this post is serving as documentation to me, as well as helping me interiorise it. But I hope you find it useful as well!

May the force be with you,
Gino

P.S. Feel free to use Rapid Software Testing category to visit the related posts!

Meet Rapid Software Testing

Rapid Software Testing is a testing methodology defined by James Bach, Michael Bolton, and Paul Holland. This methodology focuses on the skillset and experience of the tester alongside with some rules and techniques to achieve a cost efficient quality assessment of a product. It is based on heuristics, fallible methods of solving a problem of making a decision. That’s the main reason why the methodology empowers the tester, which is the one deciding how and when to use the heuristics. It also focuses on the story of the testing journey, giving as much importance to explaining how the test was, what has been found and what is missing to verify than the test itself.

In this video, James Bach briefly explains its details:
http://www.youtube.com/watch?v=2t35cEF4e6M

Personally, I agree with most of the ideas from RST. I truly believe the future of the industry would be empowering testers as high skilled professionals, and relying on automation for the most repetitive, tedious and error prone tasks. Some ideas resonate with the Quality Assistance approach as well (although it relies on teaching the developers how to be a great tester, so it can get feedback sooner).

I also find Exploratory Testing as one of the most challenging intellectual activities I’ve ever practiced. It allows me to use my knowledge of the product, the technologies involved, how the team works and my guts to dive deep in a product and get a story about the current state of its Quality. Its heuristic techniques and tips have enhanced my exploratory testing exercises, as well as making me realize some potential scenarios while developing automated verifications.

There is a lot of documentation online about this methodology (way more than we can tackle in one blog post), so I’ll try to follow up the subject on following entries. It is going to be a subjective interpretation, what I’ve learned from it and how I apply it. Feel free to use the following resources to learn about it, although it heavily relies on practice so I encourage you to give it a go. Luckily, you don’t have to change neither the process or the technologies, so you won’t need any management approval to start applying this methodology!

I want to finish with some quotes from James Bach about Software testing, life, and everything else.

“In general it can vastly simplify testing if we focus on whether the product has a problem that matters, rather than whether the product merely satisfies all relevant standards”

“That’s what training testers is all about. Not only have these ideas, but to explain and defend these ideas. “

“A lot of people think ‘Oh testing is all repetition’. It is kinda like sex, if it’s not fun, you’re not doing it right. You’re doing it wrong. If you approach testing not as the most challenging intellectual exercise you’ve ever done in your life, you don’t understand testing. Because that’s what it is.”

I hope you’ve found this as interesting as I do, and rest assured that I’ll dig deeper on the subject. That’s the best way to force myself to summarize the information I know, and it is an invaluable way to gather some precious documentation for the future.

May the force be with you,
Gino

Resources: