I consider Exploratory Testing as a adventure combined with learning and fun the tester or any team member can perform while navigating the software they are testing. But what that’s mean ?

Let’s first discover what is Exploratory Testing ?

There are interesting defintions that I want to share with you

Simultaneously designing and executing tests to learn about the system using insights from the last experiment to inform the next

Elisabeth Hendrickson

Exploratory testing is simultaneous learning, test design, and test execution

James Bach

“Exploratory software testing is a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”

Cem Kaner

It has the purpose to uncovering informations related to risks

unknown

How to use it effectively in practice ?

Exploratory Testing known also as a mixure between structure and unstructured approaches, which means that you need to define your scope, your objectives what you need to perform so that you welcome new ideas that could attract and show up hidden bugs. in other words, a way to reveal more informations that scripted test approach can’t do it, because you are not biased by the test case steps. This is where the magic start when you discover something completely outside the box.

To do that, think about those 4 elements and elaborate everyone of them:

  1. What is the purpose of your testing ?
    Define your MISSION what you want to test !
  2. How are you going to reach your objectives and explore the areas you want to ?
    Prepare your CHARTER, it’s like the plan to execute your mission
    “A charter is a component of session based testing that precisely and concisely describes what you are looking for in your test session.“

    You can prepare different charters and add them to the team backlog, it could be performed as an activity between different programming tasks so that it’s even an opportunity where developers can explore.

    A charter could be desinged this way (see next section for more details how to fill your charter)
    • <Explore> (Target : what you are exploring?)
    • <As a> (Define the personas)
    • <With> (Resources: what are you using in your exploration?)
    • <To Discover> (Information: what’s the purpose of your exploration ?)
  3. Time box your exercice and describe your findings
    Set a SESSION for that, fix a time 45min, 1h ..
    Record important informations such as
    • How you are testing?
    • What data, files, calls, personas … you are using or you created specially for this session ?
    • Bugs, problems, new ideas that comes to your mind (maybe useful for future charters), you will also have some questions to your team note all of them, and ask them after the session.
    • Time spent in testing and general time of the whole session
  4. End your session by discussing your report of findings with your team. Set a DEBRIEF.
  • Past ——— What happened during the session? 
  • Results —– What was achieved during the session? 
  • Obstacles — What got in the way of good testing? 
  • Outlook —- What still needs to be done? 
  • Feelings —- How does the tester feel about all this? 

Ideas to Strengthen Your Charters

You can prepare different charters and add them to the team backlog, it could be performed as an activity between different programming tasks so that it’s even an opportunity where developers or other team members can explore.

A charter could be desinged using the template described in previous section, my favorite part when it comes to <With>(Resources what is your toolbox?) and also <As a> (when I think about different personas that could be relevent to your context or maybe irrelevant like hackers and try to see what can go wrong in your app from that angle of users).

In this section, I’ll focus only into resources and personas could be treated in another future blog:

  • <With> (what are you using in your exploration?)
    Identify different variables that could add value to your testing :
    • Data fields/values (e.g. name)
    • Data size
    • Data format or encoding
    • Count of items (eg. Invoices)
    • Configurations
    • Position or location
    • State of entities of processes
    • Attributes on an entity
    • Sequences of actions or events
    • Methods of input
    • Response latency (network or user)
    • Contents / size of a  file
    • Number of files
    • Storage location
    • Preferences & configuration
    • Runtime environment
    • Permissions
  • Identify useful heuristics that bring new ideas
    A « Heuristic » is a problem solving strategy, Test design techniques give you a mental tool chest of heuristics, you can use to explore for risk. Examples (through testing techniques) :
    • Boundaries and constraints
    • Values that are too big, almost too bug, too small, almost too small, just right values
    • Times or dates are too early, too late, exactly on time (date)
    • Strings that are too long, too short, exactly the limit
    • Strings that include special characters (*,%,&, »,é,’, »,⍷,⍸,{,},/,\,⍹, (,),@,!,⍺,⍻,⍼,⍽,⍾,⎋) more Special characters full list: Fancy Text (fancy-text.net)
  • Consider your variables, heuristics and think about applying relevant testing techniques to play with those values !
  • Conduct a mini experiment in this phase you navigate through infinite possible variations such as
    • Use the state of an event “While the system is importing/exporting data…”, “While the server collects usage data, while the server is loading …
    • Interrupt it “Clicking user controls” (try “cancel” buttons for example), wait for session time out, Disconnecting the machine from the network, close everything and reopen …
      After you interrupt the state, consider these questions:
    • Did the software respond to the interruption the way you expected? Or is it now in an unexpected state?
    • After you recover from the interruption, is the behavior restored to normal?
      • Did the software lose or corrupt any data?
    • Experiment with sequences Reverse the order of steps, skip some steps, use a different order, stress the use of one step then move normally to the next, undo / redo some steps ..
      • Your experiments give you empirical evidence about the capabilities andmostly the limitations of your software that you need to know ! Along the way, you uncover new questions needing answers and you plan for additional types of tests.
    • Observe Use all your resources (senses, tools, logs, consoles, monitoring tools…) and capture your learning (the finding of your charter execution that I described in the SESSION (3.) )
      • Take notes when you are exploring (record how you tested, data you used)

      • Take screenshots when necessary

      • Reflect on patterns

      • Share your observations with others (Bugs/issues/problems/questions)

What are the advantages from using Exploratory Testing ?

  • Accelerate the onboarding of new memebers, they will learn faster about the product from their first session with experienced team members, so it’s also a great way to collaborate together as a team by running an ensemble testing (great writeup by @maaretp to set up your ensemble testing How to Enable Better Habits and Skills). I consider every exploratory testing session as a learning experience !
  • Effective time consumption: you don’t need a lot of preparation in advance except light ones (clear objectives, charters), you won’t spend a lot of time reading theorical documentation, you can make your session timeboxed.
  • Engaging activity that simulate your brain wisely and morivating when you see how many creative ideas you got while exploring.
  • Adapt it as an activity to special needs, Exploratory Testing can be structured enable focusing on specific risks, if you have an urgent request to test a certain part of your software or a request from your client to test a new area, you can run a session and have helpful outputs for decision making regarding possible risks.
  • A smart way to find defects beyond the specification.

Thank you for reading, I’m curious to know what other content you would like to see in my blog ? I’m planning to write how I explored APIs and describe my interesting findings in a future blog post. #StayTuned !

A special thanks to Elisabeth Hendrickson for making this video Explore it ? Explore it ?!, I used it as a great reference for this blog !

Other resources about Exploratory Testing

People you should follow to continuously learn about Exploratory Testing

Checkout this thread, where you can find very passionate exploratory testers, they tweet, blog or speak regularly about exploratory testing. I can guarantee new learning from them: