As usual in November it is ATD time, the agile testing conference takes place every year in November in Potsdam.
This time, Emna and Andrea collaborated and created together a workshop about exploratory testing. We ran the workshop for the first time this year in Potsdam at ATD.
The experience was ace. After months of preparations, countless video meetings and some dry runs finally the day of the workshop came. And we have been amazed by the attendees, their motivation and their engagement contribute even on the last day of an intense conference.
It was wonderful to experience that the attendees and we share the same passion for exploratory testing. We had great conversations with the participants of our workshop and it was simply amazing to see how everyone at their tables started collaborating in the activities. Listening to their chats, observing their commitment and overhearing the laughter was worth the evenings and weekends of preparation.
Questions we received from attendees
By the end of our workshop we asked for feedback and part of it have been some questions from the attendees. Those, we want to address in our blogpost.
- How to link requirements to Exploratory testing ?
Emna:
Requirements are certainly translated into detailed user stories by your product owners and at every sprint you have a backlog of User Stories.
We consider that ET could kick off at an early stage by visualizing those User Stories
since the reception of those user stories, where you as a tester you can challenge the product owner on different criteria and discuss how feasable
Andrea:
Domain/SUT linking:
Depending on how broad or detailed the requirements are. A requirement can be a good source to provide you with the <Explore> part of a test charter. You could use a particular requirement as the target of your exploration. While the different values, states, etc. can be utilized as variables during exploration and serve therefore the <With> section in the test charter.
Technical linking:
If your team uses some sort of management tool to organize the work, like Jira, AzureDevOps, or others, you could attach your test notes from your exploratory testing session to the specific story. Test notes can vary, it can be written notes, recordings of the test session, or a simple report that is attached and shows the team what has been tested and what information has been revealed.
- Can you give us an example of a good charter ?
A good charter offers direction without over specifying test actions.
In other words:
- They are not too specific, to spent time on test documentation with very little benefit and at the same time
- They are not too broad, to provide enough focus.
A good charter is a prompt: it suggests sources of inspiration without dictating precise actions or outcomes
A great source to learn more about writing test charters is the book “Explore it!” by Elisabeth Hendrickson.
In our workshop, many dedicated testers have written excellent test charters. We would like to share some of them. To give ideas and inspiration on what we regard as a good test charter. We used in our workshop a pizza order app and that are some of the test charters which have been created in that process:
Charter 1
Explore different pizza customization options
As a person with gluten intolerance
With different topping and dough options
To discover if the app is accessible for people with food restrictions.
Charter 2
Explore the minimum value pizza for an order
As a poor student
With downgrading all optional items and applying only included respective free options
To discover what is the lowest possible order amount.
Charter 3
Explore the order for many of pizzas
As a party host
With different pizza selections and options for a high number of pizzas and time specifications
To discover if big orders can be delivered at a specific time.
Charter 4
Explore the pizza compositions
As a hungry ninja turtle
With different topic selections
To discover if any selection, even weirds ones, are possible
Charter 5
Explore the order information
As the owner of the pizzateca
With unclear, incomplete, or non available ingredients
To discover if the contact details are saved, correct, and useful
- How many CHARTERS can we make ? Is it dependent on priority, number ?
Emna:
I’d say it depends on many factors, and I don’t recommend measuring charters by number. I would recommend asking if the created charters cover different risks related to that user story ?
You can think of different charters, then depending on the possible risks and the given time you can choose the most critical charters to execute
Andrea:
There are no limitations on how many charters you can create. The question is, will each charter provide you with useful information that will help the team to deliver the software. Personally, I would start with a small number of the most relevant charters first. After running a session there is always the possibility to come up with another charter to explore aspects further that have been revealed but not discovered during the session.
- Are there guidelines and tips to create useful charters?
No, there are no guidelines on what a test charter has to look like. There are different ways of noting down test ideas. Like a simple list, just containing a header of what shall be explored, creating mind maps, some testers like to use testing tours. That said, over time some approaches are referenced and used regularly among the testing community.
We like to use the template that is promoted by Elisabeth Hendrickson and which we also applied in our workshop. In our experience, this works very well.
In order to create a good charter, our recommendation is to focus on every element in the charter wisely, below some reflection and what need to be written
- <Explore> Target : Define your target, and be aware which area you are going to explore? it’s important to know where you have to focus to avoid interruptions
- <As a> Define for which persona perspective the exploration will be executed
We recommend using persona cards that you can design with your team before the session. Think of main motivations, and possible frustrations that this person could face and also how comfortable are with technology? - <With> Brainstorm possible resources that could be used during the exploration, with resources we are refering to different variables (including possible permissions, sequences of actions, size of data) or heuristics (checkout this link to get more ideas) that could add more value to your testing, you can also apply testing techniques to the variables you defined for better coverage
- <To Discover> Think about the purpose of your exploration, for example, your goal to find a situation where the product you are testing is not performing enough or is not secure enough, or the error messages are not as expected, you can define what kind of discovery you want to focus on?
Another tip to create charters is to practice writing them. With more training it will be easier to come up with them and you will develop a better understanding if a charter will be useful and relevant for the application you want to test.
Thank you for reading this blog post, let us know in the comments if you have any other questions related to Exploratory Testing.
Subscribe to not miss future updates
Emna Ayadi 