I’m working in a team, that’s developing a mobile application and back-end, that does not have any testers. We didn’t plan it that way, but temporarily, that’s just the way it is. While I can add my testing skills and experience to the team, that’s not the primary reason I’m there, and so I’ve been looking at ways in which the whole team can help to contribute to a quality product.

It’s been interesting. At first, when our tester left then nothing happened. Like nothing at all, tickets moved over to the ‘Ready for Testing’ column and there they stayed. Immobile, ignored and unloved. It was as if all eyes were to the left, where the fun stuff for developers was, understanding designs and coding. Although there are unit tests and some automated integration level tests for our code it clearly wasn’t enough.

Session Based Team Testing – A Bug Bash

I’ve used session based testing in the past when I managed test teams, and have found it to be a very useful way of breaking exploratory testing work into understandable pieces, and distributing that work out to the team. James and Jon Bach explain the approach far better than I can. Ministry of Testing have also got an excellent resource on SBTM.

Having read some articles from people who had broadened their approach then I thought I’d take the session based testing concept and roll it out to the whole team as a bug bash.

The key point for me was that any sort of session had to be fun. Initially I feared that not all the team were going to see the full value of such a session since it wasn’t development work per se,  so keeping the approach light-weight was key. The ideas for our first bug bash had formed.

Organisation

Getting away from the everyday work was important so the bug bash took place in a separate room, close enough to our team area that it was quick to pop out and pickup forgotten devices, etc but away from the interruptions that phones, etc provide. I made sure we had enough phones, tablets, post-it notes, pens and cake to keep us going. I also brought in some popular testing books and the Test Heuristics Cheatsheet from Elizabeth Hendrikson, in case people needed some inspiration.

We use JIRA as our project management tool and since we were testing a whole Epic then it made sense to break the testing down on a story by story basis. Working with our lead developer, we took each Epic in turn and used that to form an exploratory testing charter stored as a Google doc. You can get an example version here. Using charters enabled us to define the boundaries of the testing; ensuring there was not too much overlap. It also meant that people who were not so familiar with the story were able to quickly learn about it, and areas to test, without needing to spend time reading stories and Epics in JIRA.

So, the charters were ready, the room was ready and the people were ready. Time to start.

The Bug Bash Session

We decided on two 45 minute sessions with a tea break in between. The whole team attended; PO, designers, UX, developers and myself. Everyone was a tester for the afternoon. We agreed who would pickup which charters and made a start.

I made a Google doc in which people could record potential issues which I kept projected up on the wall throughout, so that we could all see the progress that we were making and the issues that were being raised. I encouraged the team to add screenshots as well as text and this really helped to make things clear and to prevent duplicate issues making their way into the list. Discussion was also key to the success of the session; having everyone in the room at the same time meant that potential design issues could be shown to designers straight away for review.

The lead developer for the feature was also present, not testing but helping to explain architectural and development decisions, and also to fix some bugs on-the-fly.

The session went well. We got all the charters finished and the session concluded with a quick retrospective.

Retrospective

What Went Well

Charters give structure and prevent misunderstandings

Having structure was critical to the bug bash, given that it was a new activity for the team, and they were not experienced with exploratory testing. It also meant that we got the most from the session and people were not all testing the same thing.

Hints and examples in charters are good

By including examples, and ideas on areas to test, people had a guide to start them off. Given that they were not experienced testers then this really helped.

It is good to have charters split by story

This meant that it was clear what was to be tested and there was a clear link back to the project management tool people were used to.

Some of the bugs were fixed during the session

This clearly shows the value of group pairing for me.

What Could be Improved?

Make the charters simpler

I had hoped we would get some data on setup time and test execution time so had added that to the charters but it confused most people.

Make the process of splitting up the charters and recording who is working on each one, simpler

Although it as clear to them which charter a person was working on, we didn’t keep a visible record of who was working on what. Next time I’ll write this down and pin it up on the wall.

People could swap charters halfway through the session to give multiple points of view

For each 45 minute session one person would pick up and test against a charter. Swapping round could add value and a new set of eyes onto the area.

Comments

Aim to do this for all large stories

This was great to see since it showed that the team saw the value in the approach.

Consider pairing

This will be our next step for the Bug Bash sessions. Pairing people up could help to add more value by giving multiple viewpoints.

Going off charter was a good thing

Some charters were bigger than others. People naturally started to go off-charter once their charter was done and this gave multiple viewpoints across the whole feature.

Being away from the desk is a good thing

By being one team, with one goal, isolated from our day-to-day activities, we were able to effectively test and find many issues in the feature, without the usual distractions that break concentration and flow. Having lots of cake also helped 🙂

Conclusion

We will definitely do the Bug Bash again, and I see it as becoming a really important part of our delivery processes, even when we get a full time tester back on the team. It really helped everyone to understand the feature, to explore it and to test together, giving multiple viewpoints on potential issues.

With some guidance the whole team made a great contribution. I’d encourage everyone to try a similar approach,