Tag Archives: team

Experiences Of a Testing Bug Bash

I’m working in a team, that’s developing a mobile application and back-end, that does not have any testers. We didn’t plan it that way, but temporarily, that’s just the way it is. While I can add my testing skills and experience to the team, that’s not the primary reason I’m there, and so I’ve been looking at ways in which the whole team can help to contribute to a quality product.

It’s been interesting. At first, when our tester left then nothing happened. Like nothing at all, tickets moved over to the ‘Ready for Testing’ column and there they stayed. Immobile, ignored and unloved. It was as if all eyes were to the left, where the fun stuff for developers was, understanding designs and coding. Although there are unit tests and some automated integration level tests for our code it clearly wasn’t enough.

Session Based Team Testing – A Bug Bash

I’ve used session based testing in the past when I managed test teams, and have found it to be a very useful way of breaking exploratory testing work into understandable pieces, and distributing that work out to the team. James and Jon Bach explain the approach far better than I can. Ministry of Testing have also got an excellent resource on SBTM.

Having read some articles from people who had broadened their approach then I thought I’d take the session based testing concept and roll it out to the whole team as a bug bash.

The key point for me was that any sort of session had to be fun. Initially I feared that not all the team were going to see the full value of such a session since it wasn’t development work per se,  so keeping the approach light-weight was key. The ideas for our first bug bash had formed.

Organisation

Getting away from the everyday work was important so the bug bash took place in a separate room, close enough to our team area that it was quick to pop out and pickup forgotten devices, etc but away from the interruptions that phones, etc provide. I made sure we had enough phones, tablets, post-it notes, pens and cake to keep us going. I also brought in some popular testing books and the Test Heuristics Cheatsheet from Elizabeth Hendrikson, in case people needed some inspiration.

We use JIRA as our project management tool and since we were testing a whole Epic then it made sense to break the testing down on a story by story basis. Working with our lead developer, we took each Epic in turn and used that to form an exploratory testing charter stored as a Google doc. You can get an example version here. Using charters enabled us to define the boundaries of the testing; ensuring there was not too much overlap. It also meant that people who were not so familiar with the story were able to quickly learn about it, and areas to test, without needing to spend time reading stories and Epics in JIRA.

So, the charters were ready, the room was ready and the people were ready. Time to start.

The Bug Bash Session

We decided on two 45 minute sessions with a tea break in between. The whole team attended; PO, designers, UX, developers and myself. Everyone was a tester for the afternoon. We agreed who would pickup which charters and made a start.

I made a Google doc in which people could record potential issues which I kept projected up on the wall throughout, so that we could all see the progress that we were making and the issues that were being raised. I encouraged the team to add screenshots as well as text and this really helped to make things clear and to prevent duplicate issues making their way into the list. Discussion was also key to the success of the session; having everyone in the room at the same time meant that potential design issues could be shown to designers straight away for review.

The lead developer for the feature was also present, not testing but helping to explain architectural and development decisions, and also to fix some bugs on-the-fly.

The session went well. We got all the charters finished and the session concluded with a quick retrospective.

Retrospective

What Went Well
Charters give structure and prevent misunderstandings

Having structure was critical to the bug bash, given that it was a new activity for the team, and they were not experienced with exploratory testing. It also meant that we got the most from the session and people were not all testing the same thing.

Hints and examples in charters are good

By including examples, and ideas on areas to test, people had a guide to start them off. Given that they were not experienced testers then this really helped.

It is good to have charters split by story

This meant that it was clear what was to be tested and there was a clear link back to the project management tool people were used to.

Some of the bugs were fixed during the session

This clearly shows the value of group pairing for me.

What Could be Improved?
Make the charters simpler

I had hoped we would get some data on setup time and test execution time so had added that to the charters but it confused most people.

Make the process of splitting up the charters and recording who is working on each one, simpler

Although it as clear to them which charter a person was working on, we didn’t keep a visible record of who was working on what. Next time I’ll write this down and pin it up on the wall.

People could swap charters halfway through the session to give multiple points of view

For each 45 minute session one person would pick up and test against a charter. Swapping round could add value and a new set of eyes onto the area.

Comments
Aim to do this for all large stories

This was great to see since it showed that the team saw the value in the approach.

Consider pairing

This will be our next step for the Bug Bash sessions. Pairing people up could help to add more value by giving multiple viewpoints.

Going off charter was a good thing

Some charters were bigger than others. People naturally started to go off-charter once their charter was done and this gave multiple viewpoints across the whole feature.

Being away from the desk is a good thing

By being one team, with one goal, isolated from our day-to-day activities, we were able to effectively test and find many issues in the feature, without the usual distractions that break concentration and flow. Having lots of cake also helped :)

Conclusion

We will definitely do the Bug Bash again, and I see it as becoming a really important part of our delivery processes, even when we get a full time tester back on the team. It really helped everyone to understand the feature, to explore it and to test together, giving multiple viewpoints on potential issues.

With some guidance the whole team made a great contribution. I’d encourage everyone to try a similar approach,

!(Certification) = !(New Job)

There’s a lot said in the testing press and blogs about certification. There’s some well known haters of ISTQB and a few, albeit quieter, exponents. There is of course the training providers shouting loudly about their guarenteed pass rates, how their courses are faster than all the others, and how you won’t survive in testing without the qualifications that you can get from them. Is certification as important as they say? I’m beginning to think that maybe it is, but not for the reasons their sales people present.

Firstly some background. I’m ISEB Foundation and Practitioner certified. I enjoyed the courses which I did with the excellent Grove Consultants a few years ago. OK, the exams were not fun but the courses were. I felt like I learned something and I went along because I wanted to learn. The qualification was good, but secondary. I felt it wasn’t essential. I still feel this way, I’m not an out-and-out ISTQB basher but I feel things are beginning to go too far.

Once I became a team leader, and then a test manager I continued to send people on the courses. Some didn’t want to go, but I felt it was important for them to learn something new, and more importantly to learn the same way, and using the same information, that the rest of the team had already learnt. It gave some consistency. That was useful.

Fast forward a few years. I now have a team of testers and delivery ops people. Times have been hard and training has been hard to come by, by the time these people joined the team there was no training available that would lead to the ISTQB/ ISEB certifications. Has the quality of what we do decreased? Well, no. If anything, we’ve gone out and trained ourselves, trained ourselves, and updated our ways-of-working in even better ways. We are still consistent in our approach, and as a bonus, some people can now train others. Also a good skill. Not getting the ISTQB training has annoyed some, whilst others weren’t bothered at all.

Now my team and I find ourselves in a new situation. Soon we will all lose our jobs as R&D is moved overseas. Suddenly the issue of certification slams itself forward again. Most of the job ads scream ISTQB certified, for recruiters it’s almost the first question asked “Are you ISTQB certified?”. How have we come to this?

I think a lot of the testing community is stuck in a vicious circle. If we get lazy with our recruitment then we quickly fall into a trap of just putting “ISTQB certified” in the “Essential Requirements” section of our job ads. We are the ones who caused the recruiters to ask “Are you ISTQB certified?” Certification within the industry becomes self fulfilling. And those of us recruiting testers don’t necessarily get better testers.

So what’s the solution? More certification? I think all those of us who recruit for software testers need to re-visit what we look for in a tester, to adjust our outlook and our requirements so that we are trying to find those who are good at what they do, not what they have studied. A few years ago I used to run a written interview test for candidates which was based on the ISTQB syllabus. Many of those with the qualification failed.

And to my team, without certification and needing to find new jobs? I’ve sent them on ISTQB courses. It’s only fair, they need the best start they can get in their job searches. But if I find myself in this situation again then I hope that it’s not this way….

* For those of you without any programming knowldge – ! in the title means “Not” :)

Image: jscreationzs / FreeDigitalPhotos.net