Category Archives: Testing

My Experiences of WeTest 2016

tl:dr;

I really loved being a part of WeTest 2016. I gave the opening keynote at both conferences, as well as a couple of other talks at sponsor events. If you came to any of them then this post contains some useful links you may want to read. It was great to meet so many engaged and knowledgeable testers in New Zealand.

The Opening

I was thrilled when I was offered the opportunity to travel over to New Zealand and talk at the WeTest 2016 conferences in Auckland and Wellington. I mentioned my conference speaking goal as part of my presentation at both events – this was that one day someone would invite me to speak in New Zealand. And Katrina Clokie, on behalf of WeTest did just that earlier in the year. It didn’t take long for me to agree to be a part of the conference and being offered the opening keynote was a real privilege.

Now there is one thing that’s clear and that is that it is a long way to New Zealand (28 hours to be precise from the UK). So I wanted to ensure that I made the most out of my short time there and to give a presentation that fitted in with the conference theme of “Influence and Inspire”. Given that I have made a move out of a pure test role over the last few years then this seemed like a logical theme for the presentation, and an opportunity for me to give my views, gained from outside of testing, on how testing is perceived. I also wanted to show the audiences that there’s important leadership roles within testing to be taken and made the most of.

Preparation and Arrival For WeTest

I can say without a doubt that WeTest was the best organised conference that I have spoken at and it’s testament to the effort put in by Katrina, Aaron, Shirley and Dan. All too often presenters at conferences aren’t treated brilliantly by organisers – either by having a lack of support leading up to the event, being expected to pay their own way or by needing to foot the bill for travel and accommodation then claim back. WeTest was different – they organised and paid for all travel up front and sent detailed itineraries for each speaker, plans on how the conference rooms were to be setup, as well as ensuring that the simple things like a taxi to pick you up off a long haul flight was pre-arranged. It made the whole experience easy and it made us speakers feel valued. Other conferences should take note.

A really appreciated gift at checkin. WeTest was a great example of how to run a conference
A really appreciated gift from the organisers, handed out at hotel check in. WeTest was a great example of how to run a conference.

Up and Away?

My presentation focused on my views of testing from outside of just testing and how I made a move into software management and why. It was part of a clearly well thought out programme and the other presentations, whether focused on more technical aspects such as mobile testing, or other leadership presentation, fitted together really well. You can see the programme here to get a feel.

Great sketch notes of my presentation from Yvonne Tse
Great sketch notes of my presentation from Yvonne Tse

Highlights

People arriving for WeTest in Auckland
People arriving for WeTest in Auckland

I thought all the presentations were good but a particular highlight for me was Adam Howard’s “Exploratory Testing Live” where Adam took the really brave step of doing live exploratory testing on the Trade Me website in front of a conference audience. Fair to say it went a bit better in Wellington than Auckland, where he found a bug early on that blocked a lot of the rest of the session but he coped really well and there was value in both sessions. We don’t see enough live testing at testing conferences, it’s common to get live coding at development ones but hasn’t seemed to catch on. Based upon Adam’s session then it should.

Also I really enjoyed Joshua Raine’s really personal story about “Conservation of Spoons” which he did as a noslides presentation. Deeply personal at times and spell binding. I won’t give you the details because I’m sure he’ll do this one again and to know the story would spoil it.

A New Approach to Q&A

As part of my presentation I thought I’d try out the new Q&A feature in Google Slides. It turned out that this was a good move because it enabled me to get questions from the audience as I talked and also to keep a record of all of them for later. If you speak at conferences I’d really recommend it.

For those of you who attended then here’s the questions asked, my responses, and a little more information.

What do you most miss about your testing focused roles?

I guess I miss being the expert with the safety of many years of experience of testing. I also make sure I keep connected to the testing community because it’s great and if I didn’t then that would be the biggest thing that I’d miss.

It’s hard to find a balance between adapting and burning out when trying keep up with new trends and technologies. Do you have suggestions on how to manage that balance?

I make sure I do one thing well; that together with Jerry Weinberg’s Lump Law – “If we want to learn anything, we mustn’t try to learn everything” help me to maintain a balance. Trying to find the one thing to focus on is sometimes hard, but by discussing with your stakeholders and your team then you can usually come to find out what is the most important.

Do you have any resources/suggestions for learning to speak ‘Dev’?

There’s not one clear answer to this – it depends on your context, languages and architecture that you are dealing with. When I started then I tended to use google a lot and base my searches on discussions I’d had with the dev’s in my team. I’d also ask them what they thought I should be learning and why. I’ve also found sites like Hacker News and Stack Overflow to be great for keeping up with what’s happening. Talking Code also comes recommended.

Was your career path in the same company or different? Would you suggest to change companies or progress in one?

It was in the same company and I think it’s much easier to make a change from one path to another when you are in the same company. You have that reputation to use and you are a known quantity. It’s also likely that you’d been working towards such a change as part of your personal development anyway so the company will be confident in your plans.

Since we are talking about change, at any given point during those career changes did you resist the change? How did you over come that? Were you specifically looking for change?

I talked about how it can be scary to make changes but trying something new for 6 months is key. You need to understand that you will go through a change curve just like anyone else and things will feel extremely uncertain as a result at the beginning. I knew what to expect since I’ve been through other changes.

In this case I wasn’t specifically looking for change, more for an opportunity.

Thank you for the nice talk If you had to recommend doing ONE PRACTICAL exercise to raise awareness with the team that WE ALL own quality – what would it be?

I like the idea of whole team testing and bug bashes are a great way of starting.

What do you think testing will be like in 10 years time?

I think we’ll see less traditional testing roles and the focus on automated checking will become more of a development activity. This may mean that there are less testing roles in total but good, exploratory, coaching testers will always have a place in teams. I think the biggest change will affect Test Managers, with far less of a need for them and a transition to coaching roles.

How do you avoid being typecast as just being a tester and overlooked for other career opportunities?

Show that you have an interest to others in your company. Work with your manager to map out your path to a new role and start to show that you are learning the skills that will be required in order to be successful at it.

In short – you own the responsibility for your own career.

A lot of people in this company are transitioning from Waterfall to Agile. Some of the people in the room might gain from you talking to how you see that transition working from the experience you bring, especially if they’re concerned about career development and management.

It does depend a lot on how the management structure adapts. In agile transitions that I have been a part of we’ve changed to autonomous, cross-functional teams with single managers, who have been supported by coaches for both agile, testing and development. This support network is key, as is the establishment of discipline based communities. A strong testing community for example, allows testers whose roles are changing to adapt.

I’ve spoken before about the effect of removing Test Managers from an organisation and the biggest takeaway for me was the need for a strong support network for testers afterwards.

Support from coaches allows not only testers to be supported but also means that someone with testing experience becomes responsible for establishing career paths, competency expectations, etc that can then be used by other managers who are not experienced in managing testers. It also helps maintain a ‘voice of testing’ towards senior management and to enable activities that help testers grow within an organisation.

And Repeat…

Parimala Hariprasad is full flow!
Parimala Hariprasad in full flow!

The first conference was in Auckland and we repeated the experience again in Wellington three days afterwards. This meant that the whole WeTest circus upped sticks and set off for Wellington for a repeat performance. I also did an “Understand Your Mobile Users” presentation at a sponsor the night before, plus the same WeTest presentation at a sponsors internal conference the day afterwards. Four presentations in one week was something new to me and actually pretty interesting to do as I got into the speed of things. It almost felt like being in a band on tour 🙂

Closing Thoughts

WeTest was great. I met some great people and learnt some new things. It was extremely well run by a passionate bunch of volunteers who knew how to treat their speakers well and how to organise a great programme for the conferences. The testing community in New Zealand is really engaged and it’s clear that there’s some really forward looking testers pushing the boundaries here. If you get yourself over to New Zealand then I’d certainly recommend it. It’s not that far. Really 🙂

Technical Mobile Testing At The Test Masters Academy Masterclasses

Technical Mobile Testing

I’m really excited to be partnering with Richard Bradshaw, a.k.a. Friendly Tester for some Technical Mobile Testing tutorials this year. The first one will be in New York as part of the Test Masters Academy Masterclasses on 25th and 26th April.

Technical Mobile Testing

Technical Mobile Testing builds on the Mobile Testing tutorial I’ve taught over the last couple of years, and is aimed at those who want to go deeper into mobile testing and get more technically focused.

If you have found yourself testing on mobile recently, you have probably considered or tried introducing some automation or tools into your testing efforts. You are probably thinking along the lines of, how can I make this easier? But it can appear a daunting task; there are so many frameworks and so many tools out there, so where do you start? In the tutorial we will try and help you answer that question.

What The Tutorial Covers

In this tutorial you will pick up useful hints and tips, learnt from within the industry. We plan to cover the following areas over the two days.

  • The Mobile Market and How It Affects Testing.
  • Why Get More Technical and How To Start.
  • Using Chrome Developer Tools and Safari Web Inspector to test mobile websites
  • Utilising XCode / iOS Simulator.
  • Android Virtual Devices, Emulators and Genymotion.
  • Getting the most out of Android Debug Bridge (ADB).
  • How to utilise proxying when testing mobile.
  • Recording your testing from the device.
  • Using GUI automation frameworks available.
  • Creating some GUI automated checks using Appium.

We’ll look at how to use simulators and emulators, simulate networks, fake locations and a whole lot more. You’ll pick-up tips on how to use the developer tools and SDKs, build apps, deploy apps to devices and view and change the network requests that apps make.

If you come along then you’ll leave with the knowledge needed to get far more technical with mobile and a great toolbox of hints and tips and hand-ons experience.

Coming Along? – What To Bring

We’ll be using the common SDK’s for iOS and Android so you’ll need to install XCode and Android Studio, and have Chrome and Safari installed. We’ll help you with the setup and get you started.

And bring devices. Lots of devices 🙂

Stay For The Conference

Test Masters Academy have a great line-up of both tutorials but they  also have a one day conference on 27th April. Have a look at the line-up – there’s some really interesting stuff on the program.

Hope to see you there.

State Of Testing Survey 2016

Update: The survey is now live. I’ve just completed it, hope you will too.

The guys over at QA Intelligence and Tea Time with Testers are running The State of Testing Survey 2016 again this year.

A survey which allows us to get a wider view of our profession and our community can only be a good thing in my book. It can help us understand our joint challenges far better, and to set the future direction. I’ll be a part of the survey and I hope you will be too.

The survey is not currently open but it will be soon. You can find out more information at the QA Intelligence blog.

Note: I’m not affiliated with the survey or those running it, I think the survey is a good idea so I’m supporting it.

Talking About Testing

Have you ever said “I’ll just have a play with the software….”?

As testers we are generally really bad at explaining what we do and the value it brings. In fact we are usually pretty rubbish.

This makes stakeholders take us less seriously, and can affect career prospects, position within the team, or even a job itself. Outsourcing what is perceived to be low skilled work is tempting, especially when times get tough.

We then complain that we are not being taken seriously, and we feel ignored, undervalued and sad.

And so we retreat into our bubbles and the whole thing repeats itself.

So How Can We Change This?

We can get better at explaining what we do and the value we bring as testers.

Keith Klain sums it up nicely here

What is Testing?

“Testing is the infinite process of comparing the invisible to the ambiguous so as to avoid the unthinkable happening to the anonymous”

James Bach

A great definition but not one that lands well with non-testers in my experience. It’s like when myself and Dan Ashby tried to get a company to stop saying ‘manual testing’ and instead say ‘sapient testing’. It made perfect sense but it just didn’t land. People didn’t see it as important. They didn’t see a need to change because we were trying to take a leap that was too large, from what they thought was correct language to what we thought was correct language.

When I’m explaining testing I prefer to start by asking “Do you care about quality?” The usual answer is “of course”, in which case I can then use the Weinberg/ Bach/ Bolton definition:

“Quality is value, to some person who matters”

“So what’s testing then?” they will ask.

Well.  “Testing helps us uncover risks to product quality. It’s about investigating software, in order to discover those risks, enabling others to make decisions about whether it’s suitable for release”.

We Need to Talk More Technically

But that is hard. Just see how complicated it is to explain the Heuristic Test Strategy Model in two minutes for example.

(OK – I know the whole point with this video is that it is impossible, which proves a point I think :))

What Value Do We Bring?

When explaining value, it’s all about the words we use, and the angle we take. It’s about the audience – don’t explain testing to a developer in the same way as you would to the CTO.

Again, Keith Klain nails it with this talk.

So, think very carefully about how you explain your testing. Perhaps, just perhaps, ‘playing with the software’ isn’t what you mean.

So – how do you talk about testing?

A Mobile Testing Cheatsheet from EuroSTAR 2015

I was lucky enough to be at EuroSTAR 2015 last week. As well as presenting my experiences and thoughts about the future of Test Management, I also spent most of my Wednesday in the rather excellent Test Lab.

We found lots of bugs in the DrinkedIn app during the Mobile Testing sessions that I ran. The first session was for beginners and to help us along I prepared a mini cheatsheet for mobile testing, listing some popular mnemonics and heuristics to use. I thought everyone might find it useful – so here it is.

Mobile Testing Cheatsheet – Stephen Janaway

Comments, suggestions, etc are most welcome.

Teaching Mobile Testing in the EuroSTAR 2015 Test Lab

I was lucky enough to be at EuroSTAR 2015 last week. As well as presenting my experiences and thoughts about the future of Test Management, I also spent most of my Wednesday in the rather excellent Test Lab.

testlab4

If you go to EuroSTAR, or any other conference for that matter, and you don’t take the opportunity to do some testing, and talk and learn from people whilst doing so then I think you’d be really missing out. The Test Lab is a great place where you can just hang out with other testers, speakers and guru’s. It makes everyone really accessible. You can learn a lot.

testlab1

You can also get extremely frustrated trying to figure out the various black box machines, other testing challenges and even how you might go about firing a rocket from a tank.

testlab5

I did a couple of Mobile Testing sessions, the first for beginners and then the second one as a mob testing effort. We found lots of bugs – our app of choice for the testing was DrinkedIn (the drinkers LinkedIn – check, it’s a real thing and available for both Android and iOS :)).

testlab2

I had great fun teaching, and I hope those who came along to my sessions went away knowing a bit more about mobile testing. If you are at EuroSTAR next year then be sure to get along to the Test Lab. Hopefully I’ll see you there.

 

TestBash – The Friendly Conference

It’s taken a while to write this. Not because I don’t have a head full of new ideas from TestBash but simply because it’s taken time to digest everything, put everything into a sensible order in my head, and come to come conclusions about the day.

If you hate long blog post then just read this bit 🙂

The Condensed Version

TestBash is great. It’s one of the friendliest conferences I’ve been to – people talk to each other, most people are really engaged in what they do, and it’s really easy to strike up a conversation with a complete stranger about something that you’ve seen or experienced. You should go if you haven’t already. Plus they video everything (and hopefully it’ll be online here soon).

The Long Version

Still here? Then read on….

I was lucky this year – I had the opportunity to speak again. Last time, in 2013, I was petrified. This time I’d spoken at a few more conferences and so I was altogether more calm and collected. From a personal point of view, one of the main reasons this was great was that it meant I focused more on the presentations that were on before mine. In 2013 I couldn’t tell you much about any of them. This time more of them went into my head 🙂

As usual I tried to write some mindmaps as the presentations took place. One great thing about TestBash is that there is only one track – and this means no difficult decisions about who to see, or annoying clashes. And from a presenters point of view it means you know how many people you will present to, i.e. everyone 🙂

So, here goes, my take on the day and what I saw.

Lean Coffee

If you come to TestBash and you don’t get up early and go to the Lean Coffee then you are really missing out. In fact more this time, there was also bacon 🙂 I love Lean Coffee, it’s a great way for people to get together and talk about what interests the group. We had some great discussion on a variety of testing topics and the whole event was really well attended this year, with lots of tables full of happy testers.

Then it was time to head on into the main auditorium for a day of talks.

 The Rapid Software Testing Guide to What You Meant To Say – Michael Bolton

I started off trying to mindmap Michael’s presentation and then remembered why I don’t try and write mindmaps for Michael’s presentations 🙂 There’s so much information that it’s much better to sit back, watch, and then wait for slides or videos afterwards. In this presentation Michael took us through a whole bunch of statements on testing and software development, then dissected them, put them into Rapid Software Testing context, then reassembled them to show what should have been said. There was some great stuff in there – watch the video from the TestBash website when it is available. If you want to see a very small mindmap then here it is 🙂

What you meant to say

Bug Detection – Iain McCowatt

Iain always gives a good presentation and in this one he took us through bug detection, from the perspective of tacit and explicit knowledge, the work of Harry Collins, and Dual Process Theory. Click on the thumbnail to get a full sized mindmap.

Bug detection

 

Re-running the ‘Are you a mac or a PC? battle … for iOS and Android – Sally Goble & Jon Hare-Winton

Sally and Jon from The Guardian took us through an entertaining presentation about the differences between iOS and Android, and how The Guardian approach mobile testing as a result. There were some interesting points in the presentation, and I really liked the gameification in the presentation as well, even if it did end up as a draw 🙂 Given more time then it would have been great to have seen some more detailed mobile testing information, but it was a fun presentation nonetheless.

Click on the thumbnail to get a full sized mindmap.

Mobile testing

 

What’s In a Name? Experimenting With Testing Job Titles – Martin Hynie

I’d not met Martin before TestBash but got talking to him at the tester meetup the night before. His presentation was fascinating and explained how he had experimented with job titles and the effect that it had on how testing and testers were perceived. The results he presented were really interesting and certainly made me think more about what job titles and names really mean. I’m sure we’d all like to think that being called testers is great and people will always accept the value we know we bring, but perhaps that is not always the case.

I didn’t mindmap Martin’s presentation. It was before mine 🙂

Why I Lost My Job As a Test Manager and What I Learnt As a Result – Stephen Janaway

I spoke about my experiences of losing my job as a Test Manager. and shared my experiences of going through the transition. Being someone who now works in an organisation where there are no Test Managers meant that I was able to give my take on the future of Test Management, a future that I think does not, and probably will not, include the Test Management role in the way that it does today. I spoke about the changes that the organisation made in order to keep a focus on testing; things like test communities, a coaching role, and mentoring.

No mindmap obviously. I was busy 🙂 If you’d like to see the presentation then there will be a video on the TestBash website but why not come and see it live? I’ll be presenting a longer version at both Nordic Testing Days and EuroSTAR this year.

Myths and Legends of Software Testing – Vernon Richards

The legend that is TesterFromLeicester confronted some of the myths that we all experience as part of testing. With added tutu gags. Vernon gave us some great ways of confronting common myths and challenging those who share them. Watch the video – it’s great.

Quality doesn’t belong with the tester! – Maaret Pyhäjärvi

Maaret gave a really interesting presentation on how she has tested in a team with one tester. She gave some great hints and tips, and explained how her approach changed as a sole tester, and how the developers also approached quality as a result.

Getting Rid of Release Candidate Testing – Matthew Heusser

Matt’s presentation really interested me because we’d gone through the same transition in the past year and so I’d seen the arguments for and against. Primarily it was the testers who were more concerned about getting rid of release testing, and so it was interesting to see Matt’s take. I also liked his presentation style, using no prepared slides but drawing them on a iPad as he went. Really unique and it worked.

Click on the mindmap to get a larger version.

Getting rid of release testing

Automation in Testing – Richard Bradshaw

Richard took us all through his experiences in automation. I found it really interesting that he had first tried to ‘automate everything’ before understanding more about the differences between checking and testing, and how automation should be used to add value to software testing, not ‘be’ software testing. I didn’t get the chance to mindmap Richards presentation so take a look at the video when it’s available.

The Art of Asking Questions – Karen Johnson

We finished up the presentations with Karen, who sat on the stage and quietly and confidently gave us some great advice. She explained about how to ask questions, gave us a lot of really useful reading material, and explained in lots of details why asking questions in the right way is so important. You can find her slides on Slideshare and I’d really recommend reading them and following the links.

And the Rest

The 99 second talks were really great as usual. A real mix of topics, delivery styles and delivery. If you haven’t presented before then 99 second talks are a great way to get started and seeing so many new people up there and speaking was good to see.

Overall I love TestBash. It’s a conference that is so friendly. There’s lots of social events, it’s single track which means everyone sees the same presentations, and the lunch really helps to get people together. If you went this year then you’ll know how good it was. If you didn’t then why not? You should go next year. You won’t be disappointed.

You reached the bottom. Here’s your ‘prize’. TestBash makes people feel like this sometimes.

testbash

A Testers Portfolio

portfolio

I’ve been recruiting for both testers and developers recently and one thing that struck me was the difference between the two disciplines when it came to a body of work.

As a developer you are nothing these days without a Github account. It’s expected that you have one, that it has code in it 🙂 and that an interviewer will take a look. So much so that most developers will gladly advertise their account and repo’s on their CV.

In my most recent piece of testing recruitment I was looking for an exploratory tester and a tester with more automation skills. One automation tester was able to give me a link to her Github account; this was the first time this has happened to me when interviewing testers and was a positive thing to see. None of the exploratory testers I interviewed showed me any links to previous work; now this does not necessarily make those testers better or worse of course. However, as Rob Lambert has talked in his recent series of blog posts and e-book about ‘Remaining Relevant’, as a candidate, you need to stand out, and having a body of work can really help.

For testing it’s a bit different.

So this got me thinking – why is it that testers don’t traditionally have a portfolio?

Is it because more testers are 9-5’ers?

There could be an element of truth here, perhaps there are more testers who are less motivated by their work, with testing being a job they have fallen into rather than a career. If one is not motivated by a job then one is far less likely to want to do whatever is required to remain in that career area.

Are developers more motivated to share their work?

Perhaps there’s a bit of pride in sharing something that one has created that testers don’t have so much. For testers involved in automation, particularly framework design, then there’s something to share, but this isn’t the more general case.

Do they have more to share?

I’d wouldn’t say so, it’s just what is shared is different.

Is it more difficult to build and maintain a body of work as a tester?

I think it is. Since testing is much less about creation then it does make things more difficult. But it’s far from impossible.

What could you do if you want a portfolio of work?

Myself and Dan Ashby touched upon the idea of a testers portfolio in the recent two episodes of Testing In the Pub, where we talked about recruitment. The idea is that, as a tester, you have this body of work, either built up through your day-to-day work, or from experiences in the wider testing community, that can really help you to stand out from your peers, and help when you want to make your next career move.

You may be able to directly share what you are working on at work, but this is frequently not the case. Getting involved outside of work can instead be one key way of building up a portfolio. For example you could consider:

Writing for Blogs and Magazines

Maybe your company has a tech blog. Why not write for that? Or magazines, whether printed or online are always looking for articles. Don’t think that you have nothing to write about – all experience is relevant and there’s some great people in the testing community who will help you edit an article. You just need the idea, and an ability to type 🙂

Starting Your Own Blog

Sometimes you just want to publish something, or you want to say something that’s free from any editorial tone. Start your own blog, it’s really easy and free if you use something like Blogger or WordPress. It’s also great for raising your visibility, especially if you get your blog featured on sites like Software Testing Club or Test Huddle. Add it to the directories.

Talking At Conferences and Meet-ups

Have an idea, submit a proposal, get it accepted and then panic 🙂 But seriously, talking at a conference is a great way of sharing your knowledge, raising your profile, and getting noticed. It’s daunting at first, slightly daunting even when you’ve done it before, but also really rewarding. Conference speaking looks great in a portfolio.

Sharing Your Presentations On Slideshare

Whenever you talk, take time to upload your slides to a sharing site like Slideshare. This makes them available to all and makes you far more visible to search engines like Google. Which it turn helps you build a portfolio of work.

Pulling It Altogether

Pulling it altogether means that you build up a single portfolio. This can take place on your own website or blog and these are easy and free to put together. If you don’t mind a few advertisments then a free WordPress site is a great way to start and you can easily use a theme which makes it look more like a website than a blog.

I have also found that about.me is a good place to consolidate your information together and is widely known. Let’s also not forget about LinkedIn, although I find that it’s better to keep my body of knowledge linked to LinkedIn rather than on the site itself, which allows for more flexibility.

Having a personal brand is an often overlooked but very important differentiator when looking for a new role. It won’t make up for a lack of relevant technical or social skills, but it will help you to stand out from other, similar, candidates.

So think about your portfolio. What would you include?

—-

Update

Thanks everyone who has commented, either here or on Twitter. There have been some great suggestions on other ways you can form a portfolio as a tester. For example:

  • Use Github to store useful resources. It doesn’t have to be code. Thanks to Alan Richardson for the link to this pentest example.
  • Test an OS project that is stored on Github – thanks Michael Bolton, and also @xrisfg.
  • Taking part in activities such as Weekend Testing, podcasting, etc – thanks Dan Billing.

And thanks to everyone who commented below. I’m glad people found the article useful.

Experiences Of a Testing Bug Bash

I’m working in a team, that’s developing a mobile application and back-end, that does not have any testers. We didn’t plan it that way, but temporarily, that’s just the way it is. While I can add my testing skills and experience to the team, that’s not the primary reason I’m there, and so I’ve been looking at ways in which the whole team can help to contribute to a quality product.

It’s been interesting. At first, when our tester left then nothing happened. Like nothing at all, tickets moved over to the ‘Ready for Testing’ column and there they stayed. Immobile, ignored and unloved. It was as if all eyes were to the left, where the fun stuff for developers was, understanding designs and coding. Although there are unit tests and some automated integration level tests for our code it clearly wasn’t enough.

Session Based Team Testing – A Bug Bash

I’ve used session based testing in the past when I managed test teams, and have found it to be a very useful way of breaking exploratory testing work into understandable pieces, and distributing that work out to the team. James and Jon Bach explain the approach far better than I can. Ministry of Testing have also got an excellent resource on SBTM.

Having read some articles from people who had broadened their approach then I thought I’d take the session based testing concept and roll it out to the whole team as a bug bash.

The key point for me was that any sort of session had to be fun. Initially I feared that not all the team were going to see the full value of such a session since it wasn’t development work per se,  so keeping the approach light-weight was key. The ideas for our first bug bash had formed.

Organisation

Getting away from the everyday work was important so the bug bash took place in a separate room, close enough to our team area that it was quick to pop out and pickup forgotten devices, etc but away from the interruptions that phones, etc provide. I made sure we had enough phones, tablets, post-it notes, pens and cake to keep us going. I also brought in some popular testing books and the Test Heuristics Cheatsheet from Elizabeth Hendrikson, in case people needed some inspiration.

We use JIRA as our project management tool and since we were testing a whole Epic then it made sense to break the testing down on a story by story basis. Working with our lead developer, we took each Epic in turn and used that to form an exploratory testing charter stored as a Google doc. You can get an example version here. Using charters enabled us to define the boundaries of the testing; ensuring there was not too much overlap. It also meant that people who were not so familiar with the story were able to quickly learn about it, and areas to test, without needing to spend time reading stories and Epics in JIRA.

So, the charters were ready, the room was ready and the people were ready. Time to start.

The Bug Bash Session

We decided on two 45 minute sessions with a tea break in between. The whole team attended; PO, designers, UX, developers and myself. Everyone was a tester for the afternoon. We agreed who would pickup which charters and made a start.

I made a Google doc in which people could record potential issues which I kept projected up on the wall throughout, so that we could all see the progress that we were making and the issues that were being raised. I encouraged the team to add screenshots as well as text and this really helped to make things clear and to prevent duplicate issues making their way into the list. Discussion was also key to the success of the session; having everyone in the room at the same time meant that potential design issues could be shown to designers straight away for review.

The lead developer for the feature was also present, not testing but helping to explain architectural and development decisions, and also to fix some bugs on-the-fly.

The session went well. We got all the charters finished and the session concluded with a quick retrospective.

Retrospective

What Went Well
Charters give structure and prevent misunderstandings

Having structure was critical to the bug bash, given that it was a new activity for the team, and they were not experienced with exploratory testing. It also meant that we got the most from the session and people were not all testing the same thing.

Hints and examples in charters are good

By including examples, and ideas on areas to test, people had a guide to start them off. Given that they were not experienced testers then this really helped.

It is good to have charters split by story

This meant that it was clear what was to be tested and there was a clear link back to the project management tool people were used to.

Some of the bugs were fixed during the session

This clearly shows the value of group pairing for me.

What Could be Improved?
Make the charters simpler

I had hoped we would get some data on setup time and test execution time so had added that to the charters but it confused most people.

Make the process of splitting up the charters and recording who is working on each one, simpler

Although it as clear to them which charter a person was working on, we didn’t keep a visible record of who was working on what. Next time I’ll write this down and pin it up on the wall.

People could swap charters halfway through the session to give multiple points of view

For each 45 minute session one person would pick up and test against a charter. Swapping round could add value and a new set of eyes onto the area.

Comments
Aim to do this for all large stories

This was great to see since it showed that the team saw the value in the approach.

Consider pairing

This will be our next step for the Bug Bash sessions. Pairing people up could help to add more value by giving multiple viewpoints.

Going off charter was a good thing

Some charters were bigger than others. People naturally started to go off-charter once their charter was done and this gave multiple viewpoints across the whole feature.

Being away from the desk is a good thing

By being one team, with one goal, isolated from our day-to-day activities, we were able to effectively test and find many issues in the feature, without the usual distractions that break concentration and flow. Having lots of cake also helped 🙂

Conclusion

We will definitely do the Bug Bash again, and I see it as becoming a really important part of our delivery processes, even when we get a full time tester back on the team. It really helped everyone to understand the feature, to explore it and to test together, giving multiple viewpoints on potential issues.

With some guidance the whole team made a great contribution. I’d encourage everyone to try a similar approach,