Category Archives: team

Exploring An Existing System

Recently I’ve been reading Elisabeth Hendrickson’s excellent book ‘Explore It!’. For anyone who has an interest in exploratory testing it’s a must own. I wish I’d discovered it earlier to be honest, since it gives so many useful hints and tips, as well as confirming that the ways of working that one has chosen are also recommended and used by others.

As well as using it for my own learning, I’ve been slowly going through the book and working out what parts I can use in the regular lunch and learn sessions that I run at work. While we practice exploratory testing, in fact it’s the cornerstone of our sapient testing strategy, there are some areas where I feel we could take approaches that could benefit not only testing, but also the wider business.

Working With Legacy Systems

We frequently work with legacy systems and so one chapter that was of immediate interest to me in Elisabeth’s book concerned exploring an existing system. When one has an existing system to test I find it’s all too easy to become primed by what others have already discovered, and to fall back on existing test cases (whether physically written down or in someone’s head). This can bias you, resulting in less effective testing.

Elisabeth makes the point that an existing system may well be unknown to the tester, but also may well be unknown to the whole team, or at least contain parts that are unknown. While the software fulfils a business need, how it actually goes about doing so may be less clear. That makes it ideal for exploration.

Recon Testing

James and Jon Bach like to call the initial exploration of an existing system ‘recon testing’. I like this term, by taking an initial session to explore and discover the basics of the software under test then one can then plan more effectively for future sessions, and write future charters in order to drive those plans (if you want to know more about the concept of session based testing and how charters fit in with this then have a look at James Bach’s explaination). Recon sessions help to map the territory and give insight.

During a recon session you can learn a lot, but the most important areas to ensure that you have gained insight into are:

  • The ecosystem in which the software under test resides.
  • Touchpoints to other systems.
  • Variables (things we change or can change).
  • Obvious vulnerabilities and potential risks.

Our Training Session

During our training session we carried out recon testing on the Staples ‘Easy’ button, and a standard service bell. This no doubt annoyed those in the room next door :)

Ding ding!
Ding ding!
Easy to test?
Easy to test?

By starting training with something simple, and not software based then it’s easier to pick up and learn the basics of a new technique without bias or the complication of software. We then compared our experiences, testing, (and the charters produced), with those that the Bach brothers produced when they carried out an exploratory testing exercise using the same product. Fortunately for us, their session is available on youtube, so it was an excellent addition to our de-brief. In it they explain the different testing techniques they use, and why. It’s well worth watching.

Enough Recon Testing?

One area to focus upon when conducting recon testing is whether one has conducted enough recon testing. Fortunately ‘Explore It!’ has this covered, recommending you ask yourself questions about the system that you have been exploring. If you don’t understand what the system does, how input and output works or how the environmental configuration affects the system for example, then it’s probably time to think about more recon before you move into more focused test sessions. Fortunately for the ‘Easy’ button and bell testers, (and those sitting near the meeting room where we had the lunch and learn), then this was not necessary :)

A Useful Addition

Recon testing is something that I think we’ll find is a really useful addition to our strategy. I’d certainly recommend that you check it out, and that you check out ‘Explore It!’ which contains much more useful information and techniques to use in your exploratory testing. I’ll certainly be using the book to inspire some future training sessions for the team.

Incentivising Community Engagement?

I firmly believe that in order to be as effective as possible, testers need to engage with the software testing community. Learning from others, particularly outside of the companies where we work, makes us more rounded and better informed individuals. It enables us to inspire ourselves and our colleagues in ways that we could not otherwise.

Recently I’ve been wondering why more people do not engage with the community. What is stopping them, and how can we all help change this? We can explain how brilliant the wider community is, and we can give examples from our experience. We can send people to conferences and email round blog posts. What is that does not work?

What do we do about those within a team who do not want to interact? Those who do not see it as a good use of their time, and are not willing to spend time on community matters, even if that time is given to them by the company. Should we incentivise people to do so? At least in order to push them in the direction of the wider testing community, where hopefully they will get hooked? Or should we do the opposite? Is it a valid idea to make community engagement a part of people’s role description, and therefore penalise those who hold such positions and do not exhibit such engagement?

Or is there another way of persuading everyone that the software testing community is key to their personal development? I’d be interested to know what you think.

Testing As An Activity

Recently I’ve started to come to the belief that we can solve a lot of our problems if we just start to think differently about testing. If instead of thinking about software testing only as a distinct discipline, we instead to start to think about it as an activity. After all, testing is just that, an activity. It’s something we do. Something we’d love others to do more. James Bach likes to define software testing as a performance, and what is a performance without some activity to perform?

Once we start to think of testing as an activity then it matter less that it’s not always testers who do it. Everyone in a team should test, it’s just that the tester role can be where the expertise lies, and where the test coaching comes from.

Think of all the problems we can solve if we think like this.

!(Certification) = !(New Job)

There’s a lot said in the testing press and blogs about certification. There’s some well known haters of ISTQB and a few, albeit quieter, exponents. There is of course the training providers shouting loudly about their guarenteed pass rates, how their courses are faster than all the others, and how you won’t survive in testing without the qualifications that you can get from them. Is certification as important as they say? I’m beginning to think that maybe it is, but not for the reasons their sales people present.

Firstly some background. I’m ISEB Foundation and Practitioner certified. I enjoyed the courses which I did with the excellent Grove Consultants a few years ago. OK, the exams were not fun but the courses were. I felt like I learned something and I went along because I wanted to learn. The qualification was good, but secondary. I felt it wasn’t essential. I still feel this way, I’m not an out-and-out ISTQB basher but I feel things are beginning to go too far.

Once I became a team leader, and then a test manager I continued to send people on the courses. Some didn’t want to go, but I felt it was important for them to learn something new, and more importantly to learn the same way, and using the same information, that the rest of the team had already learnt. It gave some consistency. That was useful.

Fast forward a few years. I now have a team of testers and delivery ops people. Times have been hard and training has been hard to come by, by the time these people joined the team there was no training available that would lead to the ISTQB/ ISEB certifications. Has the quality of what we do decreased? Well, no. If anything, we’ve gone out and trained ourselves, trained ourselves, and updated our ways-of-working in even better ways. We are still consistent in our approach, and as a bonus, some people can now train others. Also a good skill. Not getting the ISTQB training has annoyed some, whilst others weren’t bothered at all.

Now my team and I find ourselves in a new situation. Soon we will all lose our jobs as R&D is moved overseas. Suddenly the issue of certification slams itself forward again. Most of the job ads scream ISTQB certified, for recruiters it’s almost the first question asked “Are you ISTQB certified?”. How have we come to this?

I think a lot of the testing community is stuck in a vicious circle. If we get lazy with our recruitment then we quickly fall into a trap of just putting “ISTQB certified” in the “Essential Requirements” section of our job ads. We are the ones who caused the recruiters to ask “Are you ISTQB certified?” Certification within the industry becomes self fulfilling. And those of us recruiting testers don’t necessarily get better testers.

So what’s the solution? More certification? I think all those of us who recruit for software testers need to re-visit what we look for in a tester, to adjust our outlook and our requirements so that we are trying to find those who are good at what they do, not what they have studied. A few years ago I used to run a written interview test for candidates which was based on the ISTQB syllabus. Many of those with the qualification failed.

And to my team, without certification and needing to find new jobs? I’ve sent them on ISTQB courses. It’s only fair, they need the best start they can get in their job searches. But if I find myself in this situation again then I hope that it’s not this way….

* For those of you without any programming knowldge – ! in the title means “Not” :)

Image: jscreationzs /

KanBan for Management

Finally, a post on KanBan :)

I’ve been running teams who have been using KanBan for a few years. We find it is a really useful methodology to use, especially for our maintenance and dedicated testing/ release teams. Being a little lighter than Scrum, it enables us to quickly re-prioritise backlog items; ideal when you don’t know when the next Showstopper bug is going to come in.

Recently I’ve started to take things a little further with a pilot in my management team, which consists of test managers, defect managers and product owners who are driving various software projects to completion, mostly in the maintenance and productisation phases. The pilot involves using KanBan to run the team and prioritise the various actions and items that we need to drive forwards.

We wanted a lightweight approach to the toolset, so rather than go for a heavy tool, or something online (company policy – nothing on 3rd party servers), we picked SherePoint. Daniel Roots excellent guide on how to setup a KanBan board in SharePoint has been invaluable and what we have now certainly fits our need. OK, so you may not want to use this to run a detailed software project without some adaptation, but for prioritising and driving the management backlog then it works just fine, especially when tasks are sync’d with Outlook.

Changing our way-of-working was, as usual, initially difficult, but quickly the benefits of being able to see what everyone was working on and where it was in the cycle became apparent. Figuring out the work in progress limits for the team was a challenge, given the varied nature of the different roles, but we found that, more often than not, our tasks relied on each other, and therefore our ‘internal’ WIP limits did too. This meant that the team did have a natural velocity and this could be used to set WIP limits accordingly.

You may wonder whether this approach sounds a little heavy-handed? I’d argue that it does not, simply because you always keep an action list or backlog for a management team. Maybe you have tried to colour-code it or tabulate it, in order to make it more visible? You are halfway there. Why not go the full way and try KanBan for it instead?