Episode 7 of the software testing podcast that I record with Dan Ashby is now available. In this episode we talk about the idea of ‘schools of testing’ and compare and contrast approaches such as those from the ISTQB and context-driven communities.
You can download it from the site, via RSS or it’ll shortly be in iTunes as usual.
Myself and Dan Ashby (@danashby04) have started a software testing podcast. It’s called Testing In the Pub, primarily because we spend time in the pub talking about testing, and we thought that others in the software testing community may be interested in hearing what we talk about.
We published the first episode yesterday, called “Reviewing the Conferences 2013”, which is about the conferences that we attended in 2013, and the main learnings we took from them.
Testing in the Pub has it’s own website, and, (Apple approval permitting), will be in iTunes very soon.
It’s be great if you had a listen and gave us some feedback. This is the first time we’ve done something like this, and so all feedback will help us make it better.
If you want to appear as a guest on one of the shows then let us know as well. We’d really like to make the podcasts as varied as possible so the more the merrier 🙂
I had the pleasure of attending the London Tester Gathering Workshops last week, organised by Tony Bruce and the team at Skillsmatter. It was a good couple of days, and a good break from the presentation led conferences that I have mostly attended in the past.
As an offshoot from the London Tester Gatherings, the purpose of the workshops were to enable testers to get more hands-on and practice in a group setting, with support from some great testers and presenters. For me it was a good opportunity to get back to being a bit more hands-on, and to improve my knowledge of security testing in particular.
If you were wondering what the venue or some of the attendees looked like 🙂 then take a look at Tony’s blog. He took the pictures, I spent the time learning and talking testing.
There were a couple of workshops that really looked interesting. Black Ops Testing, run by Tony, James Lyndsay, Steve Green and Alan Richardson, and Security Testing for Mobile Apps, run by Bill Matthews.
Black Ops Testing focused on scouting, intrusion and extraction. Or, as the intro said – if you don’t like military metaphors: Thinking, Exploration, Diagnosis. It focused around exploratory techniques and a whole lot more. Using a variety of techniques on a test server, meant that we were able to quickly put into practice what we were learning. Sadly I have lost the mindmap I wrote so you’ll have have to take my word for it, and wait for the blog post from Dan Ashby.
The Black Ops Testing workshop continued on in the afternoon but sadly clashed with Bill Matthew’s Security Testing for Mobile Applications workshop. Given my focus on mobile testing, both professionally and otherwise, then this one couldn’t be missed. Bill focused the session around the Mobisec VM and gave us all a large number of hints and tips on security testing for mobile applications.
I drew a mindmap: 
We focused on testing for Android applications, learning basic tools and techniques alongside some application security concepts. It was very useful to be able to setup the Mobisec VM in particular, and then use that to test an application with known vulnerabilities Security Compass Exploit me – they have a set of labs you can follow as well on their site. Using a VM meant we got all the tools we needed in one package, and Bill was on hand to explain, answer questions and make sure we were heading in the right direction. It was a good session with lots to takeaway and practice.
If you have an interest in mobile security then I would definitely recommend that you take a look at the Mobisec VM, and then head over to the Security Compass site. They also have an iPhone version, together with labs you can go through to help learn the main concepts.
The day concluded with the London Tester Gathering, which is always a good opportunity to meet old friends and new one’s over a beer or two.
Day 2 was all about security testing again. Firstly Bug Hunting for Fun and Profit with Martin Hall, then The Evil Testers Guide to Http Proxies with Alan Richardson.
Bug Hunting for Fun and Profit was all about the tools and techniques that would enable testers to find security exploits in popular websites and applications, in order to make some money from bug bounty programs. Martin clearly knew his stuff – he gave us a lot of examples, a whole bunch of tools, and a lot of supporting information on which sites run bounty programs, the best way to approach them, and how to make some spare cash.
I mindmapped my ideas from the workshop, although, like Bill Matthew’s workshop the day before, this was just the start of things. There’s a lot of practice to do, both using the tools and the techniques before going onto any live sites. Fortunately there are a number of sites that one can practice on, and Martin gave us some great tools to use.
The afternoon was spent with Alan Richardson, talking about The Evil Testers Guide to Http Proxies. Having spent both Bill and Martin’s sessions using proxies then it was great to have Alan give his ideas and helpful advice. The session was organised around testing the Gruyere web application, a vulnerable app designed for practicing web security testing. Alan gave us a lot of documentation and support, far more than I can go through in one blog post.
The London Tester Gathering workshops were a great couple of days. I learnt a lot, and I now have a lot of great opportunities to learn and practice. The presenters were all very knowledgeable, and were happy to share that knowledge and a lot of useful tools, slides and experience. I met a lot of good testers who were keen to learn and improve their skills. It was great to meet some old friends, but equally it was good to see so many testers in the workshops that I haven’t met before. Sometimes the testing community can seem a little cliquey and this workshop certainly was not.
Thanks to Tony and all the other organisers and presenters. If you didn’t go to the workshop this year then make sure you check it out next year. It’s well worth it.
TestBash 2.0 is less than 3 weeks away. I’ll be speaking on “A Testers Hierarchy of Needs”.
What motivates you to work and improve as a tester? Why do the testers in your team work well together? Or why don’t they? Have you ever wondered what motivates professional testers?
Maslow’s Theory of Needs seeks to describe the psychology of humans by way of a hierarchical model. From physiological needs up to self-actualization, it helps explain what motivates us and how we express that motivation. A Testers Hierarchy of Needs does the same for software testers.
If you manage testers and have a keen interest in ensuring that your team are motivated and work well together, or if you are a tester wondering what is needed to make your team great, then come along and discover more. I will present a framework which helps explain tester psychology, how internal and external factors can affect their motivations, and steps that can be taken to better motivate yourself and your team.
It would be great to see you there.
A quick post to say that I’ll be speaking at the Reading Tester Gathering which is this Thursday, 28th February. I’ll be giving a presentation on mobile device testing, with ample time afterwards for questions about mobile testing, or testing in general.
It’s at Copa Bar, 76-78 Kings Road, Reading. There’s more details on the meetup page.
Hope to see you there.
I’ve been fortunate enough to have had an article published in the latest edition of Testing Experience magazine. It’s called “Beyond the Smartphone – Testing for the Developing World” and focuses on techniques and tools to use when testing feature phones.
You can download the whole magazine for free by visiting their website and subscribing to the online edition.
Hope you enjoy it.
I’ve recently been reading through my ISEB Practitioner notes, which I got when attending a course organised by Grove Consultants a few years back, as I mentioned in my previous post. It’s got me thinking about test cases, and in particular the four criteria of a good test case. Having attended both Rapid Software Testing and Rapid Test Management recently, and having rolled out an Exploratory and Session Based Test Strategy in my teams then it’s caused me to question again the validity of test cases and the need for them.
So, to a good test case. Reading my notes, a good test case is, in no particular order, apparently:
1. Exemplary.
2. Evolvable.
3. Economic.
4. Effective.
Straight from the ISTQB/ ISEB of course. But not without merit. Are these still relevant?
Good test cases can test more than one condition at the same time. This is one good reason for taking the time to design test cases in the first place. Just writing test cases because “it’s the done thing” or “because the policy says so” is time wasting, but designing them so that when the testing is carried out it is done so in the most efficient way requires exemplary test cases and can add value. It is also the case that test cases are shared and it’s difficult, especially in large organisations, to ensure that all testers have the same basic level of ability. Having test cases can help.
From a contextual point of view perhaps a good test case is not written down at all but merely in the testers head, and driving the testing into particular areas that the tester feels are worthy of time and effort. Cases where the software is the specification are becoming increasingly common; in-sufficient or non-existent requirements documentation which requires the tester to apply their previous knowledge of the system under test in order to effectively test it. Clearly in this case, if documented test cases are required then they will be need to evolve. As Rapid Software Testing mentions “How do you invent the right tests at the right time – evolve them with an exploratory strategy”.
Time is money and often in testing we have little time and sometimes little money. So using that time and that money in the most efficient way means we minimise the economic impact. Of course, sometimes the best way to get maximum value is to have a purely exploratory approach and spend more of the time and money with the software in hand. A choice which is key to a good test strategy and highly dependant on the industry area one is testing within.
Clearly a good test cases should be effective. We are not in the business of wasting time, particularly when time is precious as it often is in testing projects.
I’d argue that the definitions given in ISEB/ ISTQB are still relevant and can be a good guide as to what is required of a test case. In a lot of industries test cases are still very much required and, particularly where there is strict regulation such as in areas of financial software and even in mobile software. The ability to write a good test case is a skill which should not be forgotten.
I’ve recently been reading through my ISEB Practitioner notes, which I got when attending a course organised by Grove Consultants a few years back.
Don’t switch off yet. I know I mentioned ISEB. So before I go further, it’s worth stating my thoughts on the whole ISEB/ISTQB debate. I summed up my frustrations in a previous blog post; the fact that in the UK having ISTQB certification is practically the only way to get past the recruiters gate, but I also feel that there is some merit to the courses if taught properly, and followed up with a context driven approach such as Rapid Software Testing.
Reading back through my notes from Grove I can now see that this is what they were trying to work towards. At the time I had no idea how important the context driven school of testing was, nor the work of James Bach, Michael Bolton, Cem Kaner and others. But looking in the notes, the names are there. The techniques, albeit in nowhere near the detail that James or Michael teach in class of course, were hinted at and some approaches, particularly Exploratory Testing, are mentioned in some detail. Some slides are directly referenced from James and Michael’s work.
Unfortunately, due to the need to pass the exam, and with these areas being marked as ‘not exam’ then I didn’t pay them the attention that they warranted, and so it took a few more years to discover how and why the context driven approach can be so powerful. Which maybe shows the true problem with ISEB/ ISTQB certification after all.
Caveat – this is not a list of exactly what happens in a Rapid Test Management class, nor is it a list of exercises and course material. You won’t become good at Rapid Test Management by reading this post. Sometimes I forget things. If you really want to know about Rapid Test Management then you need to sign up for the class. It will be money well spent and this post will tell you a little about what might happen if you do take the class.
Last week I had the opportunity to attend one of the first Rapid Test Management classes, and certainly the first one to have been run in the UK. It was a great experience and the beginning of a lot more learning. There were attendees from a number of different areas of software testing and together I think we formed a very effective group, learning from each other and sharing experiences.
Rapid Test Management follows on from the Rapid Software Testing class that James Bach and Michael Bolton teach. I’d attended James’ class in Cambridge at the beginning of March and so, with the class material and my ideas rolling around in my head, I pitched up at the Hilton hotel in leafy Kensington (well almost Shepherds Bush actually) for two days of intensive learning from Michael.
Rapid Software Testing is a context driven technique, intended to enable testers to excel at our chosen craft. Since it focuses on testing as a craft itself, instead of merely on the production of test cases and then their use for testing (or it could be argued merely for checking) then this does put some questions in mind when considering how best to manage it. By way of some preparation, I noted down some of the questions that I had:
The class was split into two days, the first morning focused mostly on refreshing understanding of Rapid Software Testing and ensuring that, as a group, we had effectively collected our own desired outcomes from the class. It was good to spend some time going back over the methodology itself even though I had learnt it very recently. It was great that Michael seemed happy to tailor the class to the audience in any way that was necessary, taking a lot of time to ensure that everyone’s opinion could be heard.
What I found great about this class, and the RST course, are the little nuggets of information, stories and quotes that James and Michael pass on. Looking back through my notes from this class I find that’s what I mostly write down, quotes like:
“Quality is value to someone who matters”
“Testers tell stories about products”
We also spoke in depth about the testers Elevator Pitch, that two minutes where you have to explain what testing is all about. Being able to explain and justify the role of testers and test teams is a critical skill and Rapid Software Testing gives you that skill.
When implementing any changes, such as introducing Rapid Test Management for example, then it’s critical to take into account the existing knowledge and processes that are already in use. Michael stressed the importance of ensuring that it’s not only the documented processes that are considered; in fact it’s the informal processes that are far more important to understand. He spoke about tacit vs explicit knowledge and how it is important for managers to be aware of the tacit knowledge in a test team and not merely the explicit knowledge.
We then took a look at how to put together a good team and how to train a new tester who was coming onto the team, through guidance and suitable heuristics. It was great to discuss with the others in the class about the issues they had faced when coming into a new team or bringing new team members in and also what had gone well. Ensuring that people are trained in a fault tolerant environment and gradually introduced to new tasks may seem fairly obvious stuff but it’s often over-looked, as is the need to ensure that feedback is both given and asked for from new team members.
Day 1 concluded with a look at metrics and how important they really are, v.s. how important most organisations seem to think they are. Michael pointed us towards Cem Kaner and Walter Bond’s paper Software Engineering Metrics: What Do They Measure and How Do We Know which I would definitely recommend reading.
Find out what happened on day 2 soon…
As I’ve blogged about recently, I’ve been recently studying for the Level 5 Certificate in Management and Leadership from the Chartered Management Institute. I’m now three sessions into the four session course, and it’s getting really interesting to see and think about how one might apply general management principles more specifically to the software testing area.
The most recent session was all about Management and Leadership. As part of this we did an exercise where we had to sort out a number of different phrases into groups that either apply to ‘Management’ or ‘Leadership’. No sitting on the fence, no spending hours deciding which to put where, but a quick exercise to make you think. There were right and wrong answers (although primarily to seed further discussion).
The list, as I saw it, is below. What do you think? Do you agree?
| Leadership | Management |
| Enables others | Implements and maintains |
| Inspires vision | Focus on systems and structures |
| Encourages head and heart | Adopt short term view |
| Acts authentically | Completes transactions |
| Asks what and why | Asks how and when |
| Has long range perspective | Brings order and coordination |
| Focuses on doing the right thing | Focuses on doing things right |
| Inspires trust | Accomplishes tasks through others |
| Acts as an innovator | Focuses on performance |
| Challenges | Provides stability |
| Transforms | Controls |
| Committed to the cause | Imitates |
| Gives purpose and meaning | Complies |
| Focus on people |
After the exercise I got thinking about how I might apply this more generally to software testing. Specifically to Test Managements vs. Test Leadership, i.e. what separates those who run testing projects, probably as part of the design and delivery of a specific product, vs. those who lead groups and provide inspiration both inside those groups, and to the wider testing community. Are the skills needed in those situations different?
We clearly need within our community those who can challenge and innovate. Right now this is coming mostly from the context-driven community as I see it, where also the vision and long range perspective is clearly evident. These people are driving things along nicely in my opinion. As someone who has recently attending Rapid Software Testing with James Bach then I can say first hand that he’s certainly encouraging head and heart. I don’t need to mention challenging, right?
The question then comes, who is taking things further? Leaders need managers to take their vision and turn it into practice. They need someone who can focus on the short term view, give the control and ensure completion. They need people who implement. As a community do we currently have enough of the managers listening and implementing the vision that our leaders are sharing?