Tag Archives: techniques

Experiences, practices, team, Testing

Exploring An Existing System

3 Comments

Recently I’ve been reading Elisabeth Hendrickson’s excellent book ‘Explore It!’. For anyone who has an interest in exploratory testing it’s a must own. I wish I’d discovered it earlier to be honest, since it gives so many useful hints and tips, as well as confirming that the ways of working that one has chosen are also recommended and used by others.

As well as using it for my own learning, I’ve been slowly going through the book and working out what parts I can use in the regular lunch and learn sessions that I run at work. While we practice exploratory testing, in fact it’s the cornerstone of our sapient testing strategy, there are some areas where I feel we could take approaches that could benefit not only testing, but also the wider business.

Working With Legacy Systems

We frequently work with legacy systems and so one chapter that was of immediate interest to me in Elisabeth’s book concerned exploring an existing system. When one has an existing system to test I find it’s all too easy to become primed by what others have already discovered, and to fall back on existing test cases (whether physically written down or in someone’s head). This can bias you, resulting in less effective testing.

Elisabeth makes the point that an existing system may well be unknown to the tester, but also may well be unknown to the whole team, or at least contain parts that are unknown. While the software fulfils a business need, how it actually goes about doing so may be less clear. That makes it ideal for exploration.

Recon Testing

James and Jon Bach like to call the initial exploration of an existing system ‘recon testing’. I like this term, by taking an initial session to explore and discover the basics of the software under test then one can then plan more effectively for future sessions, and write future charters in order to drive those plans (if you want to know more about the concept of session based testing and how charters fit in with this then have a look at James Bach’s explaination). Recon sessions help to map the territory and give insight.

During a recon session you can learn a lot, but the most important areas to ensure that you have gained insight into are:

  • The ecosystem in which the software under test resides.
  • Touchpoints to other systems.
  • Variables (things we change or can change).
  • Obvious vulnerabilities and potential risks.

Our Training Session

During our training session we carried out recon testing on the Staples ‘Easy’ button, and a standard service bell. This no doubt annoyed those in the room next door 🙂

Ding ding!
Ding ding!
Easy to test?
Easy to test?

By starting training with something simple, and not software based then it’s easier to pick up and learn the basics of a new technique without bias or the complication of software. We then compared our experiences, testing, (and the charters produced), with those that the Bach brothers produced when they carried out an exploratory testing exercise using the same product. Fortunately for us, their session is available on youtube, so it was an excellent addition to our de-brief. In it they explain the different testing techniques they use, and why. It’s well worth watching.

Enough Recon Testing?

One area to focus upon when conducting recon testing is whether one has conducted enough recon testing. Fortunately ‘Explore It!’ has this covered, recommending you ask yourself questions about the system that you have been exploring. If you don’t understand what the system does, how input and output works or how the environmental configuration affects the system for example, then it’s probably time to think about more recon before you move into more focused test sessions. Fortunately for the ‘Easy’ button and bell testers, (and those sitting near the meeting room where we had the lunch and learn), then this was not necessary 🙂

A Useful Addition

Recon testing is something that I think we’ll find is a really useful addition to our strategy. I’d certainly recommend that you check it out, and that you check out ‘Explore It!’ which contains much more useful information and techniques to use in your exploratory testing. I’ll certainly be using the book to inspire some future training sessions for the team.

Experiences, Testing

Live from Testbash

2 Comments

I’m here at the awesome TestBash conference today. I’ll be posting updates here, hopefully some mindmaps too.

image

First up – Scott Barber. An excellent presentation about Managing Application Performance. My mind map is here.

Next up, Contextual Decision Making, from Mark Tomlinson. Great presentation with added spinning cats. Mindmap is here.

Jez Nicholson gave us some good tips on how to win developer friends and influence people.

Joep Schuurkes explained to us how to help a new tester to get a running start.

image

Context driven testing in an agile context from Huib Schoots. Some great stuff.

Bill Matthews kicked off the afternoon talking about Getting Out of the Testing Game.

image

Stephen Blower taught us how to inspire testers and what inspires him.

Iain McCowatt presented a great talk on changing our automation models.

image

Chris George gave us a great story from RedGate on how they improved a legacy automation suite.

And finally Keith Klain gave a great talk on how to talk to a CIO about testing.

image

image

And then 99 second talks, and that’s it. What a great day!

blog, Experiences, practices, Testing

Automated Functional Testing – A Test Activity?

7 Comments

If you are a functional test automation expert then times are good. There’s big bucks to be made in the contracting game, companies are desperate for candidates to ‘automate everything’ and to get to this oddly perceived test automation nirvana that those who are either mis-informed or have hidden agenda’s seem to feel fit to promote.

This has made me think. Primarily about how we have got to this state? Is it because, as the testing community, we have wanted to own test automation? Is it because those outside of the test community see test automation as less important than the production code that it tests? Is it that we just built up an expertise and then protected it just for the money?

Some might say that what has actually happened is that we now have a situation where second rate developers now have a great way to stay in the development game. There is a danger, in the apparent supply-side crisis that we find the industry in, that companies merely employ anyone who says they know something about test automation, without doing the same due diligence that one would do for a development position. This would be a mistake.

In my mind there is a solution to all these problems, and that solution comes from treating test automation just like production code. And that means primarily using developers to write it. Sure, you may choose to have testers involved as well, where they have the skills and expertise, but let’s not try and force skills on people who don’t want them, and let’s not accept second rate people just because they can ‘do some test automation’. One advantage to using developers is that test automation becomes a team thing, and you are less likely to spend time playing catch-up when development slips. One downside; it’s going to look like the team has slowed down. Believe me, it hasn’t. It’s just got more effective, and is playing to the right skill-sets.

Don’t believe me? 🙂 Here’s a couple more examples from Rob Lambert and Amy Phillips which show where continuous or more frequent delivery has been successfully rolled out at New Voice Media and Songkick. The common thread – in both cases the test automation is a development activity.

Experiences, Testing

London Tester Gathering Workshops 2013

Leave a comment

I had the pleasure of attending the London Tester Gathering Workshops last week, organised by Tony Bruce and the team at Skillsmatter. It was a good couple of days, and a good break from the presentation led conferences that I have mostly attended in the past.

As an offshoot from the London Tester Gatherings, the purpose of the workshops were to enable testers to get more hands-on and practice in a group setting, with support from some great testers and presenters. For me it was a good opportunity to get back to being a bit more hands-on, and to improve my knowledge of security testing in particular.

If you were wondering what the venue or some of the attendees looked like 🙂 then take a look at Tony’s blog. He took the pictures, I spent the time learning and talking testing.

Day 1

There were a couple of workshops that really looked interesting. Black Ops Testing, run by Tony, James Lyndsay, Steve Green and Alan Richardson, and Security Testing for Mobile Apps, run by Bill Matthews.

Black Ops Testing focused on scouting, intrusion and extraction. Or, as the intro said – if you don’t like military metaphors: Thinking, Exploration, Diagnosis. It focused around exploratory techniques and a whole lot more. Using a variety of techniques on a test server, meant that we were able to quickly put into practice what we were learning. Sadly I have lost the mindmap I wrote so you’ll have have to take my word for it, and wait for the blog post from Dan Ashby.

The Black Ops Testing workshop continued on in the afternoon but sadly clashed with Bill Matthew’s Security Testing for Mobile Applications workshop. Given my focus on mobile testing, both professionally and otherwise, then this one couldn’t be missed. Bill focused the session around the Mobisec VM  and gave us all a large number of hints and tips on security testing for mobile applications.

I drew a mindmap: Security Testing for Android Apps

We focused on testing for Android applications, learning basic tools and techniques alongside some application security concepts. It was very useful to be able to setup the Mobisec VM in particular, and then use that to test an application with known vulnerabilities  Security Compass Exploit me –  they have a set of labs you can follow as well on their site. Using a VM meant we got all the tools we needed in one package, and Bill was on hand to explain, answer questions and make sure we were heading in the right direction. It was a good session with lots to takeaway and practice.

If you have an interest in mobile security then I would definitely recommend that you take a look at the Mobisec VM, and then head over to the Security Compass site. They also have an iPhone version, together with labs you can go through to help learn the main concepts.

The day concluded with the London Tester Gathering, which is always a good opportunity to meet old friends and new one’s over a beer or two.

Day 2

Day 2 was all about security testing again. Firstly Bug Hunting for Fun and Profit with Martin Hall, then The Evil Testers Guide to Http Proxies with Alan Richardson.

Bug Hunting for Fun and Profit was all about the tools and techniques that would enable testers to find security exploits in popular websites and applications, in order to make some money from bug bounty programs. Martin clearly knew his stuff – he gave us a lot of examples, a whole bunch of tools, and a lot of supporting information on which sites run bounty programs, the best way to approach them, and how to make some spare cash.

I mindmapped my ideas from the workshop, although, like Bill Matthew’s workshop the day before, this was just the start of things. There’s a lot of practice to do, both using the tools and the techniques before going onto any live sites. Fortunately there are a number of sites that one can practice on, and Martin gave us some great tools to use.

Bug hunting for fun and profit - Martin Hall

The afternoon was spent with Alan Richardson, talking about The Evil Testers Guide to Http Proxies. Having spent both Bill and Martin’s sessions using proxies then it was great to have Alan give his ideas and helpful advice. The session was organised around testing the Gruyere web application, a vulnerable app designed for practicing web security testing. Alan gave us a lot of documentation and support, far more than I can go through in one blog post.

The Evil Testers Guide to HTTP Proxies and Developer Tools

Wrap Up

The London Tester Gathering workshops were a great couple of days. I learnt a lot, and I now have a lot of great opportunities to learn and practice. The presenters were all very knowledgeable, and were happy to share that knowledge and a lot of useful tools, slides and experience. I met a lot of good testers who were keen to learn and improve their skills. It was great to meet some old friends, but equally it was good to see so many testers in the workshops that I haven’t met before. Sometimes the testing community can seem a little cliquey and this workshop certainly was not.

Thanks to Tony and all the other organisers and presenters. If you didn’t go to the workshop this year then make sure you check it out next year. It’s well worth it.

"ministry of testing", Experiences, mobile, plans, Testing

Get Out and Test – My Mobile Testing Blog Post Over At The Testing Planet

1 Comment

I’ve just had an article published on The Testing Planet’s new site. It’s all about mobile testing, and why you shouldn’t just sit in your seat, in your office, when you are testing mobile.

Why not head over to their site and take a look? You can find the article on The Ministry of Testing website.

Experiences, Testing

Further Notes From A Testers Hierarchy of Needs – pt1

1 Comment

I’ve written about Maslow and links between humanistic psychology and software testing a couple of times before. I presented ‘A Testers Hierarchy of Needs’ at TestBash 2.0. This post gives a bit more information and material to support the slides which you can find on this page.

The idea is that the headings match the slides, in order to make things easy to follow. It’s split into two posts – the first talks about humanistic psychology and Maslow. The second will focus on how we can adapt this to software testing.

Abraham Maslow, Humanistic Psychology and Testing

Abraham Harold Maslow was an American psychologist who died in 1970. He was best known for creating Maslow’s hierarchy of needs, a theory of psychological health predicated on fulfilling innate human needs in priority, culminating in self-actualization. He stressed the importance of focusing on the positive qualities in people, as opposed to treating them as a “bag of symptoms”.

The needs of humans, in their most basic form

Humanistic Psychology focuses on the needs of human’s in their most basic forms. It focuses on the positive qualities in people in a positive way and is based upon observations of humans’ innate curiosity – innate curiosity, something useful for testing I’d argue. Consider the classic “Testing is questioning a product in order to evaluate it” for example.

Maslow studied what he called exemplary people such as Albert Einstein,  rather than mentally ill or neurotic people. He also  studied the healthiest 1% of the college student population.

Humanistic theories of 
self-actualization

Humanistic psychology is a perspective which rose to prominence in the mid-20th century in response to Sigmund Freud’s psychoanalytic theory. Maslow once commented: “It is as if Freud supplied us the sick half of psychology and we must now fill it out with the healthy half”. Humanistic psychologists believe that every person has a strong desire to realize his or her potential, to reach a level of “self-actualization”

It holds that people are inherently good.

Qualities of 
self-actualizing people

Maslow studied what he called self-actualised people. He realized that all the individuals he studied had similar personality traits. All were “reality centered,” able to differentiate what was fraudulent from what was genuine. We do that as testers. They were also “problem centered,” meaning that those treated life’s difficulties as problems that demanded solutions. We do that as testers. These individuals also were comfortable being alone and had healthy personal relationships. Are we like that as testers? 🙂

Self-actualizing people tend to focus on problems outside themselves; have a clear sense of what is true and what is false; and are spontaneous and creative..

The Theory of Needs

needs

The theory of human needs can be expressed or ordered in a pre-potent hierarchy—a pressing need would need to be mostly satisfied before someone would give their attention to the next highest need. Maslow described human needs as being relatively fluid—with many needs being present in a person simultaneously.

1) At the bottom of the hierarchy are the “Basic needs or Physiological needs” of a human being: food, water, sleep and sex.

2) The next level is “Safety Needs: Security, Order, and Stability.” These two steps are important to the physical survival of the person.

3) The third level of need is “Love and Belonging,” which are psychological needs.

4) The fourth level is achieved when individuals feel comfortable with what they have accomplished. This is the “Esteem” level, the need to be competent and recognized, such as through status and level of success.

5) At the top of the pyramid, “Need for Self-actualization,” occurs when individuals reach a state of harmony and understanding because they are engaged in achieving their full potential.

Once a person has reached the self-actualization state they focus on themselves and try to build their own image. They may look at this in terms of feelings such as self-confidence or by accomplishing a set goal.

Usually people in developed countries focus on the third and fourth level of needs while those in less developed worlds focus on the first and second.

Meta-motivation

Maslow used the term metamotivation to describe self actualized people who are driven by innate forces beyond their basic needs, so that they may explore and reach their full human potential. To become better people. Or better Testers.

But how does this fit in with Software Testing? We’ll find out more in part 2.

"ministry of testing", Experiences, management, recruitment, Testing

Persona Based Interviewing

5 Comments

Why you may need to do some actual testing in the interview for your next testing role

Interview

 

When I was starting to think about writing some posts to help out those who are either recruiting testers (see here for my first post), or for testers looking for new roles, the obvious first post was going to be around interviews. Interviews are scary, interviews need preparation, etc.

But to be honest, there is already so much available on the web that’ll teach you the basics, that I’d be re-inventing the wheel.

This is the second post in a series on finding a job within testing – the first one is Starting a New Job In Testing which you can also read on this site.

Basic interviewing techniques

If you need some help on basic interviewing techniques then I’d recommend spending some time on Ministry of Testing, looking at the recruitment resource section. What I want to discuss in this post is an interviewing technique we are using where I work, in order to help us recruit testers. It’s not new but it’s also not typical, and so hopefully it helps you.

 

A typical interview

A typical interview has a set of questions, and sometimes a script to follow. While the questions may not be written down, an experienced interviewer typically has a number of questions in their head that they can tailor to the interview situation, and use to steer the conversation in a particular direction. There may be a formal test, or the interview may just be conducted verbally.

Simply asking someone questions about testing, and gauging their responses, is one way of understanding in more detail what they know. You can also find out more about who they are, and what they can bring to the company. While we are following this approach, we’re also doing things a little differently.

 

Persona based interviewing

In a persona based interview, each of the interviewers play out part of a task based story, which the candidate is also a part of. One of more scenario’s are played out, normally based around a task that the candidate would typically face if they were successful in joining the company. The interviewers take different persona’s, each playing the part of a role or person that the candidate would need to interact with, in order to successfully pass the task that they are set. In this way the interviewers can understand how the candidate approaches particular tasks, how they solve problems, and how they interact with others.

For our interviewers for mobile test positions we typically play out some scenario’s based around testing our applications on real hardware. I won’t go into too much detail, for obvious reasons, but the candidate receives a certain testing task, and then is expected to start testing and exploring the application in order to successfully find bugs within it.

We play the parts of other people in the story. These could be the Product Owner deciding on re-prioritisation, other testers being able to offer advice, a developer being difficult or helpful, or even a senior manager wanting to know the progress of a testing task. As we go through we throw in these sorts of requests, and other changes to the scenario, in order to see how the tester works when requirements change and the pressure mounts.

Attempting to complete the tasks without interacting with the other roles within the scenario is very difficult and we are looking as much towards how and when the candidate asks for help, as we are to the testing skills that they show.

After the scenario’s are complete then we discuss with the candidate how they think the scenario’s went, what they thought went well and what they would do differently if faced with them again.

 

Is persona based interviewing useful for test roles?

We think it is. Mainly because:

 

It gives us a better idea of how the candidate thinks, and how they approach a testing problem

I’m a firm believer in context-driven testing and Rapid Software Testing in particular, as is the company I work for. To me, being able to observe how someone approaches a testing task, who they communicate with, and what questions they ask is very important. Being able to get an understanding of how they change their approach based upon the context is also much easier within the scenarios. Using persona based interviewing I get to discover more about ‘how they tick’ rather than what testing terms they can remember, or how much of their career story they can tell.

 

We get to see a candidates real, practical skills

By giving the candidates tasks based around testing our live applications, usually on an iPhone or iPad, we give the candidates an opportunity to demonstrate the real, practical skills that they have. We encourage the candidates to talk us through what they are doing and give them the opportunity to demonstrate what they know. Being able to demonstrate ability also puts people more at ease and we get to see what they can really do.

 

It focuses on critical assessment and improvement

By having a de-brief or lessons learnt session after the scenario’s have played out, we also get to see how a candidate critically assesses themselves, and discover what they would do differently given another chance. Making mistakes is human but learning from them is the important thing, and I don’t expect testers in my teams to get everything right first time. However I do expect them to be able to recognise ways in which they can improve.

 

It’s more fun than just answering questions

It’s certainly more fun for the interviewers, and hopefully it’s also a bit more fun for the candidate (as much as interviews can be anyway). Asking questions isn’t much fun, and only being able to show your skills in simple answers, is not the best way to spend your time. Demonstration of skills tests how a candidate works not what they can remember.

So, why not try persona based interviewing next time you are recruiting for testers. It can be a great way of finding really good people who you can have confidence will do a good job.

 

The next post in the series will be about C.V.’s, and what you need to highlight in order to get noticed.

Image courtesy of Ambro / FreeDigitalPhotos.net

 

management, mobile, practices, Testing

I’ll Be a Panelist at the Next Generation Testing Conference on 6th December

1 Comment

I’m excited to be able to announce that I’ll be a panelist at the Next Generation Testing Conference which is on Thursday 6th December in London.

The conference is in its seventh year and is continuing with the  format that was trialed at the last conference in May, with more emphasis on panels and discussions, together with case studies. This should hopefully mean that there’s some great audience participation and discussion on software testing.

I’ll be on two different panels during the day – “Retrospective on 2012” and “Looking Forward to 2013” We’ll be debating various topics including:

  • What have we learned in 2012?
  • TestDevOps – what does it mean to you?
  • Continuous Integration and Continuous Delivery – lessons learned
  • Mind Map techniques that work with Agile
  • Advances in risk-based testing
  • Changes in approach for testing cloud based solutions and mobile applications
  • The rise of TestOps
  • Innovation/renovation
  • What is the next big thing?
  • “Prespective” of 2013

Others presenting or as panelists at the event include Paul Gerrard, Julian Harty, Steve Tulk and Niels Malotaux.

It’s sure to be a great event, as the last one was. Hope to see you there. You can find out more about the event at http://next-generation-testing.com/

"ministry of testing", Experiences, Testing

I’ll be speaking at TestBash 2.0

Leave a comment

 

Some exciting news – I’ll be speaking about “A Tester’s Hierarchy of Needs” at TestBash 2.0 which is being held in Brighton, in the UK, on Friday 22nd March 2013. This will be the first time I’ve spoken at the event, and looking at the others speakers, then I’ll be certainly in very good company. James Bach will be giving the keynote.

You can find out a lot more about TestBash 2.0 at the event website, including how to get Super Early Bird tickets.

Hope to see you there.

Testing

Professional Tester: Move Closer, Transition to DevOps

Leave a comment

I’ve written the lead article in this month’s Professional Tester magazine, about my department’s transition to DevOps and the impact that had on testing.

If you fancy reading it then head over to their website and subscribe, or you can download issue 17 which contains my article, and a number of other articles about TestDevOps, directly from this link.

I hope you enjoy reading about TestDevOps 🙂