Category Archives: Experiences

A Testers Portfolio

portfolio

I’ve been recruiting for both testers and developers recently and one thing that struck me was the difference between the two disciplines when it came to a body of work.

As a developer you are nothing these days without a Github account. It’s expected that you have one, that it has code in it :) and that an interviewer will take a look. So much so that most developers will gladly advertise their account and repo’s on their CV.

In my most recent piece of testing recruitment I was looking for an exploratory tester and a tester with more automation skills. One automation tester was able to give me a link to her Github account; this was the first time this has happened to me when interviewing testers and was a positive thing to see. None of the exploratory testers I interviewed showed me any links to previous work; now this does not necessarily make those testers better or worse of course. However, as Rob Lambert has talked in his recent series of blog posts and e-book about ‘Remaining Relevant’, as a candidate, you need to stand out, and having a body of work can really help.

For testing it’s a bit different.

So this got me thinking – why is it that testers don’t traditionally have a portfolio?

Is it because more testers are 9-5’ers?

There could be an element of truth here, perhaps there are more testers who are less motivated by their work, with testing being a job they have fallen into rather than a career. If one is not motivated by a job then one is far less likely to want to do whatever is required to remain in that career area.

Are developers more motivated to share their work?

Perhaps there’s a bit of pride in sharing something that one has created that testers don’t have so much. For testers involved in automation, particularly framework design, then there’s something to share, but this isn’t the more general case.

Do they have more to share?

I’d wouldn’t say so, it’s just what is shared is different.

Is it more difficult to build and maintain a body of work as a tester?

I think it is. Since testing is much less about creation then it does make things more difficult. But it’s far from impossible.

What could you do if you want a portfolio of work?

Myself and Dan Ashby touched upon the idea of a testers portfolio in the recent two episodes of Testing In the Pub, where we talked about recruitment. The idea is that, as a tester, you have this body of work, either built up through your day-to-day work, or from experiences in the wider testing community, that can really help you to stand out from your peers, and help when you want to make your next career move.

You may be able to directly share what you are working on at work, but this is frequently not the case. Getting involved outside of work can instead be one key way of building up a portfolio. For example you could consider:

Writing for Blogs and Magazines

Maybe your company has a tech blog. Why not write for that? Or magazines, whether printed or online are always looking for articles. Don’t think that you have nothing to write about – all experience is relevant and there’s some great people in the testing community who will help you edit an article. You just need the idea, and an ability to type :)

Starting Your Own Blog

Sometimes you just want to publish something, or you want to say something that’s free from any editorial tone. Start your own blog, it’s really easy and free if you use something like Blogger or WordPress. It’s also great for raising your visibility, especially if you get your blog featured on sites like Software Testing Club or Test Huddle. Add it to the directories.

Talking At Conferences and Meet-ups

Have an idea, submit a proposal, get it accepted and then panic :) But seriously, talking at a conference is a great way of sharing your knowledge, raising your profile, and getting noticed. It’s daunting at first, slightly daunting even when you’ve done it before, but also really rewarding. Conference speaking looks great in a portfolio.

Sharing Your Presentations On Slideshare

Whenever you talk, take time to upload your slides to a sharing site like Slideshare. This makes them available to all and makes you far more visible to search engines like Google. Which it turn helps you build a portfolio of work.

Pulling It Altogether

Pulling it altogether means that you build up a single portfolio. This can take place on your own website or blog and these are easy and free to put together. If you don’t mind a few advertisments then a free WordPress site is a great way to start and you can easily use a theme which makes it look more like a website than a blog.

I have also found that about.me is a good place to consolidate your information together and is widely known. Let’s also not forget about LinkedIn, although I find that it’s better to keep my body of knowledge linked to LinkedIn rather than on the site itself, which allows for more flexibility.

Having a personal brand is an often overlooked but very important differentiator when looking for a new role. It won’t make up for a lack of relevant technical or social skills, but it will help you to stand out from other, similar, candidates.

So think about your portfolio. What would you include?

—-

Update

Thanks everyone who has commented, either here or on Twitter. There have been some great suggestions on other ways you can form a portfolio as a tester. For example:

  • Use Github to store useful resources. It doesn’t have to be code. Thanks to Alan Richardson for the link to this pentest example.
  • Test an OS project that is stored on Github – thanks Michael Bolton, and also @xrisfg.
  • Taking part in activities such as Weekend Testing, podcasting, etc – thanks Dan Billing.

And thanks to everyone who commented below. I’m glad people found the article useful.

Experiences Of a Testing Bug Bash

I’m working in a team, that’s developing a mobile application and back-end, that does not have any testers. We didn’t plan it that way, but temporarily, that’s just the way it is. While I can add my testing skills and experience to the team, that’s not the primary reason I’m there, and so I’ve been looking at ways in which the whole team can help to contribute to a quality product.

It’s been interesting. At first, when our tester left then nothing happened. Like nothing at all, tickets moved over to the ‘Ready for Testing’ column and there they stayed. Immobile, ignored and unloved. It was as if all eyes were to the left, where the fun stuff for developers was, understanding designs and coding. Although there are unit tests and some automated integration level tests for our code it clearly wasn’t enough.

Session Based Team Testing – A Bug Bash

I’ve used session based testing in the past when I managed test teams, and have found it to be a very useful way of breaking exploratory testing work into understandable pieces, and distributing that work out to the team. James and Jon Bach explain the approach far better than I can. Ministry of Testing have also got an excellent resource on SBTM.

Having read some articles from people who had broadened their approach then I thought I’d take the session based testing concept and roll it out to the whole team as a bug bash.

The key point for me was that any sort of session had to be fun. Initially I feared that not all the team were going to see the full value of such a session since it wasn’t development work per se,  so keeping the approach light-weight was key. The ideas for our first bug bash had formed.

Organisation

Getting away from the everyday work was important so the bug bash took place in a separate room, close enough to our team area that it was quick to pop out and pickup forgotten devices, etc but away from the interruptions that phones, etc provide. I made sure we had enough phones, tablets, post-it notes, pens and cake to keep us going. I also brought in some popular testing books and the Test Heuristics Cheatsheet from Elizabeth Hendrikson, in case people needed some inspiration.

We use JIRA as our project management tool and since we were testing a whole Epic then it made sense to break the testing down on a story by story basis. Working with our lead developer, we took each Epic in turn and used that to form an exploratory testing charter stored as a Google doc. You can get an example version here. Using charters enabled us to define the boundaries of the testing; ensuring there was not too much overlap. It also meant that people who were not so familiar with the story were able to quickly learn about it, and areas to test, without needing to spend time reading stories and Epics in JIRA.

So, the charters were ready, the room was ready and the people were ready. Time to start.

The Bug Bash Session

We decided on two 45 minute sessions with a tea break in between. The whole team attended; PO, designers, UX, developers and myself. Everyone was a tester for the afternoon. We agreed who would pickup which charters and made a start.

I made a Google doc in which people could record potential issues which I kept projected up on the wall throughout, so that we could all see the progress that we were making and the issues that were being raised. I encouraged the team to add screenshots as well as text and this really helped to make things clear and to prevent duplicate issues making their way into the list. Discussion was also key to the success of the session; having everyone in the room at the same time meant that potential design issues could be shown to designers straight away for review.

The lead developer for the feature was also present, not testing but helping to explain architectural and development decisions, and also to fix some bugs on-the-fly.

The session went well. We got all the charters finished and the session concluded with a quick retrospective.

Retrospective

What Went Well
Charters give structure and prevent misunderstandings

Having structure was critical to the bug bash, given that it was a new activity for the team, and they were not experienced with exploratory testing. It also meant that we got the most from the session and people were not all testing the same thing.

Hints and examples in charters are good

By including examples, and ideas on areas to test, people had a guide to start them off. Given that they were not experienced testers then this really helped.

It is good to have charters split by story

This meant that it was clear what was to be tested and there was a clear link back to the project management tool people were used to.

Some of the bugs were fixed during the session

This clearly shows the value of group pairing for me.

What Could be Improved?
Make the charters simpler

I had hoped we would get some data on setup time and test execution time so had added that to the charters but it confused most people.

Make the process of splitting up the charters and recording who is working on each one, simpler

Although it as clear to them which charter a person was working on, we didn’t keep a visible record of who was working on what. Next time I’ll write this down and pin it up on the wall.

People could swap charters halfway through the session to give multiple points of view

For each 45 minute session one person would pick up and test against a charter. Swapping round could add value and a new set of eyes onto the area.

Comments
Aim to do this for all large stories

This was great to see since it showed that the team saw the value in the approach.

Consider pairing

This will be our next step for the Bug Bash sessions. Pairing people up could help to add more value by giving multiple viewpoints.

Going off charter was a good thing

Some charters were bigger than others. People naturally started to go off-charter once their charter was done and this gave multiple viewpoints across the whole feature.

Being away from the desk is a good thing

By being one team, with one goal, isolated from our day-to-day activities, we were able to effectively test and find many issues in the feature, without the usual distractions that break concentration and flow. Having lots of cake also helped :)

Conclusion

We will definitely do the Bug Bash again, and I see it as becoming a really important part of our delivery processes, even when we get a full time tester back on the team. It really helped everyone to understand the feature, to explore it and to test together, giving multiple viewpoints on potential issues.

With some guidance the whole team made a great contribution. I’d encourage everyone to try a similar approach,

EuroSTAR 2014 – Day 3

This is the third part of my review of EuroSTAR 2014. You can also read about day 1 and day 2 in the previous posts.

Day 3

After a late night at the Gala party (well I had just presented at EuroSTAR so I reckon a celebration was in order) then I’m ashamed to say I missed the first keynote of the day. From what I hear it was good :(

My next session was ‘Stylish Mobile Testing’ with Dan Ashby and Nehir Yelkovan. I know both Dan and Nehir well – myself and Dan do the Testing In the Pub podcast together and I work with Nehir. So missing their presentation was not an option. I’d have gone along even if I hadn’t known them, the topic of the session being around mobile testing. It was a good session; they passed on lots of useful hints and tips on mobile testing and got a great double act going on.

wpid-wp-1418046411188.jpeg

After watching Rene Tuinhout talk about Passionate Dating for Testers (a talk I’d seen Rene do at the Romanian Testing Conference where it was just as funny), I settled down in the auditorium for the final keynote from Zeger Van-Hesse. My notes are below:

IMG_20141203_084836

Following the keynote, and the announcement of the EuroSTAR 2015 chair and venue (congratulations to Ruud Teunissen and let’s hope we meet in Maastricht) then then there was a decision to make. Workshop or do-over session?

I chose the do-over session, an excellent idea from the conference organisers, whereby the attendees get to vote on what session they would like to see again. This year it was won by Declan O’Riordan. Declan has had a great year, after doing his first talk at SIGIST, straight after me in fact) then he’s spoken at a number of events and also won the best paper award at EuroSTAR this year. His talk about the ‘Why, Why, Who, How of Security Testing’ was in parts exciting, parts scary and really informative. A great final session.

IMG_20141203_084845

Overall

This year’s EuroSTAR was a great event. It’s well run by a really passionate group, both the organising committee and the conference organisers themselves. The mix of speakers and topics meant that there was real variety and something different from last year. Congratulations to all who spoke, all who organised, and all who contributed in some way. I really hope I can make it back next year.

EuroSTAR 2014 – Day 2

This is the second part of my review of EuroSTAR 2014. Check here for day 1.

Day 2

Day 2 started with Isabel Evans giving her experiences of a change project that went wrong. There was a lot of things to learn from here experiences but the main thing I took away was “it’s alway about the people”. People are the main factor in software development and the main factor in change. Understanding them if the key to effective change.

IMG_20141126_090148

Here’s my notes from her session.

IMG_20141203_084746

Next up I saw Michael Bolton give a very interesting and interactive session called ‘Every Tester Has a Price: Sources of Product and Project Information’. In it we went through different information sources and produced a large and detailed mindmap. I captured some of it below:

IMG_20141203_084802

Note – it’s not complete and I need to get the rest of it from Michael.

Following Michael’s session I caught Kristoffer Nordström’s session on Gameification. This was a great personal account of how Kristoffer introduced gameification to a project he was working on, and what the results were. Certainly something that I’d like to see if I can use where I work.

IMG_20141203_084813

My final session of the morning was another case study, this time from James Christie. James gave a very detailed and interesting study of a project he worked on for the UK government, whose primary goal seemed to be for the project not to appear in the UK news/ satire magazine Private Eye. While the project did not fail, (the programme it was part of did), it placed a great strain on the people involved and certainly was not the sort of project James would want to be involved with again. There were some clear lessons to learn, hopefully captured in my notes below.

IMG_20141203_084824

After lunch it was my turn to talk about “Understanding Your Mobile User”. I was lucky, I had a good audience and they asked some great questions. IMG_20141126_130709

I spoke about ways you could understand mobile users, and why understanding the user is so important when mobile testing.

The final keynote of the day was from Julian Harty, who spoke about ‘Software Talk: Are We Listening’. Julian gave us some hints and tips on how we can listen to software, through analytics for example.

What happened next was a little bizarre. A guy from Smartbear got up on the stage and sang a song about testing, to the tune (and inspired by the lyrics) of Frozen, the Disney kids film. Not my cup of tea but some people enjoyed it I guess.

IMG_20141126_170317Then it was time for the Gala party – a great night at Croke Park, home of the GAA.

Note: Updated 11th Dec as a result of James’ comment below.

 

I’m at Mobile App Europe – Day 3

Time for the last day of Mobile App Europe.

The final keynote – The 7 Deadly Sins of Mobile Apps from Jonathan Kohl.

wpid-wp-1412172495702.jpeg

Saurabh Agarwal is now talking about ‘Tackling Fragmentation in the Mobile App World’.

wpid-wp-1412168730185.jpeg

Marc C Lange gave a really interesting presentation called ‘How to Slim Down Product Management, Gain Valuable Insights and Make Customers Early by Leaving Your Comfort Zone’. He gave some excellent tips on how to understand users and how to consider mobile prototyping. I mindmapped his session.

wpid-wp-1412166379630.jpeg

A really interesting presentation on Appium from Andreas Ludeke. He even did some live coding – a brave man – which worked! I did a little mindmap of the session.

First up – Julien Lesaicherre talking about Building the Future of Mobile Apps With Facebook. I mindmapped the session and you can find my map here.

Julien’s slides are here.

wpid-wp-1412147285425.jpeg

I’m At Mobile App Europe – Day 2

So it’s day 2. I’ll be blogging as much as I can, scroll down for the earlier sessions.

Dr Cheahan So talking about Why We Are Wrong When We Think We Are Right.

image

Next up – Peter Varhol, who is talking about Mobile Apps and the Role of Load Testing. Here’s my mindmap.

wpid-img_20140930_113431.jpg

Stefan Gwihs and Philipp Strelka talked about the use of emulators and simulators in mobile testing.

wpid-img_20140930_103213.jpg

Some interesting stuff, particularly about how a test approach should not be purely UI driven. My mindmap is here.

First the keynote. Unfortunately Daniel Knott couldn’t make it  – fortunately he put his slides up on slideshare.

Everything is not lost :) – we have a new keynote – Mobile App Quality at Paypal.

image

I’m At Mobile App Europe – Day 1

I’m at the Mobile App Europe conference in Potsdam.

This morning I gave my presentation ‘Mobile Testing, That’s Just a Smaller Screen, Right?’ – you can get the slides on slideshare.

I’m here for the rest of the conference and I’ll try and mindmap a few sessions.

Martin Wrigley is now talking about ‘Effective QA – Is It Really Necessary?’ Hint, yes it is :)

image

This afternoon I spent some time in Bill Matthew’s Security Testing Workshop. I mindmapped his slides.

Next Up – Christian Kaar from Runtastic with 10 Ingredients to Rock the App Store with Your App. Here’s my mindmap.

wpid-img_20140929_140549.jpg

The first one is ‘All Together Now – Apps for the Next Platform: Making Watches, Wearables and Web Work’ which was presented by Lars Kamp. A very interesting and informative presentation about wearables.

Exploring An Existing System

Recently I’ve been reading Elisabeth Hendrickson’s excellent book ‘Explore It!’. For anyone who has an interest in exploratory testing it’s a must own. I wish I’d discovered it earlier to be honest, since it gives so many useful hints and tips, as well as confirming that the ways of working that one has chosen are also recommended and used by others.

As well as using it for my own learning, I’ve been slowly going through the book and working out what parts I can use in the regular lunch and learn sessions that I run at work. While we practice exploratory testing, in fact it’s the cornerstone of our sapient testing strategy, there are some areas where I feel we could take approaches that could benefit not only testing, but also the wider business.

Working With Legacy Systems

We frequently work with legacy systems and so one chapter that was of immediate interest to me in Elisabeth’s book concerned exploring an existing system. When one has an existing system to test I find it’s all too easy to become primed by what others have already discovered, and to fall back on existing test cases (whether physically written down or in someone’s head). This can bias you, resulting in less effective testing.

Elisabeth makes the point that an existing system may well be unknown to the tester, but also may well be unknown to the whole team, or at least contain parts that are unknown. While the software fulfils a business need, how it actually goes about doing so may be less clear. That makes it ideal for exploration.

Recon Testing

James and Jon Bach like to call the initial exploration of an existing system ‘recon testing’. I like this term, by taking an initial session to explore and discover the basics of the software under test then one can then plan more effectively for future sessions, and write future charters in order to drive those plans (if you want to know more about the concept of session based testing and how charters fit in with this then have a look at James Bach’s explaination). Recon sessions help to map the territory and give insight.

During a recon session you can learn a lot, but the most important areas to ensure that you have gained insight into are:

  • The ecosystem in which the software under test resides.
  • Touchpoints to other systems.
  • Variables (things we change or can change).
  • Obvious vulnerabilities and potential risks.

Our Training Session

During our training session we carried out recon testing on the Staples ‘Easy’ button, and a standard service bell. This no doubt annoyed those in the room next door :)

Ding ding!
Ding ding!
Easy to test?
Easy to test?

By starting training with something simple, and not software based then it’s easier to pick up and learn the basics of a new technique without bias or the complication of software. We then compared our experiences, testing, (and the charters produced), with those that the Bach brothers produced when they carried out an exploratory testing exercise using the same product. Fortunately for us, their session is available on youtube, so it was an excellent addition to our de-brief. In it they explain the different testing techniques they use, and why. It’s well worth watching.

Enough Recon Testing?

One area to focus upon when conducting recon testing is whether one has conducted enough recon testing. Fortunately ‘Explore It!’ has this covered, recommending you ask yourself questions about the system that you have been exploring. If you don’t understand what the system does, how input and output works or how the environmental configuration affects the system for example, then it’s probably time to think about more recon before you move into more focused test sessions. Fortunately for the ‘Easy’ button and bell testers, (and those sitting near the meeting room where we had the lunch and learn), then this was not necessary :)

A Useful Addition

Recon testing is something that I think we’ll find is a really useful addition to our strategy. I’d certainly recommend that you check it out, and that you check out ‘Explore It!’ which contains much more useful information and techniques to use in your exploratory testing. I’ll certainly be using the book to inspire some future training sessions for the team.

Interviewed for Computer Weekly

I was recently interviewed for Computer Weekly, about the test strategy at Net-a-Porter, and my forthcoming talk at the Next Generation Testing Conference. It was an interesting experience and not one that I have done before.

I spoke about why I believe that Testing As An Activity is important, and why we should all test. The old axiom that “Testers Test and Programmers Code” is so outdated now and everyone needs to change. Testers are the testing experts in a team, and can help enable the whole team to own quality but they are certainly not the only one’s who should be testing.

You can read the interview itself over on the Computer Weekly site, and you can find the slides from the presentation at the Next Generation Testing conference over on Slideshare.

Interviewed For the uTest Testing Blog

I was interviewed recently for the uTest blog. I’ll be speaking about ‘Testing As An Activity’ at the forthcoming Next Generation Testing Conference, and so they asked me some questions about the topic, as well as some more general one’s about my thoughts on testing and how I started in the industry.

Worth a read I reckon, (I am biased of course). You can find the uTest blog post on their site.

More details on the Next Generation Testing Conference are on their site.