I’ve been reading a lot about test management recently. There’s some excellent posts out there, in particular I’d recommend you look at this one from Katrina Clokie, explaining the changes that are required for test management to remain relevant in the world of Agile software development and continuous delivery.
She also links some other articles which I would definitely recommend you read, if you are interested in the subject.
- The role of the Test Manager in an agile organization – Johanna Rothman
- Changing the role of test managers – Gojko Adzic
- Test Manager’s Survival Guide to Going Agile – Joel Bancroft-Connors
My Interest
So why am I interested? Well for starter I’m a Test Manager. Over the last couple of years I have seen my role change, from one of leading a separate, large test department, to one of managing testers across a number of project teams. It’s about to change again.
I’ve seen the challenges being a Test Manager in an Agile environment brings, in particular the difficulty in remaining relevant in the eyes of product and development managers, and the challenges of understanding enough about multiple areas in order to be able to support your team members. Being a Test Manager in a Agile environment can be isolating at times, particularly when the department is big, and the number of agile teams is large. It requires an ability to balance a lot of information, priorities, and tasks, across a number of areas. Stakeholder management and influence become key. Context switching comes as standard. Often it’s not much fun.
Through discussions with others, and looking at my own situation, I’m increasingly coming to the conclusion that the new ‘Agile Test Manager’ positions that Test Managers are moving/ falling into just don’t fit with the ways that teams want to work anymore. The team is more important than the manager, and, for example, choosing to keep discipline based management because it means testers are managed by testing ‘experts’ isn’t enough to justify it. Managers are not able to effectively support their people if they do not have the time and energy to keep fully in the loop with the team. As Test Managers get split across multiple teams, (primarily because having one Test Manager per Agile team is massive waste), then it becomes nearly impossible.
Continuous Delivery
Moving to Continuous Delivery complicates matters further. Giving a team complete autonomy to design, build and release it’s own code is an extremely motivating way of working. Do Test Managers fit in with this ? I’m not sure they do. Where independence and autonomy are key, management from someone from outside of the team just doesn’t fit, particularly when that management is only part-time.
Change Is Coming
So how do we change? Do Test Managers merely become people managers, desperately trying to understand what their people, spread across multiple teams, are up to? Are they there to help manage testing but not people?? What about the coaching and mentoring, the sharing of knowledge and expertise, and the personal development of testers?
As I see it I think we’re going to see a lot more of this sort of setup:
- Engineering Managers, who line manage an entire team. They understand the people best because they work with them day-to-day. No need for handovers, no need for performance feedback requests to other managers at review time, and no need to waste time and effort with coordination. Engineering Managers manage the whole delivery process and people involved. They may have come from a background of expertise is a particular discipline, but now they need to be able to represent all. But crucially they are focused on the management of a team who own a particular product or component and so share a single focus with their team.
- Test Project Managers, who manage larger testing projects/ programme’s and dedicated testing phases such as UAT and customer acceptance. No people to manage, just deliverables. This role is very dependant on the nature of the software/ hardware solution being delivered. It’s most likely not needed in a lot of companies.
- Test Coaches, who help organisations deliver the optimum testing possible. This means through coaching, mentoring, advising and working with engineering managers and whole teams in order to help them optimise their testing effort. Similar to James Bach’s idea of Test Jumpers, but with more focus on providing advice, guidance and strategy. In smaller companies they are much more likely to be exactly like the idea of Test Jumpers. Call them Test Jumpers, Test Managers, Heads of Testing or whatever, but the key point is that they are test experts who have the mandate to support testers in multiple teams but do not manage them. They can assist with recruitment and personal development if required, but are not a particular person’s official manager, and may get involved more with recruitment and personal development process, rather than people.
What Next?
The dedicated Test Manager, who manages testers and testing is not a role I can see continuing for too much longer. It is a hangover from the past, when large, dedicated test teams needed management, and it simply does not fit with how a lot of teams work anymore.
But, and this is a big but, I work in web, web services and mobile. I’ve seen the push for Agile and the push for Continuous Delivery because it fits the nature of the projects and technology used in these areas. Team’s are lean and projects are short. Almost certainly this makes me biased.
I would be interested to know what you think. Do you think the traditional Test Management role is reaching the end of the road? Or is it alive and well, and relevant in the area that you work? Why not leave a comment below and get the conversation started.
I agree with you. I am very consciously avoiding test manager as my only option for promotion as I don’t see a future in it. I don’t see any reason why an experienced tester can’t be a Team Lead with cross functional teams. I work in mobile and I see myself as a testing specialist in a mobile team – not part of a test team any more.
The only issue comes when testing is not a respected role by a dev team lead/manager. I have seen them treat testers as cheap programmers to do the boring automation tasks the developers don’t want to do. However, I’m not sure a test manager fixes this, the testers themselves have to show their value.
Thanks for reading and commenting. I agree – there is a definite danger that a bad dev mgr can ignore testing. A real risk to overall team quality.
I largely agree with you. I feel that the trend is to have less managers in general, and that the role of managers is less on controlling via substance expertise, and more on motivation based on the managee traits. How testing is done is based on teams trying to do a good job, and less by one guy calling the shots and making the rules. Good practices are shared from team to team, from senior to junior, and in specific testing communities.
Where the role of test manager will remain the longest, are the places where most dysfunctionalities are seen, where control is more important than visibility, where it is process over people, contracts over trust, decisions done on the top. So big projects, customer-vendor deals, etc. Test consultancy companies will also stick to this, as it’s easier to get more money by selling a manager than a specialist.
I also think that test managers
… will be replaced by more general dev managers, as people start to understand that testing is part of development, and not a separate phase. And the role of the tester who just does what is told will definitely vanish, hopefully quickly.
This will take time, maybe not happen at all in some places (there are entities like ISTQB who will push back).
And yea I used to be a test manager who always had to fight for my right to actually test. As I was too expensive to doing actual testing. Now I’m working as a tester in a medical device company, where we have no test managers. Or I should say, all testers are responsible for their own work.
Great to get your comments and hear your experiences. Thanks.
“Wither the Test Manager”
is both an article and later a talk by Paul Gerrard. Rephrased here: http://jlottosen.wordpress.com/2013/04/16/wither-the-test-manager/
The trend is there no doubt, especiallty in fast delivery orgs. But context plays a trick on you, because in SAP projects in infrastructe transmissions, in large scale integrated system updates – the drive to change the role of the test manager is not that high. .. but it should be.
…because change is coming! [mu hahahaha #evil grin].
But seriously, really yes.
I agra the role is changing for a Test Manager in an agile environment but there is still a need for such a profile so maybe solution is just a change of title. There is a blog by
Katrina Clokie (aka Katrina the Tester) that I feel covers the tasks and role today within a agile environment with distibuted teams..
http://katrinatester.blogspot.co.nz/2014/04/test-manager-in-agile.html
I think the 5 points given sums up the position/Role very well…
1. facilitation of inter-team communication across many agile projects within an organisation
2. presenting an aggregate view of testing utilisation to high level management
3. personal support, mentoring, and professional development for testers
4. being an escalation point for testers
5. budgeting or forecasting for testing as a service dependent on organisational process
Thanks for commenting. The need for Test Managers is organisational dependant of course, the size of the organisation being particularly important. I would argue that a good engineering manager or managers, supported by a test coach/ testing expert can also work as an effective model, especially when there is no need for testing as a service. The coach can provide mentoring, and advise the manager on development opportunities for testers, and can also run a testing community within the organisation. A good engineering manager will be capable of managing the whole team, irrespective of discipline, seeking support when required. They should be the escalation point for UX, dev, test, whoever is in their team.
It also depends on how higher level manager seeks to view their teams output – if the teams are organised essentially by Conways Law, and therefore inevitably become designed around the architecture of the product/s they deliver, then higher level management ‘should’ be more interested in the delivery capability of the teams rather than the overall testing status of the organisation.
I think this is a very good point and I agree with you, the test manager role is coming to the end at least within the Agile environments.
I rather prefer test coaches/mentors to the test managers. As a tester I want to be mentored, advised and supported for the testing that I’ve been performing.
Great blog
There is an element that I believe has not been considered in the post
The need of test managers is also strongly linked to the company mindset: I have unfortunately seen companies that did not trust their testers and did not believe that the people they hired were autonomous enough (even seen testers checking other tester’s work, insane!), hence they liked the idea of putting a test manager at the top to be accountable, and to deal with the problems of a team whose activity was not fully understood.
The more companies consider that quality is a team’s responsability and understand contribution of testers to a team, the more importance they will give to test coaching/mentoring as highlighted by Steve, and the less they will distinguish between managing a tester or another resource.
I strongly agree with the idea of test coach that inspires testers, brings along ideas: despite of the strong autonomy that some testers have, and no matter how good you are, you will always need to take a step back and get fresh ideas from outside. Peer testing in a way! Also it is optimizing the test resources in the company, the coach can favour regular test meetings, and contribute spreading an equal level of test knowledge and thoughts across the whole test team.
Now, regarding to test programme managers, I partly agree with that: this is clearly the limitation of having autonomous teams, how do you coordinate the testing across various teams? I agree you need a powerful hand to help driving the project as efficiently as possible. However since we are more and more considering testing as an activity, and the line blurs between any resource within the team, whether you are a designer, a developer or a tester, I don’t see why a global programme manager could not handle this, and I actually think the responsibility should not be divided between testing and others. This is why I don’t see test programme manager being a thriving role.
Hi,
I find this article really interesting. I am exactly in the same situation as explained by Stephen. I used to manage a team of 10-12 senior QAs directly in an organisation which was partly Agile. But now I am working in a fast paced Agile Environment and all my QAs work in their respective SCRUM teams. There are certain areas such as the following where I feel I am able to add value
– Inter team coordination and collaboration for larger projects(which come up only some times)
– Hiring and training – which I do a lot
– Managing Contractors and relationships with Agencies
– Overseeing any QA issues/QA resource issues in teams – when they occur
– Overseeing improvements in areas such as Automation
– Representing QA where required
However practically, on a day to day basis I generally feel demotivated because of the following
– Not being up to date with the happenings across the QA/IT organisation
– Not having a team to interact with on a daily basis – usually I am all by myself
– Not adding value to the day to day work in IT/QA
– As someone pointed, sometimes I wonder if people think I am valuable enough
– Not having my own resources whom I can assign certain tasks for example certain improvements – everything needs to be negotiated with the SCRUM teams as they have their work always planned
– Somtimes I feel I am not being utilised to the full potential
While I try to arrange bi weekly catchups with all the QAs – about 20 in number, sometimes I feel that though it helps me get some updates, what kind of value would I be providing(can I provide?) after gathering these updates. I am also wary about involving in each SCRUM team’s work as it might lead to taking on too much work where things are generally working fine.
I also wonder if this is how it’s going to be as more and more companies become Agile. I am a great fan of Agile but playing my role in Agile is kind of becoming tricky. Any ideas as to how others in similar environments are able cope, what kind of things they generally do would be interesting.
I agree with this blog and comment given by Usav & other people.
I feel isolated these days as each tester assigned to relevant agile team. The role is getting vanished or moving into different expectation.
One way to deal, to become more knowledgable in testing tool & be guidance what tools really fit in purpose for team. Provide technical guidance for those automation tool for team apart from just manual testing specialist. A role of automation technical consultant as such is my view.
I disagree with where you see Test Managers sitting. Swap out ‘Test’ (a rather narrow view of Quality Assurance) and replace with QA. Now your manager (or Director of QA) is responsible for the quality assurance from start to finish; assisting dev managers with their teams quality responsibilities; helping PMs understand estimates; teaching sales what happens when testing is reduced to give the client a ‘better price’; etc.
With a team of 20 testers in a consultancy, each one assigned to one or more projects, I feel very involved and hands-on with them.
What has changed is that I no longer have to worry about the project time frame, estimates or goals. I am still responsible for a tester failure, and I am tasked with ensuring QA from pre-sales right through to UAT support (inc documentation)
During the 1-2-1s I have with each tester I coach and mentor them, advising on best approach.
I also hold bi-monthly team meetings, run team competitions, and lots of other team building exercises to encourage mutual support across the testers.
To take the pressure off the testers I produced ‘pit-board’ style templates for Test documentation, meaning a Test plan could take 30 mins, a test Exit Report a maximum of 1hr to produce.
The key is to educate the Executives as to the true role of QA and not allow yourself to be pigeon holed.
Hi Pauline,
Thanks for commenting.
I feel differently, as you can tell from the article. I’m not sure how swapping out ‘test’ for ‘QA’ makes the situation any different, in fact I’d argue that it certainly would not help matters. Quality Assurance implies that the quality of a particular software deliverable lies with the ‘QA’ team. This cannot, and should not, be the case – the whole software development team is responsible for the quality of the software. I don’t see how the QA team or the Director of QA could be responsible for the quality of code that they, or their team, have not written. Testing is about discovering information that could threaten the value of software, and communicating those risks to the people within a project who are able to make decisions on release risk.
I can totally see that there is still value to having someone responsible for coaching and mentoring, improving testers skills, etc, as well as being the voice of testing within a company. But that does not necessarily have to be a Test Manager, i.e. a people manager responsible for management of only testers, rather than the whole team. The need for coaches and experts will still remain, analogous to Agile Coaching but, based upon my experiences and context, I see the Test Manager role as evolving away from people and team management.
I think the logic of this is inescapable in agile organisations. I was hired to build out a test function in a small web company that is also moving to an agile delivery model, and it’s basically impossible to do a traditional management role even when I’m not responsible for any test planning. How can I performance review people whose work I’m not familiar with? When everyone catches up with reality the Spotify model is the only thing that really makes sense in the company that I can see. The downside is that I think you’ll see a lot more companies silo without putting together the horizontal piece than those that do it right.
I have a few caveats though. The number of companies on pure agile models is still a small subset of the overall space. Lots of companies call themselves agile but have some sort of hybrid model or iterative model. And I think you are going to have a lot of companies in this space for quite some time. Test Managers are much more likely to persist to some degree there. Lots of testing gets outsourced too for various reasons, so within things like test consultancies you are likely going to have need for specialist test management and metrics.
I also do think there is an inherent web bias within Agile. Continuous delivery is great idea on infrastructures that can deployed in minutes and where customers are tolerant of shall we say “providing quick feedback”. There is still a lot of software that is still compiled and packaged and building it to support different architectures and flavours might take hours. There are also still places where customers have SLAs that require 99.99+% uptime and are not particularly tolerant of defects or missing features, There might a lot of customers with different and complex deployment models. You can pull in agile and XP practices but there might still be an extensive System test phase, performance testing et al carried out by teams that exist outside of cross functional groups. Complex integrations of software packages might lead you to a similar place where you just have to do a lot more testing than anything else. Even here though, cloud based computing is making non web software more web like, so I expect there’ll be some shrinkage there too.
The real question though is to what extent this applies to testing itself. Scrum or scrum like models are not particularly tolerant of specialisms that block flow, and testing represents a potential handoff / bottleneck within sprints. There is a danger that the critique aspects of testing get pushed to product and business analysts and the technical parts get pushed to developers. Something will be lost in that case, I think.
I must say that now testing has become a competency and skill, not a mere job of using just the common sense and run a team of 10-20 people. You need to keep yourself up to date on tools, technology, domain, latest trends…if you have all of this, no end of road for a test manager or anyone else. A test manager needs to understand this and get himself in the mainstream and not just focus on managing people issues.
Please check this
http://www.infoq.com/articles/agile-changed-test-management
Thanks. Interesting article.
I have a question here??? If am a test manager and if I finish certification for PMI ACP or PMI PMP then can I be employed as Project manager?
That would depend on the company I would have thought. You have a better chance I think.
I fully satisfied with you. I am deliberately staying away from test administrator as my choice for advertised as I don’t see a future in it. I don’t see any motivation behind why an experienced tester can’t be a Team Lead with cross practical groups. I work in mobile testing and also started blog on mobile apps testing http://www.mobileappstesting.com/.
I have been an Engineering Manager and also a Test Manager. I agree that the role of Test Manager and even the QA Engineer is changing, but it is largely ill defined in the era of the Agile.
Unfortunately Agile Scrum only allows time for ad hoc testing.
When does the QA Engineer do systematic Functional, Integration and System Testing?
If one is building bullet proof HW + SW in a regulated industry, say, a medical device, then ad hoc testing done during the very short two weeks sprint is simply inadquate.
The Integration Testing itself has become complex and time-consuming with the advent of internet enabled devices, cryptography and the multitude of iOS and Android phones and tablets.
As thinking Test Managers we have to request and ensure separate and longer sprints for systematic Functional, Integration and System Testing.
Perhaps the definition of Done needs to be redefined, viz. Done Coding and Done QA.
Thoughts?
Thanks for commenting.
I think scrum does allow for more than just ad-hoc testing, but in order for that to be successful then the whole team needs to be involved. It is about defect prevention rather than defect detection and this is a key aspect where the testing role changes. It becomes more about coaching a whole team so that they understand what quality is, rather than acting as a blocker or custodian of quality as testing has previously, and incorrectly in my opinion, become.
Definition of done is critical. At the end of a sprint the software should be production ready. If you have e a DoD that allows handoff to further testing, a testing sprint, or similar then you have merely taken a waterfall process and split it up into two week discipline based sections. If the testing doesn’t fit then maybe the teams processes need to change. Perhaps the sprint is too short? Perhaps the testing is too late and only focused on the completed software. Perhaps a risk based approach to the multitude of devices, etc using market data would be sensible?
In all these cases it comes down to coaching a whole team to own quality which I don’t believe needs a Test Manager in a role managing a separate test team. A test coach working across the team or multiple teams, managing the process of Testing but not those doing it, has worked better in my experience.
Stephen,
You have just opened the Pandora’s Box!
I hear Samsung is also using Agile blindly.
Samsung Galaxy Note 7 crisis deepens with reports of suspended production. The decision reported by South Korean news agency Yonhap follows repeated problems with the new device. https://www.theguardian.com/technology/2016/oct/10/samsung-halts-production-galaxy-note-7-phone-battery-fires
Samsung Galaxy S7 Could Come Earlier, As Samsung Uses Agile Method to Shorten Development
http://www.gsmdome.com/samsung-galaxy-s7-could-come-earlier-as-samsung-uses-agile-method-to-shorten-development
Regards,
DS
All approaches can succeed or fail and without knowing the context in great detail then it’s difficult to comment. I can say that I’ve been a part of an Agile transformation in a mobile phone manufacturer before and it proved to be pretty successful.
We are talking about tectonic cultural shifts.
Coaching is a great idea; but it will require a lot of coaching, cajoling and much more.
Defect prevention is also a great idea; it is also called writing good code, but despite all efforts the defects do creep in and thus have to be found, fixed, retested.
Many VPs and Dev Managers do not know the difference between a Test Plan and a Test Case.
Many Dev Managers and Software Developers do not know the meaning of words Unit Testing, Functional Testing, Integration Testing, End-to-End Testing, System Testing, Stress Testing, Performance Testing etc.
Most Software Engineers make very poor Test Engineers. Many SE’s simply abhor QA, when tasked with QA they will refuse to do it or flatly lie that that they have done it.
I do not see how one can find time, in a two week sprint, to read (unambiguous) requirements, design & document software and conduct Unit Testing, Functional Testing, Integration Testing, End-to-End Testing, System Testing, Stress Testing, Performance Testing etc. along with a keen oversight on test coverage and test traceability.
This is an impossible task on a software only project.
In the real world of new product development that consists of a constantly changing world of Material Science, Mechanical Engineering, Electrical Engineering and Software Engineering with n interfaces to legacy systems one is asking one to climb the Himalayas or perform neurosurgery.
Are we replacing Test Managers with Test Coaches; from those who took responsibility to those who are distributing and disbursing responsibility to the unwilling teams consisting of software “engineers” who are green behind their ears and can’t tell the difference between Ohm’s Law and Newton’s Second Law of Motion?
We should definitely take the good ideas from Agile Scrum and any other relevant disciplines.
But, it would be a bad idea to bury engineering quality under the rug of Agile manifesto.
Critical thinking still rules the world.
Ping back
https://jlottosen.wordpress.com/2016/11/01/shift-deliver/
https://jlottosen.wordpress.com/2016/10/16/shift-coach/
Excellent Article and excellent comments as well