Category Archives: mobile

blog, Experiences, mobile, Testing

Facebook Overwriting Email Addresses – Breaking Rule #1

2 Comments

As you are probably aware, Facebook has let what looks to be a pretty serious bug out into the wild. First announced over the weekend, and confirmed yesterday, was the news that users who use Facebook as their primary storage for contact information such as email addresses, had found that these addresses on their mobile devices has been overwritten with @facebook.com addresses. Without them knowing or accepting any such change.

 

What Happened?

The official line goes something like this:

Contact synchronization on devices is performed through an API. For most devices, we’ve verified that the API is working correctly and pulling the primary email address associated with the users’ Facebook account.

However, for people on certain devices, a bug meant that the device was pulling the last email address added to the account rather than the primary email address, resulting in @facebook.com addresses being pulled.

We are in the process of fixing this issue and it will be resolved soon. After that, those specific devices should pull the correct addresses.

Now let’s be clear. This is very serious. I’ve worked in the mobile industry for the last 12 years and if there is one thing that is sacrosanct it is user data. You do not change, delete, update or generally mess around with anything that the user has stored on their device without giving them the opportunity to tell you to stop it first. Users have a much greater emotional attachment to their devices than they do to their desktops. They rely on them. Suddenly finding that you cannot contact someone is extremely annoying. Finding that is because another company has changed data that is yours is far more serious.

 

So Why Did This Happen?

I’ve been musing on why this bug may have got released. After all, this seems a pretty obvious use case. Of course Facebook is famous for a) not having testers as such and b) famous for adopting a test in production methodology. Could either of these be to blame?

Would these problems have been found by using manual test techniques? Facebook is a primary user of test automation and I can’t help but feel that a bug this obvious would be found by a few skilled testers adopting an exploratory test strategy.

I also wonder how much of a place the test in production has in the mobile world. In the desktop world, where users are always connected and backups are plentiful then rolling out updates and observing what happens is OK. Just roll back. It’s not so easy on mobile. Being on a mobile device means that you probably don’t have access to backups right away. You may be in a poor signal area or away from WiFi. And so you are stuck. Stuck out of the office or away from your friends and unable to contact people. And if you are stuck then you will become far more frustrated.

It seems to me that there is a typical gap in test strategy at work here. A bug that only manifested itself on mobile, uncovered as a result of changes made at server/ desktop level. When companies start to move onto mobile then this is pretty common. Failing to adopt a combined test strategy, treating mobile and desktop equally, or pushing more testing towards mobile can leave dangerous gaps.

It’ll be interesting to see how Facebook responds to this situation. I don’t know Facebook’s test strategies in detail but it seems to me that something needs changing to adapt to the world of mobile.

 

 

Experiences, mobile

Time To Start Speaking In Public

Leave a comment

Yesterday I attended the Next Generation Testing Conference in London. Whilst on the face of it this is nothing new, I’ve been to conferences before, this time I was presenting and also took part in a panel session. This was something new for me; I’ve often presented within Nokia, and have run a number of training courses internally, but never to paying public so the pressure was on to make sure that things went well.

The Next Generation Testing Conference is not one that I had been to before, and this time the organisers UNICOM were also trying out a new format, with more emphasis on expert panel sessions, all focused around three main topics:

  • Testing Today – What are the main challenges?
  • New Tools and Techniques
  • All about Agile

The day started with coffee, introduction and some brief networking; there were people from many different areas of testing in attendance, ranging from finance to TV, all with testing and Agile as the common denominator. Then it was straight into a case study presentation about automated testing from Bertrand Meyer. He made some interesting points, some I agreed with and some I did not, and my reaction seemed to mirror the general reaction from others in the audience, as we spoke more about what we’d seen in the first coffee break.

Then it was over to my part. Dr Richard Sykes was doing a great job as facilitator and the first panel session, “Testing Today – What are the main challenges?”, with myself, Tony Bruce and Chris Ambler started. We got some good questions from the audience and some good discussions going, both with the audience and also between ourselves. The panel format worked well and made the event seem less formal than others that I have attended.

Panel session over then it was straight into my keynote “Mobile Testing, That’s Just a Smaller Screen, Right?” I spoke for about 40 minutes about mobile testing, giving the audience an overview of the mobile world as it stands today, pointing out the main challenges and areas to focus upon. This was followed by a quick overview of tools and techniques to use, and answers to questions such as “Which devices should I test on?” and “Where do I get all my devices from?” The time went quickly, it was good fun to get back to doing some presenting, and my part seemed to be well received from the feedback I got during the lunch interval. Plus some inevitable requests to help fix people’s phones 🙂

After lunch it was then time to relax and enjoy the rest of the conference. Two more panel sessions, “New Tools and Techniques” and “All about Agile” were well received and the participants were well versed in their subject areas, and able to give some great examples and tips. Sandwiched between the two panels was a bold, and very good, presentation from Colin Weaver at DB Consulting, on “10 Key Behaviours for a Successful Agile Tester”. This prompted a lot of debate in the room, regarding which behaviours were and were not applicable or specific to Agile, and which were in fact not needed at all. Healthy debate continued for the rest of the afternoon until wrap-up.

Overall it was great to be involved in a conference like this one, and I think the panel format worked very well. Thanks to UNICOM for inviting me to speak, and to all those who spoke as well. I hope this can be the start of more experience like this and I’m certainly now on the look out for more opportunities to present or run tutorials and workshops.

blog, mobile, Testing

Smartphone Sales Pass Feature Phone Sales in Japan for the First Time

Leave a comment

Something interesting has just happened in the mobile market. You may have missed it or it may surprise you if you live in the US or Western Europe and are a middle class iPhone or Android owner.

What’s happened is that smartphone sales have passed feature phone sales for the first time. That may surprise you, you probably thought this happened years ago. After all, we’re all smartphone users now, right? Nope. And it didn’t: worldwide 70% of device sold are still feature phones (meaning cheaper devices running OS’s like Nokia’s S40 or other proprietary OS’s, typically with smaller screens and a lack of multi-tasking). Although the amount of money that manufacturers make on these devices is much smaller than smartphones, they ship in their millions and billions. Nokia recently shipped it’s 1.5 billionth S40 device for example.

You can read more in the MobiLens report which surveyed over 400 Japanese customers, and was compiled by market-watcher comScore  for the three months to February 2012. A quote from the report:

“Smartphones surpassed feature phones as the most acquired device type in February 2012, signalling an important shift in Japan’s mobile market,” said Daizo Nishitani, vice president of comScore Japan KK. “The rise in smartphone adoption opens the door to tremendous opportunity for publishers and advertisers to expand their reach and increase engagement with key consumer segments through this channel. Japanese mobile phone users were already highly engaged with their devices, but with the added functionality and higher levels of mobile media consumption we should expect to see significant changes in behaviour among the Japanese mobile population in 2012.”

Why Is This a Big Deal?

Japan typically leads the smartphone market and this is therefore a good indicator that slowly but surely the tide is beginning to turn towards smartphones in mature markets.

For testers this means more opportunity – smartphones typically mean a more open OS and therefore a significantly greater number of complicated applications that require testing. Feature phones are typically tested primarily by the manufacturers themselves; the only 3rd party runtime available is normally the Java ME platform and whilst there are a lot of applications launched written in Java (check out GetJar if you want some proof), there’s no evidence to suggest a large testing population at work ensuring that they work. Feature phones are also more likely to be lower powered, with smaller screens, ITU-T keyboards and generally lower spec without hardware like GPS.

However, with this move away from feature phones also comes further testing challenges; as the market switches to smartphones then it is inevitable that this will mean greater fragmentation of the OS’s themselves as manufacturers attempt to cover more and more price segments with different products. This will mean more display sizes, more hardware configurations and more differentiation in mechanics. For testers this will mean increased complication and mobile device testing strategies will need to evolve further than previously to cover a wider range of devices under test.

Also, as the market evolves then so does the installed base on devices. This presents additional challenges and further fragmentation issues. Ignore the devices in the field at your peril.

management, mobile, Testing

I’m Speaking At the Next Generation Testing Conference on 23rd May

Leave a comment

I’m excited to be able to announce that I’ll be speaking, and sitting on a panel, at the Next Generation Testing Conference which is on Wednesday 23rd May in London.

The conference is in its seventh year and is trying out a new format this year, with more emphasis on panels and discussions, together with case studies. This should hopefully mean that there’s some great audience participation and discussion on software testing.

I’ll be on Panel 1: Testing Today – What are the main challenges? It’ll be moderated by Dr Richard Sykes and they’ll be a group of us representing various areas of software testing, including Tony Bruce of London Tester Gathering fame. We’ll be debating various topics including:

  • Testing in the cloud
  • Testing mobile applications
  • Testing Big Data migrations
  • Games Testing
  • Non-software systems testing

I’m then be presenting a case study: Mobile Testing – That’s Just a Smaller Screen, Right? This will go into the background behind mobile device testing, how it differs from the desktop world, and giving some pointers towards areas to consider when formulating a mobile device strategy.

Hope to see you there. You can find out more about the event at http://next-generation-testing.com/

 

blog, Experiences, mobile, Testing

Q4 Smartphone Sales – What Does It Mean for Testing?

Leave a comment

Gartner have recently published their Q4 2011 smartphone sales report. You can view it here: What does it tell us about the current state of the industry, and affect can this have on our work as testers in the mobile space?

Operating System 4Q11Units 4Q11 Market Share (%) 4Q10Units 4Q10 Market Share (%)
Android 75,906.1 50.9 30,801.2 30.5
iOS 35,456.0 23.8 16,011.1 15.8
Symbian 17,458.4 11.7 32,642.1 32.3
Research In Motion 13,184.5 8.8 14,762.0 14.6
Bada 3,111.3 2.1 2,026.8 2.0
Microsoft 2,759.0 1.9 3,419.3 3.4
Others 1,166.5 0.8 1,487.9 1.5
Total 149,041.8 100.0 101,150.3 100.0

Android Is Still King and iOS Is Increasing

Looking at the volume data, i.e. the number of devices running each platform, we can see a number of key points. Android increased market share from the same time last year, up to 50% of the worldwide market from only 30% a year ago. This represents a lot of devices. Android covers a wide price bracket, and devices based upon Android are available in a number of different form factors, display sizes and hardware configurations. The platform is clearly fragmenting even more significantly in order to covers these different needs.

Apple continued to increase market share, up to 24% of the market in Q4. The launch of the iPhone 4S has had an affect, as well as cost cutting of the earlier devices, a standard Apple launch policy.

As for the rest – Symbian decreased from 32% to 12%, primarily as a result of Nokia’s decision to announce the death of the platform and the knock on effect to consumer demand. RIM’s Blackberry OS lost ground, down to 9% from 14%. Microsoft’s WP platform didn’t fare well, only 1.9% of devices sold ran this OS.

 

What Does This Mean for Testers?

What’s very clear from the smartphone sales in Q4 is that there are still a large number of manufacturers in the market, and they are producing a lot of products. Growth was 47% year-on-year. This has some impacts for testing:

 

  1. Mobile applications will become impossible to ignore for a lot of companies in 2012. That means mobile applications testing will become more and more important.
  2. There are a lot of different device manufacturers and OS’s.
  3. Testers will need to cover a lot of devices and test strategies will need to reflect this.
  4. Anecdotal evidence is that the testing and QA efforts are not increasing to meet the current demand. Many applications are launched with serious bugs still present, indicating testing was not sufficient.

 

Probably the most difficult decision when designing a mobile test strategy is to decide the coverage of the OS and of the devices themselves. A typical mobile application launch strategy these days will focus on iOS and Android as the primary launch platforms for good reason – these are where the market share is, and therefore where the money is. Typically a third launch platform would then focus on Symbian, Blackberry or WP. For testers this means that there is a need to learn the skills and gain the experience with these platforms, and a clear focus on iOS and Android will initially be a good strategy.

For anyone who focuses on Android then there are additional challenges. The sheer variety of Android devices available now is staggering, from cheap, often un-licensed local brands right up to the flagship devices running the latest version OS version,  Ice Cream Sandwich. This poses many issues for anyone developing mobile applications and those testing them – being able to cover all the different configurations is difficult and care is needed to ensure sufficient coverage. It can help to know the target market for the particular application, the sort of devices available to the customers and the expectations towards key criteria such as performance and usability, as well as just ensuring that the functional aspects of the application are OK.

 

Mobile Website Testing Needs Additional Focus

For those testing mobile websites then the problem becomes even larger. A mobile test strategy for a particular application can be designed around the particular OS that application is written for, and relatively easily adapted for other OS’s at a later date. The same cannot be said of a mobile website strategy. There’s a need to provide as much coverage as possible across a significantly wider selection of OS’s and devices, some outside of smartphone scope as well, in order to ensure that the website is functioning correctly and adequately displayed. Tools such as the validator from W3 http://validator.w3.org/mobile/ can help to some extent but they are not a substitute for real testing on a real device.

 

Covering Most Devices

Covering the largest selection of devices is very important. It is not easy. Companies such as SOASTA, PerfectoMobile and DeviceAnywhere are worth checking out, since they take away the burden of device ownership from the company or tester. Crowd sourced testing will become important as well, and there are a number of companies working in this space. But there is significant scope for mobile testers here, and a significant need for more testing than currently.

Overall we can see that the smartphone sector is still growing and the market for applications is increasing. Anecdotal evidence is that the testing and QA efforts are not increasing to meet this demand, and that’s where testers can play their part.