Mobile testing is its own beast. The tests run on mobiles apps are in many ways different from those you design and perform on other types of applications. For example, there are the obvious differences in form factor, available memory, and user interface. But what can make or break a mobile app experience often has more to do with performance than anything else. Poor performance lands many mobile apps in the trash, sometimes after only a few minutes in the user's hands.
Do you know the differences in performance expectations between Wi-Fi and cell tower signals? Are you aware of what network types to test for? Do you realize that the RAM limitations of the average mobile device make your tests similar to those for a PC built in 2000?
If you’re just getting started with mobile testing, these are just a few of the issues you’ll face when working on Android or iOS apps. To help keep your team’s apps out of the trash bin, we've launched a new site, TechBeacon Learn, with a Mobile Testing track that includes a series of technical articles and tutorials that take you from the basics all the way up to advanced topics such as multi-device testing and test automation. Each track is written by experts, including Matthew David, senior manager of the Mobility Center of Excellence at Kimberly-Clarke; Antoine Aymer, mobile technologist at HPE; Justin Rohrman, consulting software tester at Excelon Development; and many others.
You probably know at least a little about software testing. So your first questions are likely to be:
- What traditional testing types can I keep using for mobile apps?
- What new testing types do I need to use for mobile apps?
- How do I begin testing mobile-specific hardware?
- What mobile-specific testing tools do I need?
The answers spell the fundamental differences between mobile and traditional testing. That’s a start. Here are the highlights from the Mobile Testing track and a few key takeaways for both mobile testing newbies and more advanced test engineers.
Test teams and mobile testing maturity
Mobile applications flip Mike Cohn’s traditional testing pyramid on its head. Mobile testers must put a greater emphasis on manual testing and real-device testing, which requires more time. If you’re moving a team of traditional testers onto the mobile avenue, you’ll most likely need to add several types of specialization to your team. Otherwise, you might consider building a mobile testing team from scratch.
A good software tester can come from anywhere, and you don’t always need a large number of people. The best strategy is to address your company's most immediate needs in mobile testing while training for the future.
As you get started, you will certainly be aware of specific areas for improvement. If you want to know your team’s strengths and weaknesses in mobile app testing, here's a maturity model that can help. Start with a self-evaluation, determine what your team needs to improve, and learn how to grow its capabilities toward world-class mobile testing prowess.
The mobile testing lab environment
Buying or renting a large number of mobile devices can be expensive, so how do you narrow down the specific devices that you test? How do you obtain all those devices for testing? You’ll need to develop some strategies for testing a broad range of devices without incurring a great deal of cost.
Just think about mobile fragmentation—the diversity that exists mostly among Android devices, since Androids use an open-source operating system. In 2015, there were more than 24,000 different Android devices. Wow. But by using a device grouping strategy, devices can be grouped by shared characteristics, and mobile testing can be made more manageable.
Understanding the options for mobile device grouping, then obtaining and maintaining devices for test, are the concerns of a mobile test lab. Other test lab issues include the use of emulators and simulators for mobile devices in testing versus real-device testing. You'll want to use both methods if you plan on having a mobile testing strategy that is both cost-effective and accurate; your decisions to use either model will partly depend on your app's specific quality assurance needs and your budget.
However, the pros and cons of emulators and real devices are definitive, and so are the best practices for using them. Emulators and simulators are great for sanity-testing an app's user interface and catching the most common bugs. But thorough testing requires an investment in real devices that, to be affordable, will require you to invest in a real-device cloud testing service or build your own test cloud.
Onward with mobile test automation
Manual testing is a critical need within a mobile app test team’s skillset. But to scale testing capabilities, you need to automate some tests. The question is, which tests should you automate?
The answer has to do with the ROI of automation. An automated test requires the overhead of writing the test, as opposed to simply running a manual test. So the ROI of a test automation project can be measured by the reusability of the constituent automation scripts of the project. To determine if the project is suitable for test automation and which test cases might be candidates for automation, you'll need to consider:
- The types of mobile apps being tested—native, web, hybrid
- The availability of the test environment—externalized or restricted
- The capabilities regarding multi-device testing—you need to identify a pool of test cases that are technically feasible to automate, then decide which of these should be automated
And what about the tools for mobile test automation? Because the mobile test landscape adds new tools all the time, you’ll want to know the most mature and successful tool types available. The different environments for test tools include:
- Emulators and simulators
- Real devices with a direct USB connection
- Mobile device clouds
Rarely will you see the case of one-size-fits-all tools in mobile testing. Since the ecosystem changes rapidly, you'll need to conduct frequent tool evaluations.
Drill down: Visit the TechBeacon Learn Mobile Testing track
These are just the highlights. To get more detail on any of the mobile testing topics above, follow the embedded links to related topics in TechBeacon Learn's Mobile Testing track. You'll find more on mobile testing requirements and techniques, including links to other sources and recommended reading.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
