Every mobile app is unique. So is the mobile device, it works on. Today, mobile technology is growing at an exponential rate. Just how fast is the growth? The number of mobile devices is growing at a rate that is 5 times more than the number of people on earth!
Ensuring that your app works flawlessly on mobile devices of various configurations is a complex & challenging task. To make your app a worldwide success, your app testing strategy needs to effectively cover device diversity.
Parameters for Choosing the Devices
As mobile technology evolves, new versions of mobile devices emerge. Each model comes with its distinct features in terms of OS, screen size and other aspects. You have to factor in not only the OS of the device but also the OS version. It is critical that you choose the right number and type of device to test your mobile app. The following are the 5 parameters for choosing devices for mobile app testing:
1) Market share
First step is to identify the devices that are popular with your target demographics. You need to select the devices based on their popularity in the geographical markets that you are targeting. Your list should include all the important devices relevant to your market. Take into account the market share of each device before shortlisting it. Do factor in the critical features you want to test and the number of releases you plan for the year.
2) OS & OS Version
To ensure that your app supports various platforms, you need to test your app on different OS and their versions. Focusing on a single platform will narrow down your market share and slow down your expansion to newer markets. Framework and characteristics of various OS vary wildly from each other. Based on OS, the biggest market share goes to Andriod based devices where device fragmentation is highest. Android’s market share is nearly 6X more that IOS devices. Popular Andriod versions include Jelly bean, KitKat, Froyo, Gingerbread and Ice cream Sandwich. As your app expands to new markets, you need to include more devices in your testing strategy. You also need to ensure the updates you release work well in different OS versions.
Top brands for android devices include Samsung, Motorola, HTC, LG & Google Nexus. You can create an app testing device matrix to identify the combination of devices that can get you the broadest coverage in terms of OS, model and brand. You can prioritize the devices as most important, secondary and additional based on their relevance to your target demographics.
Based on the phone’s model, the attributes and characteristics differ. While selecting a device for testing you need to take into account, its screen size, resolution, PPI & other hardware features that can impact your app’s performance. You should also check the disk space & data consumed by your app. Your app is also affected by the processing power of various devices.
With over 400 different mobile networks in the world supporting various network technologies like GSM & CDMA, it is crucial to factor their impact on the performance of your app. To understand the effect of network and varying conditions posed by location, it is necessary to study the app in real world conditions. Crowd sourced testing is a viable and cost-effective solution for testing your application on different networks in various locations.
Challenges In Attaining Device Coverage
Traditional approach to meet device coverage involves the use of in-house devices and lab-based emulators. The cost of acquiring all the mobile devices in the market is very high & non-viable. Further, testing your app on a handful of in-house devices will not help you in getting an accurate picture of your app’s health.
Depending only on emulators implies more hassles in configuring them and the risk of getting inaccurate results. They can never replicate the quirks and characteristics of a real device.
Another option is to depend on device farms. However, the charge for device farming is hefty and every minute of a device you farm, will cost you money.
To grow big, you need greater test coverage that can ensure your app’s success on a large number of devices. A healthy mix of emulators and real world testing can give you the device coverage you need to make your app successful.
Key Advantages Of Crowd sourced App Testing
With crowd sourced app testing, you can test your software application in real world scenarios to ensure that it works as per expectations. The key merits are as follows:
Speed: Crowd testing enables you to run multiple test cycles simultaneously, delivering faster results and reducing the time to market.
Cost: Crowd testing is a performance based and cost- effective form of testing that gives the most accurate results minus the investment in full time QA hires, expensive device farms and in-house devices.
Scale: The scope for scaling is more with crowd testing as you can access many testers from diverse back grounds to work on your project.
Coverage: With crowd testing you can test your application across different configurations and locations with a global team of crowd testers.
By analyzing your app quality goals and target demographics, you can strategize the type of mobile app testing and extend of device coverage required to make your app a winner.
Need help in mobile app testing? Shoot us an email at firstname.lastname@example.org & we’d love to help out.
This blog post was contributed by Tharika Tellicherry, from the Marketing team at 99tests.