Transition from web to responsive and mobile applications is even more rapid than from desktop to web. The first cell phone introduced to the market in 1983 was over 2 pounds heavy and made one voice call at a time. Today, most of us do not leave home without handheld devices spending over three hours daily staring and tapping at their screens. Our wearable gadgets and the Internet of Things generate data as fast as never before, and it is instantly stored in the cloud. Millions of applications and websites at fantastic network speeds struggle to capture and hold our attention. Critical bugs that go untested can bring a mobile application to a swift end. Let us figure out, what makes mobile testing a unique engineering skill.
- Dependency on Emulators
For desktop and web developers, platforms or browsers where applications are targeted are always accessible. Mobile development platforms support emulator or simulator to run the first round of testing before installing application on a real device. Such mobile testing techniques, as BrowserStack, Ghostlab, Adobe Edge Inspect CC, allowing to test devices with no physical access to, are popular today. Well-known browsers also offer built-in smart inspector tools. For example, Google Chrome has ‘override’ feature, enabling to emulate a handheld device.
Many common issues are effortlessly caught by means of emulator-based testing with no need to re-Installing apps on devices. Using emulators facilitates automation, but beware of limitations – they do not replicate advanced user interaction. Finally, in twenty two countries, there is non-profit grassroot collection of devices called Open Device Lab, which developers can come to and run their products on broad hardware range.
- Far more combinations to test.
Technology becomes outdated quickly. The rapidly increasing variety of screen size, shape, operating system, and hardware makes ensuring that mobile apps are working on all smartphones, tablets, and phablets supplied by major brands (Sony, Nokia, Samsung, Apple, HTC etc.) and on all platforms (iOS, Android, Windows, BlackBerry, etc.) extremely challenging. For example, today, there are over 4000 unique Android devices with almost endless variations of operating system on board. Many of its users do not want or have access to the updates, so testing on older versions will show respect to them.
Accommodating aesthetics on various screen sizes, layouts and configurations can be a challenging task. Readability, layout (wrapping, distortions, overlapping, spacing etc.), best use of space in landscape to portrait mode and rotation from one to another ‒ all those things are critical for usability. Ads and pop-ups must be displayed without interfering user’s shopping. Poorly designed user interface will never gain trust. Many features, navigation buttons, links that work for users on laptops and desktops may require modification to satisfy smartphone user.
- Network performance testing requires a lot of scenarios.
Most desktop and web users connect via LAN or Wireless, which are rather predictable compared to mobile networks. It is very hard to simulate real life situations: how connected application will behave with 3G, 4G, weak or powerful signal, without any network, cell, wifi network, switching from cellular to wireless? If the network is unavailable, non-communicating about this fact will frustrate the user. Many network based features require testing in motion. Ensure that all network operations work in the background mode.
- Interruptions and session management.
Default functionality of device must not be affected, keyboard behavior must not be altered by using the app. Check how much space application occupies. Back button, home button should act the same way as outside the app. Incoming calls, messages and push notifications should behave as usual while application is in use. Enter button should submit the form. Keyboard should automatically pop out upon focusing on a text field. Ensure the type of field determines the type of keyboard. Run application at absolutely low and going down battery levels, Ensure installation and application data is not damaged.
- Entry criteria and team communication.
Always make sure that the application is ready for testing. Developers, especially at initial stages of project, often edit code making application not stable. They should provide you a relatively stable build. Therefore, encourage them to use decent unit testing methods and ensure they cover code reasonably.The builds should not come too frequent; versioning should be consistent. You must have sufficient information about current state of the build, which feature is implemented and which is not. Test-driven development reduces the bug detecting cycle. Express sanity test cases should be in place to forego the complete testing scenario.
- Security concerns.
Making sure that application is protected against intrusions is equally important as assuring that it does not intrude itself or unintentionally accesses user’s data. Vulnerabilities to hacking, authentication and authorization policies, should always be clearly understood and handled. How much trail is left? How are the trails and logs cleared? – those issues are strategical. Firewall and penetration tests are carried out to ensure that nobody can access information on user’s device while using an app.
Many mobile applications exist in their sandbox, but have access to certain platform features. Phone book, pictures and multimedia are reachable to many other applications. Any flaw around unintended misuse of this data can ruin the trust. Mobile apps exchanging sensitive information are enticing targets for ‘man-in-the-middle’ (MITM) attacks where hacker can view and manage traffic. Another typical problems include SQL and command injections, insecure cryptography, poor session management, damaged access control lists etc.
- Figure out When To Stop
Any kind of testing may sometimes feel like never ending draining challenge. Absolute test coverage is an ideal, however, when it comes to mobile world, not only lifetime and cost constraints matter. Considering 100% of devices, browsers, and other condition factors is virtually impossible. Always keep in mind the intended customer’s needs and preferences.
After reaching a certain size or complexity level, bugs become inevitable in the app. They are often triggered by unusual conditions or cases. Therefore, it is not feasible to guarantee something is bug-free, but before finishing mobile testing, QA should reach a point where bugs go relatively rare, and let minor issues go. An application should be released in time and with minimum bugs, and the tester seeks for balance between those two. Keeping a list of known issues facilitates prioritizing updates and keeping users aware if some issues remain.
It is irrelevant how a visitor gets to your page or application, they should be offered the same features, interaction and appearance across popular devices and browsers. Customer doesn’t have a clue where the fault is. Even if the problem is the network, a 3rd party data provider, the corporate database or the damaged device, still the app developer will get blamed. And even worse – when there is a successful mobile service, but an infrastructure crashes under the high load.
