Mobile App Test Coverage Model : LONG FUN CUP

Smart phones changed the way we interact with software. Once mobile apps were used to kill time but now there are apps those could save lives.

They are changing the way live, thus we need to change the way we test.

Smartphone applications are developed with immense creativity and effort. Mobile users demand a sleeker experience with applications compared to desktop users. The mind set of mobile users is very different from web or desktop users. Smartphone apps are used on the move (e.g.: while walking or using a toilet), and mobile devices have a lot to offer through hardware location tracking, gyroscopes and other integrated features.

LONG FUN CUP model helps to achieve coverage at UI  level. This is designed factoring in the new touch generation mobile devices hardware, software and users mindset.

LONG FUN CUP

Location: It’s a sin to test mobile app sitting at your desk, get out!

Location tracking is a key feature offered by mobile phones, which any app must make use of it based on its core functionality. For instance, if it’s a cab booking or food ordering app, app should be smart enough to track your current location and provided suggestions surrounding your locality. Unlike web apps, where you expect user to enter location details, here the work flow should be different. Never test such apps sitting at your desk. Check if the app asks for user permission before tracking the location, does app allow users revoke the permissions? Both Android and iOS provide options to mock your locations for testing the app using emulators and simulators.

Orientation: It’s a sin to test mobile app sitting at your desk, lie in the couch.

People can change the orientation for various reasons, any mobile app should be able to provide a consistent user experience across different orientations. Test all your screens, pop-ups, toast messages, forms in all supported orientations. There are instance where the filled in form data disappears when you change the orientation. Never test a mobile app just sitting in your desk, take it to rest rooms, lie down in a couch and use it while traveling and test if it gives the same user experience with all different orientation.

Network: It’s a sin to test mobile app sitting at your desk, switch networks.

Mobile devices supports both cellular and wi-fi, they can automatically switch between any available networks. How does your app behave when the device switches between networks? These are critical especially if your apps core purpose is to send and receive data to remote server like e-commerce, banking, broadcasting apps. Make sure all such functionality behavior is tested for all such varying network switching, network strengths.

Gestures: In mobile world, app responds to gestures not clicks.

Does your app supports all standard gestures? Is it consistent across the app? If it uses any new gestures, is it easy for the users to understand? I would recommend to play The Room game (a paid game available for both iOS and Android) to understand the unlimited potential of gestures.

Function:

Any function that defines or distinguishes the product or fulfills core requirements. Test for interactions, error handling, starting and closing of the app, file access, navigation, multimedia, and sync. Did you try tapping all GUI? Did you fill in all the input fields? Did you navigate to all the screens? Are you able to navigate back?

User Scenarios:

How easy or how hard is it to complete a task using the app? Is the first time user able to understand the work flow? Any scenario should have a story that is credible, motivating, complex and easy to evaluate. List possible users and list them the ways they might use the system to accomplish a task. Try to think about disfavored users and how they might try to exploit the app. Compare with competitor or web interface to get more real life scenarios. Refer: An Introduction to Scenario Testing

Notifications:

Notifications enable an application to inform its users that it has something for them. How does your application use notifications? How easy is to turn on or turn off your app from notifications? Does the application use local or push notifications? How does your application behave if device is in sleep mode? Does the app provide too many notifications? Test for all available types of visual notifications, sound and vibration.

Communication:

These days’ people use mobile phones for various purpose, but the primary purpose for voice calls. How does the app behave after interruptions by an incoming call or an SMS? Test for such interruptions from voice communication and all necessary functions of your app. Many times app will not be able to recover after attending a voice call for a longer duration.

Updates:

Unlike desktop mobile app gets frequent updates. Not only app updates but as a tester one has to be aware of OS updates as well. How the users are informed about updates? Does the app support the silent update? Track what changes or new features are available in the latest OS update. Analyse if the app needs any modification because of the OS updates.

Platform:

It is very critical for any mobile app tester to have a good understanding of the popular mobile platforms especially Apple and Android. Be a  fan-boy of either one of these platforms (it’s impossible to be a fan of both😉 ) So, you know why Apple and Android does certain things in certain way. You need to be aware of the history and also the latest trends in the mobile platform. What tools are available for testing and what kind of test-ability layers are provided by the platform. Example Developer Options in Android, Instruments in Xcode.

Testers also should have good knowledge in App store Approval process, HIG, Android Design guidelines. This helps to take the app to the market very quick. It takes one to two weeks to get it approved by Apple App Store, so rejection wastes a lot of time. So, testers has to have a checklist handy to check those approval and guideline tests.

Testers should also should be aware of android fragmentation, you will never know in which device your app works and which it may fail. So, should keep an eye on the popular devices, new devices and the android OS market share to select an optimal device matrix to test.

12 thoughts on “Mobile App Test Coverage Model : LONG FUN CUP

  1. This talk was excellent, I will definitely be in touch over the next month to discuss this further with you. Great work DS.

  2. I agree with you, mobile is the future and testing apps has become even more important. Gallop Solutions has a upcoming event on “The Best Practices for Testing Apps” come be a part of it where presenters will share how to automate mobile app testing on different platforms. Please register and be a part of the where to get to learn many things about mobile testing: http://lf1.me/uRb/XamarinWebinar

  3. I think the word sin might have some negative connotations or might have the potential to offend some people when used out of context. I think you are trying to encourage people to diversify their testing strategies; using/testing their apps in the context of how people generally use mobile devices.

    From a UX point of view there is also accessibility to consider. I have the iOS VoiceOver short cut enabled on all of my test phones to enable quickly turning it on/off. VoiceOver isn’t just used by visually impaired users, I have it enabled on my regular phone because sometimes I like to absorb information orally rather than visually.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s