Mobile Application Testing

Challenges in Mobile Application Testing:

• Diversity of the Device Environment: The realm of mobile computing is composed of various types of mobile devices and underlying software (hundreds of device types, more than 40 mobile browsers). Some of the unique challenges involved in mobile testing as a result of this condition are:
I. Rendering of images and positioning of elements on the screen may be unsuitable in some devices due to the difference in display sizes across mobile devices and models. Exhaustive testing of user interfaces is necessary to ensure compatibility of the application.

II. Mobile devices have different application runtimes. Binary Runtime Environment for Wireless (BREW), Java, and embedded visual basic runtime are just some of the runtimes commonly available in mobile devices. Applications should be tested exhaustively for the variations specific to runtime.

• Hardware Configuration & Network-related Challenges: The mobile environment offers lesser memory and processing power for computing when compared with the traditional PC environment. Unlike the network landscape of the PC environment, the network landscape of a mobile device may have gateways (access points between the wireless internet and the cable internet). Some of the drawbacks of diverse hardware configurations and the network landscape of mobile devices are:

I. Limitations in processing speed and memory size of mobile devices lead to variations in performance of applications across different types of devices. Testing programs should ensure that the applications deliver optimum performance for all desired configurations of hardware.

II. Some devices communicate through WAP while some others use HTTP to communicate. Applications should be tested for their compatibility with WAP-enabled as well as HTTP-enabled devices.

III. Network latency (time taken for data transfer) will be unpredictable when applications communicate over network boundaries, leading to inconsistent data transfer speeds. Testing should measure the performance of applications for various network bandwidths.

IV. Gateways in a wireless network may act as data optimizers that deliver content more suitable for specific devices. This data optimization process may result in decreased performance for heavy traffic. Testing should determine the network traffic level at which gateway capabilities will impact the performance of the mobile application.

• Screens of various types: Different Mobile devices in market with all sort of shapes and sizes and all flashy looks make it difficult to exactly check the device compatibility of application.

• Keyboard versus stylus use: As we are talking here of Mobile Devices it just not consist of Mobile Phone while a big range of Mobile Devices is consisting of Personal Digital Assistance (PDA) as well as other smart devices .As PDA doesn’t consist of Keypad the things been done by using the touch screen functionality, so many a times designing and testing the applications for various such environment needed concern for such Keypad and stylus issues.

• Operating systems for mobile devices (Windows CE / Palm OS): Different operating system been loaded at different Mobile Client .The application should be well based at all the operating system and the uniformity and consistency of the application should not be lost out of running on different operating system.

• Proper internationalization: Normally we are not sure where the end user using our application is located, so it should be make a point with the Test Engineer that application doesn’t loose the motive out of running or accessing in other languages channel. The application should be in conformance with Unicode standards. Example it should display Chinese, Japanese and other characters properly.

• File formats: Mobile is just not specific to WML or CHTML, while nowadays it supports a lot more scripting language like XHTML and image formats like gif, jpeg other then wbmp and also j2me applications. So while testing the application we should make it a point to find out the support as per the device specification for all such file formats.

• File size: Different Mobile Devices comes up with different memory sizes and also their download capability varies according to that. We should make a point in our application that the downloadable image or game or any other thing should tune up with the Memory capabilities of respective device.

• Line and word-wrapping Features: Many a times using Label and Text Control we missed to take in account of screen size and display properties, although Pagination is a solution for this still we should take care of display specific issues in our application.

• Security issues: Mobile is just not a way of information sharing while e-commerce has a gateway to Mobile Phones. We have to look for the security issues when we are going to build transaction sites and other such applications. It should have proper planned session and cookie support as well WTLS and other security layer support for handling such transactions.

Guidelines for Testing Mobile Applications

1) Understand the network landscape and device landscape before testing to identify bottlenecks.
2) Conducting testing in uncontrolled real-world test conditions (field-based testing) is necessary, especially for a multi-tier mobile application.
3) Select the right automation test tool for the success of the testing program. Rules of thumb for an ideal tool are:
• One tool should support all desired platforms.
• The tool should support testing for various screen types, resolutions, and input mechanisms — such as touchpad and keypad.
• The tool should be connected to the external system to carry out end-to-end testing.

4) Use the Weighted Device Platform Matrix method to identify the most critical hardware/ platform combination to test. This method will be very useful especially when hardware/ platform combinations are high and time to test is low.
5) Check the end-to-end functional flow in all possible platforms at least once.
6) Conduct performance testing, GUI testing, and compatibility testing using actual devices. Even though these tests can be done using emulators, testing with actual devices is recommended.
7) Measure performance only in realistic conditions of wireless traffic and user load.