Test Automation Basics – Mobile Apps & Web and Games

If we are considering mobile application ecosystem of android and iOS, we find it dynamic, as both are suffer from software and hardware fragmentation. Especially, it is applicable for Android, however fragmentation also available in iOS ecosystem, as experienced with the launch of iOS 8.

There are many existing iOS applications available with its latest version that made clumsy on updated devices. Moreover, one should find typical issues in the new iPhone 6 and 6 Plus for Apple devices. Users with some options will find an important amount with older devices, especially to purchase new device for getting everything working well.

If we are considering Android, everything you find is different. Application and game developers are getting difficulty as OEMs launch software updates, latest devices, and customizations for their devices to maintain product fully supportable with all possible device alternatives. Creating some application and game work on quality devices is out of the question.

One-stop solution for common problem is professional automated testing software. It enables to create robust and reliable software with difficult technology and difficult competitive pressure. One of the affordable solutions is Automated software testing to this problem. It delivers three business advantages like:
  • Increased testing effectiveness
  • Faster time to market
  • Increased testing efficiency

Test Automation is best For Mobile App Development

Creating robust mobile apps is different from developing embedded and PC software. When developing mobile app, developer have to use tools and practices for that agility.

But doing something manually like testing an application is never agile, and thus, test automation has shown huge growth among developers and application, improving robust and better outcomes. Test automation as the part of the agile development procedure is normal for attaining better compatibility with devices, market and users.

Interestingly, many tools are available, but test automation stands on the first position. A significant value-add is offered by this tool to enable testing to be done in similar way. The next day, the tests will be confirmed and the outcomes will be ready for review.

As we all know that fixing a problem saves huge time and enable developers to finalize products earlier. Importantly, it gathers better quality with limited number of bugs.

Factors to Consider Automating Mobile App Testing

Now, we have to check how test automation can enhance the development process, increase speed up development and add value.

Costs And Assets

One will require time, people, infrastructure, tools and training irrespective of whether you go for manual or automated testing. Test automation will deliver a good ROI, depending on the project and application. Moreover, Automated Software Testing helps to minimize repetitive tests from weeks to hours. It is time saving that translates directly into cost-saving.

Incorporated Testing Cycle with App Development

Increase in efficiency and productivity with automation starts with catching a new mindset. During all development cycles, software tests have to be repeated to make sure the possible application’s quality. Software tests should be repeated when source code is different every time.

The software should be tested on all supported variants of OS for each release and all configuration of hardware. However, manually repeating would be costly and time-consuming.

Tools And Technology Used for Test Automation Frameworks

First of all, you have to choose the most robust and most cross-platform method for getting the most out of efforts and maximizing testing coverage. Moreover, one can use such automated methods for validating needs and to minimize costs through test-case generation.

But, the full automation of large software entities is available with a cost that some companies haven’t been read to pay.

Test Coverage And Reusability for Open Standards Mean

The depth and scope of tests can be increased by automated testing and importantly enhance software quality. Some long and thorough tests can be run automatically often not doable with manual testing.

Well, test cases comes with a full access to an application and test all factors, including data tables, file contents, memory contents, and internal program states to decide whether the product runs according to expectations or not.

With automated tests, one can easily execute many difficult tests when running test to deliver coverage that is not possible with manual testing. Moreover, developers have enough time to develop new automated test cases and integrated more interesting features.

Improve Usefulness And Conclude Sooner

Professional automated testing software is one of the best solutions for common problem, including how to create better and robust solution with ever growing difficult of technology and under massive competitive pressure.

Business results are improved by automated testing in three ways, i.e. greater testing effectiveness, greater testing efficiency, and a shorter time to market.

Various ways to Automate Mobile Testing

There are three ways to automating the testing of mobile applications such as:

Handwritten Test Scripts

Generally, it is one of the preferred options when you have idea about what you are doing and when you have programming-capable resource for doing the test scripts. Different alternatives are also available for test automation tools, frameworks, and integration for both open-source and commercial offerings.

Record-Playback Approach

It is the least error-prone approach as it nothing has written in code, however has limited functionality. Tests can easily and quickly recorded and after that, played repeatedly against different OS versions and device configurations. Such test concentrates on user-driven and user interactions activities.

Automatic Test Exercisers

A great way to smoke-test applications is provided by automatic test exercisers. No particular tests are required instead of focusing on testing user-interface logic like clicking buttons, opening menus, swiping and multi-gesture actions.

The least exact results yielded by automatic test exercisers, however delivering quick feedback on iteration of an application.

User Interface And Functionality

One can identify the success of application and games with users interface and its overall functionality. The two things that include visual appeal and gameplay are the important things to get right and one must ensure that device fragmentation doesn’t break any of these.

Different things in the UI require to be checked such as:

  • UI layouts and elements
  • Menu structure and functions
  • Screen orientation
  • Screen resolution

Graphics Performance

Software performance should be consistent across all devices variants among your users and test on many real devices are possible. It is advisable to create tests for determining how well your application and game responds to various levels of usage, including battery usage and performance that consider creating tests that last for hours.

Run load tests for determining whether your game runs effectively under a heavy load for a long time. Such performance test will evaluate like how responsive your game is on real devices.

Usability And User Experience

User experience, testing usability, and navigation flow cannot be completed on a desktop with a keyboard and mouse. So, it is advisable to use real devices instead of forget emulators.
You have to test how usable and entertaining your application is by considering two important things like
  • Background events
  • User interaction and responsiveness

Multi-User Features

Presently, multi-user support is common in both games and applications. Moreover, testing multi-player capabilities also play an important role and are more challenging needed real users for measure performance.

A game communicating is a typical case with the back-end server. One can also synchronize the back end with devices for better connectivity to get information about the gameplay.

Moreover, one can check test of various scenarios among them could affect on the game’s experience that results into negative feedback and lastly, game being uninstalled by players.

Social Integration

Another important factor is to integrate with social networks. In next step, you have to test this thoroughly with real iOS and Android devices with different operating system versions and different device configurations to measure functionality and ease of utilize.

Security And Liabilities

Some open-source components used by all developers in their applications. Such type of practice is used and recommended as it offloads the code’s development for non-core functionality. But, developers neglect identifying vulnerabilities and licensing restrictions with third-party code.

Breakdown: Android Test Automation Frameworks

Robotium – It is one of the popular Android test automation frameworks, supporting native and hybrid applications. With Robotium, one can easy to write robust and powerful automatic black-box UI tests for android applications.

UIautomator - delivers an efficient ways for testing UIs. An automated functional test cases are created that can be implemented against applications on real Android devices and emulators.

Calabash - If you are looking for framework for Android and iOS then Calabash is a cross-platform test automation framework. The best thing about Calabash is it understands syntax that enables non-technical people to develop and implement automated acceptance tests for apps on both of these mobile platforms.

Appium: Automation of native, hybrid and mobile web apps.

Appium is one of the best mobile test automation frameworks for mobile-web, native, and hybrid apps for iOS and Android. JSONWireProtocol is used by Appium for cooperating with Android and iOS application by using Selenium’s WebDriver.
Written in Node.js, Appium is an HTTP server, developing and handling multiple WebDriver sessions. The test is started by Appium on the device and listens for commands from the main server. It seems same as the Selenium server that gets HTTP requests from Selenium client libraries.

How to Set Up The Environment?

First of all, download our Appiumexample. You can find example in Java, Python, and Ruby. After that, choose clientlibrary according to programming language you will choose. For more information, you can switch to Appium’sdocumentation where all guidelines, help and extra material are available.


Firstly, you have to make sure that Python 2.7.x or later are installed (Note: It’s preinstalled on Mac OS X):

$ python --version

Now, you have to install Python (Linux users only):

$ sudo apt-get install python2.7

After that, you have to check Python’s “pip” module is already installed or not:

$ pip --version

Install pip if you’re on Mac OS X:

$ curl 
https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py > get-pip.py
$ sudo python get-pip.py
$ rm get-pip.py
Install pip on Linux (if it is not installed)

$ sudo apt-get install python-pip

In next step, you have to install the Selenium module for Python:

$ sudo pip install selenium

Now, you have to verify that Selenium is installed:

$ pip freeze | grep -i selenium


You have to make sure that Python 2.7.x or later are installed:

$ python version

If it’s not run properly then you have to download and run the setup from Python’sdownload center. In order to add Python environment variables, you have to go to “System properties” → “Advanced System Settings” → “Environment Variables” → “System Variables” → “Edit Path,” and after that, insert C:\Python27\;C:\Python27\Scripts\ at the end. It is supposed that you have installed Python in the default location.

You have to ensure to restart the command prompt for inviting new environment variables in to effect.

Now, you have to check if Python’s pip module is installed or not:

$ pip --version

If you don’t have install, then install pip:

$ curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py > get-pip.py
$ python get-pip.py
$ del get-pip.py

Lastly, install Python’s Selenium module:

$ pip install selenium

Running The First Tests

In that step, you have to download sampletest script and sample application.
Download the Appium sample test script and sample app. The APK for Android, the IPA for iOS and one sample test script are available in the sample package. Furthermore, three samples are

  • appium_example_ios.py
  • appium_example_android.py (for Android API level 17 and above)
  • appium_example_selendroid.py (for Android API level 16 and below)

Step: 1 Make Account And Upload The APK

You have to create an account. A freemium option is also provided by this service and you don’t require a plan for completing these samples by using real devices on a cloud service.

One can have different plans for accessing hundreds of device models. One can have to upload the APK or IPA to Testdroid’s cloud service before proceeding with running the test script through HTTP POST. Now, let’s try using cURL.

For Mac OS X and Linux:

$ curl -s --user username@example.com:password -F myAppFile=@"/absolute/path/to/app/SampleApp-iOS.ipa" http://appium.testdroid.com/upload


$ cd C:\path\to\directory\containing\the\application
$ curl -s --user username@example.com:password -F myAppFile=@"SampleApp-iOS.ipa" "http://appium.testdroid.com/upload"

You will get a JSON response with status: 0 and a unique identifier for the uploaded app upon successful upload.


status: 0,

sessionId: abcdefg-1234-5678-abcd-111122223333,

value: {

message: Uploads Successful,

uploadCount: 1,

rejectCount: 0,

expiresIn: 1800,

uploads: {

myAppFile: abcdefg-1234-5678-abcd-111122223333/SampleApp-iOS.ipa


rejects: {}



Step 2: Set Credentials And Other Parameters

Now, open the test script: Appium_sample_ios.py, in any text editor. You have to set screenshotDir to the path, where you want those screenshots to be kept on your machine. In DesiredCapabilities, you have to set your credentials on testdroid_username and testdroid_password.

After that, you have to set the myAppFile identifier against testdroid_app in Desired Capabilities. It seems like:

desired_capabilities_cloud = {
   testdroid_username:    testdroid_username,
   testdroid_password:     testdroid_password,
   testdroid_project:      Appium iOS Project 1,
   testdroid_target:       iOS,
   testdroid_description:  Appium project description,
   testdroid_testrun:      Test Run 1,
   testdroid_device:       testdroid_device,
   testdroid_app:          sample/SampleApp-iOS.ipa,
   testdroid_platformName: iOS,
   testdroid_deviceName:   iPhone device,
   testdroid_bundleId:     com.bitbar.testdroid.SampleApp-iOS

Step 3: Run The Test Script

You have to implement the below given command:

$ python sample-appium_ios.py

The output of console seems like:

Click here for see this image.

Step 4: Get Results From Cloud

You can see screenshots available locally on your machine in the directory that specified in Step 2. Now, you have to log into your cloud service and navigate to the project’s name, as defined in the _project attribute of DesiredCapabilities for accessing following log files:

  • Appium’s server log,
  • Logcat and instruments log.

Some Advanced Example: By using Appium for mobile game testing with image recognition.

We are using Supercell’s popular mobile game Clash of Clans in this example. It is one of the interesting games, many of you have played it and thus, you should be familiar with its gameplay and look. It is one of the best example of basic clicks through game’s tutorial.

## Example script that tests Clash of Clans tutorial first steps
## Works on different resolutions, both iOS and Android

import unittest

from time import sleep
from AppiumCloudTest import AppiumCloudTest, log
from selenium.common.exceptions import WebDriverException

class ClashOfClansTest(AppiumCloudTest):
   def setUp(self):
      # AppiumCloudTest takes settings from environment variables
      super(ClashOfClansTest, self).setUp()

   def test_tutorial(self):
      driver = self.get_driver() # Initialize Appium connection to device

      sleep(10) # Wait for game to load

      # Use this to get detected screen hierarchy
      # print self.driver.page_source

      # Dismiss the in-app purchases dialog if it shows
      okayButton = None
      if self.isAndroid():
            okayButton = driver.find_element_by_id('button3')
         except WebDriverException:
            log("There was no in-app purchases dialog")
      else: # iOS
         self.driver.implicitly_wait(5) # Wait only 5 seconds to find it
            okayButton = driver.find_element_by_accessibility_id('Okay')
            # No need to sleep here
         except WebDriverException:
            log("There was no in-app purchases dialog")

      # Cancel iOS Game Center login
      if self.isIOS():
         #print self.driver.page_source
            cancelButton = driver.find_element_by_accessibility_id('Cancel')
            log("Cancelling iOS Game Center login…")
         except WebDriverException:
            log("The Game Center login was not displayed")


      # Check that a goldmine is on screen
      rect = self.find_image("queryimages/tutorial_goldmine.png",
         "There should be a goldmine on screen at beginning of tutorial")
      log('Gold mine found at %s %s! Continuing…' % (rect[0], rect[1]))

      # Dismiss the bubbles
      sleep(2) # second blabla
      sleep(2) # Goblin appears

      # Go to shop
      # NOTE: tap_image does also assert, fails test if target not recognized

# Buy cannon

      # Place the cannon
      self.tap_image("queryimages/place_the_cannon.png", width_modifier=0.75)
      # Use gem to finish right away
      # Bring it on!
      self.tap_image("queryimages/bring_it_on.png", height_modifier=0.75)

      # To be continued…

if __name__ == '__main__':
Now, let’s check out some of the stages in this test script. In this test_tutorial, you will find below given steps:

It can be easily figures out whether the test implements on iOS or Android (self.isAndroid()). Moreover, it seems UI content differently. In Android, it can be tried to search a button by using an element’s ID (element_name) and iOS by using an accessibility ID along with a description (“Okay”). Such thing can be checked for iOS’ Game Center login.

You can take screenshots at various steps and easily stored in files that entered as the parameter in a function call.

A check is completed after looking whether “goldmine” seems on screen by comparing two PNG files by using a self.find_image call. Once pictures are matched then script continues implementing the tutorial of the game.

Now, the tutorial continues with below given steps:
  • Go to shop.
  • Purchase cannon.
  • Put the cannon.

You will find the information of the three items in those PNG files:
shopbutton.png, cannon.png and place_the_cannon.png.

At last, the tutorial completes and the battle starts. The application is closed after the battle.

With below given video, one can easily execute test at each step. Here, we are using three devices like iOS (iPhone 4S) and Android phones i.e. Samsung Galaxy S3 Mini and HTC One X. Click here to see video on YouTube.

How to Use Image Recognition?

Image recognition is showed in this example for recognizing features like pixels and graphic content that are shown on screen and to compare the two different images. The algorithm is developed to identify pictures that were used on real devices and on two leading platforms like iOS and Android and it also used a single test script for both platforms.

Such type of image comparison is very convenient to identify UI elements and graphics, which are rotated and resized. Let’s say template image comes with some distinctive features like text can be easily abstracted from the background content. Moreover, feature-based recognition can also be used in some cases.

In this example, if the “Button” text has been resized or rotated, it can be easily identify to take further steps. The approach of comparing images is easily explained by below given functions.

def find_image(self, queryimage_file, screenshot_match=None):
   log('Trying to find %s in current screen' % queryimage_file)

   fd,screenshot_file = tempfile.mkstemp()
      # print screenshot_file
      head, tail = os.path.split(screenshot_file)
      self.screenshot(tail, path=head)
      head,tail = os.path.split(queryimage_file)
      screenshot_match='%s/%s-match.png' %
         (self.screenshot_dir, tail.split('.')[0])
      return ImageFinder.findImages(queryimage_file,
         "%s.png" % screenshot_file,
def screenshot(self, name, path=None):
   if path:
      full_path='%s/%s.png' % (path,name)
      full_path='%s/%s.png' % (self.screenshot_dir, name)

   except WebDriverException: # for iOS, sometimes times out, so retry!
      #log("Failed taking screenshot %s - retrying" % full_path)

   width,height = get_image_size(full_path)
   if(height > width):
      if self.platform_name == 'Android':
         #log("Rotating screenshot 270 degrees")
         os.system('sips -r 270 %s >/dev/null 2&>1' % full_path)
         #log("Rotating screenshot 90 degrees")
         os.system('sips -r 90 %s >/dev/null 2&>1' % full_path)
def tap_image(self, query_image,
   width_modifier=0.5, height_modifier=0.5, retries=2):

   retries_left = retries
   rect = None
   while retries_left > 0 and not rect:
      rect = self.find_image(query_image)
      retries_left = retries_left - 1
   image_name = os.path.split(query_image)[1]
   self.assertIsNotNone(rect, "Image %s is on screen" % image_name)
   x = rect[0]+rect[2]*width_modifier
   y = rect[1]+rect[3]*height_modifier
   log('%s button found at %s %s (%sx%s), tapping at %s %s' %
      (image_name, rect[0], rect[1], rect[2], rect[3], x, y))


def tap(self, x, y):
   log('Tapping at %s,%s' % (x,y))
   driver = self.get_driver()
   if self.isAndroid():
      if self.isSelendroid():
         touch_actions = TouchActions(driver)
         touch_actions.tap_and_hold(x, y).release(x,y).perform()
         action = TouchAction(driver)
         action.press(x=x, y=y).release().perform()
      else: # iOS
         action = TouchAction(driver)
         # TODO: Temporary hack
         # On iPhone 4S = res. 480 x 320 for taps but screenshots are 960 x 640
         action.press(x=int(x/2), y=int(y/2)).release().perform()

Wrapping Text:

Testing is an important task to get success in the highly competitive landscape of mobile applications and games. If testing done poorly then it will take up to 20-50% of total development effort. In some case, it would take single budget cost of your budget.

Before starting with testing part, you should consider below given points like:
  • Automate generic processes as much as possible by planning in proper manner.
  • Choose proper testing technology according to app needs.
  • Test automation is available 24/7.
  • For global reach, it is advisable to use a cloud-based platform
Next Post »

Social Profile Icons (Do Not Edit Here!)