If we are
considering mobile application ecosystem of android and iOS, we find
it dynamic, as both are suffer from software and hardware
fragmentation. Especially, it is applicable for Android, however
fragmentation also available in iOS ecosystem, as experienced with
the launch of iOS 8.
There are
many existing iOS applications available with its latest version that
made clumsy on updated devices. Moreover, one should find typical
issues in the new iPhone 6 and 6 Plus for Apple devices. Users with
some options will find an important amount with older devices,
especially to purchase new device for getting everything working
well.
If we are
considering Android, everything you find is different. Application
and game developers are getting difficulty as OEMs launch software
updates, latest devices, and customizations for their devices to
maintain product fully supportable with all possible device
alternatives. Creating some application and game work on quality
devices is out of the question.
One-stop solution for
common problem is professional automated testing software. It enables
to create robust and reliable software with difficult technology and
difficult competitive pressure. One of the affordable solutions is
Automated software testing to this problem. It delivers three
business advantages like:
- Increased testing effectiveness
- Faster time to market
- Increased testing efficiency
Test Automation is best For Mobile App Development
But doing something
manually like testing an application is never agile, and thus, test
automation has shown huge growth among developers and application,
improving robust and better outcomes. Test automation as the part of
the agile development procedure is normal for attaining better
compatibility with devices, market and users.
Interestingly, many
tools are available, but test automation stands on the first
position. A significant value-add is offered by this tool to enable
testing to be done in similar way. The next day, the tests will be
confirmed and the outcomes will be ready for review.
As we all know that
fixing a problem saves huge time and enable developers to finalize
products earlier. Importantly, it gathers better quality with limited
number of bugs.
Factors to Consider Automating Mobile App Testing
Now, we have to check
how test automation can enhance the development process, increase
speed up development and add value.
Costs And Assets
One will require time, people, infrastructure, tools and training irrespective of whether you go for manual or automated testing. Test automation will deliver a good ROI, depending on the project and application. Moreover, Automated Software Testing helps to minimize repetitive tests from weeks to hours. It is time saving that translates directly into cost-saving.
Incorporated Testing Cycle with App Development
Increase in efficiency
and productivity with automation starts with catching a new mindset.
During all development cycles, software tests have to be repeated to
make sure the possible application’s quality. Software tests should
be repeated when source code is different every time.
The software should be
tested on all supported variants of OS for each release and all
configuration of hardware. However, manually repeating would be
costly and time-consuming.
Tools And Technology Used for Test Automation Frameworks
First of all, you have to choose the most robust and most cross-platform method for getting the most out of efforts and maximizing testing coverage. Moreover, one can use such automated methods for validating needs and to minimize costs through test-case generation.
But, the full
automation of large software entities is available with a cost that
some companies haven’t been read to pay.
Test Coverage And Reusability for Open Standards Mean
The depth and scope of
tests can be increased by automated testing and importantly enhance
software quality. Some long and thorough tests can be run
automatically often not doable with manual testing.
Well, test cases comes
with a full access to an application and test all factors, including
data tables, file contents, memory contents, and internal program
states to decide whether the product runs according to expectations
or not.
With automated tests,
one can easily execute many difficult tests when running test to
deliver coverage that is not possible with manual testing. Moreover,
developers have enough time to develop new automated test cases and
integrated more interesting features.
Improve Usefulness And Conclude Sooner
Professional automated
testing software is one of the best solutions for common problem,
including how to create better and robust solution with ever growing
difficult of technology and under massive competitive pressure.
Business results are
improved by automated testing in three ways, i.e. greater testing
effectiveness, greater testing efficiency, and a shorter time to
market.
Various ways to Automate Mobile Testing
There are three ways to
automating the testing of mobile applications such as:
Handwritten Test Scripts
Generally, it is one of
the preferred options when you have idea about what you are doing and
when you have programming-capable resource for doing the test
scripts. Different alternatives are also available for test
automation tools, frameworks, and integration for both open-source
and commercial offerings.
Record-Playback Approach
It is the least
error-prone approach as it nothing has written in code, however has
limited functionality. Tests can easily and quickly recorded and
after that, played repeatedly against different OS versions and
device configurations. Such test concentrates on user-driven and user
interactions activities.
Automatic Test Exercisers
A great way to
smoke-test applications is provided by automatic test exercisers. No
particular tests are required instead of focusing on testing
user-interface logic like clicking buttons, opening menus, swiping
and multi-gesture actions.
The least exact results
yielded by automatic test exercisers, however delivering quick
feedback on iteration of an application.
User Interface And Functionality
One can identify the
success of application and games with users interface and its overall
functionality. The two things that include visual appeal and gameplay
are the important things to get right and one must ensure that device
fragmentation doesn’t break any of these.
Different things in
the UI require to be checked such as:
- UI layouts and elements
- Menu structure and functions
- Screen orientation
- Screen resolution
Graphics Performance
Software performance
should be consistent across all devices variants among your users and
test on many real devices are possible. It is advisable to create
tests for determining how well your application and game responds to
various levels of usage, including battery usage and performance that
consider creating tests that last for hours.
Run load tests for
determining whether your game runs effectively under a heavy load for
a long time. Such performance test will evaluate like how responsive
your game is on real devices.
Usability And User Experience
User experience,
testing usability, and navigation flow cannot be completed on a
desktop with a keyboard and mouse. So, it is advisable to use real
devices instead of forget emulators.
You have to test how
usable and entertaining your application is by considering two
important things like
- Background events
- User interaction and responsiveness
Multi-User Features
Presently, multi-user
support is common in both games and applications. Moreover, testing
multi-player capabilities also play an important role and are more
challenging needed real users for measure performance.
A game communicating is
a typical case with the back-end server. One can also synchronize the
back end with devices for better connectivity to get information
about the gameplay.
Moreover, one can check
test of various scenarios among them could affect on the game’s
experience that results into negative feedback and lastly, game being
uninstalled by players.
Social Integration
Another important
factor is to integrate with social networks. In next step, you have
to test this thoroughly with real iOS and Android devices with
different operating system versions and different device
configurations to measure functionality and ease of utilize.
Security And Liabilities
Some open-source
components used by all developers in their applications. Such type of
practice is used and recommended as it offloads the code’s
development for non-core functionality. But, developers neglect
identifying vulnerabilities and licensing restrictions with
third-party code.
Breakdown: Android Test Automation Frameworks
Robotium – It is one
of the popular Android test automation frameworks, supporting native
and hybrid applications. With Robotium, one can easy to write robust
and powerful automatic black-box UI tests for android applications.
UIautomator - delivers
an efficient ways for testing UIs. An automated functional test cases
are created that can be implemented against applications on real
Android devices and emulators.
Calabash - If you are
looking for framework for Android and iOS then Calabash is a
cross-platform test automation framework. The best thing about
Calabash is it understands syntax that enables non-technical people
to develop and implement automated acceptance tests for apps on both
of these mobile platforms.
Appium: Automation of native, hybrid and mobile web apps.
Appium is one of the
best mobile test automation frameworks for mobile-web, native, and
hybrid apps for iOS and Android. JSONWireProtocol is used by Appium
for cooperating with Android and iOS application by using Selenium’s
WebDriver.
Written in Node.js,
Appium is an HTTP server, developing and handling multiple WebDriver
sessions. The test is started by Appium on the device and listens for
commands from the main server. It seems same as the Selenium server
that gets HTTP requests from Selenium client libraries.
How to Set Up The Environment?
First of all, download
our Appiumexample. You can find example in Java, Python,
and Ruby. After that, choose clientlibrary according to programming language you
will choose. For more information, you can switch to Appium’sdocumentation where all guidelines, help and
extra material are available.
MAC OS X AND LINUX
Firstly, you have to
make sure that Python 2.7.x or later are installed (Note: It’s
preinstalled on Mac OS X):
$ python --version
Now, you have to
install Python (Linux users only):
$ sudo apt-get install python2.7
After that, you have to
check Python’s “pip” module is already installed or not:
$ pip --version
Install pip if you’re
on Mac OS X:
$ curl https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py > get-pip.py $ sudo python get-pip.py $ rm get-pip.py
Install pip on Linux
(if it is not installed)
$ sudo apt-get install python-pip
In next step, you have
to install the Selenium module for Python:
$ sudo pip install selenium
Now, you have to verify
that Selenium is installed:
$ pip freeze | grep -i selenium
WINDOWS
You have to make sure
that Python
2.7.x or later are installed:
$ python –version
If it’s not run
properly then you have to download and run the setup from Python’sdownload center. In order to add Python
environment variables, you have to go to “System properties” →
“Advanced System Settings” → “Environment Variables” →
“System Variables” → “Edit Path,” and after that, insert
C:\Python27\;C:\Python27\Scripts\ at the end. It is supposed that you
have installed Python in the default location.
You have to ensure to
restart the command prompt for inviting new environment variables in
to effect.
Now, you have to check
if Python’s pip module is installed or not:
$ pip --version
If you don’t have
install, then install pip:
$ curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py > get-pip.py $ python get-pip.py $ del get-pip.py
Lastly, install
Python’s Selenium module:
$ pip install selenium
Running The First Tests
In that step, you have
to download sampletest script and sample application.
Download the Appium
sample test script and sample app. The APK for Android, the IPA for
iOS and one sample test script are available in the sample package.
Furthermore, three samples are
- appium_example_ios.py
- appium_example_android.py (for Android API level 17 and above)
- appium_example_selendroid.py (for Android API level 16 and below)
Step: 1 Make Account And Upload The APK
You have to create an
account. A freemium option is also provided by this service and you
don’t require a plan for completing these samples by using real
devices on a cloud service.
One can have different
plans for accessing hundreds of device models. One can have to upload
the APK or IPA to Testdroid’s cloud service before proceeding with
running the test script through HTTP POST. Now, let’s try using
cURL.
For Mac OS X and Linux:
$ curl -s --user username@example.com:password -F myAppFile=@"/absolute/path/to/app/SampleApp-iOS.ipa" http://appium.testdroid.com/upload
FOR WINDOWS:
$ cd
C:\path\to\directory\containing\the\application
$ curl -s --user
username@example.com:password -F myAppFile=@"SampleApp-iOS.ipa"
"http://appium.testdroid.com/upload"
You will get a JSON
response with status: 0 and a unique identifier for the uploaded app
upon successful upload.
{
“status”: 0,
“sessionId”: “abcdefg-1234-5678-abcd-111122223333”,
“value”: {
“message”: “Uploads Successful”,
“uploadCount”: 1,
“rejectCount”: 0,
“expiresIn”: 1800,
“uploads”: {
“myAppFile“: “abcdefg-1234-5678-abcd-111122223333/SampleApp-iOS.ipa“
},
“rejects”: {}
}
}
Step 2: Set Credentials And Other Parameters
Now, open the test
script: Appium_sample_ios.py, in any text editor. You have to
set screenshotDir
to the path, where you want those screenshots to be kept on
your machine. In DesiredCapabilities, you have to set your
credentials on testdroid_username and testdroid_password.
After that, you have to
set the myAppFile identifier against testdroid_app in
Desired Capabilities. It seems like:
desired_capabilities_cloud = { ‘testdroid_username’: testdroid_username, ‘testdroid_password: testdroid_password, ‘testdroid_project: ‘Appium iOS Project 1’, ‘testdroid_target: ‘iOS’, ‘testdroid_description: ‘Appium project description’, ‘testdroid_testrun: ‘Test Run 1’, ‘testdroid_device: testdroid_device, ‘testdroid_app: ‘sample/SampleApp-iOS.ipa’, ‘testdroid_platformName: ’iOS’, ‘testdroid_deviceName: ‘iPhone device’, ‘testdroid_bundleId: ‘com.bitbar.testdroid.SampleApp-iOS’ }
Step 3: Run The Test Script
You have to implement
the below given command:
$ python sample-appium_ios.py
The output of console
seems like:
Click here for see this image.
Step 4: Get Results From Cloud
You can see screenshots
available locally on your machine in the directory that specified in
Step 2. Now, you have to log into your cloud service and navigate to
the project’s name, as defined in the _project attribute of
DesiredCapabilities for accessing following log files:
- Appium’s server log,
- Logcat and instruments log.
Some Advanced Example: By using Appium for mobile game testing with image recognition.
We are using
Supercell’s popular mobile game Clash of Clans in this example. It
is one of the interesting games, many of you have played it and thus,
you should be familiar with its gameplay and look. It is one of the
best example of basic clicks through game’s tutorial.
## Example script that tests Clash of Clans tutorial first steps ## Works on different resolutions, both iOS and Android import unittest from time import sleep from AppiumCloudTest import AppiumCloudTest, log from selenium.common.exceptions import WebDriverException class ClashOfClansTest(AppiumCloudTest): def setUp(self): # AppiumCloudTest takes settings from environment variables super(ClashOfClansTest, self).setUp() def test_tutorial(self): driver = self.get_driver() # Initialize Appium connection to device sleep(10) # Wait for game to load # Use this to get detected screen hierarchy # print self.driver.page_source # Dismiss the in-app purchases dialog if it shows okayButton = None if self.isAndroid(): try: okayButton = driver.find_element_by_id('button3') okayButton.click() sleep(5) except WebDriverException: log("There was no in-app purchases dialog") else: # iOS self.driver.implicitly_wait(5) # Wait only 5 seconds to find it try: okayButton = driver.find_element_by_accessibility_id('Okay') okayButton.click() # No need to sleep here except WebDriverException: log("There was no in-app purchases dialog") self.driver.implicitly_wait(30) # Cancel iOS Game Center login if self.isIOS(): #print self.driver.page_source try: self.driver.implicitly_wait(5) cancelButton = driver.find_element_by_accessibility_id('Cancel') log("Cancelling iOS Game Center login…") cancelButton.click() sleep(2) except WebDriverException: log("The Game Center login was not displayed") self.driver.implicitly_wait(30) self.screenshot("welcome-chief") # Check that a goldmine is on screen rect = self.find_image("queryimages/tutorial_goldmine.png", screenshot_match="screenshots/goldmine_match.png") self.assertIsNotNone(rect, "There should be a goldmine on screen at beginning of tutorial") log('Gold mine found at %s %s! Continuing…' % (rect[0], rect[1])) # Dismiss the bubbles self.tap_middle() sleep(2) # second blabla self.tap_middle() sleep(2) # Goblin appears self.tap_middle() sleep(1) # Go to shop # NOTE: tap_image does also assert, fails test if target not recognized self.tap_image("queryimages/shopbutton.png") sleep(1) # Buy cannon self.screenshot('cannon') self.tap_image("queryimages/cannon.png") sleep(2) # Place the cannon self.screenshot('place_the_cannon') self.tap_image("queryimages/place_the_cannon.png", width_modifier=0.75) sleep(2) self.screenshot('finish_now') # Use gem to finish right away self.tap_image("queryimages/finish_now.png") sleep(3) # Bring it on! self.screenshot('bring_it_on') self.tap_image("queryimages/bring_it_on.png", height_modifier=0.75) sleep(10) self.screenshot('battle') sleep(10) self.screenshot('end_of_battle') # To be continued… if __name__ == '__main__': unittest.main()
Now, let’s check out
some of the stages in this test script. In this test_tutorial, you
will find below given steps:
It can be easily
figures out whether the test implements on iOS or Android
(self.isAndroid()). Moreover, it seems UI content differently.
In Android, it can be tried to search a button by using an element’s
ID (element_name) and iOS by using an accessibility ID along
with a description (“Okay”). Such thing can be checked for iOS’
Game Center login.
You can take
screenshots at various steps and easily stored in files that entered
as the parameter in a function call.
A check is completed
after looking whether “goldmine” seems on screen by comparing two
PNG files by using a self.find_image call. Once pictures are matched
then script continues implementing the tutorial of the game.
Now, the tutorial
continues with below given steps:
- Go to shop.
- Purchase cannon.
- Put the cannon.
You will find the
information of the three items in those PNG files:
shopbutton.png,
cannon.png and place_the_cannon.png.
At last, the tutorial
completes and the battle starts. The application is closed after the
battle.
With below given video,
one can easily execute test at each step. Here, we are using three
devices like iOS (iPhone 4S) and Android phones i.e. Samsung Galaxy
S3 Mini and HTC One X. Click here
to see video on YouTube.
How to Use Image Recognition?
Image recognition is
showed in this example
for recognizing features like pixels and graphic content that are
shown on screen and to compare the two different images. The
algorithm is developed to identify pictures that were used on real
devices and on two leading platforms like iOS and Android and it also
used a single test script for both platforms.
Such type of image
comparison is very convenient to identify UI elements and graphics,
which are rotated and resized. Let’s say template image comes with
some distinctive features like text can be easily abstracted from the
background content. Moreover, feature-based recognition can also be
used in some cases.
In this example, if the
“Button” text has been resized or rotated, it can be easily
identify to take further steps. The approach of comparing images is
easily explained by below given functions.
def find_image(self, queryimage_file, screenshot_match=None): log('Trying to find %s in current screen' % queryimage_file) fd,screenshot_file = tempfile.mkstemp() try: os.close(fd) # print screenshot_file head, tail = os.path.split(screenshot_file) self.screenshot(tail, path=head) head,tail = os.path.split(queryimage_file) screenshot_match='%s/%s-match.png' % (self.screenshot_dir, tail.split('.')[0]) return ImageFinder.findImages(queryimage_file, "%s.png" % screenshot_file, scale_tolerance=0.9, screenshot_match=screenshot_match) finally: os.remove(screenshot_file) def screenshot(self, name, path=None): full_path=None if path: full_path='%s/%s.png' % (path,name) else: full_path='%s/%s.png' % (self.screenshot_dir, name) try: self.get_driver().save_screenshot(full_path) except WebDriverException: # for iOS, sometimes times out, so retry! #log("Failed taking screenshot %s - retrying" % full_path) self.get_driver().save_screenshot(full_path) # width,height = get_image_size(full_path) if(height > width): if self.platform_name == 'Android': #log("Rotating screenshot 270 degrees") os.system('sips -r 270 %s >/dev/null 2&>1' % full_path) else: #log("Rotating screenshot 90 degrees") os.system('sips -r 90 %s >/dev/null 2&>1' % full_path) def tap_image(self, query_image, width_modifier=0.5, height_modifier=0.5, retries=2): retries_left = retries rect = None while retries_left > 0 and not rect: rect = self.find_image(query_image) retries_left = retries_left - 1 image_name = os.path.split(query_image)[1] self.assertIsNotNone(rect, "Image %s is on screen" % image_name) x = rect[0]+rect[2]*width_modifier y = rect[1]+rect[3]*height_modifier log('%s button found at %s %s (%sx%s), tapping at %s %s' % (image_name, rect[0], rect[1], rect[2], rect[3], x, y)) self.tap(x,y)
def tap(self, x, y): log('Tapping at %s,%s' % (x,y)) driver = self.get_driver() if self.isAndroid(): if self.isSelendroid(): touch_actions = TouchActions(driver) touch_actions.tap_and_hold(x, y).release(x,y).perform() else: action = TouchAction(driver) action.press(x=x, y=y).release().perform() else: # iOS action = TouchAction(driver) # TODO: Temporary hack # On iPhone 4S = res. 480 x 320 for taps but screenshots are 960 x 640 action.press(x=int(x/2), y=int(y/2)).release().perform()
Wrapping Text:
Testing is an important
task to get success in the highly competitive landscape of mobile
applications and games. If testing done poorly then it will take up
to 20-50% of total development effort. In some case, it would take
single budget cost of your budget.
Before starting with
testing part, you should consider below given points like:
- Automate generic processes as much as possible by planning in proper manner.
- Choose proper testing technology according to app needs.
- Test automation is available 24/7.
- For global reach, it is advisable to use a cloud-based platform
Sign up here with your email
ConversionConversion EmoticonEmoticon