Web app testing has rapidly evolved over the years, from the use of separate machines for covering different OS’s, to multiple virtual machines on a single desktop, to cloud-based cross-platform testing tools. Let's take a look at these milestones in the quest for faster, more efficient, and more comprehensive web and mobile app testing tools for software quality assurance and hardware compatibility.
The early days….you do your thing, I’ll do mine!
Earlier, testing of web applications was carried out on mainly three separate OS’s - Linux, Windows and Mac, with an array of browsers, hardware and resolutions. However, there were some limitations to managing the hardware for each OS on a separate machine:
- Every time an OS or a browser was to be installed on a system, the license for it had to be purchased anew. So, it was time-consuming and cost-prohibitive to install different OS-browser combos on separate systems with different system architectures, varying bandwidths and diverse HTML specifications.
- Also, since a particular platform-browser combination was typically available only on one machine within an in-house QA lab, developers and QA could not work on fixing and testing the issues detected therein simultaneously. This resulted in bottlenecks and slowed down the software development process.
The most obvious solution to this problem was to provide all the OS and browsers on the same QA system of an individual user to optimize the use of hardware and to avoid bottlenecks.
Virtual machines...from many down to one
It seemed that virtual machines (VMs), which provided a different OS on the users’ already existing desktop, would provide the answer to these problems. Having more than one virtual machine on the same hardware facilitated testing of the web app on the same system for compatibility with diverse platforms. VMs also allowed for cloning of platforms so that more than one user could use the same OS.
However, this slowed down the machine considerably. VMs needed more memory and CPU cycles; so a system supporting them would need faster and more powerful hardware. There were some other constraints in the use of VMs:
- As they were run on software emulators, web app testing for hardware compatibility was still a challenge.
- There were configuration and disk space constraints while installing the software of different platforms.
- Also, cost and budget was still a cause for concern as licenses for the OS’s and browsers had to be purchased by web app owners.
- Since all the VMs on a machine would be using the hardware of the host, it was as risky as the proverbial “placing all your eggs in one basket”. If the hardware platform failed, it could take out all of them.
All the concerns mentioned above were addressed by the introduction of central VM repositories on a LAN with powerful server hardware.
Central VM Repositories
This helped push the niggling issues - hardware, disk storage management and purchase of licenses for OS’s and browsers - to the server end. It became easier with the launch of plug-and-play VMs in the market. Also, since VMs allowed cloning, it helped eliminate bottlenecks during bug-tracking. If a bug was detected while testing the app on one platform, the tester had the facility to clone the same OS and work on that while the developer could look into the bug on the previous OS without disrupting the validation process.
Cloud-based, cross-platform testing….the silver lining
As internet bandwidth increased, service providers facilitated cloud-based, cross-platform testing without QA engineers having to worry about storage space, hardware issues, or software conflicts on their own individual test machines.
We will discuss two of these, namely, BrowserStack and Device Anywhere in this article.
Emulators...fake it till you make it!
With the plethora of devices (particularly Android) available today, mobile app testing is struggling to keep in sync with the latest hardware and software specifications for cross-platform compatibility. Regardless of whether it is a web app or a mobile app, for QA purposes, specialized software testing tools like emulators are needed. An emulator mimics all the software and hardware of a particular device, including screen size and resolution, which is critical for thorough test coverage.
However, emulators could not measure up to the mobile app testing specifications. Here's why:
- Testing the gyroscope feature on actual mobile devices is not possible as emulators cannot replicate it.
- It is not easy to set up an incoming call on an emulator.
- Functional testing of SIM card features of the mobile device cannot be done on an emulator.
- Most emulators do not work with automated tools and so, they must be used manually.
- Also, they get outdated very soon as mobile devices are continuously evolving and there are frequent releases of newer models.
- It is still crucial to test on real hardware before the app goes live.
Since emulators could not replace the real thing, facilitating an in-house mobile lab with real mobile devices for testing seemed to be the most obvious solution.
Real hardware for QA
Ideally, mobile app testing should be done on real devices. However, there are some problems with using real hardware for QA:
- With newer models of mobile devices being released often, it is not feasible for every web app owner to purchase all the latest models.
- Even if a QA engineer has a number of devices in his library, they can focus on a maximum of two, maybe three devices at a time. So, resource investment on hardware in case of mobile device testing is not optimally used.
- Also, if an issue is to be replicated on a different platform available on another device, the user has to wait for it to be available, thereby creating a bottleneck.
These factors compelled the thought that testers should be able to test on multiple devices without owning the hardware. The only cost effective way was through cloud-based, cross-platform testing services like BrowserStack and DeviceAnywhere.
BrowserStack
BrowserStack allows access to every possible browser-platform combination on an array of different hardware configurations.
- It offers local testing for private and protected servers, as well as public websites.
- It provides automated testing of multiple browser scripts in parallel, helping save on testing time.
- The Screenshots feature helps in testing the layout design of the website from an array of more than 650 browsers, allowing the user to select 25 browsers at one time.
- The Responsive feature lets the user see the layout of their website with respect to resolution and screen size across platform-browser combos.
- For mobile app testing, BrowserStack facilitates software and hardware compatibility tests on actual physical devices made available to the user through the cloud.
You can read more about BrowserStack at https://www.browserstack.com/mobile-browser-emulator
DeviceAnywhere
Just like BrowserStack, DeviceAnywhere empowers developers, IT and QA teams with the ability to test mobile apps from anywhere, by allowing remote access to actual mobile devices. It tests any Smart device, irrespective of its software and hardware configuration.
- A view of the device to be tested is exported to the user’s computer over the internet and all the normal functions of the device - like tapping, swiping, power on/off, mute - can be tested.
- It also provides automated testing.
- It facilitates taking videos and/or frame capture of testing sessions that can then be exported and used in issue documentation for effective bug-tracking.
Since BrowserStack and Device Anywhere allow remote access to devices, smartphone features like the gyroscope can be replicated for testing purposes.
To know more about DeviceAnywhere, go to https://www.keynote.com/solutions/testing/mobile-testing
Testing web applications - what's next?
With their comprehensive hardware and software testing services, it seems as if cloud-based, cross-platform testing tools are here to stay. No doubt, testing tools will need to evolve to keep up with newer technologies and innovations like Google Glass, the Google electric car, Artificial Intelligence (AI), drones, and advances in fields like biotechnology to ensure not only a bug-free run, but also the safety and even lives of their users.
