5. Comparison of
important part of Software Development Life Cycle is software testing.
Nowadays, there are several tools available that are web based and serve the
testing purpose very well. One reason for it’s success may be the fact that
they run on any web browser. The system with increased efficiency and less cost
is more preferable. The discussion here is about the comparison of different
web automation testing tools. This will serve two principles, one being the
understanding of all these tools and secondly, The information about the
popular and competent tools available out there. Many browsers support these
testing tools and provide
API’s and web drivers. These API’s are used in different languages to write
appropriate script for testing. One parameter to consider while choosing one of
these tools as the best tool is how well and easily it can be integrated. The
other parameters being performance and cost of each tool. The other thing to
consider is that the tool must be suitable for the application it is going to
This paper focuses on a thorough
comparison between the leading web automation testing tools. This comparison
takes into account all the important parameters necessary in determining both
the best tool for web testing and the most suitable tool for a person’s
specific application. The key features that are considered in comparison are:
OS, Language, Type, and Browser support etc. The objective here is to involve
all the important parameters that can be used to completely distinguish all the
tools and help justify the preference of one or more over others. Such as the
platform support which can determine if a certain tool can serve as a cross
platform tool or a platform specific tool.
Static and dynamic
In static testing it isn’t necessary to execute the
program while in dynamic testing the program needs to be executed.
The box approach
The box approach method involves white-box and
Manual and automation
The main difference between automation testing and
manual testing is that the manual testing does not require any tool while the
automation testing requires automation of the
manual testing process.
automation testing and tools:
software with complete focus on web applications is called web testing tool.
main goal of using web automation testing is to check if the website is flawed
or has errors in it’s basic functionality as well as evaluating other features
of the website such as responsiveness and robustness. The benefit of web
testing however, is to use the same script on different platforms, as cross
platform web browsers are easily accessible.
Saves time and money
Does what manual
is a web testing framework that is especially known for it’s portability. It enables
the tester to develop test scripts without having to learn a specific language.
Selenium supports a range of tools for a range of requirements a script
requires for web automation. Web testers mostly use only a few of the
tools supported by selenium but it is always a good idea to get full
understanding of all the provided tools. Selenium is a rich web
testing framework that provides all the web-oriented tools. This helps bind
all the operations of the tool according to the website and its functionality. The selenium’s ability to support multiple
browsers is probably it’s most important feature. The basic use of
Selenium is functional testing of web applications.Selenium is suitable for agile
testing. Selenium is a free and cross platform software meaning that it runs on
Linux, Windows and Mac.
Professional) is developed by Hewlett Packard (HP). It doesn’t need to be
monitored by the tester while it performs an automated web testing of a web
application. When the script is
written, it uses a VBScript (Visual Basic Scripting) to automate the web
application. Because of the availability of
the scripting engine in Windows Operating System, there is no need to install
own your own. The VBScript that comes
with Windows 7 is of version 5.8. VBScript is an object based language.
It features automated regression testing which
helps find errors or defects in the application’s performance that is being
tested against the required output or result. It only supports WindowsOperating
System and it lacks multiple language support for writing scripts.
Test-Complete is equipped
with a graphical interface. It uses a technique called record-playback to
automate testing. It enables the tester to use unit testing, functional
testing, GUI testing, regression testing and distributed testing. Test-complete
is used for web application testing and desktop application testing limited to
Windows application. It analyzes the features and the shortcoming of the
application and performs evaluation according to the required conditions.
It is an essentialstep in the
process of SDLC (system development life cycle). Companies that develop
softwares strictly follow the process of SDLC as a guide while testing
The type of application as well
as the cost regarding the use of the tool are the parameters that dictate
theselection of tool to be used. As it supports both the web application and
windows application testing, there is a range of types of testing regarding web
applications and windows applications, which are as follows: functional
testing, unit testing, regression testing, manual testing, data-driven testing,
object-driven, distributed testing, HTTP load testing, and stress and
scalability testing. It does that in an environment with complete automation
testing tools that support Windows, web, Java and .NET application testing. It
relives the the developers of the excessive energy and time a manual testing
It is developed with Ruby which is an
object-oriented scripting language. The is used for system testing of large
scale systems, automating user acceptance testing and functional testing. Watir uses programming language to write
scripts that enable user to control web page’s objects such as HTML and
open source software.HTML and test scripts have no layers between them which
makes it easy to create user specific scripts without having to worry about
it’s license as it is free of cost.
WET Web Tester is a web based testing tool
that drives an IE Browser directly and so the automated testing done is equivalent to how a user would drive the web pages. The
tool allows a user to perform all the operations required for testing web applications – like automatically clicking a
link, entering text in a text field, clicking a button etc. One may also
perform various checks as a part of the testing process by using Checkpoints.
The latest version of WET is 1.0. WET sits on
top of Watir, an automated test tool which uses Ruby scripting language. WET retains all the features that Watir has and
adds many usability related functionality 10.
WET started off as a small extension library for Watir – Watir
Extension Toolkit. It has come a long way and is
now bundled with many compelling features. As the WET code base has grown
multiple times, it now uses Watir just as the library to drive IE. WET
not only offers all that Watir offers but many more powerful features to make test automation effort a success. It’s base
code has received a massive increase which helped it become stand alone
software with an independent identity.
It was once considered an extension of Watir but now it is identified as WET.
It is developed by Segue. It is
used for: regression testing and functional testing for large-scaled
applicationsserving environments like .Net, Web, Java or a client-server. It
provides a range of features to choose from making it a highly efficient
automation tool for software testing. Customization, creating tests using
work-flow , validation, complete data base access, robust and flexible testing,
automated recovery system, multple platform, browser support are among these features.
Automated testing with WinRunner addresses these problems by
dramatically speeding up the testing process. You can create test scripts that
check all aspects of your application, and then run these tests on each new
build. As WinRunner runs tests, it simulates a human user by moving the mouse
cursor over the application, clicking Graphical User Interface (GUI) objects,
and entering keyboard input—but WinRunner does this faster than any human user.
4. Feature Description:
to use and adopt
need to learn test scripting language
on major platforms i.e Windows, Mac, and Linux
skilled coder is required
Oriented programming language
to use and navigate
monitoring and scheduling
of Mobile Applications
automated test recording from interactive usage of the application
key-word driven testing
test using test, Excel and Database data sources
support and cross-browser testing
It supports cross-platforms
Requires skilled coder
String and dynamic typing disciple
Written in Ruby (Programing language)
Does not run multiple scripts
Record-playback technique is used for
Setting parameters of objects
Support multiple scripts
Supports Integrated Data Driven testing
Makes creation of scripts easy
Provides 24/7 unattended testing
Basic Work flow for rapid productivity
Uses externally stored data for Data Driven Work
flow to test business logic functionality
Auto-Complete for instant test automation and
customization infrastructure development
Organizing and sharing test information with the
help of Project Workspaces
Re-usability and maintenance of scripts and
tests are made easy with the help of GUI
Scripting language is extensible, easily
maintainable and highly portable
Distributed testing for efficient usage of
excessive portability and hardware resources
Silk-Test provides reporting and execution of
tests on runtime
tool for functional regression
supports Windows platform
for applications based on GUI (Graphical User Interface)
OOT (object oriented technology)concept
static content only
5. Comparison of Tools
Name of tool
Software testing framework for
May 22, 2013,(2004)
Domain specific language
All major developer
Firefox 3.0 and later Google
Sept.3 2013/ 1999
SmartBear software Inc.
Firefox, Google Chrome
Software testing framework for
Bret Pettichard and Paul Roger
Java, .Net, c#
IE, later multiple
Web-Based Testing Tool
Gov. of Canada
4Test Scripting language
Java, 4Test, VB,C#,
IE and Firefox
Load Testing Tool
HP software division
Test Scripting language
It uses Java language and also supports
cross-platform. It has Apache 2.0 and it runs on all web browsers. It has
language support for domain specific languages.
It uses and supports VB Script and
operates on MS-Windows. It has proprietary license. It runs on IE, Firefox and
It uses Java language and has
MS-Windows support. It has proprietary license. It runs on IE, Firefox and
It uses Ruby language and supports
cross platforms. It has BSD license. It
initially had only IE support but later it received multi-browser support. It
supports java, .Net, and C#.
It uses and supports Ruby language and
has MS-Windows support. It has MIT license. It has IE (browser) support.
It uses 4Test scripting language and
has MS-Windows support. It has proprietary license. It runs on Internet
Explorer and Firefox. It supports Java,
4Test, VB, C#, and VB.Net.
It uses C language, and has MS-Windows
and Linux support. It has proprietary license. It runs on IE and Netscape. It
supports test Scripting language.
We analyzed various web
automation tools in this paper. A web automation tool tests a web application
for flaws or errors of any kind without performing any manual operation . There
are a number of web automation testing tools that serve this purpose. To decide
which tool is best, one has to tally the requirements with the tool’s features.
For instance if an organizations is looking for rapid results he will go for
speed and if an organization struggles with their budget, it is likely to
search for a cost effective tool. The requirements can be categorized according
to the feasible tools.
If an organization
requires speed in testing then the possible testing tool is Test-Complete,
but the tools comes with a cost. The Test-Complete has to be licensed and it’s
license is quite costly. So if the organization prefers speed over cost then it
will be the right call to use Test-Case. However it the organization can’t
handle the cost then they will have to manage the trade-off.
On the other hand if
an organization wants a tool that is cost effective, they may have to
settle for a tool that has a very slow testing speed. Or if the material to be
tested is on large scale the tool should be chosen according to this specific
need. For example for large-scale testing Watir is most suitable
choice. So it is clear that a tool is best for a job only if it is chosen
according to the need.
An organization that
wants a balanced tool may consider Selenium. It is both cost
effective and flexible but again it is not best known for its speed. So in
general it is a good tool but specifically- well it depends on the
Finally, all testing
tools have their strengths and weaknesses or limitations. It is important to
know the requirements to choose the best tool for a test case to perform most
effectively. All cases are to be judged by their suitability instead of their
shear performance or cost.
1. Kaner, Cem .(November 17, 2006).”Exploratory
Testing”(PDF).Florida Institute of Technology, Quality Assurance Institute Worldwide
Annual Software Testing Conference, Orlando, FL.
Pan, Software Testing, Carnegie Mellon University.
J. Myers, The Art of Software Testing, Hoboken, New Jersey: Published by John
Wiley & Sons, Inc.
Adam; Huizinga, Dorota. (2007). Automated Defect Prevention: Best Practices in
Software Management. Wiley-IEEE
Society Press. ISBN 0-470-04212-5.
Linda G. (March 2004).Automated Testing Handbook, Software Testing Institute,
Tom.(April 2015). HP snaps up Mercury Interactive,CNET. CBS Interactive
7. SudheendraHangal , Monica S. Lam.(2002).Tracking
down software bugs using automatic anomaly detection.
In Proc. 24th International
Conference on Software Engineering, pages
Scott M.( April 2015) .HP Aims to Redefine Apps Performance Testing with
Cloud Platform. ReadWrite.
Wearable World Inc.
9. David Saff, Michael D. Ernst.(2003). Reducing
wasted development time via continuous testing. Software Reliability Engineering. In
14th International Symposium, ISSRE 2003,pages 281-292..
10. Automation testing-
Rattan, Department of
Engineering College, Sonipat, Haryana,
INDIA, Comparative study
of automation testing
tools: Test Professional Selenium,
Wandan, Jiang Ningkang, Zhou Xubo, “Design and Implementation of a Web
Application Testing Framework”,
978-0-7695-3745-0/09, 2009 IEEE
13 Shauvik Roy Choudhary, Husayn Versee,
Alessandro Orso, “WEBDIFF: Automated Identification of Cross-browser Issues in Web Applications”, 978-1-4244-8628-1, 26th IEEE International conference on
Software Maintenance, 2010
14. Baowen Xu, Lei Xu, Changhai Nie1, William Chu C. H. Chang, “Applying Combinatorial Method to Test Browser Compatibility”, 0-7695-2031-6/03,
Proceedings of the IEEE Fifth International
Symposium on Multimedia Software Engineering (ISMSE’03)
15. http://Challenges in Testing Web Based Applications.htm
16. BETTER SOFTWARE, APRIL 2005 by Jonathan Kohl
and Paul Rogers www.StickyMinds.com