Visit the link below to download a free trial of TestComplete http://smartbear.com/products/qa-tools/automated-testing-tools/free-testcomplete-trial Selecting the right automated software testing tool involves having a list of requirements to review during your evaluation. Without a list of requirements, you may waste time downloading, installing and evaluating tools that only meet some of your requirements. This White Paper outlines common requirements for automated testing tools which you can build from to create a list that reflects the specifics of your software testing needs.

1. SM
Ensuring Software Success
Selecting Automated Testing Tools
A SmartBear White Paper
Selecting the right automated software testing tool involves having a list of requirements to review
during your evaluation. Without a list of requirements, you may waste time downloading, installing
and evaluating tools that only meet some of your requirements. This White Paper outlines common
requirements for automated testing tools which you can build from to create a list that reflects the
specifics of your software testing needs.
Contents
Introduction............................................................................................ 2 Logging Automated Test Results...............................................8
Support for Various Applications and Platforms...................... 2 Exporting Reports............................................................................8
Support for Various Operating Systems....................................... 3 Additional Needed Automated Testing Tool Features......9
Support for Various Testing Types.................................................. 3 Testing Multi-Language Applications......................................9
Creating Automated Tests Without Programming................. 4 Automated Testing Tool: Support for Team Work..............9
Automated Test Scripting................................................................. 5 Command Line and OLE Automation Support................. 10
Recording Automated Software Tests.......................................... 5 Integration with Team System and Build Software........ 10
Creating Automated Tests that are Resistant Technical Support......................................................................... 10
to Changes.............................................................................................. 6 Pricing Policy................................................................................... 10
Support for Controls and Objects................................................... 6 Free Trial Version........................................................................... 11
Support for Various Test Data Sources......................................... 7
Running Automated Software Tests and
Synchronization......................................................................................7
Automatically Compare Results of Automated
Test Actions..............................................................................................8
www.smartbear.com/testcomplete

2. Introduction
Testing is a critical part of the software development process. There are a lot of different automated software
testing tools currently on the market. Some of these tools are only able to perform specific kinds of testing and
only work with specific languages, like, for example, Java application unit testing. Other products support a wide
range of applications and offer more features and functionality.
When you start your search for the right automated software testing tool, it’s important to create a list of
requirements to review when choosing a tool for evaluation. If you don’t have a list of requirements, you may
waste time downloading, installing and evaluating tools that only meet some of your requirements, or may not
meet any of them.
Creating a list of requirements is not easy, since you have to take all of your testing needs into account. We have
prepared a list of requirements, which are described below. Of course, the list does not include every situation,
but this list is a good start. You can modify and add to it to get a list that reflects the specifics of your software
testing environment.
Support for Various Applications and Platforms
Some automated testing tools only support one type of application, like, for instance, only Java or .NET ap-
plications, while other tools support more application types. In general, the best option is to choose a tool that
supports all development tools currently used in your organization, or all tools that you plan on using in the near
future. Even if you build your application with one compiler, you might use other compilers in the future.
Selecting one “multi-compiler” software testing tool instead of several “compiler-specific” tools may save you
money, but you must decide which is more important and whether it will save you time and energy when creat-
ing tests for future application versions. Purchasing and using several software testing tools requires additional
time for training and creating tests. Do you have that much time? Does your company have enough budgeted
to pay for this?
TestComplete is one of the automated software testing tools that support the testing of any Windows program
created with any compiler. It works with both 32-bit and 64-bit Windows, .NET, Java and web applications. It
can also test applications running on Windows Mobile-based PDAs.
TestComplete’s automated test engine provides extended support for the following applications:
¿¿ C#, Visual Basic .NET, VCL .NET and any other .NET application for any version of .NET Framework (1.0, 1.1,
2.0, 3.0, 3.5 or 4.0)
¿¿ WPF (XAML) and XBAP applications (.NET Framework 4)
¿¿ Java applications (including Swing, SWT, AWT and WFC applications)
¿¿ Visual C++ 6.0-9.0 applications (including ATL, MFC and WTL applications)
¿¿ Visual Basic 6.0
¿¿ Delphi and C++Builder applications (both VCL and CLX applications)
2

3. ¿¿ Flex, Flash, Silverlight and JavaFX applications
¿¿ Windows Mobile applications
¿¿ Qt applications
¿¿ wxWidgets applications
This extended support means that TestComplete can access the internal objects, methods and properties of the
application under test. QA engineers can use all of these elements in their automated tests. For instance, your
tests can execute an object’s internal methods to initiate a unit test.
TestComplete also provides access to methods and properties of web page elements displayed in Internet Ex-
plorer, Firefox, Netscape Navigator or any browser based on the Microsoft Web Browser control. It can test web
pages on the client computer and is not dependant on the server technology. You can use it to test web pages
generated by IIS, ASP, ASP.NET, PHP, CGI, AJAX, Atlas and other applications.
Support for Various Operating Systems
Choose an automated testing tool that supports as many operating systems as possible. Make sure that the
tool supports recent operating system versions. The fact that your customers do not run your application under
Windows Vista does not mean that they will not use it in the future. If your application works under this operat-
ing system, your software testing tool should be able to work under it as well. In other words, the more Windows
versions the software testing tool supports, the better.
TestComplete supports both old and new Windows operating systems: Windows 2000, Windows XP, Windows
Vista, Windows 7, Windows Server 2003 and Windows Server 2008, including both 32 and 64-bit versions of
these operating systems. TestComplete also provides automated testing support for Windows Mobile applica-
tions running on PDAs. The range of supported operating systems on PDAs includes Windows Mobile 2003
(Pocket PC 2003), Windows Mobile 2003 SE (Pocket PC 2003 SE), Windows Mobile 5.0 and Windows Mobile 6.0.
Support for Various Software Testing Types
When choosing an automated testing tool, check whether it supports the test types that you are going to run.
Can it simulate user actions over the tested application’s user interface, in other words, perform functional
testing? Can it run unit tests built into the tested application? Can it run tests created with NUnit, JUnit, DUnit
or MSTest? Does it support distributed testing? Does it have special features for synchronizing tests that run
simultaneously on several network workstations? Does it support load testing?
Also, if you test your products using both manual and automated methods, you may want to choose a tool that
provides specific support for manual tests. Also, don’t forget other important factors, such as, data-driven testing
support and the ability to use the tool for regression testing. There are a lot of automated software testing tools
that support only one or two testing types, for instance, only unit testing or functional testing. Some vendors
require separate products for different testing types. For instance, support for load or distributed testing can be
implemented with separate products or add-ons. If you wish to support these testing types, you will have to add
the price of these products or add-ons to the price that you are going to pay for the product.
3

4. TestComplete supports unit, functional (or UI), regression, data-driven, manual and distributed testing of Win-
dows applications. It provides special support for unit tests created in NUnit, JUnit, DUnit, MSTest, which allows
you to run these tests directly from your TestComplete projects. This lets you use unit tests that were created
with these tools earlier in your development cycle. With TestComplete you can also perform load, stress and scal-
ability testing of Web servers and services. To perform coverage testing, check the application for resource and
memory usage and you can find bottlenecks in your application by using another one of SmartBear’s products
called AQtime. AQtime is tightly integrated with TestComplete, so you can run coverage, performance and
memory leak tests directly from your TestComplete projects.
Creating Automated Tests Without Programming
An automated testing tool must let you create tests fast and effectively. The test creation techniques must be
sufficiently powerful to allow performing various automated testing tasks. At the same time the automated
testing tool must be easy-to-use and clear even for inexperienced QA personnel. A good automated testing tool
may support several ways for creating automated tests. For instance, it may provide a simple, easy-to-use visual
automated test editor and some other features, like automated test scripts, for performing advanced tasks.
Many automated testing tools only provide scripting capabilities for creating automated tests. Creating auto-
mated test scripts require programming experience and may be difficult for inexperienced testers. TestComplete
provides a simple but powerful alternative to automated test scripts: keyword tests (also called keyword-driven
testing).
Keyword tests consist of operations that simulate user activities on the application under test (like clicking the
mouse buttons or typing on the keyboard) or perform other automated testing actions: launching tested ap-
plications, posting messages to the test log, running other automated tests (for instance, low-level procedures or
unit tests), use checkpoints to verify specific application functionality and more.
Keyword tests are very simple and easy to create. Testers do not need programming skills to create them. You
can record these keyword-driven tests or modify them visually in a special editor. The editor’s intuitive UI, wiz-
ards and dialogs help even inexperienced users create powerful automated tests with ease. Keyword tests are an
ideal feature for creating functional tests quickly.
But keyword tests are not the only automated tests that can be recorded or created visually. TestComplete can
record automated test scripts, load tests and low-level procedures (special procedures that simulate low-level
events like pressing and releasing of mouse buttons or keyboard keys). Load tests and low-level procedures can
then be modified visually with special editors. Recorded automated test scripts can also be modified in the code
editor.
4

5. Automated Test Scripting
Many automated software testing tools also provide scripting capabilities. Scripts are very important when per-
forming automated testing and they let users create automated tests for specific application features, provide a
variety of automated testing actions and perform those operations that cannot be done with standard features
built into the automated testing tool. Automated test scripts are useful even if the automated testing tool pro-
vides easier ways to create automated tests, because script code typically provides more powerful functionality,
it lets you perform those actions that other automated testing types cannot.
If you prefer scripting for your automated testing, it is important to pay attention to the scripting language the
automated testing tool uses. Using standard languages, such as VBScript or JScript rather than their dialects,
will decrease the time needed to get acquainted with a product. Another benefit is that you can find information
on these languages from third-party web sites as well as MSDN.
Two other important features to think about when choosing a tool that uses scripting is the code editor and
script debugger. Check whether the editor supports features that simplify and speed up script writing: syntax
highlighting, outlining, code completion, bookmarks, Ctrl-click navigation between script routines and many
others. Check if the tool’s script debugger includes the needed debugging functionalities, such as breakpoints,
step-over and step-into execution, Call Stack window and Evaluate dialog, etc. Using all of these features you
can easily find and correct bugs or find workarounds.
TestComplete supports several scripting languages: VBScript, JScript, DelphiScript, C++Script and C#Script, so,
you can choose the language you know best and thus save more time.
TestComplete includes a powerful code editor that supports all modern code editor features: syntax highlighting,
outlining, code completion, code templates, bookmarks and many others. TestComplete’s built-in script debug-
ger supports ordinary and conditional breakpoints, step-over, step-into and run-to-the-cursor features, the Watch
and Call Stack windows and special features for evaluating variables and objects during the script run.
Recording Automated Software Tests
Functional (or user interface) testing implies simulating user actions on the application under test. Implement-
ing these tests even for a small application is a tiresome process. When choosing an automated testing tool,
make sure the tool can record and play back user actions on the tested application. This feature will significantly
streamline test development.
TestComplete records user actions as keyword tests or automated test scripts, which you can then edit and play
back an unlimited number of times. Fast test development is not the only benefit. Recording also helps you
understand how to create a keyword test or how to write automated test script code, how to address application
objects and how to perform testing actions on them. We’d like to note that TestComplete not only records and
plays back tests; it also records and plays back HTTP traffic. This feature lets you easily create load tests for web
servers and services. TestComplete can also filter out traffic sent and received by specific applications during
traffic recording.
5

6. Creating Automated Tests that are Resistant to Changes
Normally, you start testing your application while it is still in development. So, it is quite possible that after you
make changes to the application, the current tests will become obsolete and you’ll have to update the test to
work with the changed application windows and controls. Modifying the tests may take the same amount of
time as modifying the application itself. That’s why smart teams like to create flexible tests that will be resistant
to the changes made to the application under test.
One approach that decreases the dependence on changes made to the tested application is the object-oriented
testing model. Each window or control of the application under test is represented by an appropriate object with
the name, for instance, [WinFormsObject(“Form1”)], so, to simulate user actions on this window or control, you
call the object’s methods and properties. For instance, simulating a mouse click on a button can be represented
as a call to the Click method of the test object that corresponds to this button. If you use this approach, there is
no need to change your test after the tested application has changed. For instance, if developers change a but-
ton’s width, height or position, the Click method will still work. If you simulate a click by coordinates, changing
the button’s size or position would cause changes in the automated test. Nevertheless, in some cases, you may
need to work with coordinates as well, so, it is nice if the automated testing tool supports both object-oriented
and coordinate-oriented methods.
Another approach that lets you create robust tests is object mapping. As a rule, automated software test-
ing tools address applications’ objects by their names or captions. However, during application development,
programmers may change controls’ captions, so you will have to modify your tests in order for them to match
the changed controls. That is why a good automated testing tool should be able to map an object name to a
custom name and use this custom name to address the object in tests. However, the tool must be able to map
an object not only by its caption or identifier. In this case, if a developer changes the tested application, you will
only have to modify the mapping conditions while the scripts and other tests remain the same.
TestComplete includes special features that let you map application windows, controls and objects by their prop-
erties. Testers can map a short name to the tested object and then use this mapped name in automated tests.
This simplifies working with objects, makes the created tests shorter and easier to understand. There is another
benefit of name mapping - it makes automated tests independent of object properties, which may change from
one build to another. Also, TestComplete supports the object-oriented approach to simulate user actions. Simu-
lating mouse events by coordinates is also supported.
Support for Controls and Objects
Windows applications use controls to display information and interact with users. The operating system provides
dozens of various controls that can be used in tested applications: edit boxes, list views, check boxes, menus,
buttons and much more. This number is also extended with custom components offered by third-party vendors.
These can be non-standard components, like a tree list or grid, or third-party analogs to standard components:
custom menus, toolbars and many others. When choosing an automated testing tool, make sure it includes spe-
cial features that simplify the work with controls used in your application. It is important to determine whether
6

7. these features are for both standard and third-party controls.
Another important factor is access to methods and properties of the application’s internal objects and controls.
This access provides methods and properties for simulating user actions and checking the results of test actions.
For example, you can use the internal methods and properties of a grid control to change the value in a grid’s
cell and then check if the change was successful.
TestComplete provides specific methods and properties that greatly simplify your work with standard Win32
controls, web page controls, WPF controls and third-party components provided by Microsoft, Borland, Com-
ponentOne, Developer Express, Infragistics, Syncfusion, Telerik and other vendors. In addition, TestComplete’s
automated test engine can access the internal methods and properties of application objects and use these
methods and properties to perform the desired testing tasks.
Supports Various Test Data Sources
When performing functional testing, it’s a common practice to simulate user input into the tested application’s
data forms. Thorough testing implies that you feed varied data to the input fields and that the tested applica-
tion processes it correctly. When choosing an automated testing tool, check which data formats it can use, such
as text files, XML files, database tables, and others.
With TestComplete, you can obtain data from text and XML files, Excel worksheets and databases like SQL
Server, Oracle and MySQL. TestComplete includes special program objects and test operations that allow you to
iterate through data records in external files and easily create data loops in your automated tests.
You can create and populate these data storages manually, or you can use TestComplete’s Data Generator to
generate the desired test data automatically and save it either to TestComplete’s Table variable or to the speci-
fied Excel file. This approach helps you decrease the time spent on preparing test data. For more information on
using this feature, see the Using Data Generators section in TestComplete’s help.
TestComplete also allows you to use script arrays, variables and constants to keep data for automated tests.
Running Automated Software Tests and Synchronization
The state of the application under test and the execution environment may differ from the state and conditions
you expected. For instance, some network resource may be unavailable, or the desired window is not created; the
web page loads very slowly or an unexpected modal dialog box appears. These and other similar factors may
cause the automated software testing tool to decide that the test has failed and it should be stopped. A good
automated testing tool must be able to handle these situations in the appropriate manner. For instance, it must
be able to “wait” for the needed windows, “close” unexpected dialogs automatically and perform other actions
that a human would do to continue the test.
TestComplete can handle these situations properly. For example, methods that work with windows and web
pages can wait until a window appears on screen or until a web page is loaded; methods that work with process-
es can wait until the desired process starts and so on. TestComplete also includes special features that let you
7

8. specify how the tool should handle unexpected windows without interrupting the test run.
Automatically Compare Results of Automated Test Actions
An automated testing tool should include features for automatically comparing the results of test actions. For
instance, if during the test run the tested application saves a customer list to a file, the automated testing tool
should be able to compare this file with a baseline copy and determine whether the save was successful and the
saved data is correct. The automated testing tool should be able to compare objects’ and controls’ properties,
images, files and other objects.
TestComplete includes special tools, objects and functionality that you can use to compare files, images and
object properties. The comparison can be implemented manually, or generated by TestComplete when you are
inserting a checkpoint in your test. TestComplete includes special dialogs and wizards that can be used to create
checkpoints for properties, objects, files, images, databases, tabular controls, XML documents, web services and
web pages. By using the powerful test implementation tools, you can create automated tests that will compare
complex objects and can also use regular expressions for comparisons.
Logging Automated Test Results
An automated software testing tool should keep a log of all testing actions and provide a detailed report for
them. The log of a high-end software testing tool must contain different types of messages: errors, warnings and
informative messages. In this case, you can easily determine what went wrong and why the test failed.
TestComplete includes a powerful test log that contains warning, error or informative messages as well as files
and images. The images are captured by the Test Visualizer subsystem and are automatically posted to the log
during the test run, or they can be posted using special operations or scripting methods. The captured images
illustrate test commands and simplify the analysis of the automated test results.
Exporting Reports
An automated testing tool should give users the opportunity to export test results to external files. For instance,
it may be useful, when you need to view test results on other computers that do not have this testing tool
installed. Also, you should check whether the testing tool can export the test log so that you can send the results
to your boss or colleagues.
TestComplete stores test results in XML files and allows you to easily export the test log data to several formats
(XML+HTML, Excel, MHT), pack the results with WinZip, WinRar or some other archive tool and send the results
via e-mail. You can perform these operations from the TestComplete UI or from tests.
Also, TestComplete provides access to the stored results. Since the test log uses the XML format, you can parse
the results and retrieve the desired information.
TestComplete also allows you to create a bug report for test log data and post the report to an issue-tracking
system.
8

9. Additional Needed Automated Testing Tool Features
To create powerful software tests, the test recording and playback features may not be enough. Choose a tool
that provides many additional features. For instance, you may need the following:
¿¿ A library of useful objects and functions to perform various operations. For example, reading data from and
writing it to files, working with databases, working with date and time values, regular expressions and many
others.
¿¿ The ability to call Win32 API functions.
¿¿ The ability to call functions that reside in DLLs and .NET assemblies.
¿¿ The ability to call routines that reside in Java classes.
¿¿ The ability to work with COM servers.
¿¿ Access to the shell (the WSH scripting object) and the ability to call the operating system commands.
¿¿ OCR functionality to retrieve data from applications’ windows and controls.
¿¿ The ability to create custom add-ons that will add specific functionality to your software test projects.
TestComplete includes all of these features and offers many others.
Testing Multi-Language Applications
If you have customers throughout the world and your company develops or is planning to develop localized ver-
sions of your applications, you have to check whether the software testing tool can work with localized applica-
tions and operating systems and whether it supports Unicode. If the tool does not support foreign languages,
then, most likely, you will not be able to use it to test localized software.
TestComplete supports Unicode and, therefore, it is possible to test multi-language software with it.
Automated Testing Tool: Support for Team Work
Testers are members of the development team. They work together and communicate with developers, manag-
ers and other various people. When choosing an automated testing tool, you may be interested in features that
support team work such as concurrent work with project files, sharing script code among several projects and
integration with source code control and issue-tracking systems.
TestComplete supports all of these features. It allows you to share your test projects with several testers. You
can easily link script units from one project to another. You can check project files in and check them out from a
source control system directly from TestComplete. Finally, you can add a bug report to an issue-tracking system
like Microsoft Visual Studio Team Foundation, Bugzilla, OnTime or JIRA, directly from the test log viewer, and
thus notify developers about the problem immediately.
9

10. Command Line and OLE Automation Support
Choose an automated testing tool that you can launch automatically, for instance, at a certain time or after the
application build is over. You will be able to do this only if your automated testing tool has the command line or
OLE interface where you can specify run parameters.
TestComplete supports command line arguments and can work with COM. Other applications, for instance,
SmartBear’s Automated Build Studio, use these features to call TestComplete automatically.
Integration with Team System and Build Software
Automated software testing tools are used along with build programs. For instance, the application “builder” can
run tests at the end of each build. When choosing an automated testing tool, check whether it can be used with
build programs.
TestComplete can be integrated with build software, for example, with SmartBear’s Automated Build Studio and
with Microsoft Visual Studio, including version 2010 of this popular Microsoft IDE. Also, TestComplete projects
can be called from Visual Studio test projects. TestComplete also includes a special assembly that you can use to
run TestComplete’s software tests from MSBuild projects.
Technical Support
A very important aspect of selecting an automated software testing tool is the quality of the technical support
provided. Prompt responses to your messages prepared by a professional support team will assist in resolving
problems and will save you time and energy.
Pricing Policy
When choosing an automated software testing tool, you have to determine whether its price is reasonable.
You should take into account not only the product price, but also the price of add-ons (if they are needed), the
support fee as well as training and upgrade prices. To get this information, you may need to call the company’s
office. Some vendors have special prices that you can only learn about when talking to a representative from
their sales department over the phone. SmartBear offers a clear pricing policy. You can find TestComplete prices
on our web site: TestComplete Pricing.
As you can see, SmartBear offers a feature-rich product at an affordable price. As a result, not only
large companies use TestComplete in their everyday work, but also small teams can afford it. Also, you can save
money by purchasing TestExecute licenses rather than TestComplete licenses for all of the computers that you
are going to run software tests on, but that are not for creating and debugging software tests.
Training is a paid option. For more information on this, see our TestComplete Training Web page.
10

11. Free Trial Version
An important factor is the availability of a trial version. Even if the product seems to meet your requirements, the
best way to determine this is to download and use it before purchasing. Since you are new to the product, you
may get questions and need some assistance with performing tasks. So, another important factor is whether the
company provides support for trial-version users.
SmartBear provides support for trial-version users. The trial version is a full-featured version, but it has a 30-day
limitation and there are limitations on the size of your scripts.
Try TestComplete Free for 30 Days
Scan and download your free trial to see why
testers choose SmartBear for automated testing.
You may also enjoy these other resources in the SmartBear Software Quality Series:
¿¿ 11 Best Practices for Peer Code Review
¿¿ Uniting Your Automated and Manual Test Efforts
¿¿ 6 Tips to Get Started with Automated Testing
Be Smart and join our growing community of over 100,000 development, QA and IT professionals in 90
countries.
11

12. About SmartBear Software
SmartBear Software provides tools for over one mil-
lion software professionals to build, test, and monitor
some of the best software applications and websites
anywhere – on the desktop, mobile and in the cloud.
Our users can be found worldwide, in small businesses,
Fortune 100 companies, and government agencies.
Learn more about the SmartBear Quality Anywhere
Platform, our award-winning tools, or join our active
user community at www.smartbear.com, on Facebook,
or follow us on Twitter @smartbear.
SmartBear Software, Inc. 100 Cummings Center, Suite 234N Beverly, MA 01915
+1 978.236.7900 www.smartbear.com ©2012 by SmartBear Software Inc.
Specifications subject to change. TC-WP-SATT-042012-WEB