2. Objective
⢠Identify a tool to automate the functional
testing of a Flex application. In my application,
we have the following use-cases:
â Create/Edit Invoice
â Search for a customer
3. Tools Evaluated
⢠FlexMonkey 1.0 by Gorilla Logic
⢠SilkTest 2009 R2 by Borland (Plugin for Flex)
⢠Ranorex Studio
⢠Other tools (not evaluated/less popular)
â QTP
â Selenium4Flex
â RiaTest
4. Automation support in Flex
⢠Adobe provides automation delegate implementations for the
framework components
â Automation.swc (delegation implementations)
â Automation_agent.swc (Facilitate communication with a tool)
â Automation_dmv.swc (delegates for charts and advanced datagrid)
⢠Third party tool developers leverage these and build on top of it.
â The tool developers are limited by what Adobe exposes via the
automation hooks.
⢠Custom components
⢠Components that extend framework components
â Automation delegates supplied should work
⢠Components that do NOT extend framework components (extend
UIComponent instead)
â Developer of component needs to provide automation delegates and override event
handlers.
6. Tool #1 : Flex Monkey
⢠Quick Facts
â Open Source
â Ver 1.0 GA released
â Adobe AIR application
â Scripting language â Actionscript
â Support for:
⢠Record/Playback
⢠Validation
⢠Timing delays
⢠Exporting script â integration with ANT script
⢠Pluses
â Simple to setup
â Works on both OS (Windows/Mac)
â Works with both IE and Mozilla
⢠Negatives
â Does not scale beyond simple usecases
â Sparse documentation for advanced usage
â Community is in infancy (Opensource like support )
â Recently moved out of Google code and the transition was not smooth for forum users
â Core developers offer paid consulting gigs for training and help with tool.
7. Tool #2 : SilkTest Flex plugin/Silk4J
⢠Quick facts
â 2009 R2 â Flex support since 2008 R2
â Silk4J â Eclipse plugin
â Scripting language â Java or Silk script
â Object identification using Xpath
⢠Pluses
â JUnit like test generation (Silk4J)
â Familiar tool(Silk not Silk-Flex) for QA teams in Intuit
â Enterprise support plan
⢠Negatives
â Performance hog (Seems like it is broken)
â Tool is not very developer friendly.
â Does not work with IPP native flex apps.
â Support from Ireland (timezone) â primarily e-mail based
8. Tool #3: Ranorex Studio
⢠Quick facts
â Ver 2.2 (2.3 coming out next week, 3.0 targeted for Fall â10)
â Built on .NET framework
â Scripting support â C#, Python and VB.NET
⢠Pluses
â Mature tool â very intuitive interface
â Object recognition using custom XPath
â System integration testing with Quickbooks possible (not tested out)
â Support for AJAX/Javascript testing
â In-tool support for the whole object tree once the application is instrumented.
â Has support for object preview.
â Once the test is fully baked, you can export it as an executable (.exe). This can
then be run simply by double clicking the exe.
â Executable can be passed in command line params
⢠Negatives
â ?
9. Criteria for selection
⢠Excellent support for object identification
â Custom components like custom combo-box, item
renderers
⢠Easy to setup and ramp up
⢠Mature support for scripting using standard
languages
⢠Ability to integrate with CI build
⢠Performance
⢠Good record and replay capability
⢠Support for IE and Firefox
10. Quick Summary
⢠FlexMonkey dropped from the list since it could not
recognize some of the non-trivial objects like Custom
ComboBox and DataGridColumn with item Renderer.
⢠SilkTest 2009 is a good attempt but has a long way to
go to be more developer/QA friendly in terms of the
tool. Also, it has a higher cost for the license.
⢠Ranorex satisfies our current requirements for
automated testing, goes beyond what Adobe provides
for automation support and delivers a great experience
for the developer.