Several tools and frameworks are now available to support the execution of RIA testing processes. In the following we report the characteristics of both some RIA testing tools that we developed ad hoc in our research laboratory, and some other ones we selected from the Web. All the considered tools belong to the categories presented in the previous subsection and can be freely downloaded from the Web.
6.3.1 CReRIA
CReRIA, already discussed in chapters 4, is an interactive tool for dynamicanalysis that has been designed for supporting comprehension and reverse engineering processes of RIAs implemented in Ajax [4]. In particular the tool supports the semi-automatic reverse engineering of a Finite State Machine (FSM) modelling the behaviour of an Ajax application user interface. The tool offers an integrated Web browser that allows a user to navigate the RIA and to trace and record his user sessions. Hence, this component of the tool actually implements the functionality of a Capturer that can be used in testing
105
processes too.
As to the recorded information about traced user sessions, both data about the structure and the content of user interfaces, both data about user events that were fired on user interfaces are captured by the tool. More precisely, CReRIA describes each user interface only by a selected sub-set of its widgets (such as buttons, text fields, forms, …) that have specific properties and values of these properties. As to the user events which were fired during the navigation, the user event type (i.e. click, mouseover, mouseout, etc…), and the widget on which the user event was fired are captured by the tool. From the perspective of Ajax testing, this information can be used to produce precise test cases as sequences of user interfaces and events triggered on them.
The CReRIA tool also implements several heuristic criteria for clustering together equivalent interface states, as well as equivalent events.
6.3.2 CrawlRIA
CrawlRIA is a tool belonging to the category of automatic Crawlers. It has been designed for crawling Ajax applications and explores the user interface states by automatically firing events on these interfaces. It fires events by either a depth first or a breadth first visiting strategy. CrawlRIA is also able to extract data about interfaces and triggered events and to cluster them with the same techniques implemented by the CReRIA tool, and to generate the corresponding FSM model. In testing processes, CrawlRIA can be used for automatically generating execution traces (which can be stored in the same format and in the same database used by the CReRIA tool).
6.3.3 TestRIA
TestRIA, already introduced in chapter 5, [5] is a tool designed for Ajax test case generation and management. It implements the functionalities of a Test Case Generator, Test Case Reducer, Automatic, Hybrid and Manual Assertion Generator, and of an Assertion Verifier. Moreover, it also provides the functionality of Test Case Replayer.
TestRIA generates test cases by translating the execution traces collected either by CReRIA or by CrawlRIA into test cases implemented as Java test classes that use classes
106
from the Selenium RC library [104]. The tool is an Assertion Generator that builds assertions in three distinct ways: if test cases must be used for regression testing of the application, it builds automatically assertions regarding selected properties of the user interfaces of the initial version of the application that were encountered during test case executions. TestRIA is also able to suggest potential assertions about widgets found in the interfaces, or to support the manual definition of assertions. This tool is also an Assertion Verifier that exploits the same interface equivalence criteria implemented by CReRIA and CrawlRIA for assessing the equivalence of a given interface to a cluster of similar interfaces. It verifies generic assertions associated with Html validity or Web accessibility requirements by invoking external Web services too. Eventually TestRIA is also able to replay the generated test cases either in the TestRIA context or in a standalone way.
6.3.4 DynaRIA
DynaRIA is a tool supporting the comprehension and the testing of RIAs implemented in Ajax. It is based on dynamic analysis and provides functionalities for recording (acting like a Capturer) and analysing user sessions from several perspectives and for producing various types of abstractions and visualizations about the behaviour of the application. In particular it is able to execute the test cases produced by the CReRIA and TestRIA tools (acting as a Replayer), and to implement both a Monitor of JS crashes and a Network Monitor of Http errors.
6.3.5 Crawljax and ATUSA
Crawljax [102] is an open source Java tool supporting the automatic crawling and testing of Ajax applications. It has been developed by the SERG group at the Delft University.
Initially designed for the automatic crawling of Ajax Web applications for indexing purposes [58], the most recent releases of Crawljax also supports invariant-based testing, regression testing [105], accessibility validation, security testing, broken links/images/tooltips detection [83]. The last releases of Crawljax include ATUSA, a tool originally designed for supporting AJAX testing.
Crawljax comprehends components that explore the existing application for building a
107
state-flow graph representing the dynamic DOM states and the transitions between them, and a component (a Test case generator) that builds test cases from that model. The tool also provides several functions for generating a set of pre-defined assertions about the user interface states (Assertion Generator) and for verifying them (Assertion Verifier).
6.3.6 Selenium
Selenium [104] is a framework composed of a set of tools supporting test automation of Web applications. In particular, Selenium-IDE is a Firefox add-on providing an interface for developing test cases starting from information extracted during a user navigation session. Selenium-IDE also provides assertion suggestions during the capture activity, by proposing expressions related to the presence of widgets and attributes in the captured user interfaces. The test cases produced by Selenium IDE can be executed in the context of Selenium IDE itself, or they can be automatically replayed in the context of a program written in a high-level programming language by using the Selenium-RC API.
Table 6.1 summarizes the analysed tools and reports the types of testing tool components they offer.
Table 6.1: Coverage of Tools Categories
CReRIA CrawlRIA DynaRIA TestRIA Crawljax Selenium
Crawler X X
Capturer X X X
Replayer X X X
Execution Monitor X X
Test Case Generator X X
Test Case Reducer X
Automatic Assertion Generator X X
Hybrid Assertion Generator X X
Manual Assertion Generator X X
Assertion Verifier X X X