Graphical user interface (GUI) testing is a potentially problematic area because constructing effective test cases is more difficult than the corresponding application logic. The roadblocks to effective functional GUI testing are:
Functional GUI testing needs to deal with GUI events as well as the effects of the underlying application logic that results in changes to the data and presentation.
The common methods for functional GUI testing are the "record and execute" script technique and writing test programs for different scenarios. In the "record and execute," the test designer interacts with the GUI and all the events are recorded in a script. The script can later be replayed to re-create user interactions for a particular scenario. In the test programs, the test designer tries to understand and write tests for the various GUI decision points.
This article discusses how Abbot can be used to quickly and effectively come up with a comprehensive GUI test framework for Swing GUI applications. Abbot (http://abbot.sourceforge.net/) is a JUnit extension for Swing GUI testing. It provides an interesting framework that can be used for test case generation as well as "record and execute" scripting.
Introduction to Abbot
Abbot builds upon the java.awt.Robot class to provide an automated event generation and validation framework for Swing GUI components. The framework can be used to create, record, and execute scripts and programmatic test cases in Java. Abbot also has a script editor called Costello that facilitates the creation of scripts in XML. The framework can also be well integrated to run with JUnit.
To illustrate the use of Abbot to create a GUI test infrastructure, this article will make use of the following two scenarios.
1. GUI already exists: The GUI has been coded and the application needs to be tested, but no unit tests are available. This scenario will primarily make use of the "record and execute" style and then focus on how test suites can be created and run with JUnit.
2. GUI has not been coded: The GUI has been designed on paper; however, no working code exists. This scenario will primarily focus on creating test cases in Java. This method will create a homogeneous suite of tests both for the back-end application code and the GUI.
The two scenarios focus on the end-point cases in functional GUI testing. The principles mentioned can be mixed and matched to suit the needs of the project. To get started with Abbot, download the JAR files and place them in the lib directory of the project.
Scenario One: GUI Already Exists
In this scenario, most of the GUI has already been coded. Automated testing is not available and the testing is mainly being done by hand. The goal is to come up with a test suite in the shortest time possible so that the quality metrics for the project can be met effectively.
The script editor Costello will be used to develop a battery of tests. Costello provides the "record and execute" functionality that will allow us to record different user interaction scenarios with the GUI and test those scenarios efficiently. The scripts that record the user interactions are saved as XML. The salient features of the scripts that help in rapidly creating a functional test framework are:
An example of a test script in XML is provided in Listing 1.
With this background about the features of the script, it's time to look into how to use the script editor to create the test cases. The sample GUI shown in Figure 1 is an example from the Java Swing tutorial available at java.sun.com.
The GUI is a Celsius-to-Fahrenheit converter that takes in a numeric value and on the button click displays the converted result in Fahrenheit. To test this GUI, we can write some simple GUI tests to verify the results on the input of a positive number, a negative number, and a nonnumeric input.
The script editor will be used to create the test cases. Costello, the script editor, can be started by:
java -jar lib/abbot.jar
Once the GUI is running, the first step is to create a new script (File->New Script). Each script has a launch line that specifies the method (containing the GUI) to be tested. The launch line is edited with the information that can launch the GUI. The launch information requires the method name, arguments for the method, class information, and the classpath (see Figure 2).
method="main"
args="[]"
class="CelsiusConverter"
classpath="src/demo"
Note that the XML test script example has the same information in the launch tag.
With the launch information in place, the GUI can be launched (Test->Launch). This will prepare the framework to record user actions. To record user actions, press F2 (Capture->All Actions). With this step we can record the first case, which inputs a positive number, and then press convert.
Once the user interactions have been captured, we can move the actions out of the sequence block for better readability. With the script looking the way we want, it's now time to add an assertion. To do this, we'll use the Hierarchy tab to navigate to the GUI component. Upon selection of the GUI component, the name-value pair to be validated can be selected and the Assert Property = Value button will add the assertion (see Figure 3).
The other two test cases can be created similarly to form a test suite. The test cases can run within the JUnit test harness utilizing either the command line or the GUI test runner. The junit.extensions.abbot.ScriptFixture class is subclassed to create the CelsiusConverterTest.
public class CelsiusConverterTest
extends ScriptFixture
The junit.extensions.abbot.ScriptTestSuite class is used to autogenerate a suite based on test scripts matching a certain criteria. In this case, all scripts residing in a particular directory will be used. The test ScriptTestSuite class is subclassed from the junit.Framework.TestSuite class.
public static Test suite() {
return new ScriptTestSuite
(CelsiusConverterTest.class,
"src\demo\scripts\CelsiusConverter");
}
The main method of the CelsiusConverterTest class invokes the JUnit test runners.
public static void main(String[] args){
args = Log.init(args);
String[] names = {
CelsiusConverterTest.class.getName()
};
if (args.length == 1 &&
args[0].equals("--gui"))
junit.swingui.TestRunner.main(names);
else
junit.textui.TestRunner.main(names);
}
This approach facilitates the creation of a variety of user interaction scenarios that can be easily integrated with JUnit. Another interesting and handy feature is the ability to insert existing scripts into another script. Some high-level GUI tasks can be broken down into smaller tasks. In these cases, I've created scripts for the most granular tasks and then created different interaction scenarios by inserting the scripts for different tasks in a different order or by adding additional user interactions in between the tasks. Since the scripts are XML-based, they're very easy to read and they help in rapidly generating different scenarios.
Scenario Two: GUI Is Ready to Be Coded
In this scenario, the GUI tasks have been identified and the specification is ready to be coded. A good way to begin coding is to write the tests that need to be passed before the GUI is created. The XML-based scripts can be used to create different test cases; however, to demonstrate the full power of Abbot, this section will discuss how the JUnit test cases can be written for the GUI.
The Abbot Java API will be used to write traditional JUnit test cases. The salient features of the API that help in creating test cases are:
The Celsius-to-Fahrenheit converter example presented in the previous section is used to walk through the steps involved in creating a test case using the Abbot Java API. In this test case, the input of a negative number to the Celsius input will be tested.
Like JUnit test cases, the CelsiusConverterTest class will subclass the junit.Framework.TestCase class and the constructor will initialize the name of the test.
public class CelsiusConverterTest
extends TestCase {
public CelsiusConverterTest
(String name) { super(name); }
}
The setUp() and tearDown() methods can be used to initialize the frame in which the GUI will run. The setUp() method may be used to populate any arrays or other defaults that are required by the GUI to be fully functional.
private ComponentTester tester;
protected void setUp() {
tester =
ComponentTester.getTester
(CelsiusConverter.class);
// wait for the window to be displayed
tester.waitForFrameShowing
("Convert Celsius to Fahrenheit")
}
To write the test, we create a method called testNegativeNumberInput(). This test method needs to get a reference to each of the GUI components that are being tested. One way Abbot gets a reference to a GUI component is by looking up a component of the appropriate class. This allows Abbot to continue to refer to different GUI components even if the layout changes.
ComponentReference ref =
new ComponentReference
("tempCelsius", JTextField.class);
Component tempCelsius =
getFinder().findComponent(ref);
The next step is to initialize the ComponentTester so that the user interaction event of typing in a negative number can be generated
tester = ComponentTester.
getTester(tempCelsius);
tester.actionKeyString(-45);
Once the sequence of user interactions is in place, we would need to verify if the result is correct by adding assertions. The reference to the label is obtained and then the value of the label is tested using assetEquals()
JLabel fahrenheitLabel =
(JLabel)getFinder().findComponent(ref);
assertEquals("-49 Fahrenheit",
"-49 Fahrenheit",
fahrenheitLabel.getText());
To run the test, the TestHelper class available in the test package in Abbot will be handy. The TestHelper class also provides automatic test suite generation functionality.
public static void main(String[] args){
TestHelper.runTests
(args,CelsiusConverterTest.class);
}
This method can be used in conjunction with JUnit test cases for the application logic to create a homogeneous test suite. The decision points in the application logic can be tested using normal JUnit tests. Test cases generated using Abbot will test for different user interaction scenarios.
Conclusion
Writing functional GUI tests can be a tedious task. The concepts from both of the testing scenarios mentioned can be combined to produce a comprehensive suite of tests. Abbot as a framework provides both scripting functionality and a Java API. The framework easily integrates with the JUnit test harness and therefore, during application development, the functional GUI tests can become a part of the test suite. All these features of Abbot make it an effective framework for rapidly creating a comprehensive test framework.
Resources