Taylor Results

Taylor Results is a testing framework which leverages the power of EclipseWiki, JUnit, and Maven for defining and executing test scripts. It was inspired by work we did using a tool called Fitnesse. The basic idea is to raise the level of abstraction by using wiki table syntax to define test scripts. This makes it easy for people without a programming background to write tests. These wiki pages can also serve as documentation in Test Driven Design/Development (TDD).

While we liked the overall idea of Fitnesse we believe Taylor Results is unique and an improvement because:

• No additional infrastructure, such as a wiki server, is needed
• Test scripts are edited in Eclipse just like all other artifacts
• Version control is handled not by the wiki, but by the same tool as all your other artifacts
• JUnit is already integrated with Eclipse and Maven for test execution and reporting
• It supports local container-less testing as well as remote in-container testing

Contents 1 Sample 2 Installation 3 TestCase Classes 4 Configuration 5 Variables 5.1 Expressions 5.2 Context 5.3 Types 6 Test Suite Decomposition 7 Performance Testing 8 TDD + MDA 8.1 Key Points 9 Credits

Sample

Let's start with a simple example for testing the following class.

package sample;

public class PojoServiceBean implements PojoService {

	public String echo(String value) {
		return value;
	}
}

• src/test/wiki/Sample.wiki

'''Test Echo Method'''
|*net.taylor.results.pojo.MethodTestCase*|*Test Echo*|
|*class*|sample.PojoServiceBean|
|*method*|echo|
|*set*|arg0|value1|
|*invoke*|
|*check*|response != null|true|
|*check*|response|value1|

1. Add any descriptive text to the wiki page. (see the Syntax tab in EclipseWiki editor for details)
2. Define a test using a wiki table.
3. The first row of a table directs which Taylor Results TestCase class to use and the name of the test for reporting.
4. The rest of the rows in the table are specific to the TestCase class that is used.
5. For the Pojo MethodTestCase the class name and method name are required.
6. Then added set rows for the arguments, an invoke row to call the method, and check rows to validate the response.

• EclipseWiki Editor - Browser tab

• src/test/java/sample/SampleTest.java

package sample;

import junit.framework.Test;
import junit.framework.TestCase;
import net.taylor.results.TestSuite;

public class SampleTest extends TestCase {

	public static Test suite() throws Exception {
		return new TestSuite("src/test/wiki/Sample.wiki");
	}
}

1. The TestCase class just provides a harness to execute the test via Eclipse or Maven.
2. The suite() methods is all that is needed.
3. The method uses net.taylor.results.TestSuite to process the wiki pages into a suite.
4. The test results are then rendered per usual by either Eclipse or Maven.

That's essentially all there is to it. Easy!!!

Installation

• Install the EclipseWiki Plug-in from http://eclipsewiki.sourceforge.net/
• The taylor-results Jar is installed via a maven dependency
  • Create a maven jar project for your test plan and add the following to your pom.xml

	<dependency>
		<groupId>taylor</groupId>
		<artifactId>taylor-results</artifactId>
		<version>1.3.0</version>
	</dependency>

	<repositories>
		<repository>
			<id>taylor</id>
			<name>Taylor Repository</name>
			<url>http://taylor.sourceforge.net/maven2</url>
		</repository>
	</repositories>

TestCase Classes

Method Test Cases - These allow you to test methods on your classes/beans/services:

• Pojo
• Seam
• JNDI
• Spring
• Ejb3

JSF Test Cases - These allow you to test your user interface:

• JSFUnit
• Seam

Database Test Cases - These allow you to initialize your database:

• DBUnit

See the test suite in the taylor-results project for more examples. http://taylor.svn.sourceforge.net/viewvc/taylor/components/trunk/taylor-results/src/test/

Configuration

Some test cases require special configurations. And sometimes you need to parameterize your test suite. To facilitate this a special Configuration Tables is provided.

This example shows parameters for:

• DBUnit
• JSFUnit
• JNDI access

|*net.taylor.results.model.Configuration*|
|*dbunit.driver*|org.hsqldb.jdbcDriver|
|*dbunit.url*|jdbc:hsqldb:src/test/db/sample|
|*dbunit.user*|sa|
|*dbunit.password*|blank|

|*web.user*|user|
|*web.password*|user|

|*version*|1.0.0-SNAPSHOT|
|*java.naming.factory.initial*|org.jnp.interfaces.NamingContextFactory|
|*java.naming.factory.url.pkgs*|org.jboss.naming:org.jnp.interfaces|
|*java.naming.provider.url*|jnp://localhost:1099/|
|*java.naming.security.principal*|user|
|*java.naming.security.credentials*|user|
|*j2ee.clientName*|jbossws-client|

Variables

Use expressions and context variables to parameterize your test cases.

Expressions

• ${config['variable.name']} - Returns a configuration variable
• ${random.email} - Returns a random email address. For instance: EDAF23C@gmail.com.
• ${random.int} - Returns a random integer within the range of 0..10000
• ${random.string} - Returns a random string.
• ${random.digits(int)} - Returns a random string consisted of number of digits. For instance, ${random.digits(3)} might return '023'.
• ${date.now} - Returns java.util.Date object representing today.
• ${date.after(int)} - Returns java.util.Date object representing number of days after today.
• ${date.before(int)} - Returns java.util.Date object representing number of days before today.
• ${calendar.now} - Returns java.util.Calendar object representing today.
• ${calendar.after(int)} - Returns java.util.Calendar object representing number of days after today.
• ${calendar.before(int)} - Returns java.util.Calenar object representing number of days before today.
• ${xmlcalendar.now} - Returns javax.xml.datatype.XMLGregorianCalendar object representing today.
• ${xmlcalendar.after(int)} - Returns javax.xml.datatype.XMLGregorianCalendar object representing number of days after today.
• ${xmlcalendar.before(int)} - Returns javax.xml.datatype.XMLGregorianCalendar object representing number of days before today.
• ${nil.object} - Returns null.
• ${empty.collection} - Returns the empty collection(array,List or Set) based on the type of the expression in second column of set row.

Context

When using method test cases it is useful to use the results from one test in another. Context variables allow for this.

• Declare a key name to store the response of a method

|*key*|echo2|

• Use the response in another test to set values

|*set*|entity|${echo2.response}|

Types

The types of arguments and attributes are automatically detected using Java Relection, and it is then used to construct the object by calling its newInstance method. However, if the type is an interface or an abstract type, object can't be constructed that way. In this case, you will have to specify the concrete type in the set row using ${type:concrete class name}, sample usage:

|set|aType.anInterface|${type:test.AnInterfaceImpl}|
|set|aType.anInterface.attr1|value1|

Test Suite Decomposition

You can place all your tests in a single wiki file, but it is better to decompose them into multiple reusable files.

EclipseWiki allows you to link pages together with the typical wiki UpperCamelCase syntax. Taylor Results will traverse these links to create the test suite.

Example:

1. Create a top level page to define your suite (This file maps to the JUnit TestCase class that executes the suite)
2. Link this to a Configuration Page used by all suites
3. Link this to a DBUnit Page used to initialize the database before each suite executes
4. Add test cases inline on the top level page
5. Or link to reusable test cases

Performance Testing

Taylor Results provides integration with JUnitPerf to provide basic performance testing. JUnitPerf is a great tool for finding obvious bottle necks way ahead of heavy duty performance testing. By adding this to your daily test suite you will be amazed by what you will find with so little effort!

Add the special performance rows to your test case to test for response time and simulate load. Use one or both and experiment with the order.

• Test with Load wrapping Timed

This tests that the method can execute concurrently for 2 users, with a 100ms delay between users, with each user repeating the call 2 times and that each call executes in less then 300ms.

|*net.taylor.results.pojo.MethodTestCase*|*This is my Loaded Timed Test*|
|*performance*|*time*|300|
|*performance*|*load*|*users*|2|*delay*|100|*repeat*|2|
|*class*|net.taylor.results.test.PojoServiceBean|
|*method*|echo|
|*set*|arg0|value2|
|*invoke*|
|*check*|response != null|true|
|*check*|response|value2|

• Test with Timed wrapping Load

This tests that the method can execute concurrently for 2 users, with a 100ms delay between users, with each user repeating the call 2 times and that the whole thing executes in less then 500ms.

|*net.taylor.results.pojo.MethodTestCase*|*This is my Timed Load Test*|
|*performance*|*load*|*users*|2|*delay*|100|*repeat*|2|
|*performance*|*time*|500|
|*class*|net.taylor.results.test.PojoServiceBean|
|*method*|echo|
|*set*|arg0|value2|
|*invoke*|
|*check*|response != null|true|
|*check*|response|value2|

Jamon

• See this blog entry for adding Jamon to your performance test tool kit: http://www.jroller.com/jgilbert01/entry/performance_testing_made_simple

• TODO - add support for JSFUnit Timer

TDD + MDA

Using wiki pages to define your test scripts can play an integral role in your Test Driven and Model Driven Development approach. Here is an example of how this might work.

1. Start by creating a model in Taylor
2. Model some use cases with actors and a summary description
3. Then start to enumerate all the test scenarios needed to satify the use cases
   1. Model these as use cases with a <<TestScenario>> stereotype
   2. Don't go into great detail yet, just collect all the scenarios and give a brief description
4. Once you have a reasonable list of test scenarios you can allocate them to iterations on your project plan and assign people to work on the details
   1. In other words, use test scenarios to control the scope of your iterations and define what will be delivered at each milestone
   2. Start with some simple scenarios to grease the wheels, then tackle the hard ones, and then circle back for the rest
5. Now that you have a plan you can start to drill down into the details
6. Use Taylor to generate the following from what you have in the model so far:
   1. The maven/eclipse project that will house your test plan
   2. A configuration wiki page and a sample DBUnit wiki page
   3. For each use case with the <<TestScenario>> stereotype, a top level wiki page and the accompanying JUnit test case class
   4. For each entity and service, a set of templates that can be copied & pasted into your scenarios when you get to that point
   5. NOTE: You can re-generate at any point and not lose your work, so do it as often as necessary
7. Now that you have some wiki pages created you can start to work on the details
8. Don't worry about creating wiki tables at first, just use the wiki to start creating a textual description of the flow of the test scenario
9. Collect the test data that you want to use and put in the wiki pages as plain text
10. At any point, when you identify an entity or field or service, go ahead and model what you know so far
11. Once your model starts to take form then you can start to copy and paste parts of the generated wiki table templates into your scenarios and assign the test data values that you have collected
12. Now run the build and see your test results

Key Points

• Use the test scenarios to discover what to model
• Generating the model will give you wiki table templates to paste into the test scripts
• Since the code is generated as well, you can build and run your test scripts right away
• Start small and go through the Model, Generate, Tweak, and Execute cycle as frequently as possible and automate it with a continuous integration build server
• The wiki pages also serves as documentation and as part of the model/specification
• Use test scenarios as your major planning element
• If you don't know how to test it then there is no reason to build it!!!

Credits

Leverages ideas and code from ServiceFixture