Ulrich Palha

Testing Requirements

I have a number of web services that return a status code and an XML payload. For each of these web services, I would like to write tests that meet the following criteria:

1. Test requirements
   1. Verify that the web service returns the correct status code based on the inputs (testWebserviceStatus)
   2. If the status code is correct, then I would like to verify 1 or more properties of the payload. Every property is independent, so I would like to have each property verification run as a separate test (testWebserviceProperties)
2. Other requrements
   1. The ability to add/remove the list of web services to test and add/remove the list of properties to verify for each service without updating the test code.
   2. If testWebServiceStatus fails, then its associated testWebserviceProperties should be skipped
   3. The test report should clearly show the success failure of each testWebservice/testWebserviceProperty test along with the web service/property that was being tested

Select a Testing Framework

Based on the criteria, I would need a test framework that supported data-driven tests and ideally allowed the creation of new tests based on the output of a previous test or, at minimum, allowed dependent tests.

After considering Junit and TestNG, I settled on the latter because it supports data-driven testing (as does JUnit with parametrized tests), and it also inherently supports dependent tests.

Setting up the tests with TestNG

1. Feed XML file data into test

I stored the list of web services to test and their associated properties to verify in an XML file and used a @DataProvider to read this file and pass the parameters to the testWebserviceStatus method

        @Test(dataProvider = "getWebServiceUrls")
	public void testWebserviceStatus(final TestParameters params) {
	 //test body
	}

       @DataProvider(name = "getWebServiceUrls")
	public Object[][] getWebServiceUrls() {
         //code to read test input from xml file
	}

2. Save test output as input for next test

To have the output of testWebserviceStatus, together with properties to test, passed into testWebserviceProperties, I used a List

        // used to pass data between tests
	private final List<PropertyTestInput> webServiceOutput = new ArrayList<PropertyTestInput>();

	@Test(dataProvider = "getWebServiceUrls")
	public void testWebserviceStatus(final TestParameters params) {
		final String xmlPayload = callWebservice(params.getUrl());
		final PropertyTestInput input = new PropertyTestInput(xmlPayload, params);
		// Code to verify status code of web service would go here

		// add output of test and original test params as input to next test
		webServiceOutput.add(input);
	}

3. Use previous test output as input for current test

Next, I set up a dataProvider for testWebserviceProperties to retrieve its input from the List

	@Test(dataProvider = "getPropertyTestInput")
	public void testWebserviceProperties(final PropertyTestInput input) {
		// Code to test WebServiceProperties goes here
	}

	@DataProvider(name = "getPropertyTestInput")
	public Iterator getPropertyTestInput() {
		return new TestNGIterator<PropertyTestInput>(
				this.webServiceOutput.iterator());
	}

4. Create dependency between 2 tests

Next, I needed to ensure that testWebserviceProperties ran after testWebserviceStatus. I first tried dependsOnGroups

 @Test(dataProvider = "getWebServiceUrls", groups= {"callWebService"})
	public void testWebserviceStatus(final TestParameters params) {//code }

  @Test(dataProvider = "getPropertyTestInput", dependsOnGroups = {"callWebService"})
	public void testWebserviceProperties(final PropertyTestInput input) {//code }

However, this meant that when any testWebserviceStatus test failed, then all testWebserviceProperties were skipped. I only wanted the testWebserviceProperties test associated with the failed testWebserviceStatus to be skipped.

To do this, I used priorities to manage the dependencies.

 @Test(dataProvider = "getWebServiceUrls", priority=1) //a lower number is a higher priority
	public void testWebserviceStatus(final TestParameters params) {//code }

  @Test(dataProvider = "getPropertyTestInput", priority=2)
	public void testWebserviceProperties(final PropertyTestInput input) {//code }

If testWebserviceStatus failed, then I simply skipped adding input to the List to skip running the associated testWebserviceProperties.

Test output

The test output in the runner displays each run of testWebserviceStatus and testWebserviceProperties.

Conclusion

If you need to write tests where one test depends on the output of a previous test, TestNG makes this relatively easy.

I am not sure that this is the TestNG recommended way, so I have posed this question to the TestNG users group. If you know of a better way, let me know.

Tags: data-driven testing, integration test, Junit, open source, TestNG, web service