By Jeff Rappoport

Looking back at past test automation projects that MapleWorks has been involved with leads me to conclude that test automation is no different than any other software development project.

Typically test automation tools are built using scripting or programming languages; a commercial platform such as AutomatedQA’s TestComplete or Emprix’s Hammer; an open source platform such as STAF; or an in-house development platform. Regardless of the approach taken, test automation is a product. It is a stand alone tool used to trigger events and then report whether the event triggered was handled appropriately by the system under test.

The key attributes surrounding test automation are:

• Understanding how the system under test operates
• Understanding the characteristics of the automation test tool
• Defining a precondition
• Creating an automated test
• Verifying the test case outcome
• Removing all evidence about the test

Understanding how the system under test operates is the most important component of any test automation activity. Knowing what triggers need to be injected into the system is crucial since the core component of any automated test case is to create activity on the system under test. Understanding what output is generated by the system under test is equally important because this output will be used by the test automation platform to declare whether the automated test case passes or fails.

Understanding the characteristics of an automation test tool is crucial in the construction of an action (the automated test case) from the desired intent (wanting to test the system). Each test tool is bound to some technology and each technology has limitations. The resourceful test case developer has the ability to use creativity and possibly out of the box approaches to bridge the gap between what is needed to trigger activity on the system under test and how to manipulate the test tool to generate this trigger. Understanding the automation test tool is also important in defining mechanisms for retrieving output data from the system under test.

A precondition essentially states what the starting point is prior to injecting the trigger onto the system under test. This precondition needs to be applied to the system under test as well as the automation test tool. A state machine analogy can be applied to this idea. In order to arrive at state “B”, event “B” must be applied from state “A”. Precondition requires that the system under test and the test tool be initialized and set to state “A”.

Creating an automated test case is the mechanics of applying the lessons learned from the previous two points in the construction of a tool to test the system under test. The test tool identifies the language used to create the test case and the decision making criteria in evaluating the test case outcome.

Verifying the outcome is a means to determine if the system under test is sane and behaving as expected. Once all the triggers are applied it is important to retrieve information from the system under test in order to evaluate its behavior. The automated test tool needs to obtain this data, and determine if the data meets expectations. If the expectations are satisfied then the test succeeded. If the expectations are not satisfied then the test case failed.

A failed test case is just is as valuable as a successful outcome because this suggests that the system under test is faulty.

Removing all evidence is an eye catching phrase to describe the post condition cleanup step. The goal is to place the system under test as well as the automation test tool into a known state so the next automated test case can be run. As stated previously, the test tool and system under test needs to be reset to state “A”.

At the end of the day successful test automation requires:

• Understand and prioritize your requirements

Understand the product that needs to be tested

Understand how the intended user is planning on using the tool

Understand how the system under test reports a successful or failed outcome

• Gain knowledge

Prototype the most complex procedure the intended user wants tested

Understand the technological limitations imposed by the system under test

Understand the limitations imposed by third party tools

• Go off and build the automation tool.

Design a suitable architecture

Implement the solution

This entry was posted on Wednesday, June 17th, 2009 at 3:05 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.