Some of the concepts described are not new and experienced test writers may get bored while reading this chapter. I think, however, that going through it is important in order help the reader understand the features this java testing framework provides.
The test case concept is the core of any testing framework. Everyone who ever used JUnit or TestNG is familiar with it. The idea is to have a class that:
If we have a quick look at these requirements, we notice that a class with two methods, one to create the testing data configuration and another one tear it down, accompanied by a few of coding rules to enable us to identify methods that implement the tests would be a good candidate for a test case definition. If we call the environment management methods “setUp()” and “tearDown” and say that test methods must return void, not take any parameter, not throw any exception and must be named following the “testSomething” pattern, then we obtain a set of constraints that the testing framework can use to identify and execute a test case. A test case must also contain a set of methods to assert logical conditions and force test failures.
As we start writing more and more tests, in different packages, we notice that fragments of the required data configuration may be the same for more the one test case. For example, the test for the “readUser” method and the one of “addUserToGroup”, both require a User object to be instantiated. It would be helpful if the process of adding and then removing a User to / from the data configuration could be taken out of the test case itself and placed into a separate class, so that it can be reused in another test case where User objects must be created and then removed. Such a reusable class that manages a part of the test execution environment is called “fixture”. A fixture class would have two simple methods: setUp and tearDown. Fixture classes could then be registered with one test case or the other and their setUp and tearDown methods called by the testing framework whenever the environment needs to be created and then cleaned up. Fixtures are an essential feature of any testing framework.
If you are involved with enterprise computing in general and j2ee in particular - then you surely noticed that writing tests for a component can be quite a verbose task: depending on how complex your business logic is, tens, even hundreds of tests can be written in order to fully test a session EJB. In some situations, a set of test cases may share a common invariant fragment of the data configuration, that must only be created once before the first test in the set is executed and cleaned up after the last test in the set is executed. This usually refers to creating data base tables, registering JMS listeners, file system directory structures or other resources shared by all tests in the set. For example all tests for an EJB dealing with user and user groups management would require that the User and Group tables be created in the database in order for the tests to be able to execute properly. This is a useful functionality that looks a lot like a fixture, except that instead of executing its setUp and tearDown fixtures before and after each test, setUp will be called before any test is executed and tearDown will be called after all tests have been executed. I call this concept “global fixture”, because it helps managing the global test environment, that is shared by all tests.
The concept behind any testing framework is quite simple: we invoke the code to be tested with a set of parameters which we know should produce a certain result. Then we analyze the values returned by the invoked code and decide if it conforms to our expectations. If so, the test succeeds, otherwise, the test fails. But sometimes code doesn't return any value, but alters the the environment in which the test runs. In order to decide if the code performed as expected, the testing framework will have to check if the data configuration was altered as expected. This is a simple task if the testing environment is a local resource such as a file on the local file system, but if we are in an in container testing situation and we must acecss a part of the environment that is only accessible from within that container, then it would be handy to have a simple way to analyze that remote environment. This can be achieved by placing special agents in those environments EJB container, SERVLET container, JMS listeners and send over a special object to perform the analysis and then return results to the testing framework. Such objects are supported by TESTARE and will from now on be called “test probes”.
In order to provide useful, reliable information, a test it must be executed in a coherent and trusted environment. For example, let's take the case of a method that searches and returns all User objects in the persistence environment. In order to test if getAllUsers functions properly, one could use a fixture create a certain number of User objects and persist it. Then the test would call the getAllUsers method and:
make sure the method returns exactly the same number of objects as were created by the fixtures
the identity of the returned objects matched the identity of the objects created and persisted by the fixture.
The fixture's tearDown method would be called by the testing framework to remove the User objects from the persistence layer and leave the environment in a clean state.
But there is a potential problem: if before the test is executed the persistence layer contains one or more User objects that are unaccounted for, the test would fail, even though the tested method may function properly, and this because the method will pick up not only those objects created by the fixture, but also those User objects that are unaccounted for. As a consequence, the method will return a number of objects that doesn't match the number of objects created by the fixture and the test will fail. But this gives us false information because the method functions as it should, and there is nothing the testing framework can do about it In this case it would be really useful to have a mechanism to make sure the testing environment is in the expected state before the test is actually executed, and warn that test results may not be valid if the environment is not in the expected state. This functionality can be implemented using a special kind of test case decorator, which from now on will called a “test guard”. Test guards have two functions:
when invoked by the testing framework before the test is executed, they make sure the environment is in a sane state this way we're sure we'll obtain accurate test results. This functionality is implemented by the “checkPrecondition” method.
when invoked by the testing framework after the test is completed, they make sure the environment is left in a coherent state. This may be necessary in order to make sure our test doesn't pollute the environment with objects that may affect the accuracy of other test results. This functionality is implemented by the “checkPostcondition” method.
Guards may look like a paranoid test writer's tool. The truth is that if properly used, they offer an additional level of constraints that increases your test results' reliability level and as such boost the level of confidence you can have in your tests. Most important, if your tests fail and you don't understand why, guards can really help the testing framework in giving you enough information to identify the problem.