JUnit Rules!

Rules are a simple, yet amazingly powerful, mechanism introduced in JUnit version 4.7. They allow developers to easily customize JUnit's behavior by exposing meta information regarding the currently executing test. This post provides a straightforward example for writing a custom rule that augments JUnit with some useful functionality.

My subject class is IntSet: A set of integers implementing the standard operations of add(), remove(), contains(), clear() in O(1) time. To make this performance guarantee the set needs to know (in advance) the range of the values (min..max) and its size limit (number of elements that it will accommodate).

All in all, IntSet looks something like this:

public class IntSet {
  ... // Some private fields
  public IntSet(int limit, int min, int max) { ... }
  public int size() { ... }
  public boolean contains(int n) { ... }
  public void add(int n) { ... }
  public void remove(int n) { ... }
}

One of my unit tests specifies the behavior of IntSet when its size limit is reached. If I'm only interested in the type of the exception I can specify it via the expected attribute of the @Test annotation:

  @Test(expected=IllegalStateException.class)
  public void shouldNotExceedCapacity() {
    IntSet s = new IntSet(2, -10, 100);  // Set size limit to 2
    s.add(30);
    s.add(40);
    s.add(50);   // Insertion of the 3rd element should fail
  }

There are two drawbacks with this test. First, It only asserts the type of the an exception. It does not check the error message specified for the exception. Second, it does not assert that the exception was triggered by the last add() call. In other words, if we have a bug and the 2nd add() call is failing - with the same type of exception - the test will still pass.

To overcome this limitation we want to check the error message of the thrown exception. Specifically, we want to verify that the execution of the method fires an exception whose error message is "Cannot insert '50' - The set is full". Clearly, the chances of such an exception being thrown by the 2nd call are pretty slim.

Extending JUnit in such a manner is pretty easy thanks to the rules mechanism:

public class IntSet_Tests {

  @Retention(RetentionPolicy.RUNTIME)
  @Target(ElementType.METHOD)
  @interface Throwing {
    public String value();
  }
  
  @Rule
  public MethodRule mr = new MethodRule()
  {
    @Override
    public Statement apply(final Statement base, FrameworkMethod m, Object o) {
      Throwing t = m.getAnnotation(Throwing.class);
      if(t == null)
        return base;
      
      final String message = t.value();
      return new Statement() {
        
        @Override
        public void evaluate() throws Throwable {
          try {
            base.evaluate();
            fail("No exception was thrown");
          }
          catch(AssertionError e) {
            throw e;
          }
          catch(Exception e) {
            assertEquals("Incorrect exception message", message, e.getMessage());
          }
        }
      };
    }    
  };

  // All sort of @Test methods ...


  // And now, a method that asserts the error message
  @Throwing("Cannot insert '50' - The set is full")
  @Test
  public void shouldNotExceedCapacity() {
    IntSet s = new IntSet(2, -10, 100);
    s.add(30);
    s.add(40);
    s.add(50);
  }
}

First we define a new annotation, @Throwing. Then we define a field annotated with @Rule to provide the custom handling of this annotation. Finally, we annotate the shouldNotExceedCapacity() method with a @Throwing("Cannot insert '50' - The set is full") annotation.

The mechanism works as follows: before each test method is run, JUnit creates a Statement object which is merely a command object through which the acutal method can be invoked. JUnit passes this object along with a FrameworkMethod object (a wrapper of Java's Method) and the unit test instance to all @Rule fields defined at the test class.

A @Rule field must be public and must implement the MethodRule interface (of course, you can instead extend one of several classes conveniently defined by JUnit). In the apply() method, above, we create a new Statement object that wraps the original one. The new evaluate() method will check that if an exception is thrown its message matches the text specified by the @Throwing annotation attached to the method.

Obviously, there are other ways to do that. For instance, one can use the ExpectedException class (a predefined JUnit rule) to achieve a similar effect. The purpose of this post is to surface the (mighty) powers of JUnit meta programming.