Mon, 17 Jul 06

Some realisations about testing

Having been predominantly involved in Rails development just recently, fixtures have taken on a slightly different meaning to what I now believe to be the original understanding. Having recently re-read Test Driven Development: By Example I re-discovered the ‘original’ meaning of the word (essentially shared objects used by all tests in a test case).

This has made me think about the way that I’ve been structuring my tests (or not structuring them as the case may be). I have never been a fan of the setup and teardown methods provided to us in xUnit and I now believe this to be due to my misunderstanding of the term fixture and therefore their intention. This has led to my test cases growing very large and containing many disparate tests. If there was any separation of the testcase then it was usually based on functionality of a particular method (which separately leads me to believe that they may have been better expressed in a new object). I now understand that test cases should be split to contain tests that rely on a common test fixture. As soon as I started to do this, the setup and (to a lesser degree) teardown methods have become much more useful.

I’ve also started to create my test cases as anonymous classes derived from Test::Unit::TestCase.

Class.new(Test::Unit::TestCase) do
  def test_
    assert true
  end
end

I find that this helps when trying to determine where a new test should be placed. If the class has a name I’m tempted to scan that and make a decision as to whether a new test belongs in this case. By keeping them anonymous, I have to inspect the test fixture in order to understand whether a new test belongs here.

Note I may also add a comment right below Class.new to explicitly specify the fixture for this testcase. This may seem to go against not naming the classes in the first place but I think part of the problem with class names is the lack of punctuation and therefore the difficulty in clearly expressing the test fixture.

I’ve chosen a somewhat contrived example to demonstrate my new found test structure.

Class.new(Test::Unit::TestCase) do
  # An empty stack

  def setup
    @stack = Stack.new
  end

  def test_should_accept_an_item_when_sent_push
    assert_nothing_raised { @stack.push(1) }
  end

  def test_should_complain_when_sent_top
    assert_raise(StackUnderflowError) { @stack.top }
  end

  def test_should_complain_when_sent_pop
    assert_raise(StackUnderflowError) { @stack.pop }
  end

end

The fixture for this test is an empty stack. The individual tests prod and query the empty stack to ensure that it behaves as expected. Although the example above is incredibly simple, I have consistently found that my individual test methods become much smaller.

Notice that the names of the tests all start ‘test_should’. Not only does this help us think about the responsibilities of the object; but it enables us to maintain living system specifications based on the tests. I believe this idea first came from testdox for Java but was introduced to me by Ben at work. In fact Ben has written a cool Agiledox server that displays the system specifications for your rails project.

The realisations set out above have come at around the same time as my taking more of an interest in rSpec (the stack test above has been adapted directly from one of the rSpec examples); a behaviour driven development framework for ruby. I haven’t done any real work with rSpec yet but am hoping it might improve the way I think of both testing and OO design in general.