April 18, 2021

Make a Test Fail First

When I write tests and I exactly know what I need I write a test first. Other times I experiment with the implementation and then write some tests after the code exists. And for some code the effort to write a automated tests is not worth it.

However, there is one advice I need to remind myself from time to time: Write a failing test first. Or, when you write the test afterward, make the test fail by screwing the implementation. Why? Let’s look at small test example:

Small Test in Clojure:
(t/deftest number-is-converted-to-words
  (t/is (= "one") (convert-to-words 1))
  (t/is (= "two") (convert-to-words 2))
  (t/is (= "three") (convert-to-words 3))
  (t/is (= "4") (convert-to-words 4))
  (t/is (= "10")) (convert-to-words 10))

Let’s run the tests: all green. Hooray:

All Tests Passing
Figure 1. All Test Passing

Well, did you spot my mistake? Let’s see the implementation of the tested function:

Broken Implementation:
(defn convert-to-words [num]
"TODO" )
All is fine!
Figure 2. All is fine!

Oh, so how does the test succeed? The reason is that convert-to-words is passed as the wrong parameter the is function. The is takes two arguments: The form which needs to return true for passing the test, and an optional message. We pass in (= "one") as the test, which always will succeed, and pass convert-to-words as the message. Yikes. This happened to me more than once =). The correct test code is:

Small Test in Clojure:
(t/deftest number-is-converted-to-words
  (t/is (= "one" (convert-to-words 1)))
  (t/is (= "two" (convert-to-words 2)))
  (t/is (= "three" (convert-to-words 3)))
  (t/is (= "4" (convert-to-words 4)))
  (t/is (= "10" (convert-to-words 10))))
All Tests Failing
Figure 3. Tests Failing
Fixed implementation:
(defn convert-to-words [num]
  (get {1 "one" 2 "two" 3 "three"}
       num
       (str num)))

Let Tests Fail First

To avoid this mistake, I recommend to make your test fail. So you can see that it fails in the way you expect. If you write the test first, let it run before any implementation. If you write the test for existing code, you might break the existing code temporarily to see if the test catches your broken implementation.

PS: The example is in Clojure. In Java/Scala this wrong test would probably produce a compilation error ;). However, I also wrote many 'can never fail' tests in Scala/Java, I just didn’t have an example in my head anymore.

Tags: Clojure Development