Home

Cardiogram ❤️

Cardiogram is a Common Lisp test framework that aims at being comfortable and automatic. Cardiogram is able to build tests suites hopefully eliminating such mental burdens as "How am I going to order all of this that I need to test?" or "I made a new feature that needs to be tested in the middle of all of this!" by simply telling lisp to "test this before that, now this after that, but don't test that if this other thing fails....".

To use Cardiogram, load it with Quicklisp. I recommend you use Ultralisp, to take advantage of its speed and package-inferred-system capabilities, which allow you to use cardiogram's toolkit, cardiogram's fixtures, or cardiogram's annotations as separate packages.

(ql:quickload :cardiogram)

WARNING: Cardiogram is a work in progress, and it could change without notice.

A quick introduction to Cardiogram

Let's create a file for utilities similar to cardiogram's toolkit. I'm going to call mine utils.lisp. We will write this file in a style that would allow us to load it into a REPL, this could also be a roswell script. We'll begin by quickloading cardiogram with a call to (ql:quickload :cardiogram). Next, let's define two packages of symbols, one for tests, and one for our actual code. Finally we'll turn on the @in annotation that cardiogram provides through cl-annot and change packages to the one we just defined.

;; First we quickload cardiogram
(ql:quickload :cardiogram)

;; A package to save our tests.
(defpackage :tests
  (:nicknames tst))

;; Now one to put our code in:
(defpackage :utils
  (:use :cl :cardiogram))

;; Turn on python-like annotations from cl-annot
(annot:enable-annot-syntax)

;; Finally switch to our utils package
(in-package :utils)

Writing Utilites.

We'll be writing three utilities. One, l!, is a list-making utility that does several things automatically: it collects elements, concatenates lists, and maps functions to lists. The second utility we'll be writing s! provides similar functionality as l!, but this one for strings. Finally sy! does the same as s!, but it constructs symbols. We'd like to have a convenient and ergonomical way of testing that the code we write conforms to these specifications.

Let's start with writing l!. Our specification says that l! should collect elements and concatenate lists. This means that if we did (l! 1 2 3) we should get '(1 2 3), and if we did (l! '(1 2) '(3)) we should get '(1 2 3) too. Let's see how we can write a first iteration of our function l! together with a cardiogram test that verifies it's behaviour:

The @in annotation tells Lisp to define the test in the package :tst, notice we use the nickname we give to the package, instead of the full name, this is very comfortable to use in large projects with large package names. The symbol l! for the function belongs to the package utils while we have given the same name for it's test in the tst package, enabling us to care about only one name all the while not worrying about switching packages, let alone files.

Now, the is expressions inside the deftest are valuations. Valuations are special functions from cardiogram that only work inside a cardiogram test. Their job is to verify the correctness of a single form when compared to another. Cardiogram offers several valuations for various purposes, here, the is valuation checks if the first form you pass to it is #'equal to the second one. In general, the is valuation expects an argument list like this: (is form-to-be-checked expected-result &optional (test #eql)).

Furthermore, is is a function, it evaluates it's arguments, this is so you can compare behaviours between forms as in: (is (my-is-prime? 9) (his-is-prime? 9)), which would answer: Are those functions equal at 9? This can be further customized by passing a specific test to is, one can imagine to ask: is this function faster than the other one?

To run the test, you simply call it as if it were a function. You do (tst::l!) and it runs the test, tells you if it passes, and the time it took. If a test fails, it throws an error telling you the results of the test so you can see which valuation failed.

Let's say that while writing our first sketch of l! we realized it would be cool to concatenate and append at the same time. We modify our code accordingly:

;; Second sketch of l!
(defun l! (&rest args)
  (loop while args appending
        (let ((it (pop args)))
          (typecase it
            (list it)
            (atom `(,it))))))

@in :tst
(deftest l! ()
  (is (l! 1 2 3) '(1 2 3) #'equal)
  (is (l! '(1 2) '(3)) '(1 2 3) #'equal)
  (is (l! 1 '(2) 3 '(4)) '(1 2 3 4) #'equal))

In this way, we can very comfortably modify our code, and build a detailed test suite for a unit. However, software is rarely composed of a single unit.

We have decided that this behaviour or l! is good enough for now, so we move on to the next utility: s!. What did we say it needed to do? It needed to construct strings. Alright so if we did (s! "string1" "string2") the result would be "string1string2". In Lisp:

;; First sketch of s!
(defun s! (&rest args)
  (with-output-to-string (s)
    (dolist (a args)
      (princ a s))))

@in :tst
(deftest s! (:after tst::l!)
  (is (s! "string1" "string2") "string1string2" #'string=)
  (is (s! "hello " 'world) "hello WORLD" #'string=))

Cardiogram allows us to chain tests by indicating if we want to run a test before, around, or after another. This removes the mental burden of having to order them by hand in a file, and takes advantage of the Lisper's knowledge of method combinations to provide a familiar interface. The option :after tst::l! tells cardiogram to automatically run the test tst::s! after the test tst::s! has run. This way we only need to call (tst::l!) to run both tests in sequence. If we only needed to run the tests for strings, we can call it by itself (tst::s!), without having to rewrite l!'s combination. If we needed to test l! by itself we can skip s! doing: (tst::l! :skip 'tst::s!).

@export
(defun sy! (&rest args)
  (values (intern (string-upcase (apply #'s! args)))))

@in :cxt
(deftest sy! (:after cxt::s!)
  (of-type (sy! :hello)'symbol)
  (is (symbol-name (sy! "Hi"))
      (symbol-name 'hi) #'string=)
  (is (symbol-name (sy! "H~a" "ello"))
      (symbol-name 'hello) #'string=)) 

After we write the third function in our specifications sy!, we can run the three tests in sequence by calling l!. If we needed to test l! alone, we do (l! :skip 's!), and since sy! is programmed after s! it gets automatically skipped. If we needed to test l! and s! but not sy! we only need to (l! :skip 'sy!).

Conclusion

Cardiogram's style of testing is very versatile. It allows for quick and easy prototyping of both code and unit or integration tests. The techniques I presented here are easily translatable to other style of programming like scripting or other larger in scale systems that use asdf. Particularly well suited for simple package-inferred systems.

Test combinations and skips turn simple tests definitions into flexible test combinations that the programmer can take advantage of to pinpoint which parts of the code fail. Valuations like is can make tests very granular.

Finally, having tests right next to the code is useful not only as an organizational facility but as an annotation and collaboration tool, since a developer can see clearly what is expected from the code right in the same file. This extends to other usages of Lisp as a prototyping or specification language.