FiveAM

Checks 

At the lowest level testing the system requires that certain forms be evaluated and that certain post conditions are met: the value returned must satisfy a certain predicate, the form must (or must not) signal a certain condition, etc. In FiveAM these low level operations are called 'checks' and are defined using the various checking macros.

Checks are the basic operators for collecting results. Tests and test suites on the other hand allow grouping multiple checks into logic collections.

(eval-when (:compile-toplevel :load-toplevel :execute)
  (def-special-environment run-state ()
    result-list
    current-test))

Types of test results 

Every check produces a result object.

(define-condition check-failure (error)
  ((reason :accessor reason :initarg :reason :initform "no reason given")
   (test-case :accessor test-case :initarg :test-case)
   (test-expr :accessor test-expr :initarg :test-expr))
  (:documentation "Signaled when a check fails.")
  (:report  (lambda (c stream)
              (format stream "The following check failed: ~S~%~A."
                      (test-expr c)
                      (reason c)))))

The check operators 

The IS check 

Other checks 

Random (QuickCheck-ish) testing 

FiveAM provides the ability to automatically generate a collection of random input data for a specific test and run a test multiple times.

Specification testing is done through the FOR-ALL macro. This macro will bind variables to random data and run a test body a certain number of times. Should the test body ever signal a failure we stop running and report what values of the variables caused the code to fail.

Generators 

Since this is random testing we need some way of creating random data to feed to our code. Generators are regular functions whcih create this random data.

We provide a set of built-in generators.

(defgenerator gen-integer (&key (max (1+ most-positive-fixnum))
                                (min (1- most-negative-fixnum)))
  (+ min (random (1+ (- max min)))))
(defgenerator gen-long-float (&key (max (1+ most-positive-long-float))
                                   (min (1- most-negative-long-float)))
  (+ min (random (1+ (- max min)))))
(defgenerator gen-character (&key (code (gen-integer :min 0 :max (1- char-code-limit)))
                                  (alphanumericp nil))
  (if alphanumericp
      (code-char (funcall code))
      (loop
         for char = (code-char (funcall code))
         until (alphanumericp char)
         finally (return char))))

The trivial always-produce-the-same-thing generator is done using cl:constantly.

Fixtures 

When running tests we often need to setup some kind of context (create dummy db connections, simulate an http request, etc.). Fixtures provide a way to conviently hide this context into a macro and allow the test to focus on testing.

NB: A FiveAM fixture is nothing more than a macro. Since the term 'fixture' is so common in testing frameworks we've provided a wrapper around defmacro for this purpose.

(deflookup-table fixture
  :documentation "Lookup table mapping fixture names to fixture
  objects.")