lisp-unit is a Common Lisp library that supports unit testing. There is a long history of testing packages in Lisp, usually called regression testers. More recent packages in Lisp and other languages have been inspired by JUnit for Java. For more information on both unit testing and JUnit, visit JUnit.org.
About This Package
How to Use lisp-unit
How to Organize Tests with Packages
Functions and macros for managing tests
Macros for assertions
AuthorCopyright (c) 2004-2005 Christopher K. Riesbeck
Copyright (c) 2012 Dieter Kaiser
VersionThis is the documenation of a fork of the package lisp-unit.
Mailing ListNo mailing list.
Download and Source CodeThe original code is available at
You can download a fork of lisp-unit from the repository gitorious.org/lisp-projects/lisp-unit The fork contains this documentation. Furthermore, the facilities to control how results are reported are fully implemented.
Dependencieslisp-unit does not depend on other libraries.
There are two ways to load lisp-unit. The first method is simply to load the file lisp-unit.lisp. The steps are the following:
(define-test name exp1 exp2 ...)This defines a test called name. The expressions exp1, exp2, ... can be anything, but typically most will be assertion forms. Tests can be defined before the code they test, even if they are testing macros. This is to support test-first programming. After defining your tests and the code they test, run the tests with
(run-tests)This runs every test defined in the current package. To run just certain specific tests, use:
(run-tests name1 name2 ...)Loading lisp-unit and executing tests with asdf
The second method is to load lisp-unit with the asdf facility:
(asdf:load-system :lisp-unit)Furthermore, with asdf the execution of the tests can be performed with the call
(asdf:test-system <package-name>)Here <package-name> is the package which is tested. To allow this feature of asdf a method perform has to be added to the system definition. This is an example for the package lisp-unit itself:
(asdf:defsystem :lisp-unit :name 'lisp-unit' :author 'Dieter Kaiser' :license 'LLGPL' :serial t :depends-on () :components ((:file 'lisp-unit')) :in-order-to ((asdf:test-op (asdf:load-op :lisp-unit-test))))The test package for lisp-unit has the name lisp-unit-test and has the following system definition:
(asdf:defsystem :lisp-unit-test :depends-on (:lisp-unit) :version '1.0.0' :components ((:module 'test' :components ((:file 'rtest-lisp-unit')))))In this example the test files are expected to be in a subdirectory test/ of the package itself. Because of these definitions tests in the test file rtest-lisp-unit.lisp for the package lisp-unit can be executed with the command:
> (in-package :cs325-user) #<PACKAGE CS325-USER> > (define-test pick-greater (assert-equal 5 (pick-greater 2 5)) (assert-equal 5 (pick-greater 5 2)) (assert-equal 10 (pick-greater 10 10)) (assert-equal 0 (pick-greater -5 0)) ) PICK-GREATERFollowing good test-first programming practice, we run these tests before writing any code.
> (run-tests pick-greater) PICK-GREATER: Undefined function PICK-GREATER called with arguments (2 5).This shows that we need to do some work. So we define our broken version of pick-greater.
> (defun pick-greater (x y) x) ;; deliberately wrong PICK-GREATERNow we run the tests again:
> (run-tests pick-greater) PICK-GREATER: (PICK-GREATER 2 5) failed: Expected 5 but saw 2 PICK-GREATER: (PICK-GREATER -5 0) failed: Expected 0 but saw -5 PICK-GREATER: 2 assertions passed, 2 failed.This shows two failures. In both cases, the equality test returned nil. In the first case it was because (pick-greater 2 5) returned 2 when 5 was expected, and in the second case, it was because (pick-greater -5 0) returned -5 when 0 was expected.
(assert-equal value form)This tallies a failure if form returns a value not equal to value. Both value and test are evaluated in the local lexical environment. This means that you can use local variables in tests. In particular, you can write loops that run many tests at once:
> (define-test my-sqrt (dotimes (i 5) (assert-equal i (my-sqrt (* i i))))) MY-SQRTHowever, the above output doesn't tell us for which values of i the code failed. Fortunately, you can fix this by adding expressions at the end of the assert-equal. These expression and their values will be printed on failure.
> (define-test my-sqrt (dotimes (i 5) (assert-equal i (my-sqrt (* i i)) i))) ;; added i at the end MY-SQRT > (run-tests my-sqrt) MY-SQRT: (MY-SQRT (* I I)) failed: Expected 1 but saw 1/2 I => 1 MY-SQRT: (MY-SQRT (* I I)) failed: Expected 3 but saw 9/2 I => 3 MY-SQRT: (MY-SQRT (* I I)) failed: Expected 4 but saw 8 I => 4 MY-SQRT: 2 assertions passed, 3 failed.The next most useful assertion form is
(assert-true test)This tallies a failure if test returns false. Again, if you need to print out extra information, just add expressions after test. There are also assertion forms to test what code prints, what errors code returns, or what a macro expands into. A complete list of assertion forms is in the reference section. Do not confuse assertion forms with Common Lisp's assert macro. assert is used in code to guarantee that some condition is true. If it isn't, the code halts.
(defpackage :date (:use :common-lisp) (:export #:date->string #:string->date))Your date-tests.lisp file would look like this:
(defpackage :date-tests (:use :common-lisp :lisp-unit :date))You could then run all your date tests in the test package:
(in-package :date-tests)Alternately, you could run all your date tests from any package with:
This macro defines a test called name with the expressions specified in body, in the package specified by the value of *package* in effect when define-test is executed. ...
This function returns the names of all the tests that have been defined for the package. If no package is given, the value of *package* is used.
This function returns the body of the code stored for the test name under package. If no package is given, the value of *package* is used.
This function removes the tests named for the given package. If no package is given, the value of *package* is used.
This function removes the tests for the given package. If no package is given, it removes all tests for the current package. If nil is given, it removes all tests for all packages.
This macro runs all the tests defined in the specified package and reports the results.
This macro runs the tests named and reports the results. The package used is the value of *package* in effect when the macro is expanded. If no names are given, all tests for that package are run.
By default, errors that occur while running tests are simply counted and ignored. You can change this behavior by calling use-debugger with one of three possible flag values: t (the default) means your Lisp's normal error handling routines will be invoked when errors occur; :ask means you will be asked what to do when an error occurs, and nil means errors are counted and ignored, i.e., the standard behavior.
Assertion with the predicate eq. ...
Assertion with the predicate eql ...
Assertion with the predicate equal ...
Assertion with the predicate equalp ...
Assertion with a user defined predicate. ...
assert-true tallies a failure if the test form returns false. ...
assert-false tallies a failure if the test returns true. ...
This macro tallies a failure if (macroexpand-1 form) does not produce a value equal to expansion. ...
This macro tallies a failure if form does not signal an error that is equal to or a subtype of condition-type condition. ...
Signals a failure. ...
Return T if x and y both are false or both are true.
Compare two sequences to have the same elements. ...
Compare two sequences to be unordered equal. ...
The test listener is called after each assertion form in a test is executed. The listener is passed
The error listener is called when an error occurs in running a test. The listener is passed
The summary listener is called after
The three listeners are stored in the exported global variables *test-listener*, *error-listener*, *summary-listener*. So one way to change listeners is with let. For example, to show only package-level summary counts:
(let ((*summary-listener* 'show-package-summary)) (run-tests))The above would still show failures and error messages. To hide those and just get the counts:
(let ((*error-listener* 'count-error) (*summary-listener* 'show-package-summary) (*test-listener* 'show-no-result)) (run-tests))To show no individual test results and only package summaries with failures, we need to define a function that checks the number of failures.
(defun show-failing-package (name test-count pass-count error-count) (when (or (< pass-count test-count) (> error-count 0)) (show-summary name test-count pass-count error-count)))with-listeners
A simpler way to rebind listeners is with with-listeners. The example above could be done with:
(with-listeners (show-no-result show-failing-package count-error) (run-tests))The macro with-test-listener can be used to rebind only the test listener.
(with-test-listener (show-no-result) (run-tests))