AbstractThis tutorial is intended for people who are relatively new to Common Lisp. It describes an easy way to install third-party libraries into a Lisp implementation.
Hitherto these libraries had to be installed manually, an often
complex process. However, many library authors are now packaging their
systems using the new ASDF-INSTALL standard, allowing for automatic
installation on any Lisp system that supports it.
makeprogram. ASDF works with the majority of CL implementations in use today.
A similar system which precedes ASDF is MK:DEFSYSTEM. You don't need it for ASDF-INSTALL but it won't hurt to have it available for libraries which aren't aware of ASDF. However, this document makes no effort to explain how MK:DEFSYSTEM is used. See Henrik Motakef's article "Fight The System." (Unfortunately, the link seems to be dead.)
Update: Marco Antoniotti has patched
ASDF-INSTALL to make it work with MK:DEFSYSTEM as well. See the section about MK:DEFSYSTEM below.
ASDF-INSTALL was originally written for the SBCL Common Lisp implementation. It has been recently ported to CMUCL, Allegro Common Lisp, Xanalys LispWorks, and CLISP by Edi Weitz. Marco Baringer added support for OpenMCL, James Anderson added support for Macintosh Common Lisp (MCL).
It'd be nice if users of other Lisps (like Corman Lisp, ECL, or Scieneer Common Lisp) could provide patches to make ASDF-INSTALL available on more platforms.
The original ASDF-INSTALL is distributed with SBCL. The latest incarnation of the "portable" version is available from CCLAN's CVS repository.
Note that the "portable" version can be considered a fork of the original (SBCL-only) version. Likewise, versions of ASDF-INSTALL distributed with other Lisp implementations like OpenMCL are most likely forks of the "portable" version. This document mostly describes how to use the "portable" version. If you're on a system like SBCL or OpenMCL that comes with its own version of ASDF-INSTALL you are advised to ignore the rest of this tutorial and instead consult the documentation that was supplied by your vendor. (Parts of this tutorial might apply to these implementations as well but they're not guaranteed to be kept up to date.)
This tutorial and the portable version of ASDF-INSTALL were originally
started by Edi Weitz but they're now maintained by Gary King.
Apart from one of the supported Lisps you
will need GnuPG (which is probably pre-installed on
most Linux distributions). Install it first if you don't have it already. You may also need to install the GNU version of
tar if you're not on Linux.
(GnuPG is not strictly necessary - see below - but it is recommended if you want to be reasonable sure that you're not installing arbitrary malicious code.)
Update: Beginning with version 0.14.1 ASDF-INSTALL is already included with the OpenMCL distribution. Also, AllegroCL 7.0 and higher include ASDF (but not ASDF-INSTALL.) See below for details.
Note: For MCL you must start your Lisp from a terminal.
Windows note: If you want to use ASDF-INSTALL on Windows you must install Cygwin first. You can also install GnuPG from the Cygwin setup program. If you want to use CLISP you currently have to use the Cygwin version (which can also be installed from the setup application). The good news is that if you use Cygwin you can pretty much pretend you're on Unix and skip all the Windows notes below.
(Update: Alex Mizrahi posted some notes about using the native Win32 version of CLISP to comp.lang.lisp. I asked him to send patches but he hasn't sent them yet.)
Whenever I use
~/ (the Unix shell notation for the user's
home directory) in the following text what is actually meant is the
on Unix/Linux all implementations seem to agree what this value should
be, on Windows this is not the case. Read the docs of your Lisp.
asdf.lispin a place where you want it to stay. Change into this directory and, from your Lisp, issue the command
(load (compile-file "asdf.lisp"))You should now have a new file the name of which depends on your implementation - probably something like
Note: LispWorks 4.2 (and probably earlier versions) has a bug
that prevents it from loading the compiled ASDF correctly. It is
recommended that you upgrade to 4.3 but if for some
reason you must use an older version you can skip the compilation step
above and later just load the
.lisp file instead in which
case you'll use interpreted code.
.clinit.cl(for Allegro Common Lisp),
.lispworks(for Xanalys LispWorks),
.clisprc(for CLISP), or
openmcl-init.lisp(for OpenMCL). Consult your Lisp's documentation for details.
Open this file (create it if it doesn't exist) and add this line
#-:asdf (load "/path/where/asdf/is/located/asdf")where of course you have replaced
/path/where/asdf/is/located/with the correct path to ASDF - see last section. We wrote
(load ".../asdf")and not, say,
(load ".../asdf.x86f")because this way your Lisp will load the compiled file if it is available and otherwise
asdf.lispif for some reason you didn't compile the code.
#-:asdf? After ASDF has been loaded it adds the
:ASDF to the features
list. Our use of the read-time conditional Sharpsign
Minus thus makes sure that ASDF isn't loaded a second time if it's
already there. (So you can safely save and use an image with ASDF
pre-loaded without changing your init file.)
If you're using SBCL or OpenMCL or AllegroCL 7.0 or higher don't add the line from above but use
ASDF maintains a list of places where it will look for system
definitions when it is asked to load or compile a system. (System
definitions are the files ending with
.asd.) This list is
stored in the special
ASDF:*CENTRAL-REGISTRY* and you can add new
directories to it. Open your initialization file once again and add
the following line after the line which loads ASDF:
(pushnew "/path/to/your/registry/" asdf:*central-registry* :test #'equal)You can use a directory of your choice but you should make sure it exists. You can also add several of these lines with different directories so ASDF will look into each directory in turn until it has found a system definition. Use the directory
~/.asdf-install-dir/systems/if you can't make a decision and make sure to create it. (Replace
~/with an absolute path to your home directory because not all Lisps support the tilde notation.) We will call the directory you've chosen your registry from now on.
Note: It is important that you add a directory here, not a file, so make sure the namestring ends with a slash!
Note: If you use ASDF alone the preferred way to deal with
system definitions is to create symbolic links from the
.asd files to your registry. However, you don't have to
deal with this as ASDF-INSTALL will do that for you.
Note: The free "Personal Edition" of LispWorks doesn't read
~/.lispworks on startup. You can circumvent this by
putting something like
alias lispworks="/usr/local/lib/LispWorksPersonal/lispworks-personal-4300 -init ~/.lispworks"into your
note: On Windows we can't
use a central registry because Windows doesn't have symbolic links. We
will use another mechanism (see below) to find system definitions, so
you don't have to put the
PUSHNEW line into your
.asdfile into a new directory
asdf-installwhich can be located wherever you like. Now create a symlink to your
.asdfile from your registry folder:
cd /path/to/your/registry/ ln -s /path/where/you/put/asdf-install/asdf-install.asd .
For OpenMCL you don't have to download ASDF-INSTALL because it's
already there - it's in
/path/to/ccl/ is the directory where you installed
OpenMCL. You have to provide the symlink, though.
Now start your Lisp and issue the following command:
(asdf:operate 'asdf:compile-op :asdf-install) (asdf:operate 'asdf:load-op :asdf-install)This will ask ASDF to locate the ASDF-INSTALL library, compile it, and finally load it.
Windows note: You can
leave out the
ln command. Now, before you
compile and load ASDF-INSTALL you have to put this line into your
(pushnew "/path/where/you/unpacked/asdf-install/" asdf:*central-registry* :test #'equal)and then either restart your Lisp or evaluate this expression in your current session. Afterwards, proceed with the two
#-:asdf-install (asdf:operate 'asdf:load-op :asdf-install)This will instruct ASDF to load the (compiled) ASDF-INSTALL library whenever your Lisp starts up (unless ASDF-INSTALL is already available in your image).
If you're using SBCL don't add the line from above but use
(require :asdf-install)instead. (Note: Try this from the REPL and check the messages to see whether SBCL really loads its own bundled version of ASDF-INSTALL. The "portable" version this document talks about is supposed to work with SBCL as well but in case of incompatibilities you're advised to rely on SBCL's version.)
You're now ready to use ASDF-INSTALL.
Windows note: For Windows add the following line to end of the initialization file:
(pushnew 'asdf-install:sysdef-source-dir-search asdf:*system-definition-search-functions*)As we can't use the central registry, we're using a customized search function instead. It'll scan all directories below each of the entries in
*LOCATIONS*until it finds a suitable system definition. Note that this is a sub-optimal solution because this will not necessarily find the newest one if you've installed several versions of the same library. Make sure to uninstall older versions.
The recommended setup is to use both ASDF and MK:DEFSYSTEM because this will significantly increase the number of libraries you can install with ASDF-INSTALL.
To set up your Lisp environment for this you have to do the following (after reading the sections above):
defsystem.lispfrom within the
(load (compile-file "/path/to/defsystem.lisp"))
#-:mk-defsystem (load "/path/to/defsystem") (mk:add-registry-location "/path/to/your/registry/")into your initialization file.
#-:asdf-install (asdf:operate 'asdf:load-op :asdf-install)from above with the line
#-:asdf-install (load "/path/to/asdf-install/load-asdf-install")This last step will ensure that ASDF-INSTALL will always be loaded on startup even if you only use MK:DEFSYSTEM and don't have ASDF available.
Note: Of course, the fact that a library can be installed with ASDF-INSTALL and that ASDF-INSTALL was ported to your Lisp implementation doesn't necessary mean that the library itself will work with your Lisp! Check the library's docs before you try to install it.
You can click on the name of each library to get a description. Use the library's name from the list to install it. If, say, you want to install CL-PPCRE make sure you're connected to the Internet and use this command:
(asdf-install:install :cl-ppcre)Then proceed with Where to store the library below.
Note: If you install a library by name, ASDF-INSTALL will connect to the CLiki website and from there it'll be redirected to the actual download location provided by the library's author.
Note: The argument to the
function is a string
designator, i.e. instead of
:CL-PPCRE you can also
"cl-ppcre". CLiki is case-insensitive and
therefore case doesn't matter if you install a library by name.
In order to be ASDF-installable a library has to contain a system definition for ASDF. It also has to be
packaged in a certain way: It is assumed to come as a gzipped tar
archive (usually ending in
which unpacks into one directory possibly containing
sub-directories. The system definition has to have a name
corresponding to the name of the library (so if your library is called
"foobar" the system definition is supposed to be
foobar.asd) and has to reside in the top-level
If this is the case you can download and install the library directly by providing the download URL of the package like so:
(asdf-install:install "http://weitz.de/files/cl-ppcre.tar.gz")Now proceed with Where to store the library below.
Note: Currently, ASDF-INSTALL only understands http. Other protocols like ftp or https aren't supported.
Note: It's obviously rather easy to make an existing library ASDF-installable if it isn't already. If you come across a library which you'd like to use but which isn't listed on http://www.cliki.net/asdf-install, it might be worthwhile to kindly ask the library's author to change this.
(asdf-install:install "/path/to/library/library.tar.gz")and afterwards carry on with the next section.
Note: For obvious reasons this namestring must not start with
"http://" although your operating system might
otherwise allow this.
Install where? 1) System-wide install: System in /usr/local/asdf-install/site-systems/ Files in /usr/local/asdf-install/site/ 2) Personal installation: System in /home/edi/.asdf-install-dir/systems/ Files in /home/edi/.asdf-install-dir/site/ -->Choose one of these options and enter the corresponding number, then press the
Returnkey. (Note that on Unix-like systems you usually don't have write access in
http://www.example.com/frob.tar.gzthen ASDF-INSTALL will try to download the signature from
ASDF-INSTALL will check
If one of the checks fails, you'll most likely be confronted with one of these situations:
Downloading 157777 bytes from http://weitz.de/files//cl-ppcre.tgz ... Error: Server responded 404 for GET http://weitz.de/files//cl-ppcre.tgz.asc [condition type: DOWNLOAD-ERROR] Restart actions (select using :continue): 0: Don't ckeck GPG signature for this package 1: Return to Top Level (an "abort" restart). 2: Abort entirely from this process.There was no signature corresponding to this package.
Downloading 6365 bytes from http://files.b9.com//cl-base64-latest.tar.gz ...gpg: WARNING: using insecure memory! gpg: please see http://www.gnupg.org/faq.html for more information gpg: Signature made Thu 12 Jun 2003 04:06:04 PM CEST using DSA key ID C4A3823E gpg: Can't check signature: public key not found Error: No key found for key id 0x112ECDF2C4A3823E. Try some command like gpg --recv-keys 0x112ECDF2C4A3823E [condition type: KEY-NOT-FOUND] Restart actions (select using :continue): 0: Don't ckeck GPG signature for this package 1: Return to Top Level (an "abort" restart). 2: Abort entirely from this process.The library was signed but the signer's public key wasn't found in your public keyring.
Downloading 6365 bytes from http://files.b9.com//cl-base64-latest.tar.gz ...gpg: WARNING: using insecure memory! gpg: please see http://www.gnupg.org/faq.html for more information gpg: Signature made Thu 12 Jun 2003 04:06:04 PM CEST using DSA key ID C4A3823E gpg: Good signature from "Kevin M. Rosenberg <firstname.lastname@example.org>" gpg: aka "Kevin Rosenberg <email@example.com>" gpg: aka "Kevin M. Rosenberg <firstname.lastname@example.org>" gpg: aka "Kevin Marcus Rosenberg, M.D. <email@example.com>" gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: D7A0 55B6 4768 3582 B10D 3F0C 112E CDF2 C4A3 823E Error: GPG warns that the key id 0x112ECDF2C4A3823E (Kevin M. Rosenberg <firstname.lastname@example.org>) is not fully trusted [condition type: KEY-NOT-TRUSTED] Restart actions (select using :continue): 0: Don't ckeck GPG signature for this package 1: Return to Top Level (an "abort" restart). 2: Abort entirely from this process.The signer's key is in your public keyring but you have no GPG trust relationship with him.
Downloading 157777 bytes from http://weitz.de/files//cl-ppcre.tgz ...gpg: WARNING: using insecure memory! gpg: please see http://www.gnupg.org/faq.html for more information gpg: Signature made Fri 24 Oct 2003 11:22:11 AM CEST using DSA key ID 057958C6 gpg: Good signature from "Dr. Edmund Weitz <email@example.com>" Error: Dr. Edmund Weitz <firstname.lastname@example.org> (key id 595FF045057958C6) is not on your package supplier list [condition type: AUTHOR-NOT-TRUSTED] Restart actions (select using :continue): 0: Add to package supplier list 1: Don't ckeck GPG signature for this package 2: Return to Top Level (an "abort" restart). 3: Abort entirely from this process.The signer's key is in your public keyring, you have a GPG trust relationship with him but the signer wasn't found in your list of valid suppliers of Lisp code.
As you'll have noticed, in all these cases ASDF-INSTALL offers the restart not to check the GPG signature in this particular case. How you can select this restart depends on your Lisp implementation but if you select it ASDF-INSTALL will proceed compiling and installing the package without further checks for this library.
In the last case (condition type
are also offered another restart. If you select this one the signer of
the library will be added to your package
supplier list and you won't be asked again if you install another
library signed by the same person.
Note: You might be asking yourself if all this security stuff is really necessary. Well, CLiki, the website where ASDF-INSTALL looks for the package URL if you install by name, can be edited by anyone so it would be fairly easy for a malicious hacker to redirect you to a library which once it's installed insults your boss by email or withdraws US$ 100,000 from your bank account. You better make sure this doesn't happen... See the section about customization on how to (partly) disable security checks.
Note: If you're unsure about notions like public
keyring or GPG trust relationship, please read
the GnuPG documentation. It is beyond the scope of this text to
explain these terms.
ASDF-INSTALL:INSTALLyou can immediately use the library you've just installed while you're still in the same Lisp session. If you quit your Lisp image and start it anew you have to reload the library. (Of course you don't have to install it again!) This is done like so:
(asdf:operate 'asdf:load-op :library-name)Here
:LIBRARY-NAMEis either the name you've used if you installed by name or it is the name of the main
.asdfile if you've installed by URL or from a local file. If you're not sure about the name you have to use, you can list the contents of your registry for all libraries which are available to you. So, if your registry looks like this
edi@bird:~ > ls ~/.asdf-install-dir/systems/ cl-ppcre.asd cl-ppcre-test.asd cl-who.asd html-template.asdyou can substitute
:LIBRARY-NAMEwith one of
:HTML-TEMPLATE. (CL-PPCRE-TEST was most likely automatically installed when you installed CL-PPCRE.)
If you use SBCL you can, instead of calling
You can for example from CMUCL issue the command
(asdf-install:install :osicat)and watch how ASDF-INSTALL not only downloads and installs Osicat but also UFFI.
LOADs the file
~/.asdf-installif it's there. This file (which is obviously supposed to contain Lisp code) can be used to change the values of some special variables which control ASDF-INSTALL's behaviour. Their names are exported from the
tarprogram as a string - the default is
"tar". Changing this variable has no effect if Cygwin is used.
NILby default but will be set to the value of the environment variable
$http_proxy(if it's set) prior to loading
~/.asdf-install. Set this to a non-
NILvalue if you need to go through an http proxy.
~/.asdf-installis loaded. A couple of ASDF-installable libraries are available via CCLAN and with the help of this variable you can choose another CCLAN mirror from the list at http://ww.telent.net/cclan-choose-mirror.
Tinitially which means that there'll be a security check for each library which is not installed from a local file. You can set it to
NILwhich means no checks at all or to
:UNKNOWN-LOCATIONSwhich means that only URLs which are not in
*SAFE-URL-PREFIXES*are checked. Every other value behaves like
Note: This customization option is currently not supported in the SBCL version of ASDF-INSTALL.
NILinitially. It is supposed to be a list of strings which are "safe" URL prefixes, i.e. if a download URL begins with one of these strings there's no security check. The value of
*SAFE-URL-PREFIXES*only matters if
*VERIFY-GPG-SIGNATURES*is set to
Note: This customization option is currently not supported in the SBCL version of ASDF-INSTALL.
((#p"/usr/local/asdf-install/site/" #p"/usr/local/asdf-install/site-systems/" "System-wide install") (#p"/home/edi/.asdf-install-dir/site/" #p"/home/edi/.asdf-install-dir/systems/" "Personal installation"))where
/home/edi/will obviously be replaced with your home directory. You'll notice that this corresponds to the little menu you see when ASDF-INSTALL starts to install a package. You can add elements to this list or replace it completely to get another menu. Each element is a list with three elements - a pathname denoting the directory where the (unpacked) libraries will be stored, a pathname denoting a directory where system definition symlinks will be placed, and a string describing this particular choice.
If you make changes to this value it is important that you also update
accordingly in your initialization
file or ASDF-INSTALL won't find your system definitions (unless
you are on Windows). See the example below.
Note: On SBCL the initial value of this variable is different - try it out yourself.
NIL. If it is not
NILit should be a positive integer not greater than the length of
*LOCATIONS*. By setting this value you circumvent the question about where to install a library and ASDF-INSTALL will unconditionally use the corresponding entry from
*LOCATIONS*. Note that
0) means the first entry.
Note: This customization option is currently not supported in the SBCL version of ASDF-INSTALL.
*LOCATIONS*, i.e. if it, say, contains the value
/usr/local/foo/, then the first element of
(#p"/usr/local/foo/site/" #p"/usr/local/foo/site-systems/" "System-wide install")If this variable is not set, the directory
/usr/local/asdf-install/is used. Note that this variable affects ASDF-INSTALL's behaviour before
Note: On SBCL the value of
SBCL_HOME is used
*LOCATIONS*, i.e. if it, say, contains the value
frob/and your username is
johndoe, then the second element of
(#p"/home/johndoe/frob/site/" #p"/home/johndoe/frob/systems/" "Personal installation")If this variable is not set, the value
.asdf-install-dir(note the dot) is used. Note that this variable affects ASDF-INSTALL's behaviour before
Note: On SBCL the value
.sbcl is used
~/.asdf-installcould look like:
;; use a http proxy (setq asdf-install:*proxy* "http://proxy.foo.com/") ;; use a CCLAN mirror in France (setq asdf-install:*cclan-mirror* "http://thingamy.com/cclan/") ;; only partial security checks (setq asdf-install:*verify-gpg-signatures* :unknown-locations) ;; downloads from Kevin Rosenberg and from my own server don't have to be checked (setq asdf-install:*safe-url-prefixes* '("http://files.b9.com/" "http://weitz.de/files/")) ;; add a repository for unstable libraries (pushnew '(#p"/usr/local/lisp/unstable/site/" #p"/usr/local/lisp/unstable/systems/" "Install as unstable") asdf-install:*locations* :test #'equal) ;; make sure this is also known by ASDF (pushnew "/usr/local/lisp/unstable/systems/" asdf:*central-registry* :test #'equal)
trusted-uids.lispand usually resides in the directory
~/.asdf-install-dir/but this can be customized by changing the environment variable
PRIVATE_ASDF_INSTALL_DIR. You are not supposed to edit this file manually - new entries are added automatically whenever you choose the corresponding restart during the security check.
(asdf-install:uninstall :library-name)ASDF-INSTALL will ask you to confirm this and then it'll remove the library's source directory as well as the symbolic link to the system definition (if it exists).
Windows note: Due to the way systems are found on Windows
ASDF-INSTALL will propose to delete an arbitrary version of your
library if you've installed several of them. Make sure to read
what it is about to remove before you confirm.
|2006-01-08||Tutorial and software now maintained by Gary King, removed weitz.de links|
|2005-12-21||Changed GNU tar default for FreeBSD and NetBSD (Richard Kreuter)|
|2005-12-16||Fixed bug in load dependencies (Gary King)|
|2005-12-16||Fixed file descriptor leak (John Wiseman)|
|2005-12-14||Fixed request URI (Zach Beane)|
|2005-12-05||Check GPG return values (thanks to Gary King)|
|2005-10-18||Changes for compatibility with SBCL's Unicode support (thanks to Christophe Rhodes)|
|2005-09-27||Small change for compatibility with future OpenMCL versions (thanks to Bryan O'Connor)|
|2005-07-14||Updated note about CLISP (thanks to Henri Lenzi)|
|2005-06-01||Added proxy authentication code (thanks to Sean Ross)|
|2005-02-16||More OpenMCL details (thanks to Jim Thompson)|
|2004-12-29||Added COPYRIGHT file to distribution|
|2004-09-13||Added information about AllegroCL 7.0 and OpenMCL 0.14.1|
|2004-09-08||Fixed typo in |
|2004-05-20||Changed hyphens to underlines in names of environment variables (thanks to Robert P. Goldman)|
|2004-05-19||Mentioned Alex Mizrahi's notes, added version number for MK:DEFSYSTEM in docs and SPLIT-SEQUENCE dependency in ASDF system definition (thanks to Robert P. Goldman and Robert Lehr)|
|2004-04-24||Patches by Marco Antoniotti for MK:DEFSYSTEM compatibility|
|2004-03-27||Bugfixes by Kiyoshi Mizumaru|
|2004-01-28||Improved MCL support (James Anderson)|
|2004-01-21||Support for MCL by James Anderson|
|2004-01-16||Minor edits, Cygwin CLISP support, download location for |
|2004-01-15||Preliminary Windows support, described how to uninstall a library, added |
|2004-01-13||Mentioned OpenMCL support (Marco Baringer), added some SBCL exceptions, added clarification about Windows, minor edits, changes by Dan Barlow|
$Header: /cvsroot/cclan/asdf-install/doc/index.html,v 1.22 2006/01/08 22:40:46 nhabedi Exp $
BACK TO MY HOMEPAGE