One problem I always had when hacking CL were libraries. And it was not the lack of them – contrary to popular belief, there is quite a lot of Lisp libraries. The real problem is how to comfortably find and manage them. For finding, CLiki and Common Lisp Directory work great. Managing, however, has to be done by hand. There was an attempt to solve this problem, ASDF-Install, but it relies on a very inconvenient (at least for me) assumption: that libraries are versioned .tar.gz files. In other languages it may be true, but Lisp is different (as usual). Many libraries are in a `permanently rolling mudball' state, where the current version is accessible only by various version control systems, and tarballs appear rarely and quickly become outdated.
There are other library collections and collectors:
What is more, all above are targeted towards fresh users and their goal is to provide an easy Lisp installation. There is no `library manager' which would be usable and convenient for the power user, and which would just sit on top of the ASDF managing libraries.
That is what CL-Librarian aims to be: a manager for Common Lisp libraries, focused on libraries updated dynamically by version control systems and targeted at power users. It handles downloading such libraries, and keeping them up-to-date. This may seem like a trivial task, but after maintaining an UnCommon Web installation for over a year I am quite sure it's not. Library management could use some automation.
CL-Librarian's primary platform is SBCL on Linux. It should work with other Lisp implementations (for the only non-portable part: executing external commands, it uses the EXTERNAL-PROGRAM library, which is portable). It should work on other Unix systems, including OS X. Porting it to Windows should not be very hard, but I am not a Windows expert, so most probably it won't work out of the box.
As of now, this code is under heavy development. Both implementation and interface may change. I try to document fixed interfaces, but I cannot promise much at the moment. When you upgrade Librarian, please read the commit description. You have been warned.
CL-Librarian lives in a darcs repository. To dowload the source, type:
$ darcs get http://www.pasternacki.net/repos/cl-librarian/
It depends on http://www.cliki.net/ASDF, http://www.cliki.net/SPLIT-SEQUENCE and
http://www.cliki.net/EXTERNAL-PROGRAM. Those can be installed manually, or – on
UNIX-like systems that use symlinks – the included
script can be used to download dependencies (except ASDF, which is
included in all CL implementations that I use) and
into Librarian's main directory:
cl-librarian$ sh ./bootstrap.sh
Librarian can be also downloaded with http://www.cliki.net/ASDF-INSTALL.
CL-Librarian makes extensive use of system binaries. Required binaries are:
wget(might be replaced by any other command that downloads file from a given URL)
As far as I know, all of them are ported to most of commonly used systems.
CL-Librarian is written and maintained by Maciek Pasternacki <email@example.com>. Feel free to contact me by e-mail or jabber (JID is the same as e-mail). After sending me an e-mail from an address previously unknown to me, a confirmation message will be sent back; only after the sender replies to this message I will see the original mail.
There is no mailing list at the moment.
CL-Librarian is just an ASDF library. Once it gets its
prerequisites, either with
bootstrap.sh, ASDF-Install, or by
loading the necessary libraries by hand, it doesn't need any further
set up other than being loaded:
(asdf:operate 'asdf:load-op :cl-librarian)
CL-Librarian is designed around the concept of a library shelf. A shelf is a single unit consisting of several ASDF-defined libraries managed by the Librarian. The Librarian doesn't bother moving around single libraries; only whole library shelves, which are self-contained sets of libraries, are downloaded or updated at a time.
Each shelf consists of library repositories. Libraries are
distinguished with names (symbols); it is assumed that name
identifies a library uniquely, and that it is the same as library's
.asd file. However, single library can have multiple sources
(e.g. stable tarball from release site or ASDF-Install, and
development darcs repository). Shelf can contain only one source
of every library.
Shelves and library repositories can be defined either in Lisp
.shelf extension placed in directory specified by
`*SHELVES-DIR*' variable (by default,
shelves subdirectory of
Librarian installation directory), or inline in normal Lisp code.
First method is loosely based on ASDF's
A shelf specification is a single Lisp file, with the .shelf
extension, containing a
DEFSHELF form. They resemble ASDF's .asd
DEFSYSTEM forms, which is intentional. Shelves
managed by Librarian are located in the
CL-Librarian is capable of downloading shelves from the network or
copying them from other files. The shelf contents are a directory,
shelves subdirectory of Librarian, whose name is the
.shelf file. It contains one subdirectory per each
library it includes.
A shelf can include other shelves. By these I mean literal inclusions rather than inheritance – each shelf has its own copy of every library it directly or indirectly includes. This is to keep the shelf's internal dependencies consistent and to make it possible to upgrade one shelf without touching others.
A sample Librarian's description of a shelf, describing libraries used by the Librarian itself, looks like this:
(defshelf librarian-base () ((external-program tarball-repo :source "asdf-install:external-program") (split-sequence tarball-repo :source "cclan:split-sequence.tar.gz"))
DEFSHELF form above defines a shelf, named
including no other shelves, and consisting of the EXTERNAL-PROGRAM
and SPLIT-SEQUENCE packages, downloaded as .tar.gz file (`tarball')
from CClan and ASDF-Install link at CLiki.
Shelves are distinguished by names, which are symbols. As in
ASDF systems, symbols are distinguished only by their name,
regardless of their package. In order to find and load a shelve
definitions we use the
(defun find-shelf (name &key force-reload) "Find shelf `NAME', returning its shelf object. `NAME' can be a symbol or string. If it's a symbol or string not containing the colon and dot characters, it names the shelf that is already managed. If the shelf isn't already loaded, or `FORCE-RELOAD' is true, an appropriate .shelf file from `*SHELVES-DIR*' directory is loaded, and the shelf object is returned. If `NAME' is any other string, it is treated as an URL or full pathname to a .shelf file. The file is downloaded or copied into `*SHELVES-DIR*' and then loaded." )
For downloading contents of shelved libraries, Librarian exports
(defun download-shelf (shelf) "Download libraries included in `SHELF' into shelf's subdir of `*SHELVES-DIR*'. `SHELF' may be a SHELF instance, or a symbol, or string. If it's a symbol or string, `FIND-SHELF' is invoked to get the SHELF instance." )
SEARCH-SYSTEM-IN-SHELVES function that
ASDF:*SYSTEM-DEFINITION-SEARCH-FUNCTIONS* and is used
to find single ASDF systems among used shelves. List of used
shelves is stored in
*USED-SHELVES* and can be examined and
updated manually. Convenience functions
(USE-SHELF SHELF) and
(UNUSE-SHELF SHELF) are also provided.
Example shelves are in shelves/
for easier using of the same library in different shelves
Separate shelf and systems directories. USE-SHELF and UNUSE-SHELF convenience functions added.
No authenticity checking is currently done on downloaded libraries. Care must be taken to make sure if a correct, trusted source is used and to check downloaded library's integrity.
(WITH-SHELF (name) …) -> (let ((name (find-shelf name))) …)
Date: 2008/10/18 23:12:10