cl-test-grid loads and tests every library in the Quicklisp distribution on every Common Lisp implementation it can get its hands on, and publishes the results: which library loads where, which test suite passes where, and what changed from one Quicklisp release to the next. A full pass over one distribution is some six thousand systems, each loaded in a fresh process, so it takes about a day per implementation. The machines doing it share the work through the results server: each one takes an implementation nobody else has started on and leaves the rest alone.
A machine of yours can join. It needs Docker and a few spare cores, runs one small agent, and tests only what the others have not. The results it uploads carry your email, so library authors reading the reports can see where a result came from.
What you need
- A Linux machine with Docker, or a Mac with Docker Desktop or colima. Any architecture Docker runs on: the implementations come as images for x86-64 and arm64.
- Disk: about 2 GB for each implementation's private copy of the Quicklisp distribution, plus the images, a few hundred megabytes to two gigabytes each.
- Memory: each test runs in a container capped at 4 GB. One test runs at a time unless you ask for more.
- Time: a day or so per implementation, in the background. The agent is restartable; a machine that goes to sleep or reboots picks up where it left off.
Joining
Ask for an upload token. Tokens are issued one per machine by the site administrator; write to the clo-devel mailing list with the machine's architecture and the email you want on the results. Reading the results needs no token; only uploading does.
Get the agent:
git clone -b clnet-store https://github.com/genworks/cl-test-grid.git ~/cl-test-grid
Write
~/cl-test-grid/agent.env, which the agent reads at every start (mode 0600: it holds your token):export TEST_GRID_USER_EMAIL=you@example.org export CL_TEST_GRID_TOKEN=
export TEST_GRID_DEVEL_IMAGE=cl-devel:58097950
The third line runs the implementations from an image that carries
the C libraries some Quicklisp systems need; without it those
systems fail for want of a library rather than for a fault of the
implementation. To run only some implementations, add a line like
export TEST_GRID_LISPS=sbcl,ccl,ecl; the default is every one
built for your architecture. The commercial implementations are
left out until you have read and agreed to their license; the agent
says how when it skips one.
Start it:
mkdir -p ~/cl-test-grid/work-dir cd ~/cl-test-grid && nohup ./run-agent-docker.sh > work-dir/run.log 2>&1 < /dev/null &
The first minutes pull the images and install a private Quicklisp.
Then, for each implementation, the agent asks the results server
whether that implementation has been done or claimed on the current
distribution; if not, it claims it and begins. work-dir/run.log
shows which it took and which it left to others.
Keep it running. A line in your crontab runs the agent once a day; it does nothing when there is nothing new to test, and starts a fresh pass when a new Quicklisp distribution or a new implementation release appears:
0 3 * * * $HOME/cl-test-grid/run-agent-docker.sh >> $HOME/cl-test-grid/work-dir/cron.log 2>&1
Reading what your machine did
The reports at cl-test-grid.common-lisp.dev regenerate twice a day. The test runs page lists every upload with the email it came from; the library pages and the Quicklisp diff show the results by implementation, with a log for every load and every test suite.
Leaving
Stop the agent (docker ps shows it as test-grid-agent; docker kill
it) and remove the cron line. A machine that stops in the middle of an
implementation leaves a claim on it, which other agents respect for two
days and then ignore; if your machine is gone for good, say so on the
list and the claim is released at once. Your private Quicklisp and the
images are under ~/cl-test-grid/work-dir and your Docker image store;
delete them to get the disk back.
A machine with more to give
A machine with the memory for it can run several implementations at
once: TEST_GRID_PARALLEL=4 in agent.env, with the implementations
listed in TEST_GRID_LISPS, runs four agents side by side, each in its
own work directory. Count on 4 GB per running test plus a little for
each agent.
The agent, its settings and what it does in detail are described in the repository's agent/README.md. Questions and trouble go to the clo-devel mailing list.