Embeddable Common-Lisp

Recent Content

ECL license

posted on 2017-12-15 13:30

From time to time a little misconception emerges regarding the ECL license - namely some people seem to have the impression that it is GPL-2.0, while in fact it is LGPL-2.1+. In this post I want to talk a little about the history of ECL and to describe what the practical implications of such licensing are.

The heritage of Embeddable Common Lisp is rich, as you can read here. The software has had a few maintainers throughout its history who hold copyrights to various parts of the code. ECL was licensed under GPL-2.0, but that license was changed after ECLS (or ECL-Spain) and ECoLisp were once again one project and Prof. Juanjo García-Ripoll changed the license to LGPL-2.1+ with agreement of Prof. Giuseppe Attardi. That's the point from which I took over in 2015 with blessing from the previous maintainer. I do not own all copyrights to the software and I can't change its license to anything that is incompatible with working in LGPL-2.1+. Note that parts of the codebase are licensed more liberally (like programs in the examples directory which may be used for any purpose and are licensed under the terms of BSD-2-Clause).

That said, I feel very comfortable with current licensing. It preserves a reasonable balance between producent and consumer rights and fits project goals perfectly. The meaning of our current license is, in short, the following: you can use ECL for any purpose in any setting (including commercial applications), but if you commit changes to ECL itself you are obliged to share these changes (and only them).

The core library is a shared object libecl.so and it is dynamically linked with the software. The binary called ecl is just a program which is a client of this library. Moreover, ECL compilation artifacts are usually shared objects themself (usually disguised under the fas extension):

➜  ~ ldd `which ecl`
    linux-vdso.so.1 =>  (0x00007ffff80c3000)
    libecl.so.16.1 => /home/jack/Pulpit/lisps/ecl-16.1.3/lib/libecl.so.16.1 (0x00007fc7c4665000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc7c427a000)
    libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fc7c3ffa000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fc7c3df2000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc7c3bd4000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc7c39ce000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc7c36c5000)
    /lib64/ld-linux-x86-64.so.2 (0x00005592f1735000)
➜  ~ cd .cache/common-lisp/ecl-16.1.3-21f0b92f-linux-x64/home/jack/Pulpit/repo/mcclim/Apps/Listener
➜  Listener ls listener*
listener.fas listener.o
➜  Listener ldd listener.fas 
    linux-vdso.so.1 =>  (0x00007fffb43f5000)
    libecl.so.16.1 => /home/jack/Pulpit/lisps/ecl-develop/lib/libecl.so.16.1 (0x00007fa2bbfc1000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa2bbbd6000)
    libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fa2bb956000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fa2bb74e000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa2bb530000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa2bb32a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa2bb021000)
    /lib64/ld-linux-x86-64.so.2 (0x0000563db6716000)
➜  Listener file listener.fas 
listener.fas: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=ed1cece88028eb3a388ab0a589a9ee12415532e9, not stripped

There are a few implications of this. First, I will explain (informally) what LLGPL clarification to LGPL is. Many Common Lisp implementations don't work with the notion of linking, so "static linking" and "dynamic linking" doesn't make much sense in them. Libraries are simply added to the image. That may raise questions whenever a binary with an LGPL library in it may be considered derivative work or not? My strong belief lies on the side that it is not derived work and if the author of the Lisp library gave it the LGPL license – they meant it. If we take another interpretation, then it is no different than licensing it with GPL, so it would be nonsense. I think that what is tested in court is intention and there is no rational interpretation of giving a Lisp library the LGPL license except the one that the software does not influence other image components. LLGPL additionally clarifies some wording to make it less ambiguous and clear up unnecessary doubts.

All that said, ECL is safe from concerns like that and such clarification is not necessary because it works with the very same notion LGPL talks about (static and dynamic linking). There is no program image, only objects which a binary is composed of and linked with (exactly like in C programs).

Another interesting license (with which ECL is compatible with thanks to "or later version" clause) is LGPL-3.0. This license is for example used by ECL's fork named MKCL and GMP since version 6. In short, this license adds an important restriction to LGPL-2.1+ called the anti-tivoization provision. This privision adds a new freedom for software users – they must have the ability to update the library on a device with software under this license to their own version of it. This effectively means that the device can't be a complete black box.

This leads us to another topic. ECL is licensed under LGPL-2.1+ license and it is linked with GMP. As we have noted in the paragraph above, the newest version of GMP is licensed under LGPL-3.0. In practice this means that if you use ECL and GMP in your application and any of them is LGPL-3.0 you can't put such a bundle on a camera device which doesn't allow changing its software. To prevent such situation ECL has bundled its own version of GMP, which is a slightly modified version of GMP version 4, which was licensed under the LGPL-2.1+ terms. By default, when building it tries to link against libgmp.so on your system, but given appropriate configure flags it may use the bundled GMP version and link it statically into libecl.so (ECL doesn't export symbols from libeclgmp.a to avoid symbol name conflicts with the original libgmp.so).

I think that summarises it. Now I will provide some made up FAQ to illustrate licensing implications in shorter form:

Q: Is ECL GPL-2.0?
A: No, ECL is LGPL-2.1+.

Q: Can you provide a commercial license for us?
A: No, I don't own all the copyrights.

Q: Can we use ECL in our commercial product with proprietary components?
A: Yes, but you have to keep ECL linked dynamically with them (as a shared object).

Q: Can you incorporate proprietary components in ECL?
A: God no (and I wouldn't do that even if I could).

Q: Can we use ECL in our LGPL/GPL/AGPL product?
A: Yes, you can even link ECL statically for that purpose. Your license will be intact.

Q: Can you incorporate LGPL/GPL/AGPL components in ECL?
A: If you incorporate LGPL-2.1+, then ECL remains in LGPL-2.1+ and it can be integrated in the upstream; but if you incorporate LGPL-3.0, GPL or AGPL, then your fork of ECL will become LGPL-3.0, GPL or AGPL and it won't be integrated upstream.

Q: Can we use ECL in our BSD/MIT/Apache-2.0 product?
A: Yes. If it is dynamically linked there are no further implications. If you link it statically, the overall license is LGPL-2.1+.

Q: Can you incorporate BSD/MIT/Apache-2.0 components in ECL?
A: Yes, sometimes I do that.

Q: If I compile my software with ECL is it LGPL-2.1+?
A: No, products of compilation are not influcenced by compiler license. You may compile any kind of free and proprietary software with ECL without any implications on the compilation artifacts or the code it compiles. I would appreciate not using it for something unethical though.

Q: Can I put ECL in an embedded device to which the consumer doesn't have access to?
A: Yes. You may need to build ECL with bundled GMP library to avoid LGPL-3.0 implications.

Q: If I use two libraries in my application - one being LGPL and the other MIT – what is my program license?
A: That depends. If you link them statically (monolithic programs), then resulting work will be covered at least with LGPL (you may add more restrictions if you want). If you link them dynamically (default), then you may use any license you want.

Q: If I use GPL libraries in my application – what is my program license?
A: Its license is at least GPL.

Q: Are you a lawyer?
A: Nope. You may want to consult one. But hey, you should also consult a lawyer regarding the terms of services you probably agree on surfing the web and EULAs bundled with your favourite software and hardware (i.e phone).

Q: Did you cover all LGPL-2.1+ implications?
A: No, I recommend reading the license. I have talked about things which I find relevant to this post.

Q: Can I buy something from you?
A: Yes, you may buy developer time to work on ECL or to help integrate ECL with your application. I'll probably do it anyway if you drop by on the IRC channel.

If you have more questions you may ask on the mailing list and IRC (channel #ecl on freenode).

Thank you for reading,
Daniel Kochmański


  • I want to thank Elias Mårtenson, Pascal Bourguignon and Tomek Kurcz for proofreading this text and for providing valuable remarks before publishing it on the blog.

  • In this post I've used SPDX-License-Identifier format where appropriate.

Lisp (ECL) and QML (Qt5) on Android?

posted on 2017-07-15 12:00

For up to date version of this article see [https://lights-of-holiness.eu/blog/ECL-QML-Android.htm](https://lights-of-holiness.eu/blog/ECL-QML-Android.htm).
(please note that I'm assuming a Linux/64 bit platform or VirtualBox image)

Preamble: about a month ago, I was completely void of any android experience.
This is to say: using both QML (which is easy to learn) and Common Lisp (which I assume you already know) to develop android apps is not a difficult task at all, as you'll see.

So, if you are like me just a month ago, there are no excuses not to write your own, first android app using this new "EQL5-Android" project!

We will build a small game (Sokoban), which uses Lisp for the program logic, and QML for the UI, and build an APK for the android platform.

Being the author of that very first attempt of integrating Lisp and Qt4 (see lisp-cffi-qt4), what I would like to accomplish is providing you with a ca. 3 MB download, which can be tried out instantly.

10 years ago, the lisp-cffi-qt4.zip (a runnable win32 binary version), was a 3 MB download, including both ECL and Qt4 (UPX compressed, but still).
10 years later, this time on android, what download size is to be expected?
We will see...

Since all the documentation needed for preparing the development environment is already covered in the "EQL5-Android" project itself, I will give only the link here:


So, I'm assuming that you already have everything installed and set up (Qt 5.7.1, Android NDK 10e, Android SDK, Java JDK, and obviously the EQL5-Android sources from gitlab), in order to build android packages (APK files).

(EQL5 itself, the desktop version, is not strictly needed to follow this example; but for developing your own apps, you will obviously need it; even here it's very helpful for testing and debugging, if something doesn't work as expected.)

If you already know the process of building EQL5 apps for the desktop, you will find that building (cross-compiling) to android is very similar, with only a few more steps involved.

Since we use an example of EQL5-Android itself, everything has already been set up. But I want to remember the steps that are not obvious, if you are not familiar with Qt and EQL:

  • you need to add all your external resources, like QML files, PNG files etc. to a Qt resource file (ending .qrc); this will integrate them (compressed) directly into the executable
  • you need to add all your Lisp files, in exact order of loading, to make.lisp (in a future version of EQL5, I will try to integrate this step with ASDF)

And that's it, basically (except the app name, which needs to be adapted to the *.qrc file name, to your *.pro file name and contents (see TARGET and RESOURCES), and to the contents of the third script 3-build-apk.sh (see *.json file name).

Everything else will stay the same for every project.

Now I want to call your attention on the huge advantage of using Qt for your android apps: you can first build a desktop exe, with the exact same sources, and try/debug it. If the desktop version works, the android app will generally work too (the only things that may need adaption are e.g. font sizes and similar).

So, let's get practical! In the EQL5-Android sources, switch to 'examples/sokoban/'.

Building a desktop exe would be this 3 liner:

  $ eql5 make-desktop.lisp
  $ qmake sokoban_desktop.pro
  $ make

(To test if all resources have really been included in the sokoban_desktop executable, you need to move it to a different directory, and launch it from there.)

Here's a screenshot of our app running on the desktop:

But now let's do the cross-compile dance!

First let's copy the needed shared libraries to the 'android-build/' directory.
Just run script number 1:

  $ ./1-copy-libs.sh

This step needs only be done once for every new project. It will copy the cross-compiled ECL and EQL5 libs into our android build directory.

The next steps are very similar to a desktop build:

  $ ecl-android -shell make.lisp
  $ qmake-android sokoban.pro
  $ make

Since cross-compiling requires a special "host ECL", which needs to match the target platform (that is, 32 bit, no double floats), we would be in trouble with cross-compiling EQL5 code: we certainly don't want a seperate EQL5 32 bit version, only for cross-compiling...

But there is a solution to this (see 'utils/EQL5-symbols.lisp' in sources): for cross-compiling -- which is the job of our "host ECL" -- we pretend to be the eql5 executable, by defining all packages and symbols, defining all EQL5 macros (otherwise we can't compile), and simply defining dummy-functions for all EQL5 functions, so the compiler will not complain.

So, loading 'utils/EQL5-symbols.lisp' in our host ECL will be sufficient for cross-compiling EQL5 code.

If you are interested in how the cross-compile environment is set up, please see:


(thanks to Sylvain Ageneau, who wrote the original version; this is a simplified version not depending on ASDF; the latter will be integrated in a future version)

So, the above 3 liner will build the shared library of our app, ready to be included in the android build. To copy it where the android build expects it, use script number 2:

  $ ./2-install-lib.sh

The last step, which will build our APK file, is a verbose one (for eventual debugging), and a little time consuming: it will create the whole package structure, and compile the whole thing into an APK file, ready to be installed on an android device.

There is this great tool (courtesy Qt) called androiddeployqt, which automates the whole and complex process of creating an APK file, with all the needed options already set in our 3rd script:

  $ ./3-build-apk.sh

Here the contents of the above script, where you can see all the selected options:

  $ ~/Qt5.7.1/5.7/android_armv7/bin/androiddeployqt \
        --input android-libsokoban.so-deployment-settings.json \
        --output android-build \
        --deployment ministro \
        --gradle \

If it will tell you BUILD SUCCESSFUL, then you'll find the APK file (ready for deployment) in 'android-build/build/outputs/apk/'.

The last step is copying over the APK file to your android device, and install / run it. Normally you're not allowed to do this, because it requires special developer settings (please search the web for instructions how to enable them, as they are different for every android version).

After connecting via USB and copying the APK file over to your device, just tap it from there. This will ask for installing, and then for opening the app.

Here's a screenshot of the sokoban app running on a tablet:

Above you see the splash screen, because startup will take a few seconds.

Below the game:

After seeing the result, I'd like to consider a few QML and Lisp snippets.

QML is easy to learn. You just need to declare what you want (and it will do the how behind the scenes).
Let's see this snippet for defining and placing our arrow buttons:

  // container for arrow buttons
  Item {
      id: arrows
      width: up.width * 3
      height: up.height * 3
      anchors.margins: 10
      anchors.right: parent.right
      anchors.verticalCenter: parent.verticalCenter

      Ext.Button {
          id: up
          objectName: "up"
          source: "img/up.png"
          anchors.horizontalCenter: parent.horizontalCenter

      Ext.Button {
          objectName: "left"
          source: "img/left.png"
          anchors.verticalCenter: parent.verticalCenter

      Ext.Button {
          objectName: "right"
          source: "img/right.png"
          anchors.verticalCenter: parent.verticalCenter
          anchors.right: parent.right

      Ext.Button {
          objectName: "down"
          source: "img/down.png"
          anchors.horizontalCenter: parent.horizontalCenter
          anchors.bottom: parent.bottom

So, we define an Item as container for the buttons, giving it the width (up.width * 3) and height (up.height * 3), according to the button sizes: this may be any calculation or function call, and may refer to any item of the file, referred to by its id.

For placing the container itself, and the single arrow buttons, we just need to define anchors, which can be relative to other items (here: the parent item).

The Ext.Button is a user defined item class, which can be found in 'qml/ext/Button.qml. That is, the whole directory 'ext/' is imported as Ext:

  import "ext/" as Ext

This means that every file in 'ext/' is now a new QML class, which can be referred to via Ext (like a namespace).

The definition of our image button class (see 'qml/ext/Button.qml') is:

  import QtQuick 2.0

  Image {
      signal pressed()

      MouseArea {
          anchors.fill: parent
          onPressed: { parent.pressed() }

So, we don't need a real button, but only a clickable image: adding a mouse area will allow us to capture mouse events; this event is then passed to the parent (that is, the Image class), where a Qt signal will be emitted: this will allow us to connect to it from Lisp:

  (defun connect ()
    (macrolet ((pressed (item function)
                 `(qconnect (find-quick-item ,item) "pressed()"
                            (lambda () ,function))))
      (pressed "up"       (sokoban:move :north *maze*))
      (pressed "down"     (sokoban:move :south *maze*))
      (pressed "left"     (sokoban:move :west *maze*))
      (pressed "right"    (sokoban:move :east *maze*))
      (pressed "previous" (change-level :previous))
      (pressed "next"     (change-level :next))
      (pressed "undo"     (undo))
      (pressed "restart"  (reset-maze))
      (pressed "solve"    (solve))))

If you already played the game finishing a level, you will have noticed that there are 2 animations (rotation of the player, wiggling of all boxes) which run queued.
This is a little tricky to do, but with the help of a Lisp macro, we only need these lines in Lisp (being queued a macro):

  (defun final-animation ()
    (queued (qml-set "rotate_player" "running" t)
            (qml-set-all "wiggle_box" "running" t)))

Please see the sources for all the details. And this would not be possible without a Lisp function called from QML for notifying us whenever an animation state changes, see 'qml/ext/RotationAnimation.qml':

  import QtQuick 2.0
  import EQL5 1.0

  RotationAnimation {
      onRunningChanged: Lisp.call("qsoko:animation-change", running)

What I left out to explain is the dynamic (at run time) creation of QML items (see 'qml/items/*' and 'lisp/sokoban.lisp'); let's just say that this is left as an exercise to the reader, as all sources will patiently stay there to be read...

Well. But I still didn't answer the initial question: how big of a download is to be expected, 10 years later?

Since our APK file uses the ministro service for automatically installing the needed Qt libraries at the first launch of the app, it will only need to include both the ECL and EQL5 libraries (you can therefore use different ECL and EQL5 versions for every app you deploy).

So, to finally answer the question: our download will be ca. 3.5 MB (instead of 3 MB, 10 years ago, although we obviously compare apples and oranges here).

Seems acceptable.

And since I promised you to test it instantly (if you own a device with ARM processor), here you are:






ECL 16.1.3 release

posted on 2016-12-19 11:30


Dear Community,

After almost a year of development we are proud to present a new release of ECL tagged with version 16.1.3. All changes are backward compatible fixing bugs and other issues, implementing new interfaces and cleaning up the code base.

ECL manual has been updated in a few places. Work on a new documentation is still pending. New documentation is still incomplete, but you may see it here: https://common-lisp.net/project/ecl/static/ecldoc/.

Before this release we have performed extensive tests on many platforms (Linux, FreeBSD, OpenBSD, NetBSD, OSX, Windows MSVC, Windows MinGW, Windows Cygwin, Android and Haiku). For details please consult https://gitlab.com/embeddable-common-lisp/ecl/issues/307. Extra attention has been paid to Windows testing to improve that platform support.

This release is available for download in form of a source archive:

Best regards,
ECL Development Team

API changes

  • Added better interface for package-locks.

    Introduced functions:

    ext:package-locked-p package
    ext:lock-package package
    ext:unlock-package package
    ext:without-package-locks &body body
    ext:with-unlocked-packages (&rest packages) &body body

    To use these functions user has to require the module

    (require '#:package-locks)

    defpackage accepts new option lock to allow locking package on creation:

    (defpackage foo (:lock t))
  • mp:holding-lock-p: introduce new function for multiprocessing. Function verifies if lock is hold by the thread which calls the function. Usage: (mp:holding-lock-p my-lock).

  • make-random-state: fix problem with simple-vectors. The correct initialization types for make-random-state are: (OR RANDOM-STATE FIXNUM (MEMBER T NIL)).

    Initializing a random state with an appropriate array (element type and arity dependent on platform) is also possible.

  • ext:random-state-array: new extension for random-states. Usage: (ext:random-state-array random-state).

  • ext:terminate-process: new extension for external processes. Usage: (ext:terminate-process process) with a second, optional boolean argument whenever termination should be forced or not.


  • Implemented CDR-7. https://common-lisp.net/project/cdr/document/7/index.html

  • implemented CDRs: CDR-1, CDR-5, CDR-14. Both CDR-1 and CDR-5 were already implemented, CDR-14 made us to list them in *features* (https://common-lisp.net/project/cdr/document/14/index.html).

  • if ECL is build with --with-cxx option, :CXX-CORE is present in *features*.

  • deprecated configure option --with-local-gmp has been removed - use --enable-gmp (defaults to auto).

  • configure options has been revised.

  • ASDF has been upgraded to version (with a few patches scheduled for 3.2.0).

  • bundled CLX has been purged. Lately I've fixed ECL support on portable CLX maintained by sharplispers on https://github.com/sharplispers/clx (available via QuickLisp).

  • initial port for the Haiku platform. The port is done by Kacper Kasper's work, one of Haiku developers. Threads are not supported yet.

  • refactored ECL internal tests framework. Tests in src/tests are now asdf-loadable (with load-source-op) and divided into test suites. make check target runs all regression and feature tests which aren't supposed to fail.

  • removed 15000 lines of obsolete code. Files not included in the buildsystem but lingering in the codebase or options failing to build. All info is added in the new documentation in the section "Removed interfaces".

  • improved man page and help output. Man page now contains up-to-date list of flags, as well as explanation of flag's behavior.

  • deprecated long flags with one dash, added two-dash version. Flags that aren't one-character, but start with one dash (e.g. -eval) are now deprecated; long version --eval was added instead.

  • indented C/C++ code to follow emacs's gnu C style. This is a first step towards coding standards in the documentation. Additionally all in the src/c/ directory are listed in the appropraite documentation section (new-doc).

  • refactored list_current_directory in unixfsys.d. Function was obfuscated with ifdefs with non-even pairs of #\{ and #\}.

Issues fixed

  • ECL signals floating point exceptions in top-level console.

  • mp:rwlock is treated as built-in class (previously process crashed if class-of was called on such object).

  • ECL builds now succesfully with --with-ieee-fp=no option.

  • ext:file-stream-fd: doesn't cause an internal-error if called with something not being a file-stream (signals a SIMPLE-TYPE-ERROR condtition).

  • stable-sort: bugfix and improvement in speed. Adapted from SBCL by Diogo Franco.

  • typep: accept * type specifier as abbreviation of T as described in 2.4.3 Type Specifiers of the specification.

  • MOP: fix problemes when redefining non-standard and anonymous classes. Bugs identified and fixed by Pascal Costanza.

  • getcwd: fix issue with too long pathname. This fixes the regression, which crashed ECL at start when pathname exceeded 128 characters limit.

  • make-random-state: fix a problem with simple-vectors. Until now #$ reader macro accepted simple vectors as an argument, what lead to bugs if vector didn't match specific requirements like the element type or the arity. Now we sanitize this.

  • make-load-form: provide implementation for random-state objects.

  • thread fix on msvc: on windows importing thread was closing the thread handler so the thread wakeup wasn't working because the handler is not more valid.

  • import thread wasn't set upping a proper environment: on some case the thread was mistakenly thinking that the thread was already registered.

  • ECL_HANDLER_CASE and ECL_RESTART_CASE didn't work as expected. Bug identified and fixed by Vadim Penzin.

ECL Quarterly Volume V

posted on 2016-11-08 15:00

1 Preface

Dear Readers,

I'm very pleased to present the fifth volume of the ECL Quarterly.

This issue is focused on software development. Tianrui Niu (Lexicall) written a tutorial on how to embed ECL in Qt5.7. He explains how to use ECL with C++ software and how to achieve comfortable development environment for that. Next we have a first part of the tutorial on creating Common Lisp application from scratch. I hope you enjoy it. At last some practical opinionated advice about including all dependencies in the software definitions.

ECL slowly moves towards the next release with a version number 16.1.3. Numerous bugs were fixed, some minor functions were introduced. We're currently working on simplifying the autoconf/Makefile files. Configure options were also cleaned. This may cause some regressions in the build system.

Some code base cleanup was performed with removal of the obsolete code (which wasn't compiled at all), refactor of testing framework and making man page up-to-date. One-dash long flags are deprecated in favour of two-dash alternatives. We've fixed ECL support in upstream portable CLX and now there is no reason to bundle our own separate version. Bundled module has been purged. McCLIM works now with the ECL out of the box.

We've started to implement CDRs (right now CDR1, CDR5, CDR7 and CDR14 are present in ECL with CDR9 and CDR10 on its way). Constant work on a new documentation is performed.

See the CHANGELOG for more details. I hope to release next version before the new year.

If you want to discuss some topic with the developers, please join the channel #ecl on Freenode. If you are interested in supporting ECL development please report issues, contribute new code, write documentation or just hang around with us on IRC. Writing about ECL also increases its mind-share – please do so! Preferably in ECL Quarterly. If you want to support us financially (either ECL and ECL Quarterly), please consider contributing on Bountysource.

Enjoy! And don't forget to leave the feedback at daniel@turtleware.eu.

Daniel Kochmański ;; aka jackdaniel | TurtleWare
Przemyśl, Poland
November 2016

2 Embedding ECL in Qt5.7

2.1 Introduction

ECL is a fantastic ANSI Commnon Lisp implementation that aims at embedding. It allows us to build Lisp runtime into C/C++ code as external library. That means you can both call C code from Lisp runtime and call Lisp code form the C/C++ part. In this article we focus on how to call Lisp procedures from the C/C++ side, you may also achieve the reversed process by inlining C/C++ code in ECL, but that's beyond our discussion here. Generally, this one-side work is fairly enough to enable us to exhaust any power of both Lisp and C/C++.

This article shows how you can embed ECL into a Qt project, and serve as the kernel of that program. I hope that can serve as a common example, or tutorial, for the one who want to know about the true power of ECL, and perhaps that awesome guy is you. At least we will show you that the ability to hybrid is what makes ECL different from other implementations. And if one day you need that power, you know where you gonna head for.

2.2 But why?

I know I should shoot the code and demo quickly, but let the theory come first, just for those who don't understand why we are doing this.

What we are doing is just an instance of mixed-language programming. That means you will be writing codes in different languages and glue them together to combine a program. Usually, the product program would appear to be a peanut, the kernel is written in language A and the nutshell written in language B. The most frequent combination is C++ and Lua, in game programming. This enables us to take advantage of both language A and B. But in this case we are hybridizing C++ and Common Lisp, both aiming at building large software system, an we are sure to gain even more benifits here.

Common Lisp is a masterpiece, but it lacks a good GUI toolkit. Well, if you work on Linux you definitely can take a shot on packages like CL-GTK or CommonQt, and someone will suggest you to use another package called EQL, the Qt4 binding for ECL. But any of these GUI tools look coarse on my Mac, with Retina screen. I don't want to spend my scholarships on Lispworks, so I must find another way to enable Lisp in GUI-related programming. Finally I ended up in here. The hybrid is no compromise, since I can develop fancy GUI interface without losing the power of Lisp.

We can see more benifits that are shown below:

  1. Live hotpatching. Common Lisp is a dynamic language, so it allows you to add new components at runtime. That means you can upgrade your program without re-compiling or even restart. In more advanced discussions, you may even recompile your C/C++ functions. So embedded ECL could even change the host language.
  2. With the most powerful macro system, Lisp is natively designed for complex system designing. Different from light-weight languages like Python and Lua, Lisp is suitable for huge programs. So you can always focus on Lisp, instead of seeking for help from language B. You can also use Lisp environment as a direct DSL interpreter so that you needn't write one yourself.
  3. Efficiency. There are other approaches to combine Lisp runtime with other languages, like using pipes, channels, sockets or temporary files. These are never elegant solutions, since you can only read the output by Lisp runtime manually and there's no direct access to the Lisp runtime memory. And you have to string-parse each Lisp output. So this is neither quick nor efficient. You may also use FFIs (Foreign Language Interfaces) and it's more common with the reverse direction, say call C from Lisp. Now the ECL approach is to save. The Lisp runtime in ECL shares the same part of memory with C/C++ side and there's direct way to fetch the return value of any Lisp function calls. How can they achieve this magic? Well ECL compiles Lisp to C code or bytecode so that they get the same tone.
  4. Stable and mature. ECL is currently the best for embedding. You may have heard of other implementations like Clasp, which works on LLVM and is compatible with C++. But it's not yet stable or ANSI-compatible hitherto. Meanwhile ECL has a long history and is already ready to use. When built with C++ compiler like g++ (flag –with-cxx), ECL also enables us to Call C++ functions. So stick to ECL.

I hope this should convince you that this could be a promising hybrid.

2.3 General Approach

The embedding process of ECL can be pretty simple if you understand how it works. Unfortunately the ECL official documentation of this part is not quite clear at the moment, here are some example code in the example/embed directory. Thanks to Daniel Kochmański, he helped me through the way towards my first success of hybridding. I'm still a newbie here.

The example code is enough for understanding the process of hybridizing Lisp and C codes by ECL. There is absolutely a general approach and you can use it as a pattern in your development.

ECL achieves this by compiling itself into C libraries and link it to C runtime. There is two ways to go: static library and shared library. In this article we will take the first approach. For embedding, there are a few steps:

  1. Write your Lisp files. (absolutely)
  2. Compile your Lisp files.
  3. Write a C/C++ file that includes the ECL library and boots the CL environment.
  4. Link the runtime and compile the whole project into executables.

Easy enough, isn't it? Let me explain in detail.

The first step is nothing different than general Lisp development. You can either create your own package or not. (Just leave the naked lisp file.)

The second step, well, it's time for ECL to rock. We've got two approaches, which depend on whenever you use ASDF or not. If you do not want to use it, you may follow this code:

(compile-file "<YOUR-LISP-FILE>.lisp" :system-p t)
(c:build-static-library "<LIBRARY-NAME>"
                        :lisp-files '("<THE-OBJECT-FILE>.o")
                        :init-name "<INIT-NAME>")

The first line of code compiles your .lisp file into a .o object file, say, <THE-OBJECT-FILE>.o. This file serves as the input for the next procedure. The c:build-static-library function is defined by ECL, it builds the object file into a static library, say, <LIBRARY-NAME>.a. We should pay attention to the init-name. You can define your init-name here as a string, and this is useful for step 3. We will head back when it happens.

If you choose to use ASDF, you can head for the asdf:make-build function. This can be seen in the Makefile in example:

hello.exe: hello.c hello-lisp.a
        $(CC) `ecl-config --cflags` -o $@ hello.c hello-lisp.a \
              `ecl-config --ldflags` -lecl

hello-lisp.a: hello-lisp.lisp
        ecl -norc \
            -eval '(require :asdf)' \
            -eval '(push "./" asdf:*central-registry*)' \
            -eval '(asdf:make-build :hello-lisp :type :static-library :move-here "./")' \
            -eval '(quit)'

        -rm -f hello-lisp.a hello.exe

And you may use asdf:defsystem in your lisp code. We will see this closer in my demo.

In the third step, we must dance with some C/C++ code. In your .c file where you want the ECL environment to run, you should #include <ecl/ecl.h> to make sure all the ECL symbols are linked. Then write some simple code to boot the environment:

/* Initialize ECL */
cl_boot(argc, argv);

/* Initialize the library we linked in. Each library
 * has to be initialized. It is best if all libraries
 * are joined using ASDF:MAKE-BUILD.
extern void init_lib_HELLO_LISP(cl_object);
ecl_init_module(NULL, init_lib_HELLO_LISP);

The cl_ boot procedure boots the CL environment, it takes the right args from your main entry. Now take a look at the extern declaration. Remember last time I suggest you to notice the :init-name, now it's time to use it. If you took the first approach of building library and defined your own *init-name*, now the function name should be the same with it. And if you didn't define your init-name, now the name convention should be: init_lib_<FILE_NAME_OF_THE_LIBRARY>. Say, if the static library named "hello-world–all-systems.a", then you write init_lib_HELLO_WORLD__ALL_SYSTEMS for the function name.

Notice: In C++, you should encapsule the extern code in an extern "C" block:

extern "C"{
extern void init_lib_HELLO_LISP(cl_object);

To make the link process complete. This has something to do with the naming convention that differs from C to C++. In general ECL exports symbols following C naming convention to allow seamless FFI to it from C and other languages. C++ does some weird name mangling.So if you want to call C functions from C++, you have to declare them in C++ that way indeed. The function is used by procedure ecl_init_module to load all of your user-defined Lisp symbols. Then you are freely to call your Lisp code in C/C++.

The forth step builds the whole project. So it acquires all of your C/C++ files, libraries and the ECL library. All of the work can be easily done if you are familiar with Makefile. See the example above.

2.4 Calling Lisp in C/C++

"How can I call the Lisp functions I wrote?" This should be the most urgent question you may ask. The ECL manual describes most of the functions in the Standards chapter. Apparently most of the Lisp functions, or macros have been maped into C functions, with some name convention. For example the [[https://common-lisp.net/project/ecl/static/manual/re02.html][cl_ eval]] means the corresponding Lisp code "eval". Most of the ANSI-defined procedure has the naming convention of using cl_ as prefix. So you can easily find the primitive symbol you need.

But perhaps the problem you most concern is:

  1. How can I call MY Lisp functions in C/C++?
  2. How can I translate the return value into C/C++ objects?

For the first question I suggest you to use the cl_eval function. The reason is it's simple and extensible. For the safety reasons you may use cl_funcall or cl_safe_eval. But none of them is as universal as cl_eval. The cl_funcall, as its name means, can only call functions and cannot be used to call macros. And cl_safe_eval requires more parameters in order to handle potential errors that may occur on the Lisp side. But here since I don't mean to make my code productive so I won't care about the safety or convenience. I wrote a friendlier version of cl_eval and you can call lisp code like this:

cl_eval("mapcar", list_foo, "(lambda (x) (princ x))");

And that's nearly Lisp code in appearance.

So let's head for the cl_eval. Its signature is:

cl_object cl_eval(cl_object);

It receives a cl_object and returns a cl_object. Hmm. Now you should get knowledge of how ECL manipulate Common Lisp Objects before wondering what cl_object is.

Quite simple. ECL encapsules any Lisp object into the same structure cl_object. It's a C union whose definition can be seen in object.h, line 1011. So you don't need to worry about using different types to capture the return value.

Translating C strings to cl_object is trivial: use the c_string_to_object function:

cl_object c_string_to_object (const char * s)

You just write the Lisp form in C string and the function will create Lisp object for you. So you may write

cl_eval(c_string_to_object("(princ \"hello world\")"));

To get your first hybrid-call.

The second question can be a little tough due to lack of documentation. And there's another naming convention.

Generally, you may use the ecl_to_* family to convert the cl_object to primitive C data, here is some regular examples:

char ecl_to_char(cl_object x);
int ecl_to_int(cl_object x);
double ecl_to_double(cl_object x);

I've said that these functions could only help convert cl_object to primitive C data. No array, and no string. The ECL API didn't provide them officially. So we have to implement them manually, sorry to say that. (If I missed something, correct me.)

I would show two trivial yet useful functions that may help you. The first one helps you to traverse Lisp List:

auto cl_list_traverse=[](auto& cl_lst, auto fn){

This is implemented in C++ with the convenience of C++14 standard. Can be rewritten in C like this:

void cl_list_traverse(cl_object cl_lst, void(*fn)(cl_object)){

Usage example:

void print_element(cl_object obj){
    printf("%d\n", ecl_to_int(obj));
list_traverse(foo_list, print_element);

And the second one converts the cl_object into C++ *std::string*.

std::string to_std_string(cl_object obj){
    std::string val;
    auto & str=obj->string;
    for(unsigned long i=0;i<str.fillp;i+=1)
       val+=*(typeof(str.elttype) *)(str.self+i);
    return val;

When you are using these functions to convert a cl_object to C/C++ objects, you have to know exactly what the return value is. That means, if you are trying to call ecl_to_int on a cl_object, you should be clear that the cl_object IS an integer. And for some complicate situation, a cl_object could contain more than one type at the same time. For example, if you call a function that returns a list of strings, say '("hello" "lisp") then the corresponding cl_object can both contain a string (on its car position) and a list (on its cdr position). Call cl_car and you get a cl_object containing a string, and you can call to_std_string on that object to get a C++ string. I mean, you should figure out that before you code. The secret is to just think you are still in Lisp.

2.5 Hybridizing Lisp & Qt

Now it's time to head for our ultimate goal: let Lisp rock with Qt! We have had enough knowledge of embedding ECL into C++ code in the former chapters and Qt is nothing but C++. So the work should be trivial. Sounds this is true but, there's still many things to be solved. I have stuggled much about them but now I can just write down the final progress and pretend this is simple.

The first one is, how to build a Lisp package system, instead of compiling a naked Lisp file or a single package.

2.5.1 Build Lisp Package System

If you are to build some awesome software, you must be using external packages. After all, there are plenty of excellent Lisp packages, like cl-ppcre and lparallel. Quicklisp solved the package management problem in an elegant way. But when you decide to distribute your software, you shouldn't ask Quicklisp for help, instead, you should compile all of your dependencies into your Lisp runtime, so that you can load them all by a single statement. SBCL could dump current Lisp image into a single executable file by function sb-ext:save-lisp-and-die. We need a function that does the similar thing, here in ECL.

ASDF is here to help. You can make an asdf system that defines every files and dependencies in your project. If you haven't touched that, see the tutorial.

After that, you just have one step to go: build the system into library. You may use asdf:make-build. Here comes an example:

(require 'asdf)
(push "./" asdf:*central-registry*)
(asdf:make-build :hello-lisp-system
                 :type :static-library
                 :monolithic t
                 :move-here "./")

The push expression adds current working directory into ASDF search list. Then asdf is ready to find your user-defined system in your directory.

If you have external Lisp packages as dependencies, you must set the :monolithic parameter to T. That means, you order ASDF to build your whole system into a single file. Or else you'd have to load your dependencies manually each time you start your Lisp runtime.

Unfortunately, I have to say the function is not ready for building static libraries that contains Lisp package dependencies. There is a serious bug that prevents the library from linking. So the example code shown above won't work!. Sorry to say that. But perhaps this code works fine in the future. :)

Don't be afraid. There is still two other approaches, to build a fasl file or the shared library.

I'll take the first approach since it brings some good advantages. That is, allowing us to distribute the Lisp system independently. You can debug either natively in ECL by loading the fasl file or remotely on the C/C++ side. Sometimes you need this because you don't know which side, say C or Lisp, that causes your program crash.

Since then, I have to build two different Lisp systems. The first one serves as the Lisp runtime and is build to static library. It contains just one line of Lisp code.

(princ "Lisp Environment Settled.")

This library will be linked to my C++ program. The second one will be the actual system I wrote. I'm building it into a independent fasb file.

(require 'asdf)
(push "./" asdf:*central-registry*)

(asdf:make-build :hello-lisp-system
                 :type :fasl
                 :monolithic t
                 :move-here "./")

After loading this code I will see a hello-lisp-system–all-systems.fasb file in my directory. In order to use the system, I should load that fasl file at runtime. So the init code should be:

/* init-name */
#define __cl_init_name init_lib_LISP_ENVI

extern "C"{

  extern void __cl_init_name(cl_object);


void init_cl_env(int argc, char * argv[]){
  /* Initialize CL environment */
  cl_boot(argc, argv);
  ecl_init_module(NULL, __cl_init_name);
  /* load fasb */
  cl_eval("load", CL_MAIN_FASB);
  /* set context to current package */
  cl_eval("in-package", CL_MAIN_PACKAGE_NAME);
  /* hook for shutting down cl env */

#undef __cl_init_name

There is also a function called cl_load, you may use it to load the bundle:

Signature: cl_object cl_load(cl_arg narg, cl_object path);
Usage: cl_load(1, c_string_to_object("./lisp_image.fasb"));

Notice: When you are using the Lisp runtime, you are in the :top context.

Notice: The cl_eval function I used is the optimized, or overloaded version which I will introduce in the next section.(Code is here.) If you stick to the original version, you should convert C string to cl_object manually, like:


2.5.2 Enhance ECL Bridge In C++14

ECL is written in pure C, as a result, it lacks the real object to describe Lisp data. The cl_ object structure unions the Lisp datas together but there is no method for it. Utility functions are just naked funtions. You have to write ecl_to_int(obj) to convert the object to int, but it would be friendlier if you can write that as obj.to_int(). At this moment we are going to enclosure the original cl_ object in a C++ object to implement this.

auto cl_list_traverse=[](auto& cl_lst, auto fn){

class cl_obj {
    cl_object __obj;

    cl_obj(cl_object &&obj){this->__obj=obj;}
    cl_obj(const cl_object &obj){this->__obj=obj;}

    /* list index */
    inline cl_obj car(){return cl_obj(cl_car(this->__obj));}
    inline cl_obj cdr(){return cl_obj(cl_cdr(this->__obj));}
    inline cl_obj cadr(){return this->cdr().car();}
    inline cl_obj caar(){return this->car().car();}
    inline cl_obj cddr(){return this->cdr().cdr();}

    /* predicates */
    inline bool nullp(){return Null(this->__obj);}
    inline bool atomp(){return ECL_ATOM(this->__obj);}
    inline bool listp(){return ECL_LISTP(this->__obj);}
    inline bool symbolp(){return ECL_SYMBOLP(this->__obj);}

    inline int to_int(){return ecl_to_int(this->__obj);}
    inline char to_char(){return ecl_to_char(this->__obj);}

    inline std::string to_std_string(){
        std::string val;
        auto & str=this->__obj->string;
        for(unsigned long i=0;i<str.fillp;i+=1)
           val+=*(typeof(str.elttype) *)(str.self+i);
        return val;

    template<typename function>
    inline void list_traverse(function fn){cl_list_traverse(this->__obj, fn);}

    inline cl_obj operator=(cl_object &&obj){return cl_obj(obj);}


It's just a trivial one and can only implement a small subset of ANSI Common Lisp, but anyway it's enough for our demo. After that, you can write something like obj.cdr().car().to_ int(). That is a more fluent interface.

Despite that, the original cl_eval function is not friendly enough. We are going to implement a better one so that you can call that function just as if you are in Lisp. See the overloading:

using std::string;

cl_object lispfy(string str);
 return c_string_to_object(str.data());
string __spc_expr(string first);
template <typename ...str>
string __spc_expr (string first, str ... next){
    return first+" "+__spc_expr(next...);
template<typename ...str>
string par_expr(str... all){
    return "("+__spc_expr(all...)+")";
template<typename ...str>
string par_list(str... all){
    return "'"+par_expr(all...);
template<typename ...str>
string cl_eval(str... all){
    return cl_eval(lispfy(par_expr(all...)));

Now you can call that cl_eval function like:

cl_eval("mapcar", "'(1 2 3 4 5)", "(lambda (x) (evenp x))");

Those code would compile by setting your compiler to -std=c++14.

2.5.3 Time to Hybridize!

After gaining the knowledge in the former chapter, it's trivial for us to use ECL in Qt programming. You just have to follow some small modifications and tips.

Source code of the demo being shown here can be found here.

First you should get a little knowledge about qmake. It's an automatic toolchain that helps us build our program. This time we needn't write Makefile manually since qmake is quite easy to use. You should check your .pro file and add those code to it:

QMAKE_CFLAGS += `ecl-config --cflags`
QMAKE_CXXFLAGS += `ecl-config --cflags`
QMAKE_LFLAGS += `ecl-config --ldflags` 
LIBS += -lecl

ecl-config will generate flags for your compiler. And since I used C++14, I have to add:


And we should also do a small trick. Because Qt defined the macro slots as keyword, it conflicts with the slots defined in ecl.h. So we have to undef that keyword to turn off the interference:

#ifdef slots
#undef slots
#include <ecl/ecl.h>

Now you can check out my demo. It looks like this:



It's just simple but enough to serve as a demo. The Lisp code of Fibonacci demo is based on package lparallel, the concurrent package.

(defpackage :hello-lisp
  (:use :cl :lparallel))

(in-package :hello-lisp) ;;package name hello-lisp

(setf lparallel:*kernel* (lparallel:make-kernel 4))

(lparallel:defpun pfib (n)
  (if (< n 2)
      (plet ((a (pfib (- n 1)))
             (b (pfib (- n 2))))
            (+ a b))))

You see, that's concurrent computation! This function should take use of all my four CPU cores. So that one is to show you how we can use external Lisp packages in our ECL.

The second demo is Quicksort. It just sorts the List you passed and print the result on the output line. This one demostrates the method to load and traverse Lisp list.

Click the hello-lisp button and you get an echo:


The text "Bonjour, lisp!" is returned by a Lisp function. This demostrates how to extract strings from cl_object.

Now you are ready for deeper adventure with embedding ECL. Good luck!

Note: For OSX users, after you build the source code by qmake, make, you should also run this shell code:

mv hello-lisp-system--all-systems.fasb ecl_qtdemo.app/Contents/MacOS/

To make sure the Lisp system is right in the place. For Linux users you are not bothered by this since Qt won't make application packages in default.

3 Creating a Common Lisp project – Part I

3.1 Introduction

A common question heard from the Common Lisp newcomers is:

How to create my own application with Common Lisp?

Numerous concepts like packages, Quicklisp, modules and ASDF bring the confusion, which is only deepened by a wide range of implementations and foreign to the new programmer developing paradigm of working on a live image in the memory.

This post is a humble attempt to provide a brief tutorial on creating a small application from scratch. Our goal is to build a tool to manage document collection. Due to the introductory nature of the tutorial we will name our application "Clamber".

We will start with a quick description of what should be installed on the programmer's system (assumed operating system is Linux). Later we will create a project boilerplate with the quickproject, define a protocol for the software, write the application prototype (ineffective naive implementation), provide the command line interface with Command Line Option Nuker. This is where the first part ends.

Second part will be published on McCLIM blog and will show how to create a graphical user interface for our application with McCLIM.

Afterwards (in a third part in next ECL Quarterly) we will take a look into some considerations on how to distribute the software in various scenarios:

  • Common Lisp developers perspective with Quicklisp,
  • ordinary users with system-wide package managers with ql-to-deb,
  • source code distribution to clients with Qucklisp Bundles,
  • binary distribution (closed source) with ADSF prebuilt-system,
  • as a shared library for non-CL applications with ECL.

Obviously a similar result may be achieved using different building blocks and all choices reflect my personal preference regarding the libraries I use.

3.2 How to distribute the software

Before we jump into the project creation and actual development I want to talk a little about the software distribution. We may divide our target audience in two groups – programmers and end users. Sometimes it is hard to tell the difference.

Programmers want to use our software as part of their own software as a dependency. This is a common approach in FOSS applications, where we want to focus on the problem we want to solve, not the building blocks which are freely available (what kind of freedom it is depends on the license). To make it easy to acquire such dependencies the Quicklisp project was born. It is a package manager.

End users aren't much concerned about the underlying technology. They want to use the application in the most convenient way to them. For instance average non-programming Linux user would expect to find the software with theirs system distribution package manager. Commercial client will be interested in source code with all dependencies with whom the application was tested.

Proposed solution is to use Quicklisp during the development and bundle the dependencies (also with Quicklisp) when the application is ready. After that operation our source code doesn't depend on the package manager and we have all the source code available, what simplifies further distribution.

3.3 What are Common Lisp systems?

Notion of "system" is unknown to the Common Lisp specification. It is a build-system specific concept. Most widely used build-system in 2016 is ASDF. System definition is meant to contain information essential for building the software – application name, author, license, components and dependencies. Unfortunately ADSF doesn't separate system definitions from the source code and asd format can't be considered declarative. In effect, we can't load all system definitions with certainty that unwanted side-effects will follow.

3.4 Development environment configuration

We will only outline steps which are necessary to configure the development environment. There are various tutorials on how to do that which are more descriptive.

  1. Install Emacs and SBCL1:

    These two packages should be available in your system package manager (if it has one).

  2. Install Quicklisp:

    Visit https://www.quicklisp.org/beta/ and follow the instructions. It contains steps to add Quicklisp to Lisp initialization file and to install and configure SLIME. Follow all these instructions.

  3. Start Emacs and run Slime:

    To run Slime issue M-x slime in Emacs window.

These steps are arbitrary. We propose Linux + SBCL + Emacs + Quicklisp + SLIME setup, but alternative configurations are possible.

3.5 How to create a project

Quickproject is an excellent solution for this task because it is very simple tool with a well defined goal – to simplify creating basic project structure.

The simplest way of creating a new one is loading the quickproject system with Quicklisp and calling the appropriate function. Issue the following in the REPL:

(ql:quickload 'quickproject)
(quickproject:make-project #P"~/quicklisp/local-projects/clamber/"
                           :depends-on '(#:alexandria)
                           :author "Daniel Kochmański <daniel@turtleware.eu>"
                           :license "Public Domain")

That's it. We have created a skeleton for our project. For now, we depend only on alexandria – public domain utility library. List of dependencies will grow during the development to reflect our needs. Go to the clamber directory and examine its contents.

Now we will customize the skeleton. I prefer to have one package per file, so I will squash package.lisp and clamber.lisp into one. Moreover, README.txt will be renamed to README.md, because we will use markdown format for it.

To avoid clobbering the tutorial with unnecessary code we put only interesting parts here. Complete steps are covered in the application GIT repository available here:


We propose to clone the repository and track the progress with the subsequent commits and this tutorial.

3.6 Writing the application

Here is our application Clamber informal specification:

  • Application will be used to maintain a book collection,
  • Each book has associated meta-information (disregarding the underlying book file format),
  • Books may be organized with tags and shelves,
  • Book may be only on one shelf, but may have multiple tags,
  • Both CLI and GUI interfaces are a required,
  • Displaying the books is not part of the requirements (we may use external programs for that).
  1. Protocol

    First we will focus on defining a protocol. Protocol is a functional interface to our application. We declare how external modules should interact with it. Thanks to this approach we are not tied to the implementation details (exposing internals like hash tables or class slot names would hinder us during the future refactoring, or could cause changes which are not backward compatible).

    ;;; Clamber book management protocol
    ;;; Requirements explicitly list that books has to be organized by
    ;;; shelves and tags. Book designator is used to identify books (it
    ;;; has to be unique). Protocol doesn't mandate designator type. It
    ;;; may be a unique name, pathname, URL or any arbitrary
    ;;; object. Other args (in form of keys) are meant to contain
    ;;; meta-information.
    (defgeneric insert-book (book-designator &rest args
                             &key shelf tags &allow-other-keys)
      (:documentation "Creates a book entity associated to a given ~
       `shelf' and `tags'."))
    ;;; We need to bo able to remove book. We need only the designator for
    ;;; that.
    (defgeneric delete-book (book-designator)
      (:documentation "Removes a book entity from the system."))
    ;;; We may search for books according to various
    ;;; criteria. `book-designator' is definite. It is possible to extend
    ;;; the find functionality to support other criteria. Book must match
    ;;; *all* supplied criteria.
    (defgeneric find-books (&rest args
                            &key book-designator shelf tags &allow-other-keys)
      (:documentation "Returns a `sequence' of books matching the ~
    ;;; We access books by their designators, but `find-books' returns a
    ;;; list of opaque objects. This function is needed for coercion from
    ;;; these objects to the designators. Sample usage:
    ;;; (map 'list #'book-designator (find-books :shelf "criminal"))
    (defgeneric book-designator (book)
      (:documentation "Extract book designator from opaque `book' object."))

    This code is put in clamber.lisp file. It is important to remember, that :documentation clause in defgeneric is meant only for programmers who use our library (to provide a short reminder of what the function does) and shouldn't be considered being the final documentation. Especially docstrings are not documentation.

    Comments are meant for programmers who work on our library (extend the library or just read the code for amusement). Their meaning is strictly limited to the implementation details which are irrelevant for people who use the software. Keep in mind, that comments are not reference manual.

  2. Implementation prototype

    Our initial implementation will be naive so we can move forward faster. Later we could rewrite it to use a database. During the prototyping programmer may focus on the needed functionality and modify the protocol if needed.

    This is a tight loop of gaining the intuition and adjusting rough edges of the protocol. At this phase you mustn't get too attached to the code so you can throw it away without hesitation. More time you spend on coding more attached to the code you are.

    ;;; Implementation
    ;;; At start we are going to work on in-memory database.
    (defparameter *all-books* (make-hash-table) "All defined books.")
    ;;; Note, that when we define `:reader' for the slot `designator' we
    ;;; actually implement part of the protocol.
    (defclass book ()
      ((designator :type symbol   :initarg :des :reader book-designator)
       (shelf      :type string   :initarg :shl :reader book-shelf)
       (tags       :type sequence :initarg :tgs :reader book-tags)
       (meta :initarg :meta :accessor book-information)))
    ;;; `title' and `author' are enlisted for completion.
    (defmethod insert-book ((designator symbol) &rest args
                            &key shelf tags title author &allow-other-keys
                            &aux (tags (alexandria:ensure-list tags)))
      (declare (ignore title author readedp)
               (type (shelf string)))
      (multiple-value-bind (book found?) (gethash designator *all-books*)
        (declare (ignore book))
        (if found?
            (error "Book with designator ~s already present." designator)
            (setf (gethash designator *all-books*)
                  (make-instance 'book
                                 :des designator
                                 :shl shelf
                                 :tgs (coerce tags 'list)
                                 :meta args)))))
    ;;; Trivial
    (defmethod delete-book ((designator symbol))
      (remhash designator *all-books*))
    ;;; We use `while-collecting' macro (`collect' equivalent from
    ;;; cmu-utils) to simplify the code.
    (defmethod find-books (&rest args
                             (book-designator nil designator-supplied-p)
                             (shelf nil shelf-supplied-p)
                             (tags nil tags-supplied-p)
                           &aux (tags (alexandria:ensure-list tags)))
      (declare (ignore args))
      (uiop:while-collecting (match)
        (labels ((match-book (book)
                   (and (or (null shelf-supplied-p)
                            (equalp shelf (book-shelf book)))
                        (or (null tags-supplied-p)
                            (subsetp tags (book-tags book) :test #'equalp))
                        (match book))))
          (if designator-supplied-p
              (alexandria:when-let ((book (gethash book-designator *all-books*)))
                (match-book book))
              (alexandria:maphash-values (lambda (val)
                                           (match-book val))

    Our prototype support only shelf and tags filters and allows searching with a designator. Note that book-designator function is implemented in our class definition as a reader, so we don't have to define the method manually. We add uiop to dependencies for the while-collecting macro (descendant of a collect macro in cmu-utils).

    We may check if our bare (without user interface) implementation works:

    (ql:quickload :clamber)
    ;; -> (:CLAMBER)
     :shelf "nonfiction"
     :tags '("nonfiction" "politics" "psychology")
     :title "The Captive Mind"
     :author "Czesław Miłosz")
    ;; -> #<CLAMBER::BOOK {100469CB73}>
    (clamber:find-books :tags '("politics"))
    ;; -> (#<CLAMBER::BOOK {100469CB73}>)
  3. Unit tests

    Now we will add some basic unit tests. For that we will use fiveam testing framework. For seamless integration with ASDF and to not include the tests in clamber itself we will define it as a separate system and point to it with the :in-order-to clause:

    (asdf:defsystem #:clamber
      :description "Book collection managament."
      :author "Daniel Kochmański <daniel@turtleware.eu>"
      :license "Public Domain"
      :depends-on (#:alexandria #:uiop)
      :serial t
      :components ((:file "clamber"))
      :in-order-to ((asdf:test-op
                     (asdf:test-op #:clamber/tests))))
    (asdf:defsystem #:clamber/tests
      :depends-on (#:clamber #:fiveam)
      :components ((:file "tests"))
      :perform (asdf:test-op (o s)
                 (uiop:symbol-call :clamber/tests :run-tests)))

    tests.lisp file is in the repository with clamber. To run the tests issue in the REPL:

    (asdf:test-system 'clamber/tests)
  4. Prototype data persistence

    To make our prototype complete we need to store our database. We will use for it a directory returned by uiop:xdg-data-home. To serialize a hash-table cl-store will be used.

    (defparameter *database-file* (uiop:xdg-data-home "clamber" "books.db"))
    (defun restore-db ()
      "Restore a database from the file."
      (when (probe-file *database-file*)
        (setf *all-books* (cl-store:restore *database-file*))))
    (defun store-db ()
      "Store a database in the file."
      (ensure-directories-exist *database-file*)
      (cl-store:store *all-books* *database-file*))
    (defmethod insert-book :around ((designator symbol)
                                    &rest args &key &allow-other-keys)
      (declare (ignore designator args))
      (prog2 (restore-db)
    (defmethod delete-book :around ((designator symbol))
      (declare (ignore designator))
      (prog2 (restore-db)
    (defmethod find-books :around (&rest args &key &allow-other-keys)
      (declare (ignore args))

    We read and write database during each operation (not very efficient, but it is just a prototype). find-books doesn't need to store the database, because it doesn't modify it.

    Since our database isn't only in-memory object anymore, some additional changes to tests seem appropriate. We don't want to modify user's database:

    (defparameter *test-db-file*
      (uiop:xdg-data-home "clamber" "test-books.db"))
    (defun run-tests ()
      (let ((clamber::*database-file* *test-db-file*))
        (5am:run! 'clamber)))

    Right now we have a "working" prototype, what we need is the user interface.

3.7 Creating standalone executable

There are various solutions which enable creation of standalone binaries. The most appealing to me is Clon: the Command-Line Options Nuker, which has a very complete documentation (end-user manual, user manual and reference manual) , well thought API and works on a wide range of implementations. Additionally, it is easy to use and covers various corner-cases in a very elegant manner.

Our initial CLI (Command Line Interface) will be quite modest:

% clamber --help
% clamber add-book foo \
  --tags a,b,c \
  --shelf "Favourites" \
  --meta author "Bar" title "Quux"  
% clamber del-book bah
% clamber list-books
% clamber list-books --help
% clamber list-books --shelf=bah --tags=drama,psycho
% clamber show-book bah

3.7.1 Basic CLI interface

To make our interface happen we have to define application synopsis. clon provides defsynopsis macro for that purpose:

(defsynopsis (:postfix "cmd [OPTIONS]")
  (text :contents
        "Available commands: add-book, del-book, list-books, show-book.
Each command has it's own `--help' option.")
  (flag :short-name "h" :long-name "help"
        :description "Print this help and exit.")
  (flag :short-name "g" :long-name "gui"
        :description "Use graphical user interface."))

These are all top-level flags handling main options (help and graphical mode switch). As we can see it is declarative, allowing short and long option names. Except flag other possible option types are possible (user may even add his own kind of option).

clon allows having multiple command line option processing contexts, what simplifies our task – we can provide different synopsis for each command with its own help. First though we will define a skeleton of our main function:

(defun main ()
  "Entry point for our standalone application."
  ;; create default context
    ;; if user asks for help or invokes application without parameters
    ;; print help and quit
    ((or (getopt :short-name "h")
         (not (cmdline-p)))
    ;; running in graphical mode doesn't require processing any
    ;; further options
    ((getopt :short-name "g")
     (print "Running in graphical mode!")

  (alexandria:switch ((first (remainder)) :test 'equalp)
    ("add-book"   (print "add-book called!"))
    ("del-book"   (print "del-book called!"))
    ("list-books" (print "list-books called!"))
    ("show-book"  (print "show-book called!")))


(defun dump-clamber (&optional (path "clamber"))
  (dump path main))

In our main we look for the top-level options first. After that we verify which command is called. For now our action is just a stub which prints the command name. We will expand it in the next step. Function dump-clamber is provided to simplify executable creation. To dump the executable it is enough to use this snippet:

sbcl --eval '(ql:quickload :clamber)' --eval '(clamber/cli:dump-clamber "clamber")'
./clamber --help


3.7.2 Implementing commands

Each command has to have its own synopsis. Books have unique identifiers (designators) – we force this option to be a symbol. All applications parameters following the options are treated as metadata. add-book has the following synopsis:

(defparameter +add-book-synopsis+
  (defsynopsis (:make-default nil :postfix "cmd [OPTIONS] [META]")
    (text :contents "Add a new book to the database.")
    (flag :short-name "h" :long-name "help"
          :description "Print this help and exit.")
    (lispobj :short-name "d" :long-name "ident"
             :description "Book designator (unique)."
             :typespec 'symbol)
    (stropt :short-name "s" :long-name "shelf"
            :description "Book shelf.")
    ;; comma-separated (no spaces)
    (stropt :short-name "t" :long-name "tags"
            :description "Book tags."))
  "The synopsis for the add-book command.")

We don't want duplicated options, so we filter them out in the add-book-main function, which is called in main instead of printing the message. Command entry point is implemented as follows:

(defun add-book-main (cmdline)
  "Entry point for `add-book' command."
  (make-context :cmdline cmdline
                :synopsis +add-book-synopsis+)
  (when (or (getopt :short-name "h")
            (not (cmdline-p)))

  (let ((ident (getopt :short-name "d"))
        (shelf (getopt :short-name "s"))
        (tags  (getopt :short-name "t")))

    (when (or (getopt :short-name "d")
              (getopt :short-name "s")
              (getopt :short-name "t"))
      (print "add-book: Junk on the command-line.")
      (exit 1))
    (clamber:insert-book ident
                         :shelf shelf
                         :tags (split-sequence #\, tags)
                         :meta (remainder))))

To make book listing more readable we define print-object method for books in clamber.lisp. Moreover, we tune find-books method to not rely on the fact whenever argument was supplied or not, but rather on its value (NIL vs. non-NIL).

(defmethod print-object ((object book) stream)
  (if (not *print-escape*)
      (format stream "~10s [~10s] ~s -- ~s"
              (book-designator object)
              (book-shelf object)
              (book-tags object)
              (book-information object))

list-books command is very similar, but instead of calling insert-book it prints all books found with clamber:find-books called with provided arguments. Also we don't print help if called without any options.

(defparameter +list-books-synopsis+
  (defsynopsis (:make-default nil :postfix "[META]")
    (text :contents "List books in the database.")
    (flag :short-name "h" :long-name "help"
          :description "Print this help and exit.")
    (lispobj :short-name "d" :long-name "ident"
             :description "Book designator (unique)."
             :typespec 'symbol)
    (stropt :short-name "s" :long-name "shelf"
            :description "Book shelf.")
    ;; comma-separated (no spaces)
    (stropt :short-name "t" :long-name "tags"
            :description "Book tags."))
  "The synopsis for the list-books command.")

(defun list-books-main (cmdline)
  "Entry point for `list-books' command."
  (make-context :cmdline cmdline
                :synopsis +list-books-synopsis+)
  (when (getopt :short-name "h")

  (let ((ident (getopt :short-name "d"))
        (shelf (getopt :short-name "s"))
        (tags  (getopt :short-name "t")))

    (when (or (getopt :short-name "d")
              (getopt :short-name "s")
              (getopt :short-name "t"))
      (print "add-book: Junk on the command-line.")
      (exit 1))

    (map () (lambda (book)
              (format t "~a~%" book))
         (clamber:find-books :book-designator ident
                             :shelf shelf
                             :tags tags))))

Last command we are going to implement is the simplest one – del-book:

(defparameter +del-book-synopsis+
  (defsynopsis (:make-default nil)
    (text :contents "Delete a book in the database.")
    (flag :short-name "h" :long-name "help"
          :description "Print this help and exit.")
    (lispobj :short-name "d" :long-name "ident"
             :description "Book designator (unique)."
             :typespec 'symbol))
  "The synopsis for the del-book command.")

(defun delete-book-main (cmdline)
  "Entry point for `list-books' command."
  (make-context :cmdline cmdline
                :synopsis +del-book-synopsis+)
  (when (or (getopt :short-name "h")
            (not (cmdline-p)))

  (clamber:delete-book (getopt :short-name "d")))

Of course this CLI prototype needs to be improved. For instance, it doesn't handle any errors – for if we try to add a book with already existing designator. Moreover, for testing purposes it would be nice to be able to provide database file top-level argument for testing purposes.

4 Case against implicit dependencies

Sometimes implementations provide functionality which may be expected to be present during run-time under certain conditions. For instance when we use ASDF to load a system, we probably have UIOP available in the image (because to load the system, we need ASDF which depends on UIOP at its own run-time).

It is important to remember that we can't mix two very different moments – the build time and the run-time. This difference may not be very obvious for the Common Lisp programmer because it is common practice to save the lisp image with the system, which was loaded with help of the build system (hence the build system is present in the image), or they load fasl files with the build system in question. The fact that we have only one widely adopted building facility, and that it is often preloaded, makes it even less possible to encounter any problems.

There are two main arguments against implicit dependencies. The first one is the separation of the build tool from the application. It is hardly justifiable to include autotools and make in your binary after building the system. They may have exploitable bugs, increase the application size or are simply unnecessary (unless you really depend on make at run-time).

Assuming you rely on implicit dependencies, and given that you produce a standalone application (or you cross-compile it), either your build system will inject such dependency for you (what you may not necessarily want), or your application will miss an important component which it relies upon (for instance UIOP 2) and will effectively crash.

The second argument has more to do with the declarative system definitions. If your application depends on something, you should list it, because it is a dependency. So if we switch the build system and it may read our declarative system definitions, or we have an older version of the build system which doesn't imply the dependency, then we can't load the system. It's not the build system problem, but our broken system definition.

Having that in mind, I sharply advocate listing all dependencies in system definition, despite meretricious voices saying it's rudimentary or harmful to put them there. We will take UIOP as an example. We have two possible options:

(defsystem #:system-one
  (depend-on ((:require #:uiop))))

(defsystem #:system-two
  (depend-on (#:uiop))

system-one's dependency is resolved as follows:

  1. If the system uiop is already present in the image, do nothing3,
  2. If the system uiop may be acquired as a module, require it,
  3. If the system uiop may be loaded by a build system, load it,
  4. Otherwise signal a missing-component condition.

This behavior is an elegant substitute for the implicit dependency, which relies on the UIOP version bundled with the Common Lisp implementation.

The system-two system dependency is handled in a slightly different manner:

  1. If the system uiop may be loaded from the disk and version in the image isn't up-to-date, load the system from there,
  2. If the image has a preloaded version of the system, do nothing,
  3. Otherwise signal a missing-component condition.

Both definitions are strictly declarative and any build system which "knows" the ASD file format will know your preferences disregarding if it has UIOP bundled or not. If it can't handle it correctly, then it is a bug of the build system, not your application.

UIOP here is only one example. I urge you to declare any dependencies of your system. You may know that bordeaux-threads on which you depend implies that Alexandria will be present in the image, but this assumption may turn against you if it changes this dependency in favour of something else.

I've asked one of a proponents of the implicit dependencies François-René Rideau for a comment to present the other point of view:

The dependency on ASDF is not implicit, it's explicit: you called your system file .asd.

Now, if you want to fight dependency on ASDF, be sure to also track those who put functions and variables in .asd files that they use later in the system itself. Especially version numbers.

Trying to enforce technically unenforceable constraints through shaming isn't going to fly. If you want to promote separation of software from build system, promote the use of Bazel or some other build system incompatible with ASDF.



Since we won't use any unique ECL features we suggest using SBCL here (it is faster and better supported by 3rd-party libraries). Using ECL shouldn't introduce any problems though.


UIOP doesn't depend on ASDF and it may be loaded with older versions of this widely adopted build system, or directly from the file. Quicklisp ships UIOP this way to assure compatibility with implementations which don't include new ASDF.


This is broken in ASDF as of version 3.1.7 – ASDF will load the system from the disk if it is possible. It will hopefully be fixed in version 3.1.8.

ECL Quarterly Volume IV

posted on 2016-06-15

1 Preface


I've managed to assemble the fourth volume of the ECL Quarterly. As always a bit off schedule but I hope you'll find it interesting.

This issue will revovle around ECL news, some current undertakings and plans. Additionally we'll talk about Common Lisp implementations in general and the portability layers. I believe it is important to keep things portable. Why? Keep reading!

Lately we're working with David O'Toole on making support for ECL on Android better. He wants to distribute his games on this platform and was kind enough to write an article for ECL Quarterly. Thanks to his work we've discovered various rough edges and bugs in ECL and gained some invaluable insight into the cross compilation problems of Common Lisp applications.

As the final remark – I've found some time to establish a proper RSS subscription feed for ECL and ECL Quarterly. I hope that this issue will finally land on the Planet Lisp – a well known Lisp-related blog posts aggregator maintained by Zach Beane.

I want to thank for the valuable feedback and proofreading to many people, especially Antoni Grzymała, Javier Olaechea, Michał Psota, Ilya Khaprov and David O'Toole.

Have a nice lecture,

Daniel Kochmański ;; aka jackdaniel | TurtleWare
Poznań, Poland
June 2016

2 ECL's "what's going on"

I've added a milestone with a deadline for the ECL 16.1.3 release with the bugs I want to fix. You may find it here. I'm very happy to receive a lot of positive feedback, merge requests and awesome bug reports. Thank you for that! :-)

Backporting CLOS changes from CLASP was successful but we won't incorporate them in the main branch. The recently resurrected cl-bench has shown that these changes impact performance and consing negatively (check benchmarks). If you are curious about the changes, you may checkout the branch feature-improve-clos in the repository.

I'm slowly working on the new documentation. This is very mundane task which I'm not sure I'll be able to finish. Rewriting DocBook to TexInfo and filling the missing parts is hard. I'm considering giving up and improving the DocBook instead.

In the near future I plan to make a crowdfunding campaign to improve support for cross-compilation, Android and Java interoperability in order to boost development. More details will be probably covered in the next Quarterly issue.

3 Porting Lisp Games to Android with Embeddable Common Lisp, Part 1

3.1 Introduction

Recently I ported my Common Lisp game engine "Xelf" to the Android operating system using Embeddable Common Lisp.

Some work remains to be done before I can do a proper beta test release, but ECL Quarterly provides a good opportunity to pause and share the results thus far. This is the first part of a two-part article. The focus of Part 2 will be on performance optimization, testing, and user interface concerns.

Special thanks to Daniel Kochmański, 3-B, oGMo, and the rest of the Lisp Games crew for their inspiration and assistance.

3.1.1 About the software

Xelf is a simple 2-D game engine written in Common Lisp. It is the basis of all the games I have released since 2008, and can currently be used with SBCL to deliver optimized standalone game executables for GNU/Linux, MS Windows, and Mac OSX.

I've also published a Git repository with all the work-in-progress scripts, patches, and libraries needed to compile Xelf for Android with Embeddable Common Lisp, OpenGL, and SDL.

Please note that this is a pre-alpha release and is mainly intended for Common Lisp developers looking to get a head start in building an Android game. Use with caution.

Xelf is not required; you can substitute your own Lisp libraries and applications and just use the repo as a springboard.

I would like to add support for CL-SDL2 as well, both as a prelude to porting Xelf to SDL 2.0, and as a way to help the majority who use SDL 2.0 for current projects.

3.2 Problems

3.2.1 Choosing an implementation

As I use only Free Software for my projects, I did not consider any proprietary Lisps.

Steel Bank Common Lisp now runs on Android, but SBCL as a whole cannot yet be loaded as a dynamic shared library. This is a show-stopper because Android requires the entry point of a native application to be in a shared library specially embedded in the app.

Xelf works very well with Clozure Common Lisp, but CCL's Android support is not fully functional at present. So I've been quite happy to discover Embeddable Common Lisp. Its technique of translating Common Lisp into plain C has made integration with the Android NDK toolchain relatively simple.

3.2.2 Cross-compilation

For performance reasons the Lisp stack (meaning LISPBUILDER-SDL, CL-OPENGL, CFFI, Xelf, the game, and all their dependencies) must be compiled to native ARM machine code and loaded as shared libraries.

There is a complication in this task as regards ECL. The latter produces native code by translating Common Lisp into plain C, and then invoking the C compiler. But the C compiler toolchain is not typically present on Android, and building one that is properly configured for this task has proved difficult so far.

Therefore we must cross-compile the entire Lisp stack. ECL's Android build procedure already cross-compiles the Lisp contained in ECL, but there were additional difficulties in compiling Lisp libraries which I'll cover below in the "Solutions" section.

3.2.3 Legacy code

Xelf has improved a lot over time and gained new features, but is now outdated in some respects. When I first wrote Xelf in the 2006-2007 period SDL 1.2 was current and OpenGL Immediate mode had not yet been officially deprecated. This hasn't been a terrible problem in practical terms, given that both are still widely supported on PC platforms. But porting to Android would mean I could not procrastinate any longer on updating Xelf's SDL and OpenGL support.

3.3 Solutions

3.3.1 CommanderGenius to the rescue

Help arrived for my SDL woes in the form of Sergii Pylypenko's "CommanderGenius", a fancy port of SDL 1.2/2.0 to Android. I can utilize the existing LISPBUILDER-SDL bindings for SDL, SDL-MIXER, SDL-TTF, SDL-IMAGE, and SDL-GFX. Not only that, there are extra features such as gamepad support, floating virtual joysticks, access to touchscreen gesture data and Android system events, support for the Android TV standard, and much more.

CommanderGenius is actually designed from the start to rebuild existing SDL 1.2 / 2.0 / OpenGL projects as Android applications, and includes dozens of examples to work with. So in mid-May this year I set about splicing together Daniel Kochmański's ECL-ANDROID Java wrapper and startup code (which together load ECL as a shared object from within the app) into the CommanderGenius SDL application code and build procedures.

The result is a fullscreen SDL/OpenGL application with Embeddable Common Lisp, optionally running Swank. There's even a configurable splash screen!

3.3.2 Do a little dance with ASDF

ECL can compile an entire system into one FASL file, but I ran into a snag with the ASDF-based build procedure. The typical way is to compile each Lisp file and then load the resulting compiled file. But on the cross-compiler,

(load (compile-file "myfile.lisp"))

fails because the output of COMPILE-FILE is a binary for the wrong architecture. Likewise, alien shared libraries cannot be loaded during Lisp compilation, which broke CL-OPENGL and LISPBUILDER-SDL.

My temporary solution was to redefine the function ASDF:PERFORM-LISP-LOAD-FASL in my build script. My modified version does something like this instead:

(compile-file "myfile.lisp")
(load "myfile.lisp")

I then invoke ECL's system builder, which spits out a big binary FASB file containing the whole system. But thanks to the LOAD statements, each Lisp file has had access to the macros and other definitions that preceded it in compilation.

I'm sure this is really wrong, but it works, and the resulting FASBs load very quickly. (App startup time went from over 30 seconds when loading byte-compiled FASCs, to about 3.5 seconds.)

In the end, it was simple to deal with CL-OPENGL and LISPBUILDER-SDL wanting to open shared libraries during compilation. I used Grep to find and then comment out calls to CFFI:USE-FOREIGN-LIBRARY, leaving the DEFINE-FOREIGN-LIBRARY definitions intact. This allows cross-compilation to proceed normally.

Then on Android, after the FASBs are loaded I invoke USE-FOREIGN-LIBRARY on each of the required definitions.

So tricking ASDF works. But aside from being a hack, it's not enough for some of the things I'd like to do. The INLINED-GENERIC-FUNCTION technique looks like a highly promising way to increase performance, but my cross-compilation trick led in this case to invalid FASB's with embedded FASC bytecodes. Indeed, to work with ECL in this situation would require actually loading the ARM-architecture compiled INLINED-GENERIC-FUNCTION binary before compiling systems that use inlining—which as mentioned above cannot be done during cross-compilation.

I'm exploring other potential solutions, such as installing a GNU/Linux container on my Android development unit in order to give ECL access to a native C compiler toolchain (see below). I may even attempt to write a custom cross-compilation procedure using Clang and LLVM. But this is less urgent for now, because tweaking ASDF is sufficient to produce a working application.

3.3.3 Use OpenGL ESv1 with CL-OPENGL

Luckily the the path of least resistance could prevail here. OpenGL ES version 1 is widely supported on Android devices, and is easier to port to from Immediate mode than is GLESv2. CL-OPENGL supports it right out of the box. (I'd like to thank 3-B and oGMo for their help in bridging the gap with my own code.)

Some tasks remain to be done here but most of Xelf's drawing functions are now working, including TrueType fonts and vertex coloring.

I've also written some code to partially emulate vertex coloring as a way of increasing render performance, and this will be covered in the forthcoming Part 2 of this article.

3.3.4 ProTip: Use the byte-compiler

One issue has gone unmentioned. How do I interactively redefine functions and set variables in order to develop the running game via SLIME/Swank, if everything must be cross-compiled on an X86 system?

The answer is that ECL's built-in bytecode compiler is used in these cases, and the bytecoded definitions replace the originals. I can freely use COMPILE-FILE, LOAD, and even ASDF:LOAD-SYSTEM during "live" development; under normal circumstances the only real difference is execution speed of the resulting code. The final game app will ship without Swank, of course, and with a fully native Lisp stack.

Now you have a new problem, which is how to edit the Lisp files on your Android device so that Swank can compile and load them.

3.3.5 ProTip: Use Emacs TRAMP with ADB

To make this useful you need a rooted android device.

(add-to-list 'tramp-default-user-alist '("adb" nil "root"))
(find-file "/adb::/")

This can integrate with Emacs' "bookmarks" and "desktop" features for even more convenience.

3.3.6 ProTip: Use Emacs to inspect your APK package

They're just zip files. Missing libraries or assets? Check the APK by opening it as a file in GNU Emacs.

3.3.7 ProTip: Use a GNU/Linux container for SSH and native Emacs with X11!

You can actually install a GNU/Linux "container" with Debian, Ubuntu, or some other distribution on your Android development system in order run the Secure Shell daemon and many other applications. I use it to run a graphical Emacs on the Android box, with Emacs' X11 connection forwarded through SSH so that its windows open on my desktop GNU/Linux PC's X server—right alongside my native Emacs. I use different color themes to avoid mixing them up.

This gives me full access to everything on both systems from a single mouse/keyboard/monitor, and I can cut and paste text freely between applications.

Setting up such a container is beyond the scope of this article, but I highly recommend it. It was pretty easy on a rooted device, and works very well.

3.4 Conclusion

In less than a month we went from "let's do it" to "wow, it works!" What more can you ask for?

This concludes Part 1 of my article on building Lisp games for Android with Embeddable Common Lisp. To read my running commentary and see news and test results as they are posted, you can visit the project README:


More details and all scripts and configurations can be found in that repository.

Thanks for reading,

David O'Toole (dto@xelf.me)
11 June, 2016

4 Common Lisp implementations

Some time ago I've created with the help of many kind people (most notably from Rainer Joswig and Fare Rideau) a graph presenting Common Lisp implementations and the relations between them. This version is improved over drafts presented on twitter and linkedin. If you find any errors, please contact me.


It is worth noting that LispWorks and VAX share the code with Spice Lisp which later evolved into Common Lisp implementation CMUCL. Striped lines lead to CMUCL, because I didn't want to add pre-CL implementations.

There is also suspicion that Lucid shares code with Spice Lisp and/or VAX, but I couldn't confirm that, so I'm leaving it as is.

"JavaScript Lisp Implementations" classifies some lisps as CL, but I've added only Acheron and parenscript to the list, because rest is just CL-ish, not even being a subset.

Resources I've found on the internet: CMU FAQ, ALU list, CLiki overview, Wikipedia article, JavaScript Lisp Implementations.

5 Building various implementations

I've built various lisps to perform some benchmarks and to have some material for comparison. Ultimately I've decided to polish it a little and publish. I had some problems with Clasp and Mezzano so I've decided to not include them and leave building these as an exercise for the reader ;-). Also, if you feel adventurous, you may try to build Poplog, which has Common Lisp as one of the supported languages.

If you want to read about various implementations, please consult Daniel's Weinreb Common Lisp Implementations: A Survey (material from 2010, definitely worth reading).

First we create a directory for the lisp implementations (we'll build as an ordinary user) and download the sources. Each implementation has a list of building prerequisites, but it may be not comprehensive.

export LISPS_DIR=${HOME}/lisps
mkdir -p ${LISPS_DIR}/{src,bin}
pushd ${LISPS_DIR}/src

# Obtain sources
svn co http://abcl.org/svn/trunk/abcl/ abcl
# git clone git@github.com:drmeister/clasp.git
svn co http://svn.clozure.com/publicsvn/openmcl/trunk/linuxx86/ccl ccl
hg clone http://hg.code.sf.net/p/clisp/clisp clisp
git clone git@common-lisp.net:cmucl/cmucl.git cmucl
git clone https://gitlab.com/embeddable-common-lisp/ecl.git ecl
git clone git://git.sv.gnu.org/gcl.git gcl
git clone git@github.com:davazp/jscl.git jscl
# git clone https://github.com/froggey/Mezzano.git
git clone https://gitlab.common-lisp.net/mkcl/mkcl.git mkcl
git clone git://git.code.sf.net/p/sbcl/sbcl sbcl
git clone git@github.com:wadehennessey/wcl.git wcl
git clone https://github.com/gnooth/xcl.git

5.0.1 ABCL (Armed Bear Common Lisp)

jdk, ant
pushd abcl
cp abcl ${LISPS_DIR}/bin/abcl-dev

5.0.2 CCL (Clozure Common Lisp)

gcc, m4, gnumake
pushd ccl
echo '(ccl:rebuild-ccl :full t)' | ./lx86cl64 -n -Q -b

# installation script is inspired by the AUR's PKGBUILD
mkdir -p ${LISPS_DIR}/ccl-dev
cp -a compiler contrib level-* lib* lisp-kernel objc-bridge \
   tools x86-headers64 xdump lx86cl64* examples doc \

find ${LISPS_DIR}/ccl-dev -type d -name .svn -exec rm -rf '{}' +
find ${LISPS_DIR}/ccl-dev -name '*.o' -exec rm -f '{}' +
find ${LISPS_DIR}/ccl-dev -name '*.*fsl' -exec rm -f '{}' +

cat <<EOF > ${LISPS_DIR}/bin/ccl-dev
exec ${LISPS_DIR}/ccl-dev/lx86cl64 "\$@"
chmod +x ${LISPS_DIR}/bin/ccl-dev

5.0.3 CLISP

gcc, make
don't build with ASDF (it's old and broken)
pushd clisp
./configure --prefix=${LISPS_DIR}/clisp-dev/ \
            --with-threads=POSIX_THREADS \
cd build
make && make install
ln -s ${LISPS_DIR}/clisp-dev/bin/clisp ${LISPS_DIR}/bin/clisp-dev

5.0.4 CMUCL (CMU Common Lisp)

cmucl binary, gcc, make, openmotif
it needs another CMUCL to bootstrap (release 21a)
pushd cmucl
mkdir -p prebuilt
pushd prebuilt
wget https://common-lisp.net/project/cmucl/downloads/release/21a/cmucl-21a-x86-linux.tar.bz2 \
mkdir ${LISPS_DIR}/cmucl-21a
tar -xf cmucl-21a-x86-linux.tar.bz2 -C ${LISPS_DIR}/cmucl-21a/
tar -xf cmucl-21a-x86-linux.extra.tar.bz2 -C ${LISPS_DIR}/cmucl-21a/
cat <<EOF > ${LISPS_DIR}/bin/cmucl-21a
exec ${LISPS_DIR}/cmucl-21a/bin/lisp "\$@"
chmod +x ${LISPS_DIR}/bin/cmucl-21a
# Note, that this is already a fully functional lisp now
bin/build.sh -C "" -o "cmucl-21a"
bin/make-dist.sh -I ${LISPS_DIR}/cmucl-dev/ linux-4/
cat <<EOF > ${LISPS_DIR}/bin/cmucl-dev
exec ${LISPS_DIR}/cmucl-dev/bin/lisp "\$@"
chmod +x ${LISPS_DIR}/bin/cmucl-dev

5.0.5 ECL (Embeddable Common Lisp)

gcc, make
./configure --prefix=${LISPS_DIR}/ecl-dev/
make && make install
ln -s $LISPS_DIR/ecl-dev/bin/ecl ${LISPS_DIR}/bin/ecl-dev

5.0.6 JSCL (Java Script Common Lisp)

Conforming CL implementation, web browser, nodejs
Doesn't provide LOAD yet (no filesystem), but author confirmed that this will be implemented (virtual filesystem on the browser and the physical one on the nodejs).
mkdir ${LISPS_DIR}/jscl-dev
pushd jscl

# Run in the console (node-repl)
cp jscl.js repl-node.js ${LISPS_DIR}/bin/jscl-dev
cat <<EOF > ${LISPS_DIR}/bin/jscl-dev
exec node ${LISPS_DIR}/jscl-dev/repl-node.js
chmod +x ${LISPS_DIR}/bin/jscl-dev

# Run in the web browser (optional)
cp jscl.js repl-web.js jquery.js jqconsole.min.js jscl.html style.css \
# replace surf with your favourite browser supporting JS
cat <<EOF > ${LISPS_DIR}/bin/jscl-dev-browser
exec surf ${LISPS_DIR}/jscl-dev/jscl.html
chmod +x ${LISPS_DIR}/bin/jscl-dev-browser


5.0.7 GCL (GNU Common Lisp)

gcc, make
# Doesn't work both with head and the release, luckily it works with
# the next pre-release branch
git checkout Version_2_6_13pre
./configure --prefix=${LISPS_DIR}/gcl-2.6.13-pre
make && make install
ln -s ${LISPS_DIR}/gcl-2.6.13-pre/bin/gcl ${LISPS_DIR}/bin/gcl-2.6.13-pre

5.0.8 MKCL (Man-Kai Common Lisp)

gcc, make
pushd mkcl
./configure --prefix=${LISPS_DIR}/mkcl-dev
make && make install
ln -s ${LISPS_DIR}/mkcl-dev/bin/mkcl ${LISPS_DIR}/bin/mkcl-dev

5.0.9 SBCL (Steel Bank Common Lisp)

ANSI-compliant CL implementation
  • Lisp has to close on EOF in top-level (CMUCL doesn't do that),
  • ECL has some bug regarding Lisp-to-C compiler apparently triggered by the SBCL compilation – don't use it here,
  • we could use precompiled SBCL like with the CMUCL, but let's exploit the fact, that we can compile from the C-bootstrapped implementation (we'll use already built clisp-dev),
  • it is advised to run the script in fast terminal (like xterm) or in the terminal multiplexer and to detach it – SBCL compilation process is very verbose,
  • if you build SBCL on Windows, consider using MinGW to preserve POSIX compatibility.
pushd sbcl
export GNUMAKE=make
./make.sh "clisp"
INSTALL_ROOT=${LISPS_DIR}/sbcl-dev ./install.sh
cat <<EOF > ${LISPS_DIR}/bin/sbcl-dev
SBCL_HOME=${LISPS_DIR}/sbcl-dev/lib/sbcl exec ${LISPS_DIR}/sbcl-dev/bin/sbcl "\$@"
chmod +x ${LISPS_DIR}/bin/sbcl-dev

5.0.10 WCL

tcsh, gcc, git
very incomplete implementation
pushd wcl
REV=`git rev-parse HEAD`
sed -i -e "s/WCL_VERSION = \"3.0.*$/WCL_VERSION = \"3.0-dev (git-${REV})\"/" CONFIGURATION
LD_LIBRARY_PATH=`pwd`/lib make rebuild
mkdir ${LISPS_DIR}/wcl-dev
cp -a bin/ lib/ doc/ ${LISPS_DIR}/wcl-dev/
cat <<EOF > ${LISPS_DIR}/bin/wcl-dev
LD_LIBRARY_PATH=${LISPS_DIR}/wcl-dev/lib exec ${LISPS_DIR}/wcl-dev/bin/wcl "\$@"
chmod +x ${LISPS_DIR}/bin/wcl-dev

5.0.11 XCL

last commit in 2011
pushd xcl
mkdir ${LISPS_DIR}/xcl-dev
XCL_HOME=${LISPS_DIR}/xcl-dev make
cp -a clos compiler lisp COPYING README xcl ${LISPS_DIR}/xcl-dev
# This will build in XCL_HOME, even if run in source directory
./xcl <<EOF

ln -s ${LISPS_DIR}/xcl-dev/xcl ${LISPS_DIR}/bin/xcl-dev

6 Portability libraries

It is important to know the difference between the language standard, implementation-specific extensions and the portability libraries. The language standard is something you can depend on in any conforming implementation.

Sometimes it's just not enough. You may want to do ** serializethreading*, or to *data, which is very hard to express (or even impossible) in the language provided by the standard. That's where the implementation-specific extensions kick in. Why are they called "implementation-specific"? Because the API may be different between implementations – reaching consensus is a hard thing1.

The most straightforward approach I can imagine is to reach for the documentation of the Common Lisp implementation you are currently using and to use the API provided by this implementation. I dare you not to do that! It's definitely the easiest thing to do at first, but mind the consequences. You lock yourself, and your users in the implementation you prefer. What if you want to run it on the JVM or to make it a shared library? Nope, you're locked-in.

"What can I do then?" – you may ask. Before I answer this question, I'll tell you how many people do it (or did it in the past) – they used read-time conditionals directly in the code. Something like the following:

(defun my-baz ()
  #+sbcl                        (sb-foo:do-baz-thing 'quux)
  #+ccl                         (ccl:baz-thing       'quux)
  #+(and ecl :baz-thing)        (ext:baz             'quux)
  #+abcl                        (ext:baz             'quux)
  #+(and clisp :built-with-baz) (ext:baz-thingie     'quux)
  #-(or sbcl ccl ecl abcl clisp)
  (error "Your implementation isn't supported. Fix me!"))

If the creator felt more fancy and had some extra time, they put it in the package my-app-compat. It's all great, now your application works on all supported implementations. If somebody wants theirs implementation to work, send the creator a patch, who incorporates it into the code and voila, everything works as desired.

We have one problem however. Libraries tend to depend on one another. There is also a lot of software which uses features beyond the ANSI specification (it's all good, programmers need these!). Do you see code duplication everywhere? How many times does a snippet above have to be copy-pasted, or rewritten from scratch? It's not black magic after all. APIs between ad-hoc implementations don't exactly match, covered CL implementations differ…

So you quickload your favorite library which depends on 10 other libraries which implement BAZ functionality in theirs own unique way, with a slightly different API on the unsupported implementation – that's why we have my-baz abstraction after all, right? Now, to make it work, a user has to:

  1. Find which of the ten libraries don't work (not trivial!),
  2. find and clone the repositories (we want to use git for patches),
  3. fix each one of them (grep helps!) and commit the changes,
  4. push the changes to your own forked repository and create a pull request (or send a diff to the mailing list) – *ten times*,
  5. voila, you're done, profit, get rich, grab a beer.

It's a lot of work which the user probably won't bothered to do. They will just drop the task, choose another implementation or hack their own code creating the Yet Another Baz Library for the implementations he cares for reinventing the wheel once more. It's a hacker's mortal sin.

I'm going to tell you now what is the Right Thing™ here. Of course you are free to disagree. When you feel that there is a functionality you need which isn't covered by the standard you should

  1. Look if there is a library which provides it.

    You may ask on IRC, the project's mailing list, check out the CLiki, do some research on the web. Names sometimes start with trivial-*, but it's not a rule. In other words: do your homework.

  2. If you can't find such a library, create one.

    And by creating such a library I mean comparing the API proposed by at least two CL implementations (three would be optimal IMHO), carefully designing your own API which covers the functionality (if it's trivial, this should be easy) and implementing it in your library.

    Preferably (if possible) add a fallback implementation for implementations not covered (with the appropriate warning, that it may be inefficient or not complete in one way or another).

    It may be worth reading the Maintaining Portable Lisp Programs paper written by Christophe Rhodes.

  3. Write beautiful documentation.

    A CL implementation docs may be very rough. It takes time to write them and programmers tend to prioritize code over the documentation. It's really bad, but it's very common for the documentation to be incomplete or outdated.

    Document your library, describe what it does, how to use it. Don't be afraid of the greatness! People will praise you, success will come, world will be a better place. And most importantly, your library will be useful to others.

  4. Publish the library.
  5. Make that library your project's dependency.

I know it's not easy, but in the long term it's beneficial. I guarantee you that. That's how the ecosystem grows. Less duplication, more cooperation – pure benefit.

Some people don't follow this path. They didn't think it through, or maybe they did and decided that keeping the dependency list minimal is essential to their project, or were simply lazy and hacked their own solution. There are also some old projects which exported a number of features being a very big portability library and an application at the same time (ACL-compat, McCLIM and others). What to do then?

If it's a conscious decision of the developer (who doesn't want to depend on /anything/), you can do nothing but provide a patch adding your own implementation to the supported list. It's their project, their choice, we have to respect that.

But before doing that you may simply ask if they have something against plugging these hacks with the proper portability library. If they don't – do it, everybody will benefit.

There are a few additional benefits of the presented portability library approach for the implementations itself. Having these internal details in one place makes it more probable that your implementation is already supported. If the library has a bug it's easier to fix it in one place. Also, if the CL implementation changes its API, it's easy to propagate changes to the corresponding portability libraries. New CL implementation creators have a simplified task of making their work usable with existing libraries.

It is worth noting, that creating such library paves the way to the new quasi-standard functionalities. For instance Bordeaux Threads has added recently CONDITION-WAIT function, which isn't implemented on all implementations. It is a very good stimulus to add it. This is how library creators may have real impact on the implementation creators decisions about what to implement next.

6.1 Portability layer highlights

Here are some great projects helping CL implementations be part of a more usable ecosystem. Many of these are considered being part of the de-facto standard:

Provides thread primitives, locks and conditionals
Serializing and deserializing CL objects from streams
Foreign function interface (accessing foreign libraries)
Meta-object protocol – provides it's own closer-common-lisp-user package (redefines for instance defmethod)
TCP/IP and UDP/IP socket interface.
Osicat is a lightweight operating system interface for Common Lisp on POSIX-like systems, including Windows
Portable pathname library
trivial-garbage provides a portable API to finalizers, weak hash-tables and weak pointers
trivial-features ensures consistent *FEATURES* across multiple Common Lisp implementations
trivial-gray-streams system provides an extremely thin compatibility layer for gray streams
external-program enables running programs outside the Lisp process

There are many other very good libraries which span multiple implementations. Some of them have some drawbacks though.

For instance IOlib is a great library, but piggy-backs heavily on UN*X – if you develop for many platforms you may want to consider other alternatives..

UIOP is also a very nice set of utilities, but isn't documented well, does too many things at once and tries to deprecate other actively maintained projects – that is counterproductive and socially wrong. I'd discourage using it.

There are a few arguments supporting UIOP's state – it is a direct dependency of ASDF, so it can't (or doesn't want to) depend on other libraries, but many utilities are needed by this commonly used system definition library. My reasoning here is as follows: UIOP goes beyond ASDF's requirements and tries to make actively maintained projects obsolete. Additionally it works only on supported implementations even for features which may be implemented portably.

6.2 UIOP discussion

I'm aware that my opinion regarding UIOP may be a bit controversial. I've asked the library author and a few other people for feedback which I'm very grateful for. I'm publishing it here to keep opinions balanced.

6.2.1 Fare Rideau

Dear Daniel,

while there is a variety of valid opinions based on different interests and preferences, I believe your judgment of UIOP is based on incorrect premises.

First, I object to calling UIOP "not well documented". While UIOP isn't the best documented project around, all its exported functions and variables have pretty decent DOCSTRINGs, and there is at least one automatic document extractor, HEΛP, that can deal with the fact that UIOP is made of many packages, and extract the docstrings into a set of web pages, with a public heλp site listed in the UIOP README.md. The fact that some popular docstring extractors such as quickdocs can't deal with the many packages that UIOP creates with its own uiop:define-package doesn't mean that UIOP is less documented than other projects on which these extractors work well, it's a bug in these extractors.

Second, regarding the deprecation of other projects: yes, UIOP does try to deprecate other projects, but (a) it's a good thing, and (b) I don't know that any of the projects being deprecated is "actively maintained". It's a good thing to try to deprecate other lesser libraries, as I've argued in my article Consolidating Common Lisp libraries: whoever writes any library should work hard so it will deprecate all its rivals, or so that a better library will deprecate his and all rivals (such as optima deprecating my fare-matcher). That's what being serious about a library is all about. As for the quality of the libraries I'm deprecating, one widely-used project the functionality of which is completely covered by UIOP is cl-fad. cl-fad was a great improvement in its day, but some of its API is plain broken (e.g. the :directories argument to its walk-directory function has values with bogus names, while its many pathname manipulation functions get things subtly wrong in corner cases), and its implementation not quite as portable as UIOP (that works on all known actively used implementations). There is no reason whatsoever to ever choose cl-fad over UIOP for a new project. Another project is trivial-backtrace. I reproduced most of its functionality, except in a more stable, more portable way (to every single CL implementation). The only interface I didn't reproduce from it is map-backtrace, which is actually not portable in trivial-backtrace (only for SBCL and CCL), whereas serious portable backtrace users will want to use SLIME's or SLY's API, anyway. As for external-program, a good thing it has for it is some support for asynchronous execution of subprocesses; but it fails to abstract much over the discrepancies between implementations and operating systems, and is much less portable than uiop:run-program (as for trivial-shell, it just doesn't compete).

UIOP is also ubiquitous in a way that other libraries aren't: all implementations will let you (require "asdf") out of the box at which point you have UIOP available (exception: mostly dead implementations like Corman Lisp, GCL, Genera, SCL, XCL, may require you to install ASDF 3 on top of their code; still they are all supported by UIOP, whereas most portability libraries don't even bother with any of them). This ubiquity is important when writing scripts. Indeed, all the functionality in UIOP is so basic that ASDF needed it at some point — there is nothing in UIOP that wasn't itself required by some of ASDF's functionality, contrary to your claim that "UIOP goes beyond ASDF's requirements" (exception: I added one function or two to match the functionality in cl-fad, such as delete-directory-tree which BTW has an important safeguard argument :validate; but even those functions are used if not by ASDF itself, at least by the scripts used to release ASDF itself). I never decided "hey, let's make a better portability library, for the heck of it". Instead, I started making ASDF portable and robust, and at some point the portability code became a large chunk of ASDF and I made it into its own library, and because ASDF is targetting 16 different implementations and has to actually work on them, this library soon became much more portable, much more complete and much more robust than any other portability library, and I worked hard to achieve feature parity with all the libraries I was thereby deprecating.

Finally, a lot of the functionality that UIOP offers is just not offered by any other library, much less with any pretense of universal portability.

6.2.2 David Gu

For the documentation thing, I really think Quickdocs could do a better job. The bug #24 stated that problem, however, it's remain to be solved. I will check this out if I have free time recently.

I use UIOP a lot in my previous company, the reason is simple and maybe a little naive: my manager didn't want to involve too many add-ons in the software. UIOP is shipped together with ASDF, it's really "convenient", and its robustness is the final reason why I will stick to it. If people understand how UIOP came out in the history from ASDF2 to ASDF3, I think people will understand why it's acting like deprecating several other projects – that's not the original idea of it.

But anyway, I really learned a lot from this post and also the comments. In my opinion, avoid reinventing the wheels is the right idea and directions for this community. So from that perspective, I support @fare's idea "It's a good thing to try to deprecate other lesser libraries". Including this article and along with Maintaining Portable Lisp Programs and @fare's Consolidating Common Lisp Libraries, we should let more people involved in this topic.



If you are Common Lisp implementer and plan to add a feature beyond ANSI specification, please consider writing a proposal and submitting it to Common Lisp Document Repository. It will make everybody's life easier.

New website look

posted on 2016-04-22

I've imported the old archives and genearated ECL website with help of the coleslaw and the sclp. Now we have a proper RSS feed and posting news is less annoying then before.

For posterity, here is the ugly hack I've used to import archives from JSON:

(defparameter *archives-template*
title: ~A
date: ~A
author: ~A
format: md


(setf *json-posts*
  (with-open-file (f #P"/home/jack/linki/repo/ecl-website/static/files/misc/news-ecl-backup-2015-08-25.json"
                     :direction :input
                     :external-format '(:line-termination :cr :character-encoding :utf-8))
    (cl-json:decode-json f)))

(mapcar (let ((cnt 0))
          #'(lambda (post)
              (with-open-file (f (format nil "/tmp/archives/archive-~A.post" (incf cnt))
                                 :direction :output
                                 :if-exists :supersede
                                 :external-format (make-external-format :line-termination :unix))
                (format f *archives-template*
                        (cdr (assoc :title post))
                        ;; (cdr (assoc :labels post))
                        (substitute #\- #\/
                                    (subseq (cdr (assoc :url post)) 40 47))
                        (let ((author (cdr (assoc :author post))))
                          (if (string-equal author "dkochmanski")
                              "jackdaniel" author))
                        (remove #\Return (cdr (assoc :text post)))))))
        (cdar *json-posts*))

You may find a guide how to use the Sample Common Lisp Project template for your own project here. The clnet theme is inspired by the css in most of the common-lisp.net projects.

Best regards, Daniel

Bountysource Salt account

posted on 2016-03-04

We have set up an account on the bountysource to simplify the financial contributions to the project.

Additionally you may send money contributions with a PayPal account:

ECL 16.1.2 release

posted on 2016-02-29

We are happy to inform that the official ECL 16.1.2 release is available for download:

ECL Quarterly Volume III

posted on 2016-02-06

1 Preface

Dear all,

I'm proud to publish the third ECL Quarterly volume. I'm aware that it's a bit late, but very happy that delay didn't extend any further. Some exciting stuff is going on. First of all Embeddable Common-Lisp has been successfully ported to the Android platform. Additionally we have untested ports to NaCL and PNaCL (any volunteer?) and an alpha quality Android application - ecl-android (fully fledged swank-accessible Common Lisp running in Dalvik via JNI with all the goodies ECL provides).

I'm very happy to open this volume with a great guide written and contributed by Earl Ducaine – Stupid ECL tricks. This is set of a very useful hints for the development with ECL. We invite everyone to contribute to the repository located at Earl's GitHub repository. Moreover Angelo Rossi has finished his project embodying ECL on the embedded board SBC MIPS Creator CI20 and written very nice summary to share with us.

Third chapter will resolve around the Android port and how to build the ecl-android application.

I want to apology for this delay everyone, who waited for this volume. I'm quite busy lately and despite having some unfinished material I couldn't find time to polish it, so I've decided to skip it for the next volume. I'm very grateful to both Earl and Angelo for providing good material for a readers. I hope the next volume will be published very soon, or at least on schedule :-). Thank you for waiting.

We're testing now a new release of ECL (version number 16.1.2) and if everything goes fine it will be released on February 29th. If you are curious about the changes, you may skim the "Pending changes" section here.

*Please* send all the feedback to the mailing list or directly to me. It is really ensuring to know, that someone reads this. Also if you want to be published here please let me know. Thank you!

Daniel Kochmański ;; aka jackdaniel | TurtleWare
Poznań, Poland
February 2016

2 Stupid ECL tricks

Mostly these are half baked hacks. But hopefully they stimulate the imagination of real programmers by providing a glims of what ELC is cabable of.

2.1 Running ECL in gdb.

I've always had a total mental block when it comes to C pointers. It makes no sense to my brain that * indicates a variable is a pointer when used in a declaration, but retrieves a value when used as an operator. And an array of pointers to a character string makes total sense to me in words but char** str[] causes my mind to go blank. As a consequence any C code I write, or even look at too intently immediately blows up when compiled and run. A big inconvenience when embedding Lisp. Replacing the usual,

(setq inferior-lisp-program "ecl")


(setq inferior-lisp-program
      "gdb --eval-command=run --eval-command=quit --args ecl")

Will run ecl under gdb, which will provide you the normal gdb environment with c runtime errors, while throwing you into the lisp debugger for Lisp errors. Note that gdb by default breaks on SIGPWR and SIGXCPU which ecl uses for internal processing. So, you'll also want to add the following to your .gdbinit file.

handle SIGPWR nostop noprint
handle SIGXCPU nostop noprint

2.2 Embedding Swank in a c application.

Swank is a Lisp program that provides remote access to a Lisp instance. It started as client/server application layer in CMUCL and the Hemlock editor it ran. It's since been ported to most Lisps. Slime is the Emacs front-end client to Swank. Together the two tools provide a powerful Lisp development environment in Emacs. The easiest way to install Swank and Slime is simply to get it from quicklisp. See:


Swank and slime work in following way:

+----------+     launch ecl in                +--------------------+ 
| emacs    |---- process buffer, tell ------> | ecl process buffer |
+----------+     ecl to start swank           +-----+--------------+                         
   |                                                       |
   |                                         start swank server:
create slime                                 (swank-loader:init)
buffer                                       (swank:start-server)
   |                                                |
   |                                                |
  \/                                              \/
+--------------+      integrated      +--------------------------------+
| repl:        +<---- lisp repl   --->| swank server listening         |
| slime buffer |      interaction     | on some arbitrary              |
+--------------+                      | TCP/IP port e.g.               |
                                      | "Swank started at port: 46493" |
+--------------------------+                           |
| edit:                    +<--------------------------+
| buffer with Lisp source  |

To embed swank in a C application we need the application to launch Swank and then for Emacs to establish the connection to the swank server using slime-connect. Below is the C code that launches Swank.

Note, the following example is for a GNU/Linux type system. ecl needs to explicitly load load a shared library in order to access binary symbols such as C functions or C variables in the process, this is a hackish way of handling it since the library was already loaded when the applicaiton started, and could cause problems on platforms that put different constraints on loading shared libraries.

/* -*- mode: c;  -*-
   file: main.c

#include "app_main.h"
/* a.out wrapper for call into a shared library. */
int main() {
  return app_main();
/* -*- mode: c;  -*-
   file: app_main.h

#ifndef __APP_MAIN_H__
#define __APP_MAIN_H__

#include <ecl/ecl.h>

int app_main();

#endif /* APP_MAIN_H */

The following creates the shared library app_main used by both the C program and ECL for symbols. The embedded ECL code initializes the ECL environment and calls the Common Lisp load function to load a local Lisp file with the code to run swank.

/* -*- mode: c;  -*-
   file: app_main.c

#include <stdlib.h>
#include <math.h>
#include "app_main.h"

void run_swank();

/* TODO: Are embedded quotes really needed? */
char start_swank[] =

char* argv;
char** pargv;

int app_main() {
  argv = "app";
  pargv = &argv;

  cl_boot(1, pargv);

  /* Set up handler for Lisp errors to prevent buggy Lisp (an */
  /* imposibility, I know!) from killing the app. */
  const cl_env_ptr l_env = ecl_process_env();

  return 0;


void run_swank() {
  cl_object cl_start_swank_path = c_string_to_object(start_swank);
  cl_object cl_load =  ecl_make_symbol("LOAD","CL");
  cl_funcall(2, cl_load, cl_start_swank_path);

The following Lisp file, loaded by appmain, contains a couple of snippets of code I copied from the Emacs Slime client that launches the Swank server. When Swank launches it will print out the socket you can use to connect to it, e.g.

;; Swank started at port: 58252.

you can then connect to it in Emacs using Slime:

M-x slime-connect

;;; -*- mode: lisp ; syntax: ansi-common-lisp -*-

;; standard quicklisp init file, since with be launching ecl without ~/.eclrc
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
  (when (probe-file quicklisp-init)
    (load quicklisp-init)))

(when (probe-file  "/tmp/slime.2565")
  (delete-file "/tmp/slime.2565"))

 :verbose t)

(funcall (read-from-string "swank-loader:init"))
(funcall (read-from-string "swank:start-server")

A quick and dirty script file to build a shared library.

# -*- mode: bash;  -*-

rm -f *.o *.so app

export libs="-lm"

# Note, the -Wl,-R flags will make our shared library available to the
# executable app from the location that it was compiled, rather than
# having to be installed globably or adding the build path to

export ldflags="-L. -Wl,-R -Wl,."
export cflags="-DGC_LINUX_THREADS -D_REENTRANT -fPIC  -g -pipe -Wall"

gcc $cflags -c app_main.c
gcc -shared -Wl,-soname,libapp_main.so $ldflags -lecl -o libapp_main.so *o $libs
gcc main.c $cflags $ldflags -lapp_main -lecl -o app

To build and run

$ ./build_app.sh
$ ./app

2.3 Troubleshooting compilation problems with ffi:c-inline

ECL provide a facility for embedding C code directly in Lisp code like the following:

(defun c-sin (x)
  (ffi:clines "#include \"ecl/ecl.h\"")
  ;; Whoops!  mathh.h should be math.h
  (ffi:clines "#include <mathh.h>")
  (ffi:clines  "#include \"app_main.h\"")
  (ffi:c-inline (x) (:double) :double "{
@(return 0)= sin(#0);
}" :one-liner nil))

To use this function you need to compile the defun. When you issue the explicit compile,

(compile 'c-sin)

ECL will invoke your underlying C compiler. However, C syntax and header include errors, like we included in the above example, will cause compilation to fail. Unfortunately, ECL doesn't pass along the compilers output. You'll get something like the following:

;;; OPTIMIZE levels: Safety=2, Space=0, Speed=3, Debug=3
;;; End of Pass 1.
;;; Internal error:
;;;   ** Error code 1 when executing
;;; (RUN-PROGRAM "gcc" ("-I." "-I/usr/local/include/" "-D_GNU_SOURCE" "-D_FILE_OFFSET_BITS=64" "-g" "-O2" "-fPIC" "-D_THREAD_SAFE" "-Dlinux" "-O2" "-c" "/tmp/ecl001QoKf80.c" "-o" "/tmp/ecl001QoKf80.o"))

if you try to recreate the error by invoking the implied shell command:

$ gcc -I. -I/usr/local/include/ -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 \
    -g -O2 -fPIC -D_THREAD_SAFE -Dlinux -O2 -c /tmp/ecl001QoKf8.c \
    -o /tmp/ecl001QoKf80.o

You'll get the error:

gcc: error: /tmp/ecl001QoKf80.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.

Because ECL has already cleaned it from /tmp.

But, ECL has a special variable, compiler::*delete-files* that controls cleaning up c output files. By setting it to nil, (setf compiler::*delete-files* nil) you can troubleshoot compilation errors. Re-running above gcc command on from the Unix shell gives us the following:

In file included from /tmp/ecl001QoKf80.c:6:0:
/tmp/ecl001QoKf80.eclh:8:19: fatal error: mathh.h: No such file or directory
#include <mathh.h>
compilation terminated.

2.4 Cache Files

Swank and ECL's embedded C in Lisp facility seem to have some issues with caching where compiled C snippets and a Swank images don't get refreshed when they should (at least on GNU/Linux). If you start noticing strange issues with changes to ffi:c-inline not taking effect or Swank having the wrong image, try deleting the following cache files:

rm -rf ~/.cache/common-lisp/ecl-15.2.21-ee989b97-linux-x64
rm -rf ~/.slime

/Earl Ducaine/

3 ADIO project with ECL and a bunch of electronic devices

3.1 Synopsis

This document describe my experience with the SBC MIPS Creator CI20 from Imagination and ecl (Embeddable Common Lisp) interpreter/compiler. The goal is to program a distributed diagnostic/supervision system for DVB transmitter equipment's.


3.2 The experience

The Creator CI20 comes with a linux Debian Jessie distro preinstalled on its 8 GB internal flash drive. After upgrading it and install some packages I was ready to deploy ECL. The version choosen for the prototype was the 16.1.0, compilation was really straightforward (I modified a bash script written for SBCL to download, compile and install the git version of ECL) and after 20/30 minutes of code crunching I've got a shiny working ECL. I must say that problems didn't come from ECL itself, but from other packages. As I installed quicklisp soon came troubles, for example: slime does not add the correct architecture to the *features* variable and some other minor problems with iolib too. The Lisp community is the most active and helpful among those living on irc and with their help I've got fixed and ready in short time. The project I'm currently working on is divided in two main areas: the hardware design and test phase and the software one. The first, at this stage, contemplates the design and production of a limited number of ADIO boards (designed by me). More basic an ADIO board is an insulated analog and digital i/o board, it is interfaced with the Creator CI20 via the i2c bus and externally powered. The software part is the more vast part of the project. The idea is to provide a system that is able to interface to external users and feed them with sensors data regarding the state of the DVB apparatuses plus every ADIO + Creator CI20 hold the configuration for the apparatus they're going to monitor/supervise. In addition to that the system should be responsible to warn about anomalies and take some pre-emptive actions just to not make the whole thing blow up. So the Customer was facing a dilemma: Arduino + RS485 and a lot of amulets or rely on a modern and saner idea? My advice for the landing party was: "why do not use a cheap sbc (80 Eur) with an embedded OS powered by a Lisp interpreter?" here are the benefits:

  • ethernet based communication hardware layer is proven to be cost effective and reliable, plus every oses and cheap sbcs support it;
  • an embedded os, in most cases, is better than start a baremetal application: e.g. tcp/ip connections and standard network services like http, ftp, ssh, samba and snmp require a lot time to program, debug and test even if you use on the shelf solutions. on the contrary with an embedded os you've got these things already coded and ready to use, shortening the deployment time of about one order of magnitude (in man hours);
  • using lisp is useful for almost three aspects: portability, remote debugging and last but not least the possibility to create compiled code for mips;
  • possibility to expand hardware and software via external modules.


So lisp at last, the first prototype is now running a test code which provide basic functionalities like: a general i2c library to access the linux i2c device via /dev/i2c-*, a simple i2c gpio expander library for the mcp23017 and another library to cope with hitachi compatible 16x2 lcd and a simple looping program which displays some random characters on it. next improvements are ready to be tested too, like a machine learning algorithm to detect anomalies and so on.

3.3 Last words

At this stage, ECL was truly the right choice: first of all the time to develop this small set of libraries was incredibly short compared to c/c++ due to that now I've got a test prototype running on my desk happily showing on a 16x2 display some test patterns in just about one month circa from Lisp environment setup. I must thank the friend PJB for his invaluable help and patience and jackdaniel for his work on ECL and the patience too.

/Angelo Rossi/

4 ECL Android port

I'm happy to announce, that the android port has been merged to the develop branch of the main ECL repository. This work is mainly based on Sylvain Ageneau's work. I've also adjusted and incorporated NaCL and pNaCL patches but these are not tested (yet).

To use ECL fully on the android, user want's to have an application which may be started from within the phone, not in the terminal. That's why I've created a separate project ECL Android which is based on Sylvain's integration example code (further adjusted by Evrim Ulu). For now it is in alpha state and it's discouraged to use it in applications meant for the end user.

The following sections will explain current compilation process of the ECL and how to build ECL Android application and deploy it to the computer.

4.1 Building ECL

ECL may be built as a library. This great feature allows us to dynamically link ECL with arbitrary applications capable of using shared objects. We take advantage of this and the fact, that the android platform allows us to embed shared objects with the .apk and use it with the Java Native Interface (JNI).

To use libecl we first need to cross compile it for the target platform. There is some work pending to simplify the cross compilation process, but for a time being you have to build a host ECL compiler and after that the target library.

Before that, however, you have to build a proper android toolchain. If you don't have a prebuilt one you may use android NDK scripts. Both Android SDK and NDK are a prerequisites and installing them is an exercise for the reader.

export PLATFORM_PREFIX=/opt/toolchains/android-ndk9/
export NDK_PATH=/opt/android-ndk/
export NDK_PLATFORM=android-12

/opt/android-ndk/build/tools/make-standalone-toolchain.sh \
    --platform=${NDK_PLATFORM} \
    --install-dir=${PLATFORM_PREFIX} \


Now you have to build the host compiler1 and the final library for the android. Note, that using the preexisting ECL binary could work if you it is a 32 bit installation with disabled longdouble.

./configure ABI=32 CFLAGS="-m32 -g -O2" LDFLAGS="-m32 -g -O2" \
            --disable-longdouble \
            --prefix=`pwd`/ecl-android-host \

make && make install
export ECL_TO_RUN=`pwd`/ecl-android-host/bin/ecl
rm -r build

./configure --host=arm-linux-androideabi \
            --prefix=`pwd`/ecl-android-target \
            --with-cross-config=`pwd`/src/util/android.cross_config \

# You have to adjust build/cross_config to your settings (especially
# set ECL_TO_RUN to your host ecl)
make && make install

You should have libecl.so and the other necessary files in the ecl-android/ directory. Some pre-compiled modules are located in the ecl-android/lib/ecl-16.1.0 directory. You will want them on the target system.

4.2 ECL Android (application/service)

This application isn't stable yet and documentation is still rather scarce. API isn't stable and it is discouraged to base any work on it for now. It is provided on terms of AGPL-3.0+ license, however alternative licensing is possible.

First thing to do is to clone the repository and adjust the project configuration to your local setup.

git clone https://gitlab.common-lisp.net/ecl/ecl-android.git
cd ecl-android
# update the project (sets sdk path and the other android "magic")
android update project -t android-10 -p .
# create symlinks (sets ECL directories). For instance:
ln -s ../ecl-android-target ecl-android
ln -s ecl-android/lib/ecl-*.*.* ecl-libdir

Now it's worth to explain how the application works on the target platform. We put the library libecl.so in the apk file, but rest of the module and other ECL-related files are packed as a resource in the assets/lisp/ directory at which *default-pathname-defaults* points to.

Initialization is performed by assets/lisp/etc/init.lisp file, which loads the user.lisp. The latter contains some sample code loading swank (if it's put in the home/slime-2.14/ directory – swank is not bundled with the repository) and defines auxiliary functions: #'get-quicklisp, #'start-swank and #'stop-swank.

Function #'get-quicklisp will download and install the Quicklisp. It will also replace it's bundled compression tools with a prebuilt one (it is essential for performance – GCC produces faster code then the byte compiler).

Before you build the ecl-android you may want to copy some files for further use and edit the initialization script:

mkdir assets/lisp/home
cp -rf ~/things/slime-2.14 assets/lisp/home
cp ~/things/my-awesome-lisp-app/awesome.lisp assets/lisp/
emacs assets/etc/user.lisp

When you are ready you may build and deploy the application on the phone:

ant debug install

ECL Android launcher should appear on your phone.



If your host platform is darwin, then the host compiler should be built with the Apple's GCC (not the GCC from Macports). Using the MacPort command:

sudo port select --set gcc none

Hint provided by Pascal J. Bourguignon.

ECL Android 0.0.1

posted on 2015-11-07

ECL Android 0.0.1 has been released. Keep in mind that this is an alpha quality software for the preview purposes. Prebuilt apk is based on the ECL repository, not the 16.0.0 release. More in-depth information will be provided in the upcoming ECL Quarterly. Please send a feedback to the mailing list or directly to me.

Enjoy :-).