Embeddable Common-Lisp

Recent Content

ECL Quarterly Volume II

posted on 2015-09-16

1 Preface

Hello!

Three months has passed (plus a few days - 25 actually) and a new volume of ECL Quarterly arises — as promised.

Highlights of this volume:

Embeddable Common-Lisp 16.0.0 (new release)
A few words about new ECL release
ECL future
Things we consider implementing in the future
On Lisp (not a book!)
Three short essays about unique ECL C/C++ inlining feature (in the form of a tutorial), the ANSI CL specification in the context of LET, FLET and LABELS block, and non ANSI compliant extensions of the reader syntax

I've recently started my own company – TurtleWare. There is a shameless plug at the end of the volume with information that I'm open for consultancy work. Everyone is free to skip the last section.

I also want to remind everyone that the main project site (containing all further resource pointers) is located at:

https://www.common-lisp.net/project/ecl

If you have any suggestions regarding the Quarterly – you like it? or maybe you hate it? – please tell me either by writing to the mailing list or by writing an e-mail directly to me. If you want your own article or tutorial to be published in this e-zin – please drop me an e-mail (daniel[at]turtleware.eu). Thank you!


Daniel Kochmański ;; aka jackdaniel
Poznań, Poland
August 2015

2 Sourceforge migration

Until now we had a few remaining resources kept on Sourceforge:

  • Mailing lists
  • Release source archive
  • Announcements / blog channel

All these have now been moved to the common-lisp.net servers (https://www.common-lisp.net/project/ecl). All members of the mailing list did get an invitation e-mail to the new one. The obsolete list will be shut down at September 1st. Mailing list archives are already imported, so we won't lost any accumulated threads.

I owe a big thanks to Erik Huelsmann for help with the migration and for his constant work on the common-lisp.net platform. Without him, such migration would be somewhere between a nightmare and the impossible. Thanks!

3 Embeddable Common-Lisp 16.0.0 (new release)

3.1 Announcement

We are happy to announce that the new ECL release has been published. Version 16.0.0 has various improvements over the previous one mostly focused on the interoperability with the existing CL libraries ecosystem, pushing forward ANSI compliance and increasing portability across various platforms.

Significant (ongoing) efforts have been made to improve general code quality – removing dead blocks and interfaces, untabifying sources and refactoring parts of the code base. Also we've refreshed the testing framework. Documentation has been verified and updated.

We owe big "thank you" to many people who helped us with understanding the ANSI spec and pointing us in the right direction when in doubt, and to those who bothered to report issues and provide test cases. These discussions took place mostly on IRC and via the "Issues" tab on the GitLab platform. Unfortunately I don't remember all the nicks and names, so it would be unfair to list only a few I remember.

People who have contributed to this release are (alphabetically): Daniel Kochmański, Philipp Mark, Roger Sen and Evrim Ulu.

Without further ado – the changes:

3.1.1 Known issues

  • In Windows, ECL comes with the bytecode compiler by default, because C compilers are normally not available. Unfortunately several libraries out there are not prepared for this. If you plan to use quicklisp and have a C compiler accessible to ECL, you may use (ext:install-c-compiler) to switch back to the Lisp-to-C compiler.
  • In order to test a package, programmer has to install ECL on a desired destination (specified with "–prefix" parameter given to the configure script).

3.1.2 API changes

  • There is no UFFI nickname for the FFI package - we piggyback on cffi-uffi-compat for UFFI dependent systems (our UFFI wasn't version 2.0 compatible and there were problems with ADSF dependencies on UFFI - it wasn't a system)
  • CLOS has the new nickname "MOP"
  • The new ext:run-program :error argument can automatically create a separate stream if provided with the :stream keyword. The external-process structure also has a new field to hold that stream.
  • ext:run-program accepts new arguments - :if-input-does-not-exist, :if-error-exists and :external-format
  • ext:system no longer binds ** standard-outputstandard-input* and *and now ignores input and output (use ext:run-program for more control)
  • methods can be specialized on both single-float and double-float (built-in classes were added for them)
  • LET/FLET/LABELS will signal an error if the parameter of the same name appears multiple times
  • lambda lists with repeated required parameters name are considered invalid
  • deprecated configure options "–with-system-boehm=x" and "–enable-slow-config" removed

3.1.3 Enhancements

  • Verification if manual is up-to-date, providing corrections for outdated parts
  • Documentation is now included in the main repository under the top-level directory `doc'
  • Update libffi to version 3.2.1
  • Update asdf to version 3.1.5.4
  • Update Boehm-Demers-Weiser garbage collector to version 7.4.2
  • Pathname string-parts internal representation is now character, not base-char
  • Dead code removal, tabulators were replaced by spaces
  • Better quality of generated code (explicit casting when necessary)

3.1.4 Issues fixed

  • Various fixes of bogus declarations
  • Remove deprecated GC calls
  • ROTATEF, SHIFTF, PSETF reworked to conform to the ANSI standard. Places were handled improperly in regard of multiple values.
  • Improved Unicode support in character handling
  • Format handles floats and exponentials correctly (major format rework)
  • Stack limits refinements and cleanup, inconsistency and bug fixes
  • Duplicate large block deallocation with GMP 6.0.0a fixed
  • ECL builds on OpenBSD with threads enabled
  • Closures put in mapcar work as expected in both compiled and interpreted code
  • Improved readtable-case handling (:invert and character literals now conform)
  • Library initialization functions have unique names - no risk of clashing symbol names in object files
  • Format float bug fixed, when width and fdigits were not set, but k was
  • `logical-pathname-translations' now signals an error if logical pathname wasn't defined yet, to conform with ANSI (it used to return NIL)
  • Wildcards in logical pathname translations are replaced correctly
  • Regression testing framework and unit tests cleanup
  • deftype ANSI conformity fix (deftype accepts macro labda-lists)
  • ECL built with MSVC doesn't crash when Control-C is pressed
  • Other minor tweaks

3.2 New version numbering scheme

The data-based version numbering scheme is ceased from this release. From now on all the releases will follow the rules described in https://autotools.io/libtool/version.html.

Basically release numbers will follow the scheme X.Y.Z, where X increases when the API changes, Y if interfaces are added (but not removed) or changed in mostly backward compatible and Z is the "patch level" part, which changes for fixes not affecting API.

All new releases are considered ABI incompatible, so the sources have to be recompiled with each new release (this is the default when using ASDF).

4 ECL future

There is plenty of work to be done and many ideas to realize. We can't do it all at the same time and all the further points are rather loose ideas than plans for the next release. Just a food for thought.

  • Dynamic Foreign Function Interface

    ECL has a decent DFFI for a small set of platforms (x86, x8664 and PPC32) hand coded with assembly. ECL on other platforms doesn't support DFFI with the bytecode compiler and requires the C backend (function calls are inlined in the generated C code). On the other hand, we already depend on libffi with regard to closures. We can use it for DFFI as well – it has a really impressive list of supported platforms and using it for that will be a big win.

  • Cross-compilation framework

    We are able to cross-compile applications with host the ECL and GCC toolchain, but this is somewhat painful and we have no convenient interface for doing that at run-time from the REPL user perspective. Compilation flags and parameters are stored as global variables and API for cross compilation is documented.

    Juan Jose Garcia Ripoll started to make the target machine description separate from the host implementation. On the other hand Sylvain Ageneau created the cross-cmp compiler package. I haven't investigated either work in great detail, but I see an opportunity to combine both these works for greater support for the cross comilation. Sylvain's repository (https://github.com/ageneau/ecl-android) seems to be a place full of real treasures worth merging back to the main repository.

    Cross-compiling applications for Android to native code is definitely something I could put to good use ;-)

  • Documentation improvements

    Many exported functions and interfaces are undocumented or existing documentation is scarce. More usage examples and verification of "examples/" src directory would be a good thing too. The official manual layout could be refined to allow html "one-page" builds (source is in DocBook).

  • C compiler on-board

    Providing an optional C compiler package to be bundled as fallback with ECL would be a nice thing to have. Both GCC and TCC can be built as libraries. Didn't investigate it much yet.

  • More platforms

    The repository mentioned above ("ecl-android") isn't only about Android support. It has iOS and NaCL ports too. Creating a Minix port would be rather easy – only the bdwgc port is missing, and porting it would be as easy as adding two defines since it's advertised as "API compatible" with NetBSD which is already supported.

  • Introspection facilities

    ECL has a few problems when it comes to introspection. While it works nicely with SLIME when it comes to the user-defined functions, many functions defined at ECL build time don't have any hints. It's even worse when it comes to macros – no hints at all. This issues have to be addressed and fixed by fixing ECL interfaces and improving SLIME integration. It will also create a good background for writing simplified (not emacs-dependant) IDE (read below).

  • Graphical integrated development environment

    This should work via Swank backend and must be easy to grasp for a programming beginners. I truly believe that Emacs (while being editor of my choice) doesn't have to be the only solution for CL programmers who use free implementations. It's a big entrance barrier which people often fail to conquer.

    I imagine it being written in EQL and supporting Swank, so any implementation can be attached. Some ideas from Bret Victor's "Learnable Programming" might be incorporated, especially contextual information in the case when documentation is in a specific format (for instance providing description for each parameter).

    Decent support from line-edit would also be nice for quick hacking from the console – auto-completion and hints are a killer feature here. Imagine BPython (it's ncurses).

  • Improving existing interfaces

    Gray streams and MOP has a few remaining issues (implementation isn't fully conforming). For instance: Gray Streams #'close isn't a method but function, MOP's complex methods are terribly slow or even non-functional. This issues has to be addressed.

  • Compiler thread-safety

    ECL compiler isn't thread safe (at least that's what I've been told). Verifying and fixing it is one of the things to be done.

  • Central Document Repository

    We plan to incorporate suitable extensions into ECL. Starting from CDR-14 (we already have CDR-5). It's a nice place for quasi-standard proposals and we think it's a good way to achieve consensus among various CL implementations about a common API for future extensions to allow portability.

    We know many facilities are practically standardized by portability libraries (ASDF, Bordeaux Threads, CFFI etc.), but future extensions might have an actual specification – we all know warts of "reference implementation" standards *cough-python-cough*.

    I would love to see "Extensible Sequences" in the Central Document Repository.

5 On Lisp (not a book!)

5.1 Inlining C/C++ code in Common-Lisp with ECL

Like many compilers ECL offers a facility to inline it's assembly in the source code for a convenience and performance reasons. ECL's assembler is C/C++°.

To achieve inlining, ECL offers three constructs.

(ffi:clines                           &body strings)
(ffi:c-inline args arg-types ret-type &body others)
(ffi:c-progn  args                    &body forms-and-strings)

The most basic is ffi:clines, which allows you to just drop in some C/C++ code (for instance include a header you need for further use).

ffi:c-inline is much more useful, allowing you to pass values from Lisp to the block and receive output in return. This construct allows returning multiple values declared in the third clause. You may also declare whenever it has side effects and if it is one-liner°° (more details in ECL manual). Short example:

(defun ctrunc (number divisor)
  (ffi:c-inline (number divisor) (:int :int) (values :int :int) "{
    int num = #0, div = #1;
    @(return 0) = num/div;
    @(return 1) = num%div;
}"))

It's worth mentioning that the number and divisor types are checked when the function #'ctrunc is called and if incorrect – proper lisp condition is signaled.

ffi:c-progn is the last construct. It allows you to intermix both C and lisp code – it doesn't return any value so it is called purely for it's side-effects (assigning variable with computed result for instance). Don't forget to declare variable types! If you don't, results might be surprising.

To illustrate potential gain of using inlined C language we'll take the trivial Fibbonachi algorithm and benchmark ECL against itself. It's not a proper benchmark, nor a good implementation, but both implementations are comparable and this should prove a point, that ECL might be quite fast when we want it to be°°°.

(defun fib-1 (n)
  "Borrowed from http://www.cliki.net/fibonacci"
  (loop for f1 = 0 then f2
     and f2 = 1 then (+ f1 f2)
     repeat n finally (return f1)))

(defun fib-2 (n)
  (let ((f1 0) (f2 1) (n n))
    (declare (:int f1 f2 n))
    (ffi:c-progn (n f1 f2) "
     int aux = 0;
     for( ; #0>0; #0--, aux=#1) {
         #1 = #2;
         #2 = aux + #2;
     }")
    f1))

(defun bench ()
  (compile 'fib-1)
  (compile 'fib-2)
  (prog1 nil
    (print "Common Lisp:")
    (time (dotimes (x 10000000) (fib-1 20)))
    (print "Common Lisp with inlined C:")
    (time (dotimes (x 10000000) (fib-2 20)))))

On my computer This yields:

"Common Lisp:" real time : 9.583 secs run time : 9.596 secs gc count : 1 times consed : 271531840 bytes

"Common Lisp with inlined C:" real time : 0.657 secs run time : 0.656 secs gc count : 1 times consed : 271567088 bytes

So, as we can see, the speed improvement is pretty decent. Proper declarations would speed up #'fib-1 a little, but the C version will still be faster (note the fact that we are comparing ECL with ECL, other implementations might theoretically outperform ECL's C version using provided #'fib-1 definition).

For more information please consult manual: https://common-lisp.net/project/ecl/static/manual/ch28.html

° None of these constructs will work with the bytecode compiler (it will signal an error).

°° C doesn't treat each statement as a valid R-value. By "one-liner" we mean something, what might be used as such.

°°° We can't say it's fast Common Lisp, since it's no longer Common Lisp – if we use this technique we're using ECL and it's not portable by any means.

5.2 Case on LET / FLET / LABELS (aka Nasal Demons Reborn)

Consider an example

(let ((x 'foo)
      (x 'bar))
  x)

what symbol does this form evaluate to: **BARFOO* or *?

Well, it's not defined – in CCL and CLISP it is **FOOBAR*, ECL and ABCL make it *, while SBCL signals an error. It isn't clear what should happen because it's not specified in the spec. I think that the SBCL approach is the most sane. I can hardly imagine a programmer doing that, not as a typo, but as a conscious decision°.

The same argument applies to **FLET* FLET*FLET* (except the side-effect thingy – all but one definition of the same name can be optimized out) – which function definition is the valid one? It isn't specified – note that the operator name isn't *but *.

The last case – the most dangerous and the least comprehensible. *LABELS* allows mutual recursion and it's far less obvious what would happen even from an implementation point of view. If we write a code:

(labels ((function1 ()
           (im-dangerous 2))
         (im-dangerous (x)
           (format t "FIRST X=~A~%" x)
           (if (zerop x)
               'first
               (im-dangerous (1- x))))
         (function2 ()
           (im-dangerous 2))
         (im-dangerous (x)
           (format t "SECOND X=~A~%" x)
           (if (zerop x)
               'second
               (im-dangerous (1- x))))
         (function3 ()
           (im-dangerous 2)))
  (list (function1)
        (function2)
        (function3)
        (im-dangerous 1)))

CLISP take the first definition of ** IM-DANGEROUSIM-DANGEROUS*, while others the second one. SBCL on the other hand behaves inconsistently – inside function definitions references to *are bound to the first definition, while references from inner block are bound to the second one.

It is important to say that the behavior in these situations is undefined by the spec and each implementation is free to do what it considers most reasonable, easiest to implement, or best to optimize – and none of these is wrong (CCL allows all constructs, but issues a style warning for each – it is the only implementation which does that – bravo).

Curious minds may find the following article amusing (warning, C code involved): http://blogs.msdn.com/b/oldnewthing/archive/2014/06/27/10537746.aspx.

My conclusion is as follows: you can't rely on ** LABELS* FLETLET*, *and *constructs if multiple definitions of the same name exist – they are ambiguous and should be considered being an error. The new release of ECL treats them that way°°.

° Multiple variables can't be optimized out, because the initialization form might have side effects

(let ((x (side-effects1))
      (x (side-effects2)))
  x)

;; More deterministic form:
(let ((x (progn (side-effects1)
                (side-effects2))))
  x)

°° ANSI defines LAMBDA-LIST in terms of LET*, so repeating the parameters of the same name is theoretically correct. There is no valid use-case for two required parameters of the same name (there is no initialization form) though, so ECL signals an error on such situation.

5.3 Case for portability and against unusual reader syntax

Why we shouldn't use all these implementation-specific niceties

Lately, Zach Bane posted a nifty trick for accessing symbols from packages you're not in, literally:

foo::(list 'these 'symbols 'are 'from "foo" 'package)

It's nice, intuitive, practical… and not compliant. It would be a minor problem, if it weren't a syntax hack, which is hard to implement portably as a library (at least I don't see any elegant solution). In my opinion it requires digging into implementation innards and tweaking the reader to support it. Manipulating strings before they are actually read is an option too. Also – unlike the metaobject protocol, gray streams or extensible sequences – it doesn't bring anything new to the table.

"But hey! It's just syntactic sugar for REPL interaction! Don't be such a grumpy guy!"

OK, fine, if you promise me that this "syntactic sugar" won't land in any library *ever* – it's advertised after all. Nobody knows who will pick up this hint. And if critical mass will prevail, then this dubious syntactic sugar will become de-facto standard, so other implementations will be forced to implement it, or fail to load libraries using it.

Such a situation happened once. CLtL had an example of the ** UNTIL* FORLOOP* usage, where a *clause landed after an *clause, which isn't ANSI compliant. The fact that it was supported by a few implementations lead to the situation imagined above.

If you really want some syntactic sugar for using other packages locally – I propose a little uglier, yet portable, solution:

(defun sharp-l (stream char subchar)
  (declare (ignore char subchar))
  (let ((*package* (find-package (read stream))))
    (read stream nil nil t)))

(set-dispatch-macro-character #\# #\l #'sharp-l)

#l foo (list 'these 'symbols 'are 'from "foo" 'package)

It's only five lines of code and works everywhere (unless someone binds #l to his own reader macro). If someone wants to be a little fancier, then he may mimic the SLIME prompt in his syntax:

(defun sharp-x (stream char subchar)
  (declare (ignore char subchar))
  (let ((*readtable* (copy-readtable))
        (right #\))
        (rpar #\>))
    (set-macro-character rpar (get-macro-character right))
    (let ((*package* (find-package (read stream))))
      (peek-char #\> stream)
      (read-char stream)
      (read stream nil nil t))))

(set-dispatch-macro-character #\# #\[ #'sharp-x)

#[foo> (list 'these 'symbols 'are 'from "foo" 'package)

And, honestly I really like proposed syntax, but for my taste it's totally unportable and harmful for reasons I've mentioned above.

6 Advertisement

Here comes the shameless plug – I've recently (officially) launched the company named TurtleWare. I am open for consultancy. If anyone:

  • wants to pay for a dreamed ECL feature,
  • prioritizes his own issues with implementation,
  • likes to pay for support or system maintenance,
  • has some Lisp and/or embedded systems work to do

then reach me with further details at: hello[at]turtleware.eu.

ECL Quarterly Volume I

posted on 2015-05-01

1 Preface

Hello everyone!

From time to time there are misleading signals, that ECL is "floating dead". This FUD is mostly spread by IRC newcomers with little orientation, but it's good! This is a signal, that this particular implementation of Common Lisp needs to make little more noise, instead of silently fixing ** stuffstuff* (however it obviously needs a bit of work on *as well ;-)) .

Some projects make new release each month to prevent such disinformation. It solves this particular problem, but introduces a new one - it's a bit fuzzy, if a new release introduces any significant improvements, or if it's just a release number bump. Yet, updating it requires recompilation of personal projects, which depending on machine might take a while and is surely a bit annoying.

This is where ECL Quarterly idea has born. To show a bit activity from time to time - post will be published every three months. I want to make it an e-zine containing info about development, tutorials, comparisons, benchmarks and anything, what is at least slightly related to ECL. Everyone is welcome to contribute - such materials will be also published on wiki, added to git tree, and if appropriate - incorporated in documentation. If you have such material, don't hesitate to contact me at dkochmanski[at]turtle-solutions[dot]eu.

This chapter highlights:

  • Changes, development status etc.
  • Contributing to ECL

If you have any suggestions regarding ECL Quarterly, you like it, or maybe hate it - please tell me either by commenting this post, or writing an e-mail. Thank you!


Daniel Kochmański
Poznań, Poland
May 2015

2 Changes and development status

After 15.3.7 release there are a few changes going on. We are now hosted at gitlab.com (however it would be nice to move to gitlab.common-lisp.net, just /not now/), a few wiki entries are added, official site is https://common-lisp.net/project/ecl/. Domain embeddable-common-lisp.net is already bought (not configured properly yet).

Until now we have updated libraries, which we depend on. Namely:

  • libffi to version 3.2.1,
  • asdf to version 3.1.4,
  • bdwgc to version 7.4.2.

There was also update of GMP to more recent version, but we won't include it - yes, we are staying LGPLv2, to remain GPLv2 compatible. All changes were tested on Windows with MSVC2012.

For now we have also reworked ROTATEF, SHIFTF and PSETF macros, to conform ANSI standard in regard to places (multiple values weren't held properly).

Experimental branch contains Android (as name suggests - experimental) port, which is preview only for now (this branch is LGPLv3, because included GMP enforces it, and doesn't build on Windows).

Loads of things are to be done, and we're slowly making progress. After all we won't change release versioning scheme. No ETA for next release, but I assure you - if you contribute, it will be sooner ;-). What leads us to the next part.

3 Contributing to ECL

3.1 Documentation, tutorials, wiki, others

You can contribute in numerous ways. One way is to code and fix bugs - and it's very important part. But equally important is to find bugs and report them, so developers can improve codebase. Feature requests are also welcome (however we are focused now on fixing stuff rather than adding functionality, so even good proposition might wait in queue behind something maybe less exciting).

Testing on various platform and architectures is essential. Thanks to Anton Vodonosov and cl-test-grid it is as easy as setting environment, tweaking two configuration files and invoking ./run-agent.sh. Run may take a while (depending on computer) and is limited to operating systems and architectures supported by Clozure Common Lisp.

If ECL doesn't build, crashes, or works non-conforming to specification, please do report at https://gitlab.com/embeddable-common-lisp/ecl/issues. It requires account on GitLab, but setting it up is free. I'm still struggling to produce some time and move all tickets from https://sourceforge.net/p/ecls/_list/tickets to above mentioned issue tracking site - volunteers are more then welcome to do it.

If you encounter problem, please write to mailing list. There are many kind souls there more then willing to help - and usually they do. On the other hand, if someone asks for help and you know the answer - act! :-)

Wiki is a place, where many resources are gathered. It is incomplete and barely usable, because I don't have time to improve it. If you successfully build ECL on Android - great! Please add page to wiki, so others may reproduce your work! Do you have some nifty idea, and you believe it is worth to keep it there - do it. See a typo? Bug? Outdated information? You know what to do! You spot some nice blog post about ECL? Please share with others - wiki is best place to do so. You did successful project using ECL? Share this information! All this, and so much more, may be done here: https://gitlab.com/embeddable-common-lisp/ecl/wikis/home.

You can also write something for ECL Quarterly (my e-mail is mentioned in preface).

3.2 Source code contributions

Development takes place on git repository located at git@gitlab.com:embeddable-common-lisp/ecl.git.

If you want your code in Embeddable Common-Lisp project, please send a patch to mailing list with additional tag [PATCH] in subject. Generally we want to follow convention used by U-Boot development team (http://www.denx.de/wiki/U-Boot/Patches), which borrows a lot from Linux policy. Please read this guide, it's really worthwhile reading. If you submit a significant change, please report it in CHANGELOG located in top directory.

Basically, if you want to contribute to code, you have at least two choices. You may improve C code (which is probably less interesting for lispers). Most of the sources are written in lisp however, and require no knowledge about C. Problems vary from fairly easy even for seasoned developer to hard conceptual riddles, so everyone can find something interesting for himself.. Please consult appropriate directories under src/ sub-directory (i.e. src/lsp, src/clos etc.) - it's pure lisp! And it's fun to hack. Improving both C and Lisp sources might be a great learning experience. To figure what's wrong requires often getting your hands dirty, and then cleaning proposed solution. This, connected with peer review, might be a next step to become a better programmer.

There is also a third part, which is tiresome (for me at least) - improving build system - it is buggy and inconsistent. First person who will fix this has granted a free dinner when we meet in person. Remember however, that we support many operating systems, so it might be tricky to do it properly without introducing *new* problems.

If you are a library or application developer, _please_ test against as many implementations as possible - it's hard, it takes time, but in my humble opinion it is essential for CL ecosystem. Each implementation has it's weak and strong sides, and you never know, when you'll need one, or who and in what purpose is using your code :-).

ECL Quarterly - Volume I

posted on 2015-05

ECL Quarterly - Volume I

Preface

Hello everyone!

From time to time there are misleading signals, that ECL is "floating dead". This FUD is mostly spread by IRC newcomers with little orientation, but it's good! This is a signal, that this particular implementation of Common Lisp needs to make a bit more noise, instead of silently fixing stuff (however it obviously needs a bit of work on stuff as well ;-)).

Some projects make new release each month to prevent such disinformation. It solves this particular problem, but introduces a new one - it's a bit fuzzy, if a new release introduces any significant improvements, or if it's just a release number bump. Yet, updating it requires recompilation of personal projects, which depending on machine might take a while and is surely a bit annoying.

This is how the ECL Quarterly idea was born. To show a bit activity from time to time - post will be published every three months. I want to make it an e-zine containing info about development, tutorials, comparisons, benchmarks and anything what is at least slightly related to ECL. Everyone is welcome to contribute - such material will also be published on wiki, added to git tree, and if appropriate - incorporated in documentation. If you have such material, don't hesitate to contact me at dkochmanski[at]turtle-solutions[dot]eu.

This chapter highlights:

  • Changes, development status etc.
  • Contributing to ECL

If you have any suggestions regarding ECL Quarterly, you like it, or maybe hate it - please tell me either by commenting this post, or writing an e-mail. Thank you!

Daniel Kochmaski Pozna, Poland May 2015

Changes and development status

After the 15.3.7 release there are a few changes going on. We are now hosted at gitlab.com (however it would be nice to move to gitlab.common-lisp.net, just not now), a few wiki entries are added, official site is https://common-lisp.net/project/ecl/. Domain embeddable-common-lisp.net is already bought (not configured properly yet).

Until now we have updated libraries that we depend on. Namely:

  • libffi to version 3.2.1,
  • asdf to version 3.1.4,
  • bdwgc to version 7.4.2.

There was also an update of GMP to more recent version, but we won't include it - yes, we are staying LGPLv2, to remain GPLv2 compatible. All changes were tested on Windows with MSVC2012.

For now we have also reworked the ROTATEF, SHIFTF and PSETF macros, to conform ANSI standard in regard to places (multiple values weren't held properly).

The experimental branch contains Android (as name suggests - experimental) port, which is preview only for now (this branch is LGPLv3, because included GMP enforces it, and doesn't build on Windows).

Loads of things are to be done, and we're slowly making progress. After all we won't change release versioning scheme. No ETA for next release, but I assure you - if you contribute, it will be sooner ;-). What leads us to the next part.

Contributing to ECL

Documentation, tutorials, wiki, others

You can contribute in numerous ways. One way is to code and fix bugs - and it's very important part. But equally important is to find bugs and report them, so developers can improve codebase. Feature requests are also welcome (however we are focused now on fixing stuff rather than adding functionality, so even good proposition might wait in queue behind something maybe less exciting).

Testing on various platform and architectures is essential. Thanks to Anton Vodonosov and cl-test-grid it is as easy as setting environment, tweaking two configuration files and invoking ./run-agent.sh. Run may take a while (depending on computer) and is limited to operating systems and architectures supported by Clozure Common Lisp.

If ECL doesn't build, crashes, or works non-conforming to specification, please do report at https://gitlab.com/embeddable-common-lisp/ecl/issues. It requires account on GitLab, but setting it up is free. I'm still struggling to produce some time and move all tickets from https://sourceforge.net/p/ecls/_list/tickets to above mentioned issue tracking site - volunteers are more then welcome to do it.

If you encounter problem, please write to mailing list. There are many kind souls there more then willing to help - and usually they do. On the other hand, if someone asks for help and you know the answer - act! :-)

Wiki is a place, where many resources are gathered. It is incomplete and barely usable, because I don't have time to improve it. If you successfully build ECL on Android - great! Please add page to wiki, so others may reproduce your work! Do you have some nifty idea, and you believe it is worth to keep it there - do it. See a typo? Bug? Outdated information? You know what to do! You spot some nice blog post about ECL? Please share with others - wiki is best place to do so. You did successful project using ECL? Share this information! All this, and so much more, may be done here: https://gitlab.com/embeddable-common-lisp/ecl/wikis/home.

You can also write something for ECL Quarterly (e-mail me at dkochmanski[at]turtle-solutions[dot]eu).

Source code contributions

Development takes place on git repository located at git@gitlab.com:embeddable-common-lisp/ecl.git.

If you want your code in Embeddable Common-Lisp project, please send a patch to mailing list with additional tag [PATCH] in subject. Generally we want to follow convention used by U-Boot development team (http://www.denx.de/wiki/U-Boot/Patches), which borrows a lot from Linux policy. Please read this guide, it's really worthwhile reading. If you submit a significant change, please report it in CHANGELOG located in top directory.

Basically, if you want to contribute to code, you have at least two choices. You may improve C code (which is probably less interesting for lispers). Most of the sources are written in lisp however, and require no knowledge about C. Problems vary from fairly easy even for seasoned developer to hard conceptual riddles, so everyone can find something interesting for himself.. Please consult appropriate directories under src/ sub-directory (i.e. src/lsp, src/clos etc.) - it's pure lisp! And it's fun to hack. Improving both C and Lisp sources might be a great learning experience. To figure what's wrong requires often getting your hands dirty, and then cleaning proposed solution. This, connected with peer review, might be a next step to become a better programmer.

There is also a third part, which is tiresome (for me at least) - improving build system - it is buggy and inconsistent. First person who will fix this has granted a free dinner when we meet in person. Remember however, that we support many operating systems, so it might be tricky to do it properly without introducing new problems.

If you are a library or application developer, please test against as many implementations as possible - its hard, it takes time, but in my humble opinion it is essential for CL ecosystem. Each implementation has it's weak and strong sides, and you never know, when you'll need one, or who and in what purpose is using your code :-).

ECL 15.3.7 released

posted on 2015-03

My deep apologies to all Windows users for misfortunate unicode character at top screen, which caused an exception at the very start of ECL startup. Version 15.2.21 was released little prematurely. Proper fix is released a week after problem was noticed, because we did want to test it first, and to fix a few regressions between 13.5.1 and 15.2.21.

This release is at least as good as 15.2.21. Enchantments include:

  • ECL now builds easily with Visual Studio 2012, including builds with custom codepages,
  • Autoconf scripts are upgraded to version 2.69,
  • Fixed stack direction, so builds against upcoming gcc 5.0 should work,
  • Broken --with-sse=yes configure flag works once again,
  • Fixed stable-sort bug (corner case on sorting strings).

Big thanks to everyone, who helped with testing and development :-)

Regarding other news, we're still on a move. We barely settled on gitorious and now it's aquired by gitlab, so we're moving forward, to gitlab. But not everything at once. Full migration will be performed at 14th march, but repositories are already synced, and wiki is moved as well (some links are broken, but it's a matter of a few edits). Anyone interested on participating, is invited to join a team at https://gitlab.com/embeddable-common-lisp .

ECL 15.2.21 released, new maintainer found

posted on 2015-02

New release, which is mainly current state of git HEAD (plus a few fixes). Contains numerous bug-fixes in comparison to 13.5.1 and is last, which follows date-based version convention. It's time to finally release ECL 1.0 ;-).

Development moves to gitorious (https://gitorious.org/embeddable-common-lisp), same as wiki (previous content is inaccessible now, but once subscription is renewed, I'll start to migrate content from there). In GIT topic - current permissions will be preserved. Just drop me a line with gitorious login, and I'll add person on corresponding permission level to project.

Mailing list and website are staying at SF for now, but I'd really like to switch the latter to something more manageable. Also, SF have lately problems with stability, what is quite annoying.

More on maintainer topic: My name is Daniel Kochmaski (you may meet me on IRC and over internet under the nick "jackdaniel") and I'm willing to spare at least a few hours a week for this amazing project to keep it alive. More on progress ideas and myself might be found on mailing list archive - I ask for comments, suggestions and discussion (and forgiveness for some potential dumb ideas I might propose), to develop them better. Also, I do ask for help.

While I will try to set up Linux/Unix environment to check builds and try to fix problems on various operating systems (I'm thinking about putting vagrant in use), I have no access to neither Windows or OSX environments, so I will have no clue, if any commit will break builds on these. Testers for these platforms are crucial imo.

Best regards, Daniel

New maintainer sought

posted on 2013-10

Reasons here: http://article.gmane.org/gmane.lisp.ecl.general/10264

ECL 13.5.1

posted on 2013-05

Known issues

  • Cygwin's library is still broken: fork/exec fails to reload the cygwin library, or ECL's compiled libraries in a completely random fashion. For this reason we recommend using ext:system instead of ext:run-program in that platform.
  • In Windows ECL comes with bytecodes compiler by default, because C compilers are normally not avaiable. Unfortunately several libraries out there are not prepared for this. If you plan to use quicklisp and have a C compiler accessible to ECL, you may use (ext:install-c-compiler) to switch back to the Lisp-to-C compiler.

Changes since last release

Some highlights of this release are:

  • ECL now ships with the whole of the Unicode names database, optimally compressed using constant C tables. This means ECL now recognizes all valid Unicode (and ASCII) names for the whole range of characters, and can print them as well.
  • ECL has changed the way it stores compiled data in the C files, so that it works around the limit of 65k characters per string.
  • ECL now builds with Microsoft Visual Studio C++ 2012.
  • We bend the knee and accept WHILE/IF statements intertwined with FOR/AS, though this is still not valid ANSI Common Lisp.

See file src/CHANGELOG or browse it online:

http://ecls.cvs.sourceforge.net/viewvc/ecls/ecl/src/CHANGELOG?view=markup

Stumpwm can be built with ECL

posted on 2013-01

For those complaining about this, I finally found time and there were only three small bugs:

  1. CLX requires recursive locks, but ECL now makes non-recursive the default. I fixed CLX in ECL.

  2. Stumpwm uses the old ASDF-ECL building routines instead of the new and improved ASDF-BUNDLE which is standard and supported by ASDF (Faré).

  3. Stumpwm's RELOAD kept a reference to ASDF which should not be there for standalone programs, since ASDF is not used in that case.

The patches are here https://dl.dropbox.com/u/23177754/stumpwm.patches

ECL 12.12.1 released

posted on 2012-12

Some highlights of this release are:

  • Lots of bugs fixed.
  • The MOP has been fixed to work with the upcoming release of Closer-MOP
  • ECL now produces a much more readable C code, with indentation and more explicit declarations of variables.

See file src/CHANGELOG or browse it online

http://ecls.cvs.sourceforge.net/viewvc/ecls/ecl/src/CHANGELOG?view=markup

PS: If you read ECL news through Planet Lisp, please follow the links to the original page, https://sourceforge.net/p/ecls/news , because SF's news feed breaks all the beautiful formatting of its markup code.

Involvement in ECL

posted on 2012-11

Would you like to get acquainted with a Common Lisp implementation? Do you have free time and would like to start by some small tasks?

ECL is currently offering a simple way for you to do so. Thanks to Anton Vodonosov's cl-test-grid and Eric Marsden's benchmark collection, which are offered here http://ecls.sourceforge.net/reports-generated/ecl/index.html and here http://ecls.sourceforge.net/pictures/index_js.html , you may now see in real time what problems are left to be solved in terms both of compatibility and performance.

Usually, the things to be corrected are small details, like a core function that does not have an optimised inline expansion (SEARCH, FIND, etc), or some tiny mismatch between the expectations of library developers and what I, as an implementor of ECL, understood as standard. Many of those bugs or performance regressions are also easy to debug (*).

If you wish to contribute, please do so through the appropriate channels, which include the ECL mailing list or, most notably, the bug/patch/feature/support reporting database in Sourceforge (https://sourceforge.net/p/ecls/news/new)

Cheers,

Juanjo

(*) If you just want to optimise the compilation of certain functions, performance of CLOS, etc, etc, this can be done knowing only Common Lisp, for the Lisp->C compiler is based entirely on Common Lisp functions and relies on compiler macros for optimisations. For other tasks, some C skills and acquaintance with a C debugger is needed.