Embeddable Common-Lisp

Content from 2012-05

Bug reports: please do them!

posted on 2012-05

I am sorry. Really, I am. I cannot follow you all in all your usual communication channels: comp.lang.lisp, the ecl forum, #lisp irc channel... But please consider this:

First of all, ECL is maintained in free time. In this sense, it is more beneficial for you if the maintainers spend more time coding than trolling about lisp.

Second, an IRC channel is not a place to look for support. Sourceforge offers you a bug reporting facility (https://sourceforge.net/tracker/?group_id=30035), a feature request database (https://sourceforge.net/tracker/?group_id=30035&atid=398056), a mailing list (http://news.gmane.org/gmane.lisp.ecl.general/)...

Response times are usually very fast, and only in the most difficult cases (it involves legacy code, requires some nonstandard architecture, etc), or when a release is on queue, does it takes longer.

In particular, in order to ease your life, bug reporting and feature requests are now possible for people without an account in Sourceforge. This is an experiment and I will try to keep it that way as long as spammers allow. Please make good use of it.

Finally, if you find something does not work the way you expect it, instead of spending a week on it and then complaining about it anywhere, just leave a message in any of those communication channels I mentioned above. It will be more productive for you and for the free software community in general.

Ah, and before I forget it: a good bug report for ECL needs at least the following information: 1) platform information and operating system, 2) configuration options, 3) if it failed at configuration time, the log, 4) if it failed at build time (i.e. during "make") a log of that, 5) if it fails with a standard (quicklisp library), steps to reproduce it, 6) if it fails with your code, ideally some sample to reproduce it.

Experimenting: Windows console

posted on 2012-05

Not long ago a user reported problems with ECL when running under Windows with the Chinese codepage 936. I was really surprised to verify that indeed, ECL was aborting on any multibyte character... but only when typed in a terminal window (the so called Windows console).

A bit of investigation revealed that the Microsoft C runtime is quite broken (http://alfps.wordpress.com/2011/12/08/unicode-part-2-utf-8-stream-mode/) or at least not very POSIX / ANSI conformant. Roughly, some versions of the runtime act differently when the standard input or output of a program is connected to a console. In this case they try to be very clever and translate the Unicode input/output generated by the console into the selected codepage. This also leads the CRT to reject multibyte characters because read() demands a buffer with as many bytes as a whole character needs. For instance, if a character with codes F7 8C is input at the console, read(0, buffer, 1), instead of returning F7, will abort. Or put some other way: characters cannot be read one by one.

All these problems disappear when the user forgets about POSIX/ANSI file descriptors and works with the Windows API and indeed there are MSDN blogs out there asking you to do so... And so we did.

As of now, ECL incorporates new ANSI streams for handling the Windows console. Currently they cannot be created by the user and are activated by ECL whenever the input or output of a program are connected to a console. The stream is interactive and read, write, listen, etc, they all work as expected.

This also has the added benefit that ECL now detects which codepage is used by the console and in that case it activates by default the expected external format --- which you are free to correct with (SETF STREAM-EXTERNAL-FORMAT) IIRC.

I would like to ask people working with Windows to test and help debug this new feature.

Ha! We beat iconv!

posted on 2012-05

Seems ECL supports more codepages than iconv on certain platforms

;;; Testing ISO-2022-JP-1 iconv: conversion from ISO-2022-JP-1' is not supported Tryiconv --help' or `iconv --usage' for more information. (From regressions in http://ecls.sourceforge.net/logs.html )

This is so in some not too old Linuxes, OpenBSD, NetBSD, etc.

Multithreading is now on by default

posted on 2012-05

Today or tomorrow I will upload a set of changes that will make ECL build with support for threads by default. If you want a single-threaded ECL, use "--disable-threads" when configuring it.

On a completely different matter, thanks to user contributed patches, ECL now seems to build in 64 bit mode using Visual Studio C++ (express) or Microsoft's SDK for Windows v7.1 (free).

Periodic reports: multithreading & tests

posted on 2012-05

Probably nobody cares about this, but there has been some activity in ECL in various fronts. I periodically report that to the ECL mailing list, but this seems to lead other people to think the project is silent or dead. From now on, expect a little bit more noise through the ECL news RSS.

The biggest change ECL will suffer with the upcoming release is the multithreading library. So far we were relying on the POSIX primitives for that (pthread_mutex_lock, condition variables, etc), but this has proven to be a nightmare to maintain. The primitives are not interruptible and they do not interact well with mp:process-interrupt, a feature unfortunately everybody expects in a Lisp implementation.

The upcoming library will be based on a combination of hardware-coded atomic operations (via libatomic-ops) with a new waiting queue that makes clever use of the operating system facilities. The result should be a library that is fair (different threads have equal opportunities), it recognizes fast paths (when a synchronization object is free it is quickly acquired), and in the worst case it is not CPU consuming (relies on sigsuspend or SleepEx to really sleep a waiting thread).

Everything is right now in git/CVS (http://ecls.sourceforge.net/download.html) and under heavy testing with new dedicated tests. Help is welcome from anybody who is interesting in ensuring stability.

Another side effect is that "make check" now works in ECL. I have abandoned the development of "ecl-tests" and instead the tests are now integral part of ECL. In order to save space, the ANSI test suite is not shipped with ECL but rather automatically downloaded thanks to ECL's new components: ecl-curl and ecl-minitar. The test suite also allows downloading of quicklisp and automated testing of selected libraries.

In the end this was the last missing ingredient to resucitate our compiler farm, which is now up and running again: http://ecls.sourceforge.net/logs.html

Expect a new release soon, and also a few more posts on other upcoming features.



P.S.: I would like to thank the ELS organizers and steering committee for a wonderful meeting in Zadar. Great location, amazing food, and even better discussions.