Next: , Previous: , Up: Top   [Contents][Index]

2 Developer’s guide

Next: , Up: Developer's guide   [Contents][Index]

2.1 Sources structure

2.1.1 src/c

alloc_2.dmemory allocation based on the Boehm GC
all_symbols.dname mangler and symbol initialization
apply.dinterface to C call mechanism
arch/*architecture dependant code
array.darray routines
backq.dbackquote mechanism
big.dbignum routines based on the GMP
big_ll.dbignum emulation with long long
cfun.dcompiled functions
cfun_dispatch.dtrampolines for functions
character.dcharacter routines
char_ctype.dcharacter properties.
cinit.dlisp initialization
clos/accessor.ddispatch for slots
clos/cache.dthread-local cache for a variety of operations
clos/gfun.ddispatch for generic functions
clos/instance.dCLOS interface
cmpaux.dauxiliaries used in compiled Lisp code
compiler.dbytecode compiler
disassembler.dbytecodes disassembler utilities
dpp.cdefun preprocessor
ecl_constants.hcontstant values for all_symbols.d
features.hnames of features compiled into ECL
error.derror handling
ffi/backtrace.dC backtraces
ffi/cdata.ddata for compiled files
ffi/libraries.dshared library and bundle opening / copying / closing
ffi/mmap.dmapping of binary files
ffi.duser defined data types and foreign functions interface
file.dfile interface (implementation dependent)
format.dformat (this isn’t ANSI compliant, we need it for bootstrapping though)
hash.dhash tables
interpreter.dbytecode interpreter
iso_latin_names.hcharacter names in ISO-LATIN-1
list.dlist manipulating routines
load.dbinary loader (contains also open_fasl_data)
macros.dmacros and environment
main.decl boot proccess
Makefile.inMakefile for ECL core library
newhash.dhashing routines
num_arith.darithmetic operations
number.dconstructing numbers
numbers/*.darithmetic operations (abs, atan, plusp etc)
num_co.doperations on floating-point numbers (implementation dependent)
num_log.dlogical operations on numbers
num_pred.dpredicates on numbers
num_rand.drandom numbers
package.dpackages (OS dependent)
printer/*.dprinter utilities and object representations
read.dread.d - reader
reference.dreference in Constants and Variables
sequence.dsequence routines
serialize.dserialize a bunch of lisp data
sse2.dSSE2 vector type support
stacks.dbinding/history/frame stacks
string.dstring routines
structure.dstructure interface
symbols_list2.hThe latter is generated from the first. The first has to contain all symbols on the system which aren’t local.
tcp.dstream interface to TCP
time.dtime routines
typespec.dtype specifier routines
unicode/*unicode definitions
unixfsys.dUnix file system interface
unixsys.dUnix shell interface
vector_push.dvector optimizations
atomic.datomic operations
barrier.dwait barriers
condition_variable.dcondition variables for native threads
ecl_atomics.halternative definitions for atomic operations
mailbox.dthread communication queue
mutex.dmutually exclusive locks.
process.dnative threads
queue.dwaiting queue for threads
rwlock.dPOSIX read-write locks
semaphore.dPOSIX-like semaphores

Next: , Previous: , Up: Developer's guide   [Contents][Index]

2.2 Contributing

Next: , Previous: , Up: Developer's guide   [Contents][Index]

2.3 Manipulating Lisp objects

If you want to extend, fix or simply customize ECL for your own needs, you should understand how the implementation works.

C/C++ identifier: cl_lispunion cons big ratio SF DF longfloat complex symbol pack hash array vector base_string string stream random readtable pathname bytecodes bclosure cfun cfunfixed cclosure d instance process queue lock rwlock condition_variable semaphore barrier mailbox cblock foreign frame weak sse

Union containing all first-class ECL types.

Next: , Up: Manipulating Lisp objects   [Contents][Index]

2.3.1 Objects representation

In ECL a lisp object is represented by a type called cl_object. This type is a word which is long enough to host both an integer and a pointer. The least significant bits of this word, also called the tag bits, determine whether it is a pointer to a C structure representing a complex object, or whether it is an immediate data, such as a fixnum or a character.


Figure 2.1: Immediate types

The topic of the immediate values and bit fiddling is nicely described in Peter Bex’s blog describing Chicken Scheme internal data representation. We could borrow some ideas from it (like improving fixnum bitness and providing more immediate values). All changes to code related to immediate values should be carefully benchmarked.

The fixnums and characters are called immediate data types, because they require no more than the cl_object datatype to store all information. All other ECL objects are non-immediate and they are represented by a pointer to a cell that is allocated on the heap. Each cell consists of several words of memory and contains all the information related to that object. By storing data in multiples of a word size, we make sure that the least significant bits of a pointer are zero, which distinguishes pointers from immediate data.

In an immediate datatype, the tag bits determine the type of the object. In non-immediate datatypes, the first byte in the cell contains the secondary type indicator, and distinguishes between different types of non immediate data. The use of the remaining bytes differs for each type of object. For instance, a cons cell consists of three words:

| CONS    |          |
|     car-pointer    |
|     cdr-pointer    |

Note, that this is on of the possible implementations of cons. The second one (currently default) uses the immediate value for the list and consumes two words instead of three. Such implementation is more memory and speed efficient (according to the comments in the source code):

 * We implement two variants. The "small cons" type carries the type
 * information in the least significant bits of the pointer. We have
 * to do some pointer arithmetics to find out the CAR / CDR of the
 * cons but the overall result is faster and memory efficient, only
 * using two words per cons.
 * The other scheme stores conses as three-words objects, the first
 * word carrying the type information. This is kept for backward
 * compatibility and also because the oldest garbage collector does
 * not yet support the smaller datatype.
 * To make code portable and independent of the representation, only
 * access the objects using the common macros below (that is all
 * except ECL_CONS_PTR or ECL_PTR_CONS).
C/C++ identifier: cl_object

This is the type of a lisp object. For your C/C++ program, a cl_object can be either a fixnum, a character, or a pointer to a union of structures (See cl_lispunion in the header object.h). The actual interpretation of that object can be guessed with the macro ecl_t_of.


For example, if x is of type cl_object, and it is of type fixnum, we may retrieve its value:

if (ecl_t_of(x) == t_fixnum)
    printf("Integer value: %d\n", fix(x));


If x is of type cl_object and it does not contain an immediate datatype, you may inspect the cell associated to the lisp object using x as a pointer. For example:

if (ecl_t_of(x) == t_vector)
    printf("Vector's dimension is: %d\n", x->dim);

You should see the following sections and the header object.h to learn how to use the different fields of a cl_object pointer.

C/C++ identifier: cl_type

Enumeration type which distinguishes the different types of lisp objects. The most important values are:

t_cons t_fixnum, t_character, t_bignum, t_ratio, t_singlefloat, t_doublefloat, t_complex, t_symbol, t_package, t_hashtable, t_array, t_vector, t_string, t_bitvector, t_stream, t_random, t_readtable, t_pathname, t_bytecodes, t_cfun, t_cclosure, t_gfun, t_instance, t_foreign and t_thread.

Function: cl_type ecl_t_of (cl_object x)

If x is a valid lisp object, ecl_t_of(x) returns an integer denoting the type that lisp object. That integer is one of the values of the enumeration type cl_type.

Function: bool ECL_FIXNUMP (cl_object o)
Function: bool ECL_CHARACTERP (cl_object o)
Function: bool ECL_BASE_CHAR_P (cl_object o)
Function: bool ECL_CODE_CHAR_P (cl_object o)
Function: bool ECL_BASE_CHAR_CODE_P (cl_object o)
Function: bool ECL_NUMBER_TYPE_P (cl_object o)
Function: bool ECL_REAL_TYPE_P (cl_object o)
Function: bool ECL_CONSP (cl_object o)
Function: bool ECL_LISTP (cl_object o)
Function: bool ECL_ATOM (cl_object o)
Function: bool ECL_SYMBOLP (cl_object o)
Function: bool ECL_ARRAYP (cl_object o)
Function: bool ECL_VECTORP (cl_object o)
Function: bool ECL_BIT_VECTOR_P (cl_object o)
Function: bool ECL_STRINGP (cl_object o)

Different macros that check whether o belongs to the specified type. These checks have been optimized, and are preferred over several calls to ecl_t_of.

Function: bool ECL_IMMEDIATE (cl_object o)

Tells whether x is an immediate datatype.

Previous: , Up: Manipulating Lisp objects   [Contents][Index]

2.3.2 Constructing objects

On each of the following sections we will document the standard interface for building objects of different types. For some objects, though, it is too difficult to make a C interface that resembles all of the functionality in the lisp environment. In those cases you need to

  1. build the objects from their textual representation, or
  2. use the evaluator to build these objects.

The first way makes use of a C or Lisp string to construct an object. The two functions you need to know are the following ones.

Function: cl_object c_string_to_object (const char *s)
Function: cl_object string_to_object (cl_object o)

c_string_to_object builds a lisp object from a C string which contains a suitable representation of a lisp object. string_to_object performs the same task, but uses a lisp string, and therefore it is less useful.


Using a C string

cl_object array1 = c_string_to_object("#(1 2 3 4)");

Using a Lisp string

cl_object string = make_simple_string("#(1 2 3 4)");
cl_object array2 = string_to_object(string);


Common-Lisp distinguishes two types of integer types: bignums and fixnums. A fixnum is a small integer, which ideally occupies only a word of memory and which is between the values MOST-NEGATIVE-FIXNUM and MOST-POSITIVE-FIXNUM. A bignum is any integer which is not a fixnum and it is only constrained by the amount of memory available to represent it.

In ECL a fixnum is an integer that, together with the tag bits, fits in a word of memory. The size of a word, and thus the size of a fixnum, varies from one architecture to another, and you should refer to the types and constants in the ecl.h header to make sure that your C extensions are portable. All other integers are stored as bignums, they are not immediate objects, they take up a variable amount of memory and the GNU Multiprecision Library is required to create, manipulate and calculate with them.

C/C++ identifier: cl_fixnum

This is a C signed integer type capable of holding a whole fixnum without any loss of precision. The opposite is not true, and you may create a cl_fixnum which exceeds the limits of a fixnum and should be stored as a bignum.

C/C++ identifier: cl_index

This is a C unsigned integer type capable of holding a non-negative fixnum without loss of precision. Typically, a cl_index is used as an index into an array, or into a proper list, etc.


These constants mark the limits of a fixnum.

Function: bool ecl_fixnum_lower (cl_fixnum a, cl_fixnum b)
Function: bool ecl_fixnum_greater (cl_fixnum a, cl_fixnum b)
Function: bool ecl_fixnum_leq (cl_fixnum a, cl_fixnum b)
Function: bool ecl_fixnum_geq (cl_fixnum a, cl_fixnum b)
Function: bool ecl_fixnum_plusp (cl_fixnum a)
Function: bool ecl_fixnum_minusp (cl_fixnum a)

Operations on fixnums (comparison and predicates).

Function: cl_object ecl_make_fixnum (cl_fixnum n)
Function: cl_fixnum ecl_unfix (cl_object o)

ecl_make_fixnum converts from an integer to a lisp object, while the ecl_fixnum does the opposite (converts lisp object fixnum to integer). These functions do not check their arguments.

Function: cl_fixnum fixint (cl_object o)
Function: cl_index fixnint (cl_object o)

Safe conversion of a lisp fixnum to a C integer of the appropriate size. Signals an error if o is not of fixnum type.

fixnint additionally ensure that o is not negative.


ECL has two types of characters – one fits in the C type char, while the other is used when ECL is built with a configure option --enable-unicode.

C/C++ identifier: ecl_character

Immediate type t_character. If ECL built with Unicode support, then may be either base or extended character, which may be distinguished with the predicate ECL_BASE_CHAR_P.

Additionally we have ecl_base_char for base strings, which is an equivalent to the ordinary char.


    printf("Base character: %c\n", ECL_CHAR_CODE(o));

Each character is assigned an integer code which ranges from 0 to (ECL_CHAR_CODE_LIMIT-1).

Function: cl_fixnum ECL_CHAR_CODE (cl_object o)
Function: cl_fixnum ECL_CODE_CHAR (cl_object o)

ECL_CHAR_CODE, ecl_char_code and ecl_base_char_code return the integer code associated to a lisp character. ecl_char_code and ecl_base_char_code perform a safe conversion, while ECL_CHAR_CODE doesn’t check it’s argument. ecl_base_char_code is an optimized version for base chars. Checks it’s argument.

ECL_CODE_CHAR returns the lisp character associated to an integer code. It does not check its arguments.

Function: bool ecl_char_eq (cl_object x, cl_object y)
Function: bool ecl_char_equal (cl_object x, cl_object y)

Compare two characters for equality. char_eq take case into account and char_equal ignores it.

Function: bool ecl_char_cmp (cl_object x, cl_object y)
Function: bool ecl_char_compare (cl_object x, cl_object y)

Compare the relative order of two characters. char_cmp takes care of case and char_compare converts all characters to uppercase before comparing them.


An array is an aggregate of data of a common type, which can be accessed with one or more non-negative indices. ECL stores arrays as a C structure with a pointer to the region of memory which contains the actual data. The cell of an array datatype varies depending on whether it is a vector, a bit-vector, a multidimensional array or a string.

Function: bool ECL_ADJUSTABLE_ARRAY_P (cl_object x)
Function: bool ECL_ARRAY_HAS_FILL_POINTER_P (cl_object x)

All arrays (arrays, strings and bit-vectors) may be tested for being adjustable and whenever they have a fill pointer with this two functions.

C/C++ identifier: ecl_vector

If x contains a vector, you can access the following fields:


The type of the elements of the vector.


Boolean indicating if it is displaced.


The maximum number of elements.


Actual numer of elements in the vector or fill pointer.


Union of pointers of different types. You should choose the right pointer depending on x->vector.elttype.

C/C++ identifier: ecl_array

If x contains a multidimensional array, you can access the following fields:


The type of the elements of the array.


The number of array dimensions.


Boolean indicating if it is displaced.


The maximum number of elements.


Array with the dimensions of the array. The elements range from x->array.dim[0] to x->array.dim[x->array.rank-1].


Actual numer of elements in the array or fill pointer.


Union of pointers of different types. You should choose the right pointer depending on x->array.elttype.

C/C++ identifier: cl_elttype ecl_aet_object ecl_aet_sf ecl_aet_df ecl_aet_bit ecl_aet_fix ecl_aet_index ecl_aet_b8 ecl_aet_i8 ecl_aet_b16 ecl_aet_i16 ecl_aet_b32 ecl_aet_i32 ecl_aet_b64 ecl_aet_i64 ecl_aet_ch ecl_aet_bc

Each array is of an specialized type which is the type of the elements of the array. ECL has arrays only a few following specialized types, and for each of these types there is a C integer which is the corresponding value of x->array.elttype or x->vector.elttype. We list some of those types together with the C constant that denotes that type:



















Function: cl_elttype ecl_array_elttype (cl_object array)

Returns the element type of the array o, which can be a string, a bit-vector, vector, or a multidimensional array.


For example, the code

ecl_array_elttype(c_string_to_object("\"AAA\""));  /* returns ecl_aet_ch */
ecl_array_elttype(c_string_to_object("#(A B C)")); /* returns ecl_aet_object */
Function: cl_object ecl_aref (cl_object x, cl_index index)
Function: cl_object ecl_aset (cl_object x, cl_index index, cl_object value)

These functions are used to retrieve and set the elements of an array. The elements are accessed with one index, index, as in the lisp function ROW-MAJOR-AREF.


cl_object array = c_string_to_object("#2A((1 2) (3 4))");
cl_object x = aref(array, 3);
cl_print(1, x);	/* Outputs 4 */
aset(array, 3, MAKE_FIXNUM(5));
cl_print(1, array); /* Outputs #2A((1 2) (3 5)) */
Function: cl_object ecl_aref (cl_object x, cl_index index)
Function: cl_object ecl_aset (cl_object x, cl_index index, cl_object value)

These functions are similar to aref and aset, but they operate on vectors.


cl_object array = c_string_to_object("#(1 2 3 4)");
cl_object x = aref1(array, 3);
cl_print(1, x);	    /* Outputs 4 */
aset1(array, 3, MAKE_FIXNUM(5));
cl_print(1, array); /* Outputs #(1 2 3 5) */


A string, both in Common-Lisp and in ECL is nothing but a vector of characters. Therefore, almost everything mentioned in the section of arrays remains valid here.

The only important difference is that ECL stores the base-strings (non-Unicode version of a string) as a lisp object with a pointer to a zero terminated C string. Thus, if a string has n characters, ECL will reserve n+1 bytes for the base-string. This allows us to pass the base-string self pointer to any C routine.

C/C++ identifier: ecl_string
C/C++ identifier: ecl_base_string

If x is a lisp object of type string or a base-string, we can access the following fields:

x->string.dim x->base_string.dim

Actual number of characters in the string.

x->string.fillp x->base_string.fillp

Actual number of characters in the string.

x->string.self x->base_string.self

Pointer to the characters (appropriately integers and chars).

Function: bool ECL_EXTENDED_STRING_P (cl_object object)
Function: bool ECL_BASE_STRING_P (cl_object object)

Verifies if an objects is an extended or base string. If Unicode isn’t supported, then ECL_EXTENDED_STRING_P always returns 0.


Bit-vector operations are implemented in file src/c/array.d. Bit-vector shares the structure with a vector, therefore, almost everything mentioned in the section of arrays remains valid here.


Streams implementation is a broad topic. Most of the implementation is done in the file src/c/file.d. Stream handling may have different implementations referred by a member pointer ops.

Additionally on top of that we have implemented Gray Streams (in portable Common Lisp) in file src/clos/streams.lsp, which may be somewhat slower (we need to benchmark it!). This implementation is in a separate package GRAY. We may redefine functions in the COMMON-LISP package with a function redefine-cl-functions at run-time.

C/C++ identifier: ecl_file_ops write_* read_* unread_* peek_* listen clear_input clear_output finish_output force_output input_p output_p interactive_p element_type length get_position set_position column close
C/C++ identifier: ecl_stream
ecl_smmode mode

Stream mode (in example ecl_smm_string_input).

int closed

Whenever stream is closed or not.

ecl_file_ops *ops

Pointer to the structure containing operation implementations (dispatch table).

union file

Union of ANSI C streams (FILE *stream) and POSIX files interface (cl_fixnum descriptor).

cl_object object0, object1

Some objects (may be used for a specific implementation purposes).

cl_object byte_stack

Buffer for unread bytes.

cl_index column

File column.

cl_fixnum last_char

Last character read.

cl_fixnum last_code[2]

Actual composition of the last character.

cl_fixnum int0 int1

Some integers (may be used for a specific implementation purposes).

cl_index byte_size

Size of byte in binary streams.

cl_fixnum last_op

0: unknown, 1: reading, -1: writing

char *buffer

Buffer for FILE

cl_object format

external format

cl_eformat_encoder encoder
cl_eformat_encoder decoder
cl_object format_table
in flags

Character table, flags, etc

ecl_character eof_character
Function: bool ECL_ANSI_STREAM_P (cl_object o)

Predicate determining if o is a first-class stream object. Doesn’t check type of it’s argument.

Function: bool ECL_ANSI_STREAM_TYPE_P (cl_object o, ecl_smmode m)

Predicate determining if o is a first-class stream object of type m.


Structures and instances share the same datatype t_instance ( with a few exceptions. Structure implementation details are the file src/c/structure.d.

Function: cl_object ECL_STRUCT_TYPE (cl_object x)
Function: cl_object ECL_STRUCT_SLOTS (cl_object x)
Function: cl_object ECL_STRUCT_LENGTH (cl_object x)
Function: cl_object ECL_STRUCT_SLOT (cl_object x, cl_index i)
Function: cl_object ECL_STRUCT_NAME (cl_object x)

Convenience functions for the structures.


Function: cl_object ECL_CLASS_OF (cl_object x)
Function: cl_object ECL_SPEC_FLAG (cl_object x)
Function: cl_object ECL_SPEC_OBJECT (cl_object x)
Function: cl_object ECL_CLASS_NAME (cl_object x)
Function: cl_object ECL_CLASS_SUPERIORS (cl_object x)
Function: cl_object ECL_CLASS_INFERIORS (cl_object x)
Function: cl_object ECL_CLASS_SLOTS (cl_object x)
Function: cl_object ECL_CLASS_CPL (cl_object x)
Function: bool ECL_INSTANCEP (cl_object x)

Convenience functions for the structures.


A bytecodes object is a lisp object with a piece of code that can be interpreted. The objects of type t_bytecode are implicitly constructed by a call to eval, but can also be explicitly constructed with the make_lambda function.

Function: cl_object si_safe_eval (cl_object form, cl_object env, ...)

si_safe_eval evaluates form in the lexical environment env, which can be ECL_NIL. Before evaluating it, the expression form must be bytecompiled.

DEPRECATED cl_object cl_eval (cl_object form)

cl_eval is the equivalent of si_safe_eval but without environment and with err_value set to nil. It exists only for compatibility with previous versions.

DEPRECATED cl_object cl_safe_eval (cl_object form)

Equivalent of si_safe_eval (macro define).


si_object form = c_string_to_object("(print 1)");
si_safe_eval(form, ECL_NIL);
si_safe_eval(form, ECL_NIL, 3); /* on error function will return 3 */
Function: cl_object si_make_lambda (cl_object name, cl_object def)

Builds an interpreted lisp function with name given by the symbol name and body given by def.


For instance, we would achieve the equivalent of

(funcall #'(lambda (x y)
             (block foo (+ x y)))
         1 2)

with the following code

cl_object def = c_string_to_object("((x y) (+ x y))");
cl_object name = _intern("foo")
cl_object fun = si_make_lambda(name, def);
return funcall(fun, MAKE_FIXNUM(1), MAKE_FIXNUM(2));

Notice that si_make_lambda performs a bytecodes compilation of the definition and thus it may signal some errors. Such errors are not handled by the routine itself so you might consider using si_safe_eval instead.

Next: , Previous: , Up: Developer's guide   [Contents][Index]

2.4 Environment implementation

Next: , Previous: , Up: Developer's guide   [Contents][Index]

2.5 The interpreter

2.5.1 ECL stacks

ECL uses the following stacks:

Frame Stackconsisting of catch, block, tagbody frames
Bind Stackfor shallow binding of dynamic variables
Interpreter Stackacts as a Forth data stack, keeping intermediate arguments to interpreted functions, plus a history of called functions.
C Control Stackused for arguments/values passing, typed lexical variables, temporary values, and function invocation.

2.5.2 Procedure Call Conventions

ECL employs standard C calling conventions to achieve efficiency and interoperability with other languages. Each Lisp function is implemented as a C function which takes as many argument as the Lisp original plus one additional integer argument which holds the number of actual arguments. The function sets NValues to the number of Lisp values produced, it returns the first one and the remaining ones are kept in a global (per thread) array (VALUES).

To show the argument/value passing mechanism, here we list the actual code for the Common-Lisp function cons.

   cl_cons(int narg, object car, object cdr)
   {       object x;
   x = alloc_object(t_cons);
   CAR(x) = car;
   CDR(x) = cdr;
   NValues = 1;
   return x;

ECL adopts the convention that the name of a function that implements a Common-Lisp function begins with a short package name (cl for COMMON-LISP, si for SYSTEM, etc), followed by L, and followed by the name of the Common-Lisp function. (Strictly speaking, ‘-’ and ‘*’ in the Common-Lisp function name are replaced by ‘_’ and ‘A’, respectively, to obey the syntax of C.)

check_arg(2) in the code of cl_cons checks that exactly two arguments are supplied to cons. That is, it checks that narg is 2, and otherwise, it causes an error. allocate_object(t_cons) allocates a cons cell in the heap and returns the pointer to the cell. After the CAR and the CDR fields of the cell are set, the cell pointer is returned directly. The number assigned to NValues set by the function (1 in this case) represents the number of values of the function.

In general, if one is to play with the C kernel of ECL there is no need to know about all these conventions. There is a preprocessor that takes care of the details, by using a lisp representation of the statements that output values, and of the function definitions. For instance, the actual source code for cl_cons in src/c/lists.d

   @(defun cons (car cdr)
   @(return CONS(car, cdr))

2.5.3 The lexical environment

The ECL interpreter uses two A-lists (Association lists) to represent lexical environments.

When a function closure is created, the current two A-lists are saved in the closure along with the lambda expression. Later, when the closure is invoked, the saved A-lists are used to recover the lexical environment.

2.5.4 The interpreter stack

The bytecodes interpreter uses a stack of its own to save and restore values from intermediate calculations. This Forth-like data stack is also used in other parts of the C kernel for various purposes, such as saving compiled code, keeping arguments to FORMAT, etc.

However, one of the most important roles of the Interpreter Stack is to keep a log of the functions which are called during the execution of bytecodes. For each function invoked, the interpreter keeps three lisp objects on the stack:

  | function | lexical environment | index to previous record |

The first item is the object which is funcalled. It can be a bytecodes object, a compiled function or a generic function. In the last two cases the lexical environment is just NIL. In the first case, the second item on the stack is the lexical environment on which the code is executed. Each of these records are popped out of the stack after function invocation.

Let us see how these invocation records are used for debugging.

  >(defun fact (x)                ;;;  Wrong definition of the
  (if (= x 0)                  ;;;  factorial function.
  one                      ;;;  one  should be  1.
  (* x (fact (1- x)))))

  >(fact 3)                       ;;;  Tries  3!
  Error: The variable ONE is unbound.
  Error signalled by IF.
  Broken at IF.
  >>:b                            ;;;  Backtrace.
  Backtrace: eval > fact > if > fact > if > fact > if > fact > IF
  ;;;  Currently at the last  IF.
  >>:h                            ;;;  Help.

  Break commands:
  :q(uit)         Return to some previous break level.
  :pop            Pop to previous break level.
  :c(ontinue)     Continue execution.
  :b(acktrace)    Print backtrace.
  :f(unction)     Show current function.
  :p(revious)     Go to previous function.
  :n(ext)         Go to next function.
  :g(o)           Go to next function.
  :fs             Search forward for function.
  :bs             Search backward for function.
  :v(ariables)    Show local variables, functions, blocks, and tags.
  :l(ocal)        Return the nth local value on the stack.
  :hide           Hide function.
  :unhide         Unhide function.
  :hp             Hide package.
  :unhp           Unhide package.
  :unhide-all     Unhide all variables and packages.
  :bds            Show binding stack.
  :m(essage)      Show error message.
  :hs             Help stack.
  Top level commands:
  :cf             Compile file.
  :exit or ^D     Exit Lisp.
  :ld             Load file.
  :step           Single step form.
  :tr(ace)        Trace function.
  :untr(ace)      Untrace function.

  Help commands:
  :apropos        Apropos.
  :doc(ument)     Document.
  :h(elp) or ?    Help.  Type ":help help" for more information.

  >>:p                        ;;;  Move to the last call of  FACT.
  Broken at IF.

  Backtrace: eval > fact > if > fact > if > fact > if > FACT > if
  ;;;  Now at the last  FACT.
  >>:v                        ;;;  The environment at the last call
  Local variables:            ;;;  to  FACT  is recovered.
  X: 0                      ;;;  X  is the only bound variable.
  Block names: FACT.          ;;;  The block  FACT  is established.

  0                           ;;;  The value of  x  is  0.

  >>(return-from fact 1)      ;;;  Return from the last call of
  6                           ;;;  FACT  with the value of  0.
  ;;;  The execution is resumed and
  >                           ;;;  the value  6  is returned.
  ;;;  Again at the top-level loop.

Next: , Previous: , Up: Developer's guide   [Contents][Index]

2.6 The compiler

2.6.1 The compiler translates to C

The ECL compiler is essentially a translator from Common-Lisp to C. Given a Lisp source file, the compiler first generates three intermediate files:

The ECL compiler then invokes the C compiler to compile the C-file into an object file. Finally, the contents of the Data-file is appended to the object file to make a Fasl-file. The generated Fasl-file can be loaded into the ECL system by the Common-Lisp function load. By default, the three intermediate files are deleted after the compilation, but, if asked, the compiler leaves them.

The merits of the use of C as the intermediate language are:

The demerits are:

2.6.2 The compiler mimics human C programmer

The format of the intermediate C code generated by the ECL compiler is the same as the hand-coded C code of the ECL source programs. For example, supposing that the Lisp source file contains the following function definition:

(defvar *delta* 2)
(defun add1 (x) (+ *delta* x))

The compiler generates the following intermediate C code.

/*	function definition for ADD1                                  */
static cl_object L1(cl_object V1)
cl_object value0;
value0=number_plus(symbol_value(VV[0]),V1); NVALUES=1;
return value0;
/*      initialization of this module                                 */
void init_CODE(cl_object flag)
cl_object value0;
if (!FIXNUMP(flag)){
flag-> = VV;
flag->cblock.data_size = VM;
flag->cblock.data_text = compiler_data_text;
flag->cblock.data_text_size = compiler_data_text_size;
VV = Cblock->;
if(SYM_VAL(T0)!=OBJNULL) cl_setq(VV[0],T0);

The C function L1 implements the Lisp function add1. This relation is established by cl_def_c_function in the initialization function init_CODE, which is invoked at load time. There, the vector VV consists of Lisp objects; VV[0] and VV[1] in this example hold the Lisp symbols *delta* and add1. VM in the definition of L1 is a C macro declared in the corresponding H-file. The actual value of VM is the number of value stack locations used by this module, i.e., 2 in this example. Thus the following macro definition is found in the H-file.

#define VM 2

2.6.3 Implementation of Compiled Closures

The ECL compiler takes two passes before it invokes the C compiler. The major role of the first pass is to detect function closures and to detect, for each function closure, those lexical objects (i.e., lexical variable, local function definitions, tags, and block-names) to be enclosed within the closure. This check must be done before the C code generation in the second pass, because lexical objects to be enclosed in function closures are treated in a different way from those not enclosed.

Ordinarily, lexical variables in a compiled function f are allocated on the C stack. However, if a lexical variable is to be enclosed in function closures, it is allocated on a list, called the "environment list", which is local to f. In addition, a local variable is created which points to the lexical variable’s location (within the environment list), so that the variable may be accessed through an indirection rather than by list traversal.

The environment list is a pushdown list: It is empty when f is called. An element is pushed on the environment list when a variable to be enclosed in closures is bound, and is popped when the binding is no more in effect. That is, at any moment during execution of f, the environment list contains those lexical variables whose binding is still in effect and which should be enclosed in closures. When a compiled closure is created during execution of f, the compiled code for the closure is coupled with the environment list at that moment to form the compiled closure.

Later, when the compiled closure is invoked, a pointer is set up to each lexical variable in the environment list, so that each object may be referenced through a memory indirection.

Let us see an example. Suppose the following function has been compiled.

(defun foo (x)
(let ((a #'(lambda () (incf x)))
(y x))
(values a #'(lambda () (incf x y)))))

foo returns two compiled closures. The first closure increments x by one, whereas the second closure increments x by the initial value of x. Both closures return the incremented value of x.

>(multiple-value-setq (f g) (foo 10))
#<compiled-closure nil>

>(funcall f)

>(funcall g)


After this, the two compiled closures look like:

  second closure       y:                     x:
  |-------|------|      |-------|------|       |------|------| 
  |  **   |    --|----->|  10   |    --|------>|  21  | nil  |
  |-------|------|      |-------|------|       |------|------| 
  first closure             |
  |-------|------|          |
  |   *   |    --|----------| 

  * : address of the compiled code for #'(lambda () (incf x))
  ** : address of the compiled code for #'(lambda () (incf x y))

2.6.4 Use of Declarations to Improve Efficiency

Declarations, especially type and function declarations, increase the efficiency of the compiled code. For example, for the following Lisp source file, with two Common-Lisp declarations added,

(eval-when (compile)
(proclaim '(function tak (fixnum fixnum fixnum) fixnum))

(defun tak (x y z)
(declare (fixnum x y z))
(if (not (< y x))
(tak (tak (1- x) y z)
(tak (1- y) z x)
(tak (1- z) x y))))

The compiler generates the following C code:

/*      local entry for function TAK                                  */
static int LI1(register int V1,register int V2,register int V3)
if (V2 < V1) {
goto L2;}
{ int V5;
V5 = LI1((V1)-1,V2,V3);
{ int V6;
V6 = LI1((V2)-1,V3,V1);
V3 = LI1((V3)-1,V1,V2);
V2 = V6;
V1 = V5;}}
goto TTL;
;;; Note: Tail-recursive call of TAK was replaced by iteration.

2.6.5 Inspecting generated C code

Common-Lisp defines a function disassemble, which is supposed to disassemble a compiled function and to display the assembler code. According to Common-Lisp: The Language,

This is primary useful for debugging the compiler, ..\\

This is, however, useless in our case, because we are not concerned with assembly language. Rather, we are interested in the C code generated by the ECL compiler. Thus the disassemble function in ECL accepts not-yet-compiled functions only and displays the translated C code.

> (defun add1 (x) (1+ x))
> (disassemble *)
;;; Compiling (DEFUN ADD1 ...).
;;; Emitting code for ADD1.

/*      function definition for ADD1                                  */
static L1(int narg, object V1)
VALUES(0) = one_plus((V1));

Next: , Previous: , Up: Developer's guide   [Contents][Index]

2.7 Porting ECL

To port ECL to a new architecture, the following steps are required:

  1. Ensure that the GNU Multiprecision library supports this machine.
  2. Ensure that the Boehm-Weiser garbage collector is supported by that architecture. Alternatively, port ECL’s own garbage collector src/c/alloc.d and src/c/gbc.d to that platform.
  3. Fix src/, src/h/ and src/h/ecl.h so that they supply flags for the new host machine.
  4. Fix the machine dependent code in src/c/. The most critical parts are in the unix*.d and thread*.d files.
  5. Compile as in any other platform.
  6. Run the tests and compare to the results of other platforms.

Previous: , Up: Developer's guide   [Contents][Index]

2.8 Removed features

In-house DFFI

Commit 10bd3b613fd389da7640902c2b88a6e36088c920. Native DFFI was replaced by a libffi long time ago, but we have maintained the code as a fallback. Due to small number of supported platforms and no real use it has been removed in 2016.

In-house GC

Commit 61500316b7ea17d0e42f5ca127f2f9fa3e6596a8. Broken GC is replaced by BoehmGC library. This may be added back as a fallback in the near future.

3bd9799a2fef21cc309472e604a46be236b155c7 removes a leftover (apparently gbc.d wasn’t bdwgc glue).

Green threads

Commit 41923d5927f31f4dd702f546b9caee74e98a2080. Green threads (aka light weight processes) has been replaced with native threads implementation. There is an ongoing effort to bring them back as an alternative interface.

Compiler newcmp

Commit 9b8258388487df8243e2ced9c784e569c0b34c4f This was abandoned effort of changing the compiler architecture. Some clever ideas and a compiler package hierarchy. Some of these things should be incorporated during the evolution of the primary compiler.

Old MIT loop

Commit 5042589043a7be853b7f85fd7a996747412de6b4. This old loop implementation has got superseeded by the one incorporated from Symbolics LOOP in 2001.

Support for bignum arithmetic (earith.d)

Commit edfc2ba785d6a64667e89c869ef0a872d7b9704b. Removes pre-gmp bignum code. Name comes probably from “extended arithmetic”, contains multiplication and division routines (assembler and a portable implementation).

Unification module

Commit 6ff5d20417a21a76846c4b28e532aac097f03109. Old unifiction module (logic programming) from EcoLisp times.

Hierarchical packages

Commit 72e422f1b3c4b3c52fa273b961517db943749a8f. Partially broken. Tests left in package-extensions.lsp.

Previous: , Up: Developer's guide   [Contents][Index]