On most platforms, ECL supports native multithreading. That means there can be several tasks executing lisp code on parallel and sharing memory, variables and files. The interface for multitasking in ECL, like those of most other implementations, is based on a set of functions and types that resemble the multiprocessing capabilities of old Lisp Machines.
This backward compatibility is why tasks or threads are called "processes". However, they should not be confused with operating system processes, which are made of programs running in separate contexts and without access to each other’s memory.
The implementation of threads in ECL is purely native and based on Posix Threads wherever available. The use of native threads has advantages. For instance, they allow for non-blocking file operations, so that while one task is reading a file, a different one is performing a computation.
As mentioned above, tasks share the same memory, as well as the set of open files and sockets. This manifests on two features. First of all, different tasks can operate on the same lisp objects, reading and writing their slots, or manipulating the same arrays. Second, while threads share global variables, constants and function definitions they can also have thread-local bindings to special variables that are not seen by other tasks.
The fact that different tasks have access to the same set of data allows both for flexibility and a greater risk. In order to control access to different resources, ECL provides the user with locks, as explained in the next section.
Process is a primitive representing native thread.
cl_object mp_all_processes () ¶Returns the list of processes associated to running tasks. The list is a fresh new one and can be destructively modified. However, it may happen that the output list is not up to date, because some of the tasks have expired before this copy is returned.
cl_object mp_exit_process () ecl_attr_noreturn ¶When called from a running task, this function immediately causes the
task to finish. When invoked from the main thread, it is equivalent to
invoking ext:quit with exit code 0.
cl_object mp_interrupt_process (cl_object process, cl_object function) ¶Interrupt a task. This function sends a signal to a running process. When the task is free to process that signal, it will stop whatever it is doing and execute the given function.
WARNING: Use with care! Interrupts can happen anywhere,
except in code regions explicitely protected with
mp:without-interrupts. This can lead to dangerous situations
when interrupting functions which are not thread safe. In particular,
one has to consider:
throw or return-from in the interrupting code will handle unwind-protect forms like usual. However, the cleanup forms of an unwind-protect can still be interrupted. In that case the execution flow will jump to the next unwind-protect.
Note also that no guarantees are made that functions from the Common
Lisp standard or ECL extensions are interrupt safe (although most of
them will be). In particular, the compiler (compile and
compile-file functions), FFI calls and aquire/release functions
for multithreading synchronization objects like mutexes or condition
variables should not be interrupted by mp:interrupt-process.
Example:
Kill a task that is doing nothing (See mp:process-kill).
(flet ((task-to-be-killed ()
;; Infinite loop
(loop (sleep 1))))
(let ((task (mp:process-run-function 'background #'task-to-be-killed)))
(sleep 10)
(mp:interrupt-process task 'mp:exit-process)))
cl_object mp_make_process (cl_narg narg, ...) ¶Create a new thread. This function creates a separate task with a name
set to name and no function to run. See also
mp:process-run-function. Returns newly created process.
If initial-bindings is false, the new process inherits local
bindings to special variables (i.e. binding a special variable with
let or let*) from the current thread, otherwise the new
thread possesses no local bindings.
cl_object mp_process_active_p (cl_object process) ¶Returns t when process is active, nil
otherwise. Signals an error if process doesn’t designate a valid
process.
cl_object mp_process_enable (cl_object process) ¶The argument to this function should be a process created by
mp:make-process, which has a function associated as per
mp:process-preset but which is not yet running. After invoking
this function a new thread will be created in which the associated
function will be executed. Returns process if the thread
creation was successful and nil otherwise.
(defun process-run-function (process-name process-function &rest args)
(let ((process (mp:make-process name)))
(apply #'mp:process-preset process function args)
(mp:process-enable process)))
cl_object mp_process_yield () ¶Yield the processor to other threads.
cl_object mp_process_join (cl_object process) ¶Suspend current thread until process exits. Return the result values of the process function.
cl_object mp_process_kill (cl_object process) ¶Try to stop a running task. Killing a process may fail if the task has disabled interrupts.
Example:
Kill a task that is doing nothing
(flet ((task-to-be-killed ()
;; Infinite loop
(loop (sleep 1))))
(let ((task (mp:process-run-function 'background #'task-to-be-killed)))
(sleep 10)
(mp:process-kill task)))
cl_object mp_process_suspend (cl_object process) ¶Suspend a running process. May be resumed with
mp:process-resume.
Example:
(flet ((ticking-task ()
;; Infinite loop
(loop
(sleep 1)
(print :tick))))
(print "Running task (one tick per second)")
(let ((task (mp:process-run-function 'background #'ticking-task)))
(sleep 5)
(print "Suspending task for 5 seconds")
(mp:process-suspend task)
(sleep 5)
(print "Resuming task for 5 seconds")
(mp:process-resume task)
(sleep 5)
(print "Killing task")
(mp:process-kill task)))
cl_object mp_process_resume (cl_object process) ¶Resumes a suspended process. See example in
mp:process-suspend.
cl_object mp_process_name (cl_object process) ¶Returns the name of a process (if any).
cl_object mp_process_preset (cl_narg narg, cl_object process, cl_object function, ...) ¶Associates a function to call with the arguments function-args, with a stopped process. The function will be the entry point when the task is enabled in the future.
See mp:process-enable and mp:process-run-function.
cl_object mp_process_run_function (cl_narg narg, cl_object name, cl_object function, ...) ¶Create a new process using mp:make-process, associate a function
to it and start it using mp:process-preset.
Example:
(flet ((count-numbers (end-number)
(dotimes (i end-number)
(format t "~%;;; Counting: ~i" i)
(terpri)
(sleep 1))))
(mp:process-run-function 'counter #'count-numbers 10))
cl_object mp_current_process () ¶Returns/holds the current process of a caller.
cl_object mp_block_signals () ¶Blocks process for interrupts and returns the previous sigmask.
See mp:interrupt-process.
cl_object mp_restore_signals (cl_object sigmask) ¶Enables the interrupts from sigmask.
See mp:interrupt-process.
Executes body with all deferrable interrupts disabled. Deferrable interrupts arriving during execution of the body take effect after body has been executed.
Deferrable interrupts include most blockable POSIX signals, and
mp:interrupt-process. Does not interfere with garbage
collection, and unlike in many traditional Lisps using userspace
threads, in ECL mp:without-interrupts does not inhibit
scheduling of other threads.
Binds mp:allow-with-interrupts,
mp:with-local-interrupts and
mp:with-restored-interrupts as a local macros.
mp:with-restored-interrupts executes the body with interrupts
enabled if and only if the mp:without-interrupts was in an
environment in which interrupts were allowed.
mp:allow-with-interrupts allows the
mp:with-interrupts to take effect during the dynamic scope of
its body, unless there is an outer mp:without-interrupts
without a corresponding mp:allow-with-interrupts.
mp:with-local-interrupts executes its body with interrupts
enabled provided that there is an mp:allow-with-interrupts for
every mp:without-interrupts surrounding the current one.
mp:with-local-interrupts is equivalent to:
(mp:allow-with-interrupts (mp:with-interrupts ...))
Care must be taken not to let either mp:allow-with-interrupts
or mp:with-local-interrupts appear in a function that escapes
from inside the mp:without-interrupts in:
(mp:without-interrupts ;; The body of the lambda would be executed with WITH-INTERRUPTS allowed ;; regardless of the interrupt policy in effect when it is called. (lambda () (mp:allow-with-interrupts ...))) (mp:without-interrupts ;; The body of the lambda would be executed with interrupts enabled ;; regardless of the interrupt policy in effect when it is called. (lambda () (mp:with-local-interrupts ...)))
Executes body with deferrable interrupts conditionally enabled. If there are pending interrupts they take effect prior to executing body.
As interrupts are normally allowed mp:with-interrupts only
makes sense if there is an outer mp:without-interrupts with a
corresponding mp:allow-with-interrupts: interrupts are not
enabled if any outer mp:without-interrupts is not accompanied
by mp:allow-with-interrupts.
Locks are used to synchronize access to the shared data. Lock may be owned only by a single thread at any given time. Recursive locks may be re-acquired by the same thread multiple times (and non-recursive locks can’t).
cl_object ecl_make_lock (cl_object name, bool recursive) ¶C/C++ equivalent of mp:make-lock without key arguments.
See mp:make-lock.
Creates a lock named name. If recursive is true, a recursive lock is created that can be locked multiple times by the same thread.
cl_object mp_recursive_lock_p (cl_object lock) ¶Predicate verifying if lock is recursive.
cl_object mp_holding_lock_p (cl_object lock) ¶Predicate verifying if the current thread holds lock.
cl_object mp_lock_name (cl_object lock) ¶Returns the name of lock.
cl_object mp_lock_owner (cl_object lock) ¶Returns the process owning lock or nil if the mutex is not
owned by any process. For testing whether the current thread
is holding a lock see mp:holding-lock-p.
cl_object mp_lock_count (cl_object lock) ¶Returns number of times lock has been locked.
cl_object mp_get_lock_wait (cl_object lock) ¶Grabs a lock (blocking if lock is already taken). Returns
ECL_T.
cl_object mp_get_lock_nowait ¶Grabs a lock if free (non-blocking). If lock is already taken
returns ECL_NIL, otherwise ECL_T.
Tries to acquire a lock. wait indicates whether function should
block or give up if lock is already taken. If wait is
nil, immediately return, if wait is a real number
wait specifies a timeout in seconds and otherwise block until the
lock becomes available. If lock can’t be acquired return
nil. Successful operation returns t. Will signal an error
if the mutex is non-recursive and current thread already owns the lock.
cl_object mp_giveup_lock (cl_object lock) ¶Releases lock and returns t. May signal an error if the
lock is not owned by the current thread.
Acquire lock for the dynamic scope of body, which is executed with the lock held by current thread. Returns the values of body.
Readers-writer (or shared-exclusive ) locks allow concurrent access
for read-only operations, while write operations require exclusive
access. mp:rwlock is non-recursive and cannot be used together
with condition variables.
cl_object ecl_make_rwlock (cl_object name) ¶C/C++ equivalent of mp:make-rwlock without key arguments.
See mp:make-rwlock.
Creates a rwlock named name.
cl_object mp_rwlock_name (cl_object lock) ¶Returns the name of lock.
cl_object mp_get_rwlock_read_wait (cl_object lock) ¶Acquires lock (blocks if lock is already taken with
mp:get-rwlock-write. Lock may be acquired by multiple
readers). Returns ECL_T.
cl_object mp_get_rwlock_read_nowait ¶Tries to acquire lock. If lock is already taken with
mp:get-rwlock-write returns ECL_NIL, otherwise
ECL_T.
Tries to acquire lock. wait indicates whenever function
should block or give up if lock is already taken with
mp:get-rwlock-write.
cl_object mp_get_rwlock_write_wait (cl_object lock) ¶Acquires lock (blocks if lock is already taken). Returns
ECL_T.
cl_object mp_get_rwlock_write_nowait ¶Tries to acquire lock. If lock is already taken returns
ECL_NIL, otherwise ECL_T.
Tries to acquire lock. wait indicates whenever function should block or give up if lock is already taken.
cl_object mp_giveup_rwlock_read (cl_object lock) ¶cl_object mp_giveup_rwlock_write (cl_object lock) ¶Acquire rwlock for the dynamic scope of body for operation operation, which is executed with the lock held by current thread. Returns the values of body.
Valid values of argument operation are :read or
:write (for reader and writer access accordingly).
Condition variables are used to wait for a particular condition becoming true (e.g new client connects to the server).
cl_object mp_make_condition_variable () ¶Creates a condition variable.
cl_object mp_condition_variable_wait (cl_object cv, cl_object lock) ¶Release lock and suspend thread until
mp:condition-variable-signal or
mp:condition-variable-broadcast is called on cv. When
thread resumes re-aquire lock. Always returns t. May signal
an error if lock is not owned by the current thread.
Note: In some circumstances, the thread may wake up even if no
call to mp:condition-variable-signal or
mp:condition-variable-broadcast has happened. It is
recommended to check for the condition that triggered the wait in a loop
around any mp:condition-variable-wait call.
Note: While the condition variable is blocked waiting for a
signal or broadcast event, calling mp:condition-variable-wait
from further threads must be done using the same mutex as that used by
the threads that are already waiting on this condition variable. The
behaviour is undefined if this constraint is violated.
cl_object mp_condition_variable_timedwait (cl_object cv, cl_object lock, cl_object seconds) ¶mp:condition-variable-wait which timeouts after seconds
seconds. Returns nil on timeout and t otherwise. May
signal an error if lock is not owned by the current thread.
cl_object mp_condition_variable_signal (cl_object cv) ¶Wake up at least one of the waiters of cv. Usually, this will wake
up only a single thread, but it may also wake up multiple threads.
Always returns t.
cl_object mp_condition_variable_broadcast (cl_object cv) ¶Wake up all waiters of cv. Always returns t.
Semaphores are objects which allow an arbitrary resource count. Semaphores are used for shared access to resources where number of concurrent threads allowed to access it is limited.
cl_object ecl_make_semaphore (cl_object name, cl_fixnum count) ¶C/C++ equivalent of mp:make-semaphore without key
arguments.
See mp:make-semaphore.
Creates a counting semaphore name with a resource count count.
cl_object mp_semaphore_name (cl_object semaphore) ¶Returns the name of semaphore.
cl_object mp_semaphore_count (cl_object semaphore) ¶Returns the resource count of semaphore.
cl_object mp_semaphore_wait_count (cl_object semaphore) ¶Returns the number of threads waiting on semaphore.
cl_object mp_sempahore_wait(cl_object semaphore, cl_object count, cl_object timeout) ¶Decrement the count of semaphore by count if the count would not be negative.
Else blocks until the semaphore can be decremented. Returns the old count of semaphore on success.
If timeout is not nil, it is the maximum number of seconds to
wait. If the count cannot be decremented in that time, returns
nil without decrementing the count.
cl_object mp_wait_on_semaphore (cl_narg n, cl_object sem, ...) ¶Waits on semaphore until it can grab count resources.
Returns resource count before semaphore was acquired.
This function is equivalent to (mp:semaphore-wait semaphore count timeout)
cl_object mp_try_get_semaphore (cl_narg n, cl_object sem, ...) ¶Tries to get a semaphore (non-blocking).
If there is no enough resource returns nil, otherwise returns
resource count before semaphore was acquired.
This function is equivalent to (mp:semaphore-wait semaphore count 0)
cl_object mp_signal_semaphore (cl_narg n, cl_object sem, ...); ¶Releases count units of a resource on semaphore. Returns no values.
Barriers are objects which for a group of threads make them stop and they can’t proceed until all other threads reach the barrier.
cl_object ecl_make_barrier (cl_object name, cl_index count) ¶C/C++ equivalent of mp:make-barrier without key
arguments.
See mp:make-barrier.
Creates a barrier name with a thread count count.
Returns the count of barrier.
Returns the name of barrier.
Returns the number of threads waiting on barrier.
The caller thread waits on barrier. When the barrier is saturated
then all threads waiting on it are unblocked. Returns t if the
calling thread had to wait to pass the barrier, :unblocked if the
barrier is enabled but could be passed without waiting and nil if
the barrier is disabled.
Forcefully wakes up all processes waiting on the barrier.
reset-count when used resets barrier counter.
disable disables or enables barrier. When a barrier is
disabled then all calls to mp:barrier-wait immedietely
return.
kill-waiting is used to kill all woken threads.
Returns no values.
ECL supports both compare-and-swap and fetch-and-add (which may be
faster on some processors) atomic operations on a number of different
places. The compare-and-swap macro is user extensible with a protocol
similar to setf.
cl_object ecl_compare_and_swap (cl_object *slot, cl_object old, cl_object new) ¶Perform an atomic compare and swap operation on slot and return
the previous value stored in slot. If the return value is equal
to old (comparison by ==), the operation has succeeded.
This is a inline-only function defined in "ecl/ecl_atomics.h".
cl_object ecl_atomic_incf (cl_object *slot, cl_object increment) ¶cl_object ecl_atomic_incf_by_fixnum (cl_object *slot, cl_fixnum increment) ¶Atomically increment slot by the given increment and return the
previous value stored in slot. The consequences are undefined if
the value of slot is not of type fixnum.
ecl_atomic_incf signals an error if increment is not of
type fixnum. This is a inline-only function defined in
"ecl/ecl_atomics.h".
cl_index ecl_atomic_index_incf (cl_index *slot); ¶Atomically increment slot by 1 and return the new value stored in slot.
cl_object ecl_atomic_get (cl_object *slot) ¶Perform a volatile load of the object in slot and then
atomically set slot to ECL_NIL. Returns the value
previously stored in slot.
void ecl_atomic_push (cl_object *slot, cl_object o) ¶cl_object ecl_atomic_pop (cl_object *slot) ¶Like push/pop but atomic.
Atomically increments/decrements the fixnum stored in place by
the given increment and returns the value of place before
the increment. Incrementing and decrementing is done using modular
arithmetic, so that mp:atomic-incf of a place whose value is
most-positive-fixnum by 1 results in
most-negative-fixnum stored in place.
Currently the following places are supported:
car, cdr, first, rest, svref,
symbol-value, slot-value,
clos:standard-instance-access,
clos:funcallable-standard-instance-access.
For slot-value, the object should have no applicable methods
defined for slot-value-using-class or (setf
slot-value-using-class).
The consequences are undefined if the value of place is not of
type fixnum.
Atomically stores new in place if old is eq
to the current value of place. Returns the previous value of
place: if the returned value is eq to old, the swap
was carried out.
Currently, the following places are supported:
car, cdr, first, rest, svref,
symbol-plist, symbol-value, slot-value,
clos:standard-instance-access,
clos:funcallable-standard-instance-access, a structure slot
accessor4 or any other place for which a compare-and-swap
expansion was defined by mp:defcas or
mp:define-cas-expander.
For slot-value, slot-unbound is called if the slot is
unbound unless old is eq to si:unbound, in which
case old is returned and new is assigned to the slot.
Additionally, the object should have no applicable methods defined for
slot-value-using-class or (setf slot-value-using-class).
Atomically updates the CAS-able place to the value returned by
calling update-fn with arguments and the old value of
place. update-fn must be a function accepting (1+
(length arguments)) arguments. Returns the new value which was stored
in place.
place may be read and update-fn may be called more than once if multiple threads are trying to write to place at the same time.
Example:
Atomic update of a structure slot. If the update would not be atomic, the result would be unpredictable.
(defstruct test-struct
(slot1 0))
(let ((struct (make-test-struct)))
(mapc #'mp:process-join
(loop repeat 100
collect (mp:process-run-function
""
(lambda ()
(loop repeat 1000 do
(mp:atomic-update (test-struct-slot1 struct) #'1+)
(sleep 0.00001))))))
(test-struct-slot1 struct))
=> 100000
Like push/pop, but atomic. place must be CAS-able
and may be read multiple times before the update succeeds.
Define a compare-and-swap expander similar to
define-setf-expander. Defines the compare-and-swap-expander for
generalized-variables (accessor ...). When a form
(mp:compare-and-swap (accessor arg1 ... argn) old new) is
evaluated, the forms given in the body of
mp:define-cas-expander are evaluated in order with the
parameters in lambda-list bound to arg1 ... argn. The
body must return six values
(var1 ... vark) (form1 ... formk) old-var new-var compare-and-swap-form volatile-access-form
in order (Note that old-var and new-var are single
variables, unlike in define-setf-expander). The whole
compare-and-swap form is then expanded into
(let* ((var1 from1) ... (vark formk)
(old-var old-form)
(new-var new-form))
compare-and-swap-form).
Note that it is up to the user of this macro to ensure atomicity for the resulting compare-and-swap expansions.
Example
mp:define-cas-expander can be used to define a more
convienient compare-and-swap expansion for a class slot. Consider the
following class:
(defclass food ()
((name :initarg :name)
(deliciousness :initform 5 :type '(integer 0 10)
:accessor food-deliciousness)))
(defvar *spätzle* (make-instance 'food :name "Spätzle"))
We can’t just use mp:compare-and-swap on
*spätzle*:
> (mp:compare-and-swap (food-deliciousness *x*) 5 10) Condition of type: SIMPLE-ERROR Cannot get the compare-and-swap expansion of (FOOD-DELICIOUSNESS *X*).
We can use symbol-value, but let’s define a more convenient
compare-and-swap expander:
(mp:define-cas-expander food-deliciousness (food)
(let ((old (gensym))
(new (gensym)))
(values nil nil old new
`(progn (check-type ,new (integer 0 10))
(mp:compare-and-swap (slot-value ,food 'deliciousness)
,old ,new))
`(food-deliciousness ,food))))
Now finally, we can safely store our rating:
> (mp:compare-and-swap (food-deliciousness *spätzle*) 5 10) 5
Define a compare-and-swap expansion similar to the short form
of defsetf. Defines an expansion
(compare-and-swap (accessor arg1 ... argn) old new) => (cas-fun arg1 ... argn old new)
Note that it is up to the user of this macro to ensure atomicity for the resulting compare-and-swap expansions.
Remove a compare-and-swap expansion. It is an equivalent of
fmakunbound (setf symbol) for cas expansions.
Returns the compare-and-swap expansion forms and variables as defined in
mp:define-cas-expander for place as six values.
The creation of atomic structure slot accessors can be
deactivated by supplying a (:atomic-accessors nil) option to
defstruct.