Previous: Multithreaded Web Applications, Up: Design Patterns
This section contains a collection of case studies or overviews of read-world applications that have exploited Elephant.
Elephant is used by Konsenti(tm), a for-profit company of Robert L. Read, one of the maintainers of Elephant. It can be visited at http://konsenti.com.
Konsenti uses the Data Collection Management (DCM) package, found in
the src/contrib/rread directory
. DCM provides
prevalence-style in-memory write-through caching. The most enjoyable
feature about Elephant for this project is that new Business Layer
objects can be created without having to deal with an
Object-Relational Mapping, enabling extremely rapid development.
All Business objects are persisted via a director
in DCM (which
sits on top of Elephant.) Many of these business objects are in fact
finite state machines decorated with functions. The functions are
represented by lambda s-expressions stored in slots on the business
objects. A complete Message Factory and double-entry accounting
system are also implemented as DCM objects. Binary objects, such as
uploaded PDFs, can be attached to objects as comments and are stored
directly in Elephant. Konsenti is based on utf-8, and unicode
characters outside of the ISO-8859-1 character set are routinely
stored in Elephant. Konsenti uses Postgres as a backend for licensing
reasons; but use of other data stores is possible.
Conceptminer is an Elephant-based web-mining framework developed by Ian Eslick (http://www.media.mit.edu/~eslick) that performs large-scale text analysis over the web to identify semantic relationships such as “PartOf”, “DesireOf” and “EffectOf” between English phrases.
Elephant's persistence capability is used to keep full records of all source material, extracted relationships and search queries so that it is always possible to trace the source of a learned relation and to avoid repeated queries to web search engines. Conceptminer used Elephant 0.6.0 and the development branch of Elephant 0.9 to perform months of analysis consisting of millions of pages and a page/query database of over ten gigabytes.
There are several interesting uses and extensions of Elephant in Conceptminer:
The most interesting use of Elephant was extending its transactional
architecture to cover in-memory lisp operations. PCOMP
(Process
Components) is a framework for constructing and managing simple,
dataflow-style multi-threaded applications in Common Lisp. The goal
is to simplify the process sufficiently so that the ordinary user can
hide from many of the details associated with aborting transactions.
To this end, the model provides for safe, asynchronous communications
among a set of components which may be scheduled together in a single
process or communicate across separate threads (and potentially
processes). Components are packaged into a system inside a Container
object which schedules execution and mediates communications.
Communications between components can be in a dataflow style or using messages. Each component has a single port for receiving incoming data items. These items, if access is shared among components, should have the proper synchronization protections on mutating accesses. There is also an asynchronous communications method allowing you to send messages to components with particular names.
The basic building block is a component. Components are defined using the defcomponent form and contain several major elements, such as:
(defcomponent counter (:vars (count 0) end (increment 1)) (:initialize (assert end)) (:body (when (>= (incf count increment) end) (terminate))))
The arguments to defcomponent behave as follows:
:vars
- Values that the body wants to retain between invocations
:initialize
- A reserved message handler called at the begining of time
:body
- A body expression that is executed whenever data has arrived
The body and messages are evaluated in a very specific environment. Within the body certain variables and functions are bound:
data
- The current data item
self
- The component object
"vars"
- all variables named in :vars
are bound using symbol-macrolet
and available as in a let
statement. Any side effects to those vars are visible, but not saved to th component state until the component commits (see below).
(terminate)
(send data)
(receive data)
(get-ctrl-msg target type data)
(pause)
(abort)
Each component execution is bound in a transactional framework. No
variables are written, messages consumed or messages sent until the
body or control handler has exited normally. Users can tap into this
transactional framework by overriding start-transaction
,
commit-transaction
and abort-transaction
methods for the
component class. Transactional variables are implemented via
:after
methods on these generic functions.
When signals are asserted by the body or a message handler, they are also wrapped in restart handlers called: