| Chapter 9 |
Interprocess
Communication under LISP |
|
by William Lott and Bill Chiles
CMUCL offers a facility for interprocess communication (IPC) on top
of using Unix system calls and the complications of that level of
IPC. There is a simple remote-procedure-call (RPC) package build on
top of TCP/IP sockets.
The remote package provides simple RPC
facility including interfaces for creating servers, connecting to
already existing servers, and calling functions in other Lisp
processes. The routines for establishing a connection between two
processes, create-request-server and
connect-to-remote-server, return wire structures. A wire maintains the current state
of a connection, and all the RPC forms require a wire to indicate
where to send requests.
| 9.1.1 |
Connecting
Servers and Clients |
|
Before a client can connect to a server, it must know the network
address on which the server accepts connections. Network addresses
consist of a host address or name, and a port number. Host
addresses are either a string of the form VANCOUVER.SLISP.CS.CMU.EDU or a 32 bit unsigned
integer. Port numbers are 16 bit unsigned integers. Note:
port in this context has nothing to do
with Mach ports and message passing.
When a process wants to receive connection requests (that is,
become a server), it first picks an integer to use as the port.
Only one server (Lisp or otherwise) can use a given port number on
a given machine at any particular time. This can be an iterative
process to find a free port: picking an integer and calling
create-request-server. This function signals
an error if the chosen port is unusable. You will probably want to
write a loop using handler-case, catching
conditions of type error, since this function does not signal more
specific conditions.
[Function]
wire:create-request-server port &optional on-connect
create-request-server sets up the current
Lisp to accept connections on the given port. If port is
unavailable for any reason, this signals an error. When a client
connects to this port, the acceptance mechanism makes a wire
structure and invokes the on-connect
function. Invoking this function has a couple of purposes, and
on-connect may be nil in which case the system foregoes invoking any
function at connect time.
The on-connect function is both a hook
that allows you access to the wire created by the acceptance
mechanism, and it confirms the connection. This function takes two
arguments, the wire and the host address of the connecting process.
See the section on host addresses below. When on-connect is nil, the
request server allows all connections. When it is non-nil, the function returns two values, whether to accept
the connection and a function the system should call when the
connection terminates. Either value may be nil, but when the first value is nil, the acceptance mechanism destroys the wire.
create-request-server returns an object that
destroy-request-server uses to terminate a
connection.
[Function]
wire:destroy-request-server server
destroy-request-server takes the result of
create-request-server and terminates that
server. Any existing connections remain intact, but all additional
connection attempts will fail.
[Function]
wire:connect-to-remote-server host port &optional on-death
connect-to-remote-server attempts to connect
to a remote server at the given port on
host and returns a wire structure if it
is successful. If on-death is
non-nil, it is a function the system invokes
when this connection terminates.
After the server and client have connected, they each have a wire
allowing function evaluation in the other process. This RPC
mechanism has three flavors: for side-effect only, for a single
value, and for multiple values.
Only a limited number of data types can be sent across wires as
arguments for remote function calls and as return values: integers
inclusively less than 32 bits in length, symbols, lists, and
remote-objects (see section 9.1.3). The system sends symbols as two strings,
the package name and the symbol name, and if the package doesn't
exist remotely, the remote process signals an error. The system
ignores other slots of symbols. Lists may be any tree of the above
valid data types. To send other data types you must represent them
in terms of these supported types. For example, you could use
prin1-to-string locally, send the string, and
use read-from-string remotely.
[Macro]
wire:remote wire
{call-specs}*
The remote macro arranges for the process at
the other end of wire to invoke each of
the functions in the call-specs. To make
sure the system sends the remote evaluation requests over the wire,
you must call wire-force-output.
Each of call-specs looks like a function
call textually, but it has some odd constraints and semantics. The
function position of the form must be the symbolic name of a
function. remote evaluates each of the
argument subforms for each of the call-specs locally in the current context, sending
these values as the arguments for the functions.
Consider the following example:
(defun write-remote-string (str)
(declare (simple-string str))
(wire:remote wire
(write-string str)))
The value of str in the local process is
passed over the wire with a request to invoke write-string on the value. The system does not expect
to remotely evaluate str for a value in the
remote process.
[Function]
wire:wire-force-output wire
wire-force-output flushes all internal
buffers associated with wire, sending the
remote requests. This is necessary after a call to remote.
[Macro]
wire:remote-value wire
call-spec
The remote-value macro is similar to the
remote macro. remote-value only takes one call-spec, and it returns the value returned by the
function call in the remote process. The value must be a valid type
the system can send over a wire, and there is no need to call
wire-force-output in conjunction with this
interface.
If client unwinds past the call to remote-value, the server continues running, but the
system ignores the value the server sends back.
If the server unwinds past the remotely requested call, instead of
returning normally, remote-value returns two
values, nil and t.
Otherwise this returns the result of the remote evaluation and
nil.
[Macro]
wire:remote-value-bind wire ({variable}*)
remote-form {local-forms}*
remote-value-bind is similar to multiple-value-bind except the values bound come from
remote-form's evaluation in the remote
process. The local-forms execute in an
implicit progn.
If the client unwinds past the call to remote-value-bind, the server continues running, but
the system ignores the values the server sends back.
If the server unwinds past the remotely requested call, instead of
returning normally, the local-forms never
execute, and remote-value-bind returns
nil.
The wire mechanism only directly supports a limited number of data
types for transmission as arguments for remote function calls and
as return values: integers inclusively less than 32 bits in length,
symbols, lists. Sometimes it is useful to allow remote processes to
refer to local data structures without allowing the remote process
to operate on the data. We have remote-objects to support this without the need to
represent the data structure in terms of the above data types, to
send the representation to the remote process, to decode the
representation, to later encode it again, and to send it back along
the wire.
You can convert any Lisp object into a remote-object. When you send
a remote-object along a wire, the system simply sends a unique
token for it. In the remote process, the system looks up the token
and returns a remote-object for the token. When the remote process
needs to refer to the original Lisp object as an argument to a
remote call back or as a return value, it uses the remote-object it
has which the system converts to the unique token, sending that
along the wire to the originating process. Upon receipt in the
first process, the system converts the token back to the same
(eq) remote-object.
[Function]
wire:make-remote-object object
make-remote-object returns a remote-object
that has object as its value. The
remote-object can be passed across wires just like the directly
supported wire data types.
[Function]
wire:remote-object-p object
The function remote-object-p returns
t if object is a
remote object and nil otherwise.
[Function]
wire:remote-object-local-p remote
The function remote-object-local-p returns
t if remote refers
to an object in the local process. This is can only occur if the
local process created remote with
make-remote-object.
[Function]
wire:remote-object-eq obj1 obj2
The function remote-object-eq returns
t if obj1 and
obj2 refer to the same (eq) lisp object, regardless of which process created
the remote-objects.
[Function]
wire:remote-object-value remote
This function returns the original object used to create the given
remote object. It is an error if some other process originally
created the remote-object.
[Function]
wire:forget-remote-translation object
This function removes the information and storage necessary to
translate remote-objects back into object, so the next gc can
reclaim the memory. You should use this when you no longer expect
to receive references to object. If some
remote process does send a reference to object, remote-object-value
signals an error.
The wire package provides for sending data
along wires. The remote package sits on top
of this package. All data sent with a given output routine must be
read in the remote process with the complementary fetching routine.
For example, if you send so a string with wire-output-string, the remote process must know to use
wire-get-string. To avoid rigid data
transfers and complicated code, the interface supports sending
tagged data. With tagged data, the system
sends a tag announcing the type of the next data, and the remote
system takes care of fetching the appropriate type.
When using interfaces at the wire level instead of the RPC level,
the remote process must read everything sent by these routines. If
the remote process leaves any input on the wire, it will later
mistake the data for an RPC request causing unknown lossage.
When using these routines both ends of the wire know exactly what
types are coming and going and in what order. This data is
restricted to the following types:
- 8 bit unsigned bytes.
- 32 bit unsigned bytes.
- 32 bit integers.
- simple-strings less than 65535 in length.
[Function]
wire:wire-output-byte wire byte
[Function]
wire:wire-get-byte wire
[Function]
wire:wire-output-number wire number
[Function]
wire:wire-get-number wire
&optional signed
[Function]
wire:wire-output-string wire string
[Function]
wire:wire-get-string wire
These functions either output or input an object of the specified
data type. When you use any of these output routines to send data
across the wire, you must use the corresponding input routine
interpret the data.
When using these routines, the system automatically transmits and
interprets the tags for you, so both ends can figure out what kind
of data transfers occur. Sending tagged data allows a greater
variety of data types: integers inclusively less than 32 bits in
length, symbols, lists, and remote-objects (see section 9.1.3). The system sends symbols as two strings,
the package name and the symbol name, and if the package doesn't
exist remotely, the remote process signals an error. The system
ignores other slots of symbols. Lists may be any tree of the above
valid data types. To send other data types you must represent them
in terms of these supported types. For example, you could use
prin1-to-string locally, send the string, and
use read-from-string remotely.
[Function]
wire:wire-output-object wire object &optional cache-it
[Function]
wire:wire-get-object wire
The function wire-output-object sends
object over wire preceded by a tag indicating its type.
If cache-it is non-nil, this function only sends object the first time it gets object. Each end of the wire associates a token
with object, similar to remote-objects,
allowing you to send the object more efficiently on successive
transmissions. cache-it defaults to
t for symbols and nil
for other types. Since the RPC level requires function names, a
high-level protocol based on a set of function calls saves time in
sending the functions' names repeatedly.
The function wire-get-object reads the
results of wire-output-object and returns
that object.
| 9.2.3 |
Making Your Own
Wires |
|
You can create wires manually in addition to the remote package's interface creating them for you. To
create a wire, you need a Unix file descriptor. If you are
unfamiliar with Unix file descriptors, see section 2 of the Unix
manual pages.
[Function]
wire:make-wire descriptor
The function make-wire creates a new wire
when supplied with the file descriptor to use for the underlying
I/O operations.
[Function]
wire:wire-p object
This function returns t if object is indeed a wire, nil
otherwise.
[Function]
wire:wire-fd wire
This function returns the file descriptor used by the wire.
The TCP/IP protocol allows users to send data asynchronously,
otherwise known as out-of-band data. When
using this feature, the operating system interrupts the receiving
process if this process has chosen to be notified about out-of-band
data. The receiver can grab this input without affecting any
information currently queued on the socket. Therefore, you can use
this without interfering with any current activity due to other
wire and remote interfaces.
Unfortunately, most implementations of TCP/IP are broken, so use of
out-of-band data is limited for safety reasons. You can only
reliably send one character at a time.
The Wire package is built on top of CMUCLs networking support. In
view of this, it is possible to use the routines described in
section 10.6 for handling
and sending out-of-band data. These all take a Unix file descriptor
instead of a wire, but you can fetch a wire's file descriptor with
wire-fd.