Chapter 9 Interprocess Communication under LISP

by William Lott and Bill Chiles

CMUCL offers a facility for interprocess communication (IPC) on top of using Unix system calls and the complications of that level of IPC. There is a simple remote-procedure-call (RPC) package build on top of TCP/IP sockets.

9.1 The REMOTE Package

The remote package provides simple RPC facility including interfaces for creating servers, connecting to already existing servers, and calling functions in other Lisp processes. The routines for establishing a connection between two processes, create-request-server and connect-to-remote-server, return wire structures. A wire maintains the current state of a connection, and all the RPC forms require a wire to indicate where to send requests.

9.1.1 Connecting Servers and Clients

Before a client can connect to a server, it must know the network address on which the server accepts connections. Network addresses consist of a host address or name, and a port number. Host addresses are either a string of the form VANCOUVER.SLISP.CS.CMU.EDU or a 32 bit unsigned integer. Port numbers are 16 bit unsigned integers. Note: port in this context has nothing to do with Mach ports and message passing.

When a process wants to receive connection requests (that is, become a server), it first picks an integer to use as the port. Only one server (Lisp or otherwise) can use a given port number on a given machine at any particular time. This can be an iterative process to find a free port: picking an integer and calling create-request-server. This function signals an error if the chosen port is unusable. You will probably want to write a loop using handler-case, catching conditions of type error, since this function does not signal more specific conditions.

[Function]
wire:create-request-server port &optional on-connect

create-request-server sets up the current Lisp to accept connections on the given port. If port is unavailable for any reason, this signals an error. When a client connects to this port, the acceptance mechanism makes a wire structure and invokes the on-connect function. Invoking this function has a couple of purposes, and on-connect may be nil in which case the system foregoes invoking any function at connect time.

The on-connect function is both a hook that allows you access to the wire created by the acceptance mechanism, and it confirms the connection. This function takes two arguments, the wire and the host address of the connecting process. See the section on host addresses below. When on-connect is nil, the request server allows all connections. When it is non-nil, the function returns two values, whether to accept the connection and a function the system should call when the connection terminates. Either value may be nil, but when the first value is nil, the acceptance mechanism destroys the wire.

create-request-server returns an object that destroy-request-server uses to terminate a connection.

[Function]
wire:destroy-request-server server

destroy-request-server takes the result of create-request-server and terminates that server. Any existing connections remain intact, but all additional connection attempts will fail.

[Function]
wire:connect-to-remote-server host port &optional on-death

connect-to-remote-server attempts to connect to a remote server at the given port on host and returns a wire structure if it is successful. If on-death is non-nil, it is a function the system invokes when this connection terminates.

9.1.2 Remote Evaluations

After the server and client have connected, they each have a wire allowing function evaluation in the other process. This RPC mechanism has three flavors: for side-effect only, for a single value, and for multiple values.

Only a limited number of data types can be sent across wires as arguments for remote function calls and as return values: integers inclusively less than 32 bits in length, symbols, lists, and remote-objects (see section 9.1.3). The system sends symbols as two strings, the package name and the symbol name, and if the package doesn't exist remotely, the remote process signals an error. The system ignores other slots of symbols. Lists may be any tree of the above valid data types. To send other data types you must represent them in terms of these supported types. For example, you could use prin1-to-string locally, send the string, and use read-from-string remotely.

[Macro]
wire:remote wire {call-specs}^*

The remote macro arranges for the process at the other end of wire to invoke each of the functions in the call-specs. To make sure the system sends the remote evaluation requests over the wire, you must call wire-force-output.

Each of call-specs looks like a function call textually, but it has some odd constraints and semantics. The function position of the form must be the symbolic name of a function. remote evaluates each of the argument subforms for each of the call-specs locally in the current context, sending these values as the arguments for the functions.

Consider the following example:
(defun write-remote-string (str)
  (declare (simple-string str))
  (wire:remote wire
    (write-string str)))
The value of str in the local process is passed over the wire with a request to invoke write-string on the value. The system does not expect to remotely evaluate str for a value in the remote process.

[Function]
wire:wire-force-output wire

wire-force-output flushes all internal buffers associated with wire, sending the remote requests. This is necessary after a call to remote.

[Macro]
wire:remote-value wire call-spec

The remote-value macro is similar to the remote macro. remote-value only takes one call-spec, and it returns the value returned by the function call in the remote process. The value must be a valid type the system can send over a wire, and there is no need to call wire-force-output in conjunction with this interface.

If client unwinds past the call to remote-value, the server continues running, but the system ignores the value the server sends back.

If the server unwinds past the remotely requested call, instead of returning normally, remote-value returns two values, nil and t. Otherwise this returns the result of the remote evaluation and nil.

[Macro]
wire:remote-value-bind wire ({variable}^*) remote-form {local-forms}^*

remote-value-bind is similar to multiple-value-bind except the values bound come from remote-form's evaluation in the remote process. The local-forms execute in an implicit progn.

If the client unwinds past the call to remote-value-bind, the server continues running, but the system ignores the values the server sends back.

If the server unwinds past the remotely requested call, instead of returning normally, the local-forms never execute, and remote-value-bind returns nil.

9.1.3 Remote Objects

The wire mechanism only directly supports a limited number of data types for transmission as arguments for remote function calls and as return values: integers inclusively less than 32 bits in length, symbols, lists. Sometimes it is useful to allow remote processes to refer to local data structures without allowing the remote process to operate on the data. We have remote-objects to support this without the need to represent the data structure in terms of the above data types, to send the representation to the remote process, to decode the representation, to later encode it again, and to send it back along the wire.

You can convert any Lisp object into a remote-object. When you send a remote-object along a wire, the system simply sends a unique token for it. In the remote process, the system looks up the token and returns a remote-object for the token. When the remote process needs to refer to the original Lisp object as an argument to a remote call back or as a return value, it uses the remote-object it has which the system converts to the unique token, sending that along the wire to the originating process. Upon receipt in the first process, the system converts the token back to the same (eq) remote-object.

[Function]
wire:make-remote-object object

make-remote-object returns a remote-object that has object as its value. The remote-object can be passed across wires just like the directly supported wire data types.

[Function]
wire:remote-object-p object

The function remote-object-p returns t if object is a remote object and nil otherwise.

[Function]
wire:remote-object-local-p remote

The function remote-object-local-p returns t if remote refers to an object in the local process. This is can only occur if the local process created remote with make-remote-object.

[Function]
wire:remote-object-eq obj1 obj2

The function remote-object-eq returns t if obj1 and obj2 refer to the same (eq) lisp object, regardless of which process created the remote-objects.

[Function]
wire:remote-object-value remote

This function returns the original object used to create the given remote object. It is an error if some other process originally created the remote-object.

[Function]
wire:forget-remote-translation object

This function removes the information and storage necessary to translate remote-objects back into object, so the next gc can reclaim the memory. You should use this when you no longer expect to receive references to object. If some remote process does send a reference to object, remote-object-value signals an error.

9.2 The WIRE Package

The wire package provides for sending data along wires. The remote package sits on top of this package. All data sent with a given output routine must be read in the remote process with the complementary fetching routine. For example, if you send so a string with wire-output-string, the remote process must know to use wire-get-string. To avoid rigid data transfers and complicated code, the interface supports sending tagged data. With tagged data, the system sends a tag announcing the type of the next data, and the remote system takes care of fetching the appropriate type.

When using interfaces at the wire level instead of the RPC level, the remote process must read everything sent by these routines. If the remote process leaves any input on the wire, it will later mistake the data for an RPC request causing unknown lossage.

9.2.1 Untagged Data

When using these routines both ends of the wire know exactly what types are coming and going and in what order. This data is restricted to the following types:

8 bit unsigned bytes.
32 bit unsigned bytes.
32 bit integers.
simple-strings less than 65535 in length.

[Function]
wire:wire-output-byte wire byte

[Function]
wire:wire-get-byte wire

[Function]
wire:wire-output-number wire number

[Function]
wire:wire-get-number wire &optional signed

[Function]
wire:wire-output-string wire string

[Function]
wire:wire-get-string wire

These functions either output or input an object of the specified data type. When you use any of these output routines to send data across the wire, you must use the corresponding input routine interpret the data.

9.2.2 Tagged Data

When using these routines, the system automatically transmits and interprets the tags for you, so both ends can figure out what kind of data transfers occur. Sending tagged data allows a greater variety of data types: integers inclusively less than 32 bits in length, symbols, lists, and remote-objects (see section 9.1.3). The system sends symbols as two strings, the package name and the symbol name, and if the package doesn't exist remotely, the remote process signals an error. The system ignores other slots of symbols. Lists may be any tree of the above valid data types. To send other data types you must represent them in terms of these supported types. For example, you could use prin1-to-string locally, send the string, and use read-from-string remotely.

[Function]
wire:wire-output-object wire object &optional cache-it

[Function]
wire:wire-get-object wire

The function wire-output-object sends object over wire preceded by a tag indicating its type.

If cache-it is non-nil, this function only sends object the first time it gets object. Each end of the wire associates a token with object, similar to remote-objects, allowing you to send the object more efficiently on successive transmissions. cache-it defaults to t for symbols and nil for other types. Since the RPC level requires function names, a high-level protocol based on a set of function calls saves time in sending the functions' names repeatedly.

The function wire-get-object reads the results of wire-output-object and returns that object.

9.2.3 Making Your Own Wires

You can create wires manually in addition to the remote package's interface creating them for you. To create a wire, you need a Unix file descriptor. If you are unfamiliar with Unix file descriptors, see section 2 of the Unix manual pages.

[Function]
wire:make-wire descriptor

The function make-wire creates a new wire when supplied with the file descriptor to use for the underlying I/O operations.

[Function]
wire:wire-p object

This function returns t if object is indeed a wire, nil otherwise.

[Function]
wire:wire-fd wire

This function returns the file descriptor used by the wire.

9.3 Out-Of-Band Data

The TCP/IP protocol allows users to send data asynchronously, otherwise known as out-of-band data. When using this feature, the operating system interrupts the receiving process if this process has chosen to be notified about out-of-band data. The receiver can grab this input without affecting any information currently queued on the socket. Therefore, you can use this without interfering with any current activity due to other wire and remote interfaces.

Unfortunately, most implementations of TCP/IP are broken, so use of out-of-band data is limited for safety reasons. You can only reliably send one character at a time.

The Wire package is built on top of CMUCLs networking support. In view of this, it is possible to use the routines described in section 10.6 for handling and sending out-of-band data. These all take a Unix file descriptor instead of a wire, but you can fetch a wire's file descriptor with wire-fd.