Network Programming in ANSI Common Lisp with IOLib
                   by: Peter Keller (psilord@cs.wisc.edu)
                               Version 0.0
                                4/02/2010


What is IOLib?
--------------

IOLib is a portable I/O library for ANSI Common Lisp. It includes socket
interfaces for network programming with IPV4/IPV6 TCP and UDP, an I/O
multiplexer that includes nonblocking I/O, a DNS resolver library, and a
pathname library.

Where do I get IOLib?
---------------------

The current version of IOLib is found here:

http://common-lisp.net/project/iolib/download.shtml

Please use the repository located in the Live Sources section for the most up
to date version of IOLib.

Introduction
------------

This tutorial loosely follows the exposition of network programming in "UNIX
Network Programming, Networking APIs: Sockets and XTI 2nd Edition" by W.
Richard Stevens. Many examples are derived from the source codes in that book.
Major deviations from the C sources include converting the concurrent examples
which use fork() into threaded examples which use the portable Bordeaux Threads
package, more structured implementations of certain concepts such as data
buffers and error handling, and general movement of coding style towards a
Common Lisp viewpoint.

The scope of this version of the tutorial is:
    0. Exposition suitable for programmers unfamiliar with ANSI Common Lisp
    1. IPV4 TCP
    2. Client/Server architecture
    3. Iterative vs Concurrent (via threading) vs Multiplexed Server Design
    4. Blocking and nonblocking I/O

It is intended, however, that this tutorial grows to contain the entirety of
IOLib's API as detailed in the Future Directions section of this tutorial. As
newer revisions of this tutorial are released, those gaps will be filled until
the whole of the IOLib API has been discussed.

Finally, the example code in this tutorial is algorithmically cut from the
actual example programs and inserted into the tutorial via a template
generation method. The example codes have embedded in them a tiny markup
language which facilitates this in the form (on a single line) of ';; ex-NNNb'
to begin an example section, and ';; ex-NNNe' to end an example section--NNN
stands for an enumeration integer for which each section's begin and end must
match.

Acknowledgements
----------------

I would like to greatly thank Stelian Ionescu, the author of IOLib
for his exposition of the various features of IOLib and his patience
in our sometimes long conversations.

Supporting Code
---------------

The file package.lisp contains a small library of codes used widely in the
examples. The supporting code implements:

    0. The package containing the examples, called :iolib.examples.

    1. The variables *host* and *port*, set to "localhost" and 9999
       respectively. This is the default name and port to which
       client connect and servers listen. Servers usually bind
       to 0.0.0.0, however.

    2. A small, but efficient, queue implementation, from "ANSI Common Lisp"
       by Paul Graham. The interface calls are:
       (make-queue)
       (enqueue obj q)
       (dequeue q)
       (empty-queue q)

    3. :iolib.examples currently depends upon IOLib alone and uses 
       packages :common-lisp, :iolib, and :bordeaux-threads.

Running the Examples
--------------------

These examples were developed and tested on SBCL 1.0.33.30 running on an x86
Ubuntu 8.10 machine. They were ran with two sessions of SBCL running, one
acting as a client, and the other as a server.

Supposing we'd like to start up the first example of the daytime server and
connect to it with the first daytime client example. Initially, the server will
bind to *host* and *port* and wait for the client to connect. We connect with
the client to *host* and *port*, get the time, and exit.

First we'll start up a server:

Linux black > sbcl
This is SBCL 1.0.33.30, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (require :iolib.examples) ; much output!
* (in-package :iolib.examples)

#<PACKAGE "IOLIB.EXAMPLES">
* (run-ex1-server)
Created socket: #<passive IPv4 stream socket, unbound {BF84B99}>[fd=5]
Bound socket: #<passive IPv4 stream socket bound to 0.0.0.0/9999 {BF84B99}>
Listening on socket bound to: 0.0.0.0:9999
Waiting to accept a connection...
[ server is waiting for the below client to connect! ]
Got a connection from 127.0.0.1:34794!
Sending the time...Sent!
T
*

Now we'll start up the client which connected to the above server:

Linux black > sbcl
This is SBCL 1.0.33.30, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (require :iolib.examples) ; much output!
* (in-package :iolib.examples)

#<PACKAGE "IOLIB.EXAMPLES">
* (run-ex1-client)
Connected to server 127.0.0.1:9999 via my local connection at 127.0.0.1:34794!
2/27/2010 13:51:48
T
*

In each client example, one can specify which host or port to which it should
connect:

* (run-ex1-client :host "localhost" :port 9999)
Connected to server 127.0.0.1:9999 via my local connection at 127.0.0.1:34798!
2/27/2010 13:53:7
T
*

The servers can be told a port they should listen upon and in this tutorial,
unless otherwise specified, will always bind to 0.0.0.0:9999 which means across
all interfaces on the machine and on port 9999.


                                   CHAPTER 1
                                   ---------
                             IPV4 TCP Client/Server
                          Blocking and nonblocking I/O

Overview of Examples
--------------------

The examples consist of a collection of clients and servers. They are split
into two groups: a set of daytime clients and server, and echo clients and
servers. In some of the examples, a certain network protocol, suppose
end-of-file handling, must be matched between client and server causing further
delineation.

Client protocols are matched to server protocols thusly:

Clients: ex1-client, ex2-client, ex3-client, can work
with servers: ex1-server, ex2-server, ex3-server, ex4-server.

Clients: ex4-client, ex5a-client, can work with servers: ex5-server,
ex6-server.

Clients: ex5b-client, can work with servers: ex7-server, ex8-server

Some clients and servers use the "daytime" series of protocols, those
are ex1-client, ex2-client, ex3-client, and ex1-server, ex2-server,
ex3-server, and ex4-server.

Some clients and servers use the "echo a line" series of protocols,
those are ex4-client, ex5a-client, ex5b-client, and ex5-server,
ex6-server, ex7-server, and ex8-server.

Even though much of the example source is included in the tutorial, it is
recommended that the example sources be carefully read and understood in order
to gain the most benefit from the tutorial.

Daytime Clients
---------------

In this section we show the evolution of a client which connects to a server
and gets the time of day. Each example shows some kind of an incremental
improvement to the previous one.

Daytime Client IVP4/TCP: ex1-client.lisp
----------------------------------------

This example is a very simple daytime client program which contacts a server,
by default at *host* and *port*, returns a single line of text that is the
current date and time, and then exits. It is written in more of a C style just
to make it easy to compare with similar simple examples in other languages. It
uses blocking, line oriented I/O.

The steps this program performs are:

0. The ex1-client.lisp entrance call:

+------------------------------------------------------------------------------+
|                                                                              |
(defun run-ex1-client (&key (host *host*) (port *port*))
|                                                                              |
+------------------------------------------------------------------------------+

1. Create an active TCP socket:

	The socket creation function (MAKE-SOCKET ...) is the method by which one
	creates a socket in IOLib.  It is very versatile and can be used to both
	create and initialize the socket in a single call.

	In this case, we use it simply and create an active IPV4 Internet stream
	socket which can read or write utf8 text and that understands a particular
	newline convention in the underlying data.

	One small, but important, deviation of IOLib sockets from Berkeley sockets
	is that when a socket is created, it is predestined to forever and
	unalterably be either an active or passive socket. Active sockets are used
	to connect to a server and passive sockets are used for a server's
	listening socket.

+------------------------------------------------------------------------------+
|                                                                              |
  ;; Create a internet TCP socket under IPV4
  (let ((socket
         (make-socket
          :connect :active
          :address-family :internet
          :type :stream
          :external-format '(:utf-8 :eol-style :crlf)
          :ipv6 nil)))
|                                                                              |
+------------------------------------------------------------------------------+

2. Specify the Server's IP address and port and establish a connection 
    with the server:

	This bit of code contains many calls into IOLib and we shall examine each
	of them. 

    The function LOOKUP-HOSTNAME takes as a string the DNS name
    for a machine and returns 4 values:
        A. an address
        B. a list of additional addresses(if existent)
        C. the canonical name of the host
        D. an alist of all the host's names with their respective addresses 

	We use only the first return value, the address component, to pass to the
	function CONNECT.

	The function CONNECT will connect the socket to the address, but to a
	random port if the :port keyword argument is not specified. The average
	client codes usually use :wait t to block until the connect can resolve
	with a connected fd or an error. The exception to always using :wait t is
	if the client needs to connect to many servers at once, suppose a web
	client, or if a server is also a client in other contexts and wishes not to
	block.

	The functions REMOTE-HOST and REMOTE-PORT return the ip address and port of
	the remote connection associated with the connected socket. LOCAL-HOST and
	LOCAL-PORT return the information of the client's end of the connected
	socket. Analogous calls REMOTE-NAME and LOCAL-NAME each return two values
	where the first value is the equivalent of *-host and the second value is
	the equivalent of *-port.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; do a blocking connect to the daytime server on the port.
    (connect socket (lookup-hostname host) :port port :wait t)
    (format t "Connected to server ~A:~A via my local connection at ~A:~A!~%"
            (remote-host socket) (remote-port socket)
            (local-host socket) (local-port socket))
|                                                                              |
+------------------------------------------------------------------------------+

3. Read and display the server's reply:

	Now that the socket has been connected to the server, the server will send
	a line of text to the client. The client uses the standard Common Lisp
	function READ-LINE to read the information from the socket. The function
	READ-LINE blocks and will only return when an *entire line* is read. Once
	read, the line is emitted to *standard-output* via the function call
	FORMAT.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; read the one line of information I need from the daytime
    ;; server.  I can use read-line here because this is a TCP socket.
    (let ((line (read-line socket)))
      (format t "~A" line))
|                                                                              |
+------------------------------------------------------------------------------+

4. End program:

	We close the socket with the standard function CLOSE and return true so the
	return value of this example is t.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; all done
    (close socket)
    t))
|                                                                              |
+------------------------------------------------------------------------------+

While this program works, it has some major flaws in it. First and foremost is
that it doesn't handle any conditions that IOLib signals in common use cases.
An example would be to run the ex1-client.lisp example without a daytime server
running. In most, if not all, Common Lisp toplevels, you'll be dropped into the
debugger on an unhandled SOCKET-CONNECTION-REFUSED-ERROR condition. Secondly,
it isn't written in the Common Lisp style.

Daytime Client IVP4/TCP: ex2-client.lisp
----------------------------------------

In this example, we simply tackle the fact ex1-server.lisp can be shortened
with an IOLib form to something where the application writer has less to do
concerning cleaning up the socket object. It also uses line oriented blocking
I/O.

The introduced macro WITH-OPEN-SOCKET calls MAKE-SOCKET with the arguments in
question and binds the socket to the variable 'socket'. When this form returns,
it will automatically close the socket.

This shortens the program so much, that the example can be included in its 
entirety:

+------------------------------------------------------------------------------+
|                                                                              |
(defun run-ex2-client (&key (host *host*) (port *port*))

  ;; We introduce with-open-socket here as a means to easily wrap
  ;; usually synchronous and blocking communication with a form that
  ;; ensures the socket is closed no matter how we exit it.
  (with-open-socket
      (socket :connect :active
              :address-family :internet
              :type :stream
              :external-format '(:utf-8 :eol-style :crlf)
              :ipv6 nil)

    ;; Do a blocking connect to the daytime server on the port.  We
    ;; also introduce lookup-hostname, which converts a hostname to an
    ;; 4 values, but in our case we only want the first, which is an
    ;; address.
    (connect socket (lookup-hostname host) :port port :wait t)
    (format t "Connected to server ~A:~A from my local connection at ~A:~A!~%"
            (remote-name socket) (remote-port socket)
            (local-name socket) (local-port socket))

    ;; read the one line of information I need from the daytime
    ;; server.  I can use read-line here because this is a TCP
    ;; socket. It will block until the whole line is read.
    (let ((line (read-line socket)))
      (format t "~A" line)
      t)))
|                                                                              |
+------------------------------------------------------------------------------+

This shorthand can go even further, if we add this to the WITH-OPEN-SOCKET
flags

    :remote-host (lookup-hostname host)
    :remote-port port

then the underlying MAKE-SOCKET call will in fact connect the socket directly
to the server before it is available for the body of the macro allowing us to
remove the connect call entirely! In the early examples, however, we don't
utilize IOLib's shorthand notations to this degree in order to make apparent
how the library maps into traditional socket concepts. After one gains
familiarity with the IOLib API, the situations where application of the
shortcuts are useful become much easier to see.

Daytime Client IVP4/TCP: ex3-client.lisp
----------------------------------------

Now we come to condition handling, which can moderately affect the layout of
your IOLib program. Any real program using IOLib must handle IOLib's signaled
conditions which are common to the boundary cases of network programming.
We've already seen one of these boundary cases when we tried to connect a
daytime client to a server that wasn't running.  The condition signaled in that
case was: SOCKET-CONNECTION-REFUSED-ERROR.  The stream interface has a set of
conditions which IOLib will signal, and another lower level IOLib layer--which
we'll come to in the nonblocking I/O examples have another set of conditions.
There is some intersection between them and we will explore that later. For
now, we'll just use the conditions associated with a stream.

Our rewrite of ex2-client.lisp into ex3-client.lisp (continuing to use line
oriented blocking I/O) proceeds thusly:

0. We create a helper function which connects to the server and reads the
    daytime line:

	Notice the HANDLER-CASE macro around the portion of the function which
	reads the date from the server. In looking at the boundary conditions from
	the server given this protocol, we can receive an END-OF-FILE condition if
	the client connected, but before the server could respond it exited,
	closing the connection. Since in this case we're inside of a
	WITH-OPEN-SOCKET form, we simply note that we got an END-OF-FILE and let
	the cleanup forms of WITH-OPEN-SOCKET close the connection. If we don't
	catch this condition, then the program will break into the debugger and
	that isn't useful.  It is usually debatable as to where one should handle
	conditions: either near to or far away from the generating calls. In these
	simple examples, no choice has any significant pros or cons. As your IOLib
	programs become more and more complex, however, it becomes more obvious at
	what abstraction level to handle signaled conditions.

+------------------------------------------------------------------------------+
|                                                                              |
(defun run-ex3-client-helper (host port)

  ;; Create a internet TCP socket under IPV4
  (with-open-socket
      (socket :connect :active
              :address-family :internet
              :type :stream
              :external-format '(:utf-8 :eol-style :crlf)
              :ipv6 nil)

    ;; do a blocking connect to the daytime server on the port.
    (connect socket (lookup-hostname host) :port port :wait t)
    (format t "Connected to server ~A:~A from my local connection at ~A:~A!~%"
            (remote-name socket) (remote-port socket)
            (local-name socket) (local-port socket))

    (handler-case
        ;; read the one line of information I need from the daytime
        ;; server.  I can use read-line here because this is a TCP
        ;; socket. It will block until the whole line is read.
        (let ((line (read-line socket)))
          (format t "~A" line)
          t)

      ;; However, let's notice the signaled condition if the server
      ;; went away prematurely...
      (end-of-file ()
        (format t "Got end-of-file. Server closed connection!")))))
|                                                                              |
+------------------------------------------------------------------------------+

1. Some conditions which are complete show-stoppers to the functioning of the
    code are caught at a higher level:

	Notice we catch the possible SOCKET-CONNECTION-REFUSED-ERROR from the
	connect inside of the function run-ex3-client-helper.

+------------------------------------------------------------------------------+
|                                                                              |
;; The main entry point into ex3-client
(defun run-ex3-client (&key (host *host*) (port *port*))
  (handler-case

      (run-ex3-client-helper host port)

    ;; handle a commonly signaled error...
    (socket-connection-refused-error ()
      (format t "Connection refused to ~A:~A. Maybe the server isn't running?~%"
              (lookup-hostname host) port))))
|                                                                              |
+------------------------------------------------------------------------------+

Here are some common conditions in IOLib (some from ANSI Common Lisp too) and
under what situations they are signaled.  In any IOLib program, *at least*
these conditions should be handled where appropriate.

END-OF-FILE:
    When a stream function such as READ, READ-LINE, etc...(but not
    RECEIVE-FROM), reads from a socket where the other end has been closed.

HANGUP:
    When writing to a socket with a stream function such as WRITE,
    FORMAT, etc...(but not SEND-TO), if the socket is closed then this
    condition is signaled.

SOCKET-CONNECTION-RESET-ERROR:
    When doing I/O on a socket and the other side of the socket sent a
    RST packet, this condition is signaled.  It can also happen with
    the IOLIb function ACCEPT and similar.

SOCKET-CONNECTION-REFUSED-ERROR:
    Signaled by connect if there is no server waiting to accept the incoming
    connection.


Daytime Servers
---------------

Now that we have completed the evolution of the daytime client, let's look at
the daytime servers.

The exposition of the servers follows in style of the clients.

Daytime Server IVP4/TCP: ex1-server.lisp
----------------------------------------

This first example is an iterative server which handles a single client and
then exits. The I/O is blocking and no error handling is performed.  This is
similar in scope to the ex1-client.lisp example.

0. Create the server socket:

	We see that the socket is :passive. Every socket in IOLib is predestined to
	be either an active or passive socket and since this is a server socket, it
	is passive. Also here we see that we can ask for the underlying fd of the
	socket with the function SOCKET-OS-FD.

+------------------------------------------------------------------------------+
|                                                                              |
(defun run-ex1-server (&key (port *port*))
  ;; Create a passive (server) TCP socket under IPV4 Sockets meant to
  ;; be listened upon *must* be created passively. This is a minor
  ;; deviation from the Berkeley socket interface.
  (let ((socket
         (make-socket
          :connect :passive
          :address-family :internet
          :type :stream
          :external-format '(:utf-8 :eol-style :crlf)
          :ipv6 nil)))
    (format t "Created socket: ~A[fd=~A]~%" socket (socket-os-fd socket))
|                                                                              |
+------------------------------------------------------------------------------+

1. Bind the socket

	Binding a socket is what gives it an endpoint to which clients can connect.
	The IOLib constant +IPV4-UNSPECIFIED+ represents 0.0.0.0 and means if a
	connection arrives on any interface, it will be accepted if it comes to the
	:port specified. The :reuse-addr keyword represents the socket option
	SO_REUSEADDR and states (among other things) that if the socket is in the
	TIME_WAIT state it can be reused immediately.  It is recommended that all
	servers use :reuse-addr on their listening socket.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; Bind the socket to all interfaces with specified port.
    (bind-address socket
                  +ipv4-unspecified+ ; which means INADDR_ANY or 0.0.0.0
                  :port port
                  :reuse-addr t)
    (format t "Bound socket: ~A~%" socket)
|                                                                              |
+------------------------------------------------------------------------------+

2. Listen on the socket

	Listening on a socket allows clients to connect. In this example, we've
	specified that 5 pending connection can be queued up in the kernel before
	being accepted by the process.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; Convert the sockxet to a listening socket
    (listen-on socket :backlog 5)
    (format t "Listening on socket bound to: ~A:~A~%"
            (local-host socket) (local-port socket))
|                                                                              |
+------------------------------------------------------------------------------+

3. Accept the client connection.

	Here we finally call the IOLib function ACCEPT-CONNECTION. We would like it
	to block, so we pass it :wait t. When ACCEPT-CONNECTION returns it will
	return a new socket which represents the connection to the client.
	ACCEPT-CONNECTION can return nil under some situations, such as on a slow
	server when the client sent a TCP RST packet in between the time the kernel
	sees the connection attempt and ACCEPT-CONNECTION is actually called.  We
	also opt to use the function REMOTE-NAME, which returns two values, the ip
	address and port of the remote side of the socket.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; Block on accepting a connection
    (format t "Waiting to accept a connection...~%")
    (let ((client (accept-connection socket :wait t)))
      (when client
        ;; When we get a new connection, show who it is from.
        (multiple-value-bind (who rport)
            (remote-name client)
          (format t "Got a connection from ~A:~A!~%" who rport))
|                                                                              |
+------------------------------------------------------------------------------+

4. Write the time to the client.

	Here we've figured out the time string and wrote it to the client.  Notice
	we call the function FINISH-OUTPUT. This ensures that all output is written
	to the client socket. For streams using blocking I/O, it is recommended
	that every write to a blocking socket be followed up with a call to
	FINISH-OUTPUT.

+------------------------------------------------------------------------------+
|                                                                              |
        ;; Since we're using a internet TCP stream, we can use format
        ;; with it. However, we should be sure to call finish-output on
        ;; the socket in order that all the data is sent. Also, this is
        ;; a blocking write.
        (multiple-value-bind (s m h d mon y)
            (get-decoded-time)
          (format t "Sending the time...")
          (format client "~A/~A/~A ~A:~A:~A~%" mon d y h m s)
          (finish-output client))
|                                                                              |
+------------------------------------------------------------------------------+

5. Close the connection to the client.

    We're done writing to the client, so close the connection so the client
	knows it got everything.

+------------------------------------------------------------------------------+
|                                                                              |
        ;; We're done talking to the client.
        (close client)
        (format t "Sent!~%"))
|                                                                              |
+------------------------------------------------------------------------------+

6. Close the server's socket.

	Since this is a one shot server, we close the listening socket and exit. In
	this and all other servers we call FINISH-OUTPUT to flush all pending
	message to *standard-output*, if any.

+------------------------------------------------------------------------------+
|                                                                              |
      ;; We're done with the server socket too.
      (close socket)
      (finish-output)
      t)))
|                                                                              |
+------------------------------------------------------------------------------+

The above code is the basic idea for how a very simple TCP blocking I/O server
functions. Like ex1-client, this server suffers from the inability to handle
common signaled conditions such as a HANGUP from the client--which means the
client went away before the server could write the time to it.

However, one major, and subtle, problem of this particular example is that the
socket to the client is *not immediately closed* if the server happens to exit,
say by going through the debugger back to toplevel--or a signaled condition,
before writing the date to the client. If this happens, it can take a VERY long
time for the socket to be garbage collected and closed. In this scenario, the
client will hang around waiting for data which will never come until the Lisp
implementation closes the socket when it gets around to collecting it. Garbage
collection is an extremely nice feature of Common Lisp, but non-memory OS
resources in general should be eagerly cleaned up.  Clients can suffer from
this problem too, leaving open, but unmanipulable, sockets to servers.

All clients or servers written against IOLib should either use some IOLib
specific macros to handle closing of socket, Common Lisp's condition system
like handler-case to catch the signaled conditions, or some other manual
solution.

Daytime Server IVP4/TCP: ex2-server.lisp
----------------------------------------

Similarly to ex2-client, this server uses the macro WITH-OPEN-SOCKET to open
the server socket. We introduce WITH-ACCEPT-CONNECTION to accept the client and
convert this server from a single shot server to an iterative server which can
handle, in a serial fashion only, multiple clients.

0. Serially accept and process clients:

	This portion of ex2-server shows the infinite loop around the accepting of
	the connection.  The macro WITH-ACCEPT-CONNECTION takes the server socket
	and introduces a new binding: client, which is the accepted connection. We
	ensure to tell the accept we'd like to be blocking. If for whatever reason
	we exit the body, it'll clean up the client socket automatically.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; Keep accepting connections forever.
    (loop
       (format t "Waiting to accept a connection...~%")

       ;; Using with-accept-connection, when this form returns it will
       ;; automatically close the client connection.
       (with-accept-connection (client server :wait t)
         ;; When we get a new connection, show who it is from.
         (multiple-value-bind (who rport)
             (remote-name client)
           (format t "Got a connnection from ~A:~A!~%" who rport))

         ;; Since we're using a internet TCP stream, we can use format
         ;; with it. However, we should be sure to finish-output in
         ;; order that all the data is sent.
         (multiple-value-bind (s m h d mon y)
             (get-decoded-time)
           (format t "Sending the time...")
           (format client "~A/~A/~A ~A:~A:~A~%" mon d y h m s)
           (finish-output client)
           (format t "Sent!~%")
           (finish-output)
           t)))))
|                                                                              |
+------------------------------------------------------------------------------+

For very simple blocking I/O servers like this one, serially accepting and
handling client connections isn't so much of a problem, but if the server does
anything which takes a lot of time or has to send lots of data back and forth
to many persistent clients, then this is a poor design. The means by which you
exit this server is by breaking evaluation and returning to the toplevel. When
this happens, the WITH-* forms automatically close the connection to the
client.

Daytime Server IVP4/TCP: ex3-server.lisp
----------------------------------------

In this iterative and blocking I/O server example, we add the handling of the
usual signaled conditions in network boundary cases often found with sockets.
Like the earlier client where we introduced HANDLER-CASE, this involves a
little bit of restructuring of the codes.

0. A helper function which opens a passive socket, binds it, and
    listens on it:

	There is nothing new in this portion of the code. We've seen this pattern
	before. In production code, we could probably shorten this further by
	having WITH-OPEN-SOCKET do the binding and connecting with appropriate
	keyword arguments.

+------------------------------------------------------------------------------+
|                                                                              |
(defun run-ex3-server-helper (port)
  (with-open-socket
      (server :connect :passive
              :address-family :internet
              :type :stream
              :ipv6 nil
              :external-format '(:utf-8 :eol-style :crlf))

    (format t "Created socket: ~A[fd=~A]~%" server (socket-os-fd server))

    ;; Bind the socket to all interfaces with specified port.
    (bind-address server +ipv4-unspecified+ :port port :reuse-addr t)
    (format t "Bound socket: ~A~%" server)

    ;; start listening on the server socket
    (listen-on server :backlog 5)
    (format t "Listening on socket bound to: ~A:~A~%"
            (local-host server)
            (local-port server))
|                                                                              |
+------------------------------------------------------------------------------+

1. Repeatedly handle clients in a serial fashion:

	The new material in this function is the HANDLER-CASE around sending the
	client the time information. The boundary conditions when writing to a
	client include the server getting a reset (RST) from the client or
	discovering the client had gone away and there is no-one to which to write.
	Since the write is contained within the WITH-ACCEPT-CONNECTION form, if any
	of these conditions happen, we simply notice that they happened and let the
	form clean up the socket when it exits.  If we didn't catch the conditions,
	however, we'd break into the debugger.

	One might ask what the value of catching these conditions here is at all
	since we don't actually do anything with them--other than printing a
	message and preventing the code from breaking into the debugger. For the
	purposes of the tutorial, it is intended that the reader induce the
	boundary cases manually and see the flow of the code and to understand
	exactly what conditions may be signaled under what conditions and how to
	structure code to deal with them. In production code where the author might
	not care about these conditions at all, one might simply ignore all the
	signaled conditions that writing to the client might cause.

	Of course, the appropriateness of ignoring network boundary conditions is
	best determined by context.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; keep accepting connections forever.
    (loop
       (format t "Waiting to accept a connection...~%")

       ;; Here we see with-accept-connection which simplifies closing
       ;; the client socket when are done with it.
       (with-accept-connection (client server :wait t)
         ;; When we get a new connection, show who it
         ;; is from.
         (multiple-value-bind (who rport)
             (remote-name client)
           (format t "Got a connnection from ~A:~A!~%" who rport))

         ;; Since we're using an internet TCP stream, we can use format
         ;; with it. However, we should be sure to finish-output in
         ;; order that all the data is sent.
         (multiple-value-bind (s m h d mon y)
             (get-decoded-time)
           (format t "Sending the time...")

           ;; Catch the condition of the client closing the connection.
           ;; Since we exist inside a with-accept-connection, the
           ;; socket will be automatically closed.
           (handler-case
               (progn
                 (format client "~A/~A/~A ~A:~A:~A~%" mon d y h m s)
                 (finish-output client))

             (socket-connection-reset-error ()
               (format t "Client reset connection!~%"))

             (hangup ()
               (format t "Client closed conection!~%")))

           (format t "Sent!~%"))))
|                                                                              |
+------------------------------------------------------------------------------+

2. End of the helper function, returns T to whomever called it:

+------------------------------------------------------------------------------+
|                                                                              |
    t))
|                                                                              |
+------------------------------------------------------------------------------+

3. The entry point into this example:

	We handle the condition SOCKET-ADDRESS-IN-USE-ERROR which is most commonly
	signaled when we try to bind a socket to address which already has a server
	running on it or when the address is in the TIME_WAIT state. The latter
	situation is so common--usually caused by a server just having exited and
	another one starting up to replace it, that when binding addresses, one
	should supply the keyword argument :reuse-addr with a true value to
	BIND-ADDRESS to allow binding a socket to an address in TIME_WAIT state.

+------------------------------------------------------------------------------+
|                                                                              |
;; This is the main entry point into the example 3 server.
(defun run-ex3-server (&key (port *port*))
  (handler-case

      (run-ex3-server-helper port)

    (socket-address-in-use-error ()
      ;; Here we catch a condition which represents trying to bind to
      ;; the same port before the first one has been released by the
      ;; kernel.  Generally this means you forgot to put ':reuse-addr
      ;; t' as an argument to bind address.
      (format t "Bind: Address already in use, forget :reuse-addr t?")))

  (finish-output))
|                                                                              |
+------------------------------------------------------------------------------+

Daytime Server IVP4/TCP: ex4-server.lisp
----------------------------------------

This is the first of our concurrent servers and the last of our daytime
protocol servers. Usually concurrency is introduced (in the UNIX environment)
with the fork() library call which creates an entirely new process with
copy-on-write semantics to handle the connection to the client. In this
tutorial environment, we've chosen to render this idea with the portable
threading library Bordeaux Threads.  The I/O is still line oriented and
blocking, however, when a thread blocks another can run giving the illusion of
a server handling multiple clients in a non-blocking fashion.

We also introduce UNWIND-PROTECT ensures that various sockets are closed under
various boundary conditions in the execution of the server.  An UNWIND-PROTECT
executes a single form, and after the evaluation, or interruption, of that
form, evaluates a special cleanup form. The cleanup form is *always* evaluated
and we use this to cleanup non-memory system resources like sockets.

Threads present their own special problems in the design of a server. Two
important problems are: data races and thread termination. The tutorial tries
very hard to avoid any data races in the examples and this problem is
ultimately solvable using Bordeaux-Threads mutexes or condition variables.  Our
simple examples do not need mutexes as they do not share any data between
themselves. 

The harder problem is thread termination. Since the tutorial encourages
experimentation with the clients and servers in a REPL, threads may leak when
the server process' initial thread stops execution and goes back to the REPL.
We use three API calls from the Bordeaux Threads: THREAD-ALIVE-P, ALL-THREADS,
and DESTROY-THREAD--which are not to be used in normal thread programming.  We
do this here in order to try and clean up leaked threads so the clients know
immediately when the server process stopped and we don't pollute the REPL with
an ever increasing number of executing threads. The employed method of
destroying the threads, on SBCL specifically, allows the invocation of the
thread's UNWIND-PROTECT's cleanup form, which closes the socket to the client
before destroying the thread.  On other implementations of Common Lisp, we are
not guaranteed that the thread's UNWIND-PROTECT cleanup form will be evaluated
when we destroy it.

This method is also extremely heavy handed in that it uses the function
IGNORE-ERRORS to ignore any condition that Bordeaux Thread's DESTROY-THREAD may
have signaled, including important conditions like HEAP-EXHAUSTED-ERROR, an
SBCL specific condition. In a real threaded server, the exiting of the initial
thread (which means exiting of the runtime and termination of the entire Lisp
process) will destroy all other threads as the process tears itself down and
exits. This is the recommended way a threaded server should exit.

Since threading is implementation dependent for what guarantees are provided,
any non-toy threaded network server will probably use the native implementation
of threads for a specific Common Lisp implementation.  An example difficult
situation would be trying to terminate a thread which is blocked on I/O.
Different implementations would handle this in different ways.

The two provided examples, ex4-server and ex5-server, provide a general idea
for the structuring of the code to utilize threads.

Here is the dissection of ex4-server:

0. A special variable which will allow the initial thread to pass a client
    socket to a thread handling said client:

+------------------------------------------------------------------------------+
|                                                                              |
;; This variable is the means by which we transmit the client socket from
;; the initial thread to the particular thread which will handle that client.
(defvar *ex4-tls-client* nil)
|                                                                              |
+------------------------------------------------------------------------------+

1. A helper function which begins with the usual recipe for a server:

+------------------------------------------------------------------------------+
|                                                                              |
(defun run-ex4-server-helper (port)
  (with-open-socket
      (server :connect :passive
              :address-family :internet
              :type :stream
              :ipv6 nil
              :external-format '(:utf-8 :eol-style :crlf))

    (format t "Created socket: ~A[fd=~A]~%" server (socket-os-fd server))

    ;; Bind the socket to all interfaces with specified port.
    (bind-address server +ipv4-unspecified+ :port port :reuse-addr t)
    (format t "Bound socket: ~A~%" server)

    ;; start listening on the server socket
    (listen-on server :backlog 5)
    (format t "Listening on socket bound to: ~A:~A~%"
            (local-host server)
            (local-port server))
|                                                                              |
+------------------------------------------------------------------------------+

2. Forever more, accept a client connection on the listening socket
    and start a thread which handles it:

	There is a lot going on in this piece of code. The first thing to notice is
	the UNWIND-PROTECT and its cleanup form. The form which UNWIND-PROTECT is
	guarding is an infinite loop which does a blocking accept to get a client
	socket, rebinds *default-special-bindings* adding to its assoc list the
	binding for *ex4-tls-client*, and creates a thread which handles the
	client.

	The cleanup form walks all of the active client threads and destroys them,
	ignoring any conditions that may have arose while doing so. Destroying the
	threads prevents them from piling up and eventually causing havoc if many
	servers start and exit over time. In addition, it forces an eager close on
	the client sockets allowing any clients to know the server went away
	immediately.

+------------------------------------------------------------------------------+
|                                                                              |
    ;; Here we introduce unwind-protect to ensure we properly clean up
    ;; any leftover threads when the server exits for whatever reason.
    ;; keep accepting connections forever, but if this exits for
    ;; whatever reason ensure to destroy any remaining running
    ;; threads.
    (unwind-protect
         (loop                         ; keep accepting connections...
            (format t "Waiting to accept a connection...~%")
            (finish-output)
            (let* ((client (accept-connection server :wait t))
                   ;; set up the special variable according to the
                   ;; needs of the Bordeaux Threads package to pass in
                   ;; the client socket we accepted to the about to be
                   ;; created thread.  *default-special-bindings* must
                   ;; not be modified, so here we just push a new scope
                   ;; onto it.
                   (*default-special-bindings*
                    (acons '*ex4-tls-client* client
                           *default-special-bindings*)))

              ;; ...and handle the connection!
              (when client
                (make-thread #'process-ex4-client-thread
                             :name 'process-ex4-client-thread))))