Storage allocation is handled differently by different computer systems. In many languages, the programmer must spend a lot of time thinking about when variables and storage units are allocated and deallocated. In Lisp, freeing of allocated storage is normally done automatically by the Lisp system; when an object is no longer accessible to the Lisp environment, the garbage collector reuses its storage for some other object. This relieves the programmer of a great burden, and makes writing programs much easier.

However, automatic freeing of storage incurs an expense: more computer resources must be devoted to the garbage collector. If a program is designed to allocate temporary storage, which is then left as garbage, more of the computer must be devoted to the collection of garbage; this expense can be high. In some cases, the programmer may decide that it is worth putting up with the inconvenience of having to free storage under program control, rather than letting the system do it automatically, in order to prevent a great deal of overhead from the garbage collector.

It usually is not worth worrying about freeing of storage when the units of storage are very small things such as conses or small arrays. Numbers are not a problem, either; fixnums and short floats do not occupy storage, and the system has a special way of garbage-collecting the other kinds of numbers with low overhead. But when a program allocates and then gives up very large objects at a high rate (or large objects at a very high rate), it can be worthwhile to keep track of that one kind of object manually. Within the Lisp Machine system, there are several programs that are in this position. The Chaosnet software allocates and frees ``packets'', which are moderately large, at a very high rate. The window system allocates and frees certain kinds of windows, which are very large, moderately often. Both of these programs manage their objects manually, keeping track of when they are no longer used.

When we say that a program ``manually frees'' storage, it does not really mean that the storage is freed in the same sense that the garbage collector frees storage. Instead, a list of unused objects is kept. When a new object is desired, the program first looks on the list to see if there is one around already, and if there is it uses it. Only if the list is empty does it actually allocate a new one. When the program is finished with the object, it returns it to this list.

The functions and special forms in this section perform the above function. The set of objects forming each such list is called a resource; for example, there might be a Chaosnet packet resource. defresource defines a new resource; allocate-resource allocates one of the objects; deallocate-resource frees one of the objects (putting it back on the list); and using-resource temporarily allocates an object and then frees it.

The defresource special form is used to define a new resource. The form looks like this: (defresource name parameters doc-string keyword value keyword value ...) name should be a symbol; it is the name of the resource and gets a defresource property of the internal data structure representing the resource. parameters is a lambda-list giving names and default values (if &optional is used) of parameters to an object of this type. For example, if one had a resource of two-dimensional arrays to be used as temporary storage in a calculation, the resource would typically have two parameters, the number of rows and the number of columns. In the simplest case parameters is (). The documentation string is recorded for (documentation name 'resource) to access. It may be omitted. The keyword options control how the objects of the resource are made and kept track of. The following keywords are allowed:

:constructor	The value is either a form or the name of a function. It is responsible for making an object, and will be used when someone tries to allocate an object from the resource and no suitable free objects exist. If the value is a form, it may access the parameters as variables. If it is a function, it is given the internal data structure for the resource and any supplied parameters as its arguments; it will need to default any unsupplied optional parameters. This keyword is required.
:free-list-size	The value is the number of objects which the resource data structure should have room, initially, to remember. This is not a hard limit, since the data structure will be made bigger if necessary.
:initial-copies	The value is a number (or nil which means 0). This many objects will be made as part of the evaluation of the defresource; thus is useful to set up a pool of free objects during loading of a program. The default is to make no initial copies. If initial copies are made and there are parameters, all the parameters must be &optional and the initial copies will have the default values of the parameters.
:initializer	The value is a form or a function as with :constructor. In addition to the parameters, a form here may access the variable object (in the current package). A function gets the object as its second argument, after the data structure and before the parameters. The purpose of the initializer function or form is to clean up the contents of the object before each use. It is called or evaluated each time an object is allocated, whether just constructed or being reused.
:finder	The value is a form or a function as with :constructor and sees the same arguments. If this option is specified, the resource system does not keep track of the objects. Instead, the finder must do so. It will be called inside a without-interrupts and must find a usable object somehow and return it.
:matcher	The value is a form or a function as with :constructor. In addition to the parameters, a form here may access the variable object (in the current package). A function gets the object as its second argument, after the data structure and before the parameters. The job of the matcher is to make sure that the object matches the specified parameters. If no matcher is supplied, the system will remember the values of the parameters (including optional ones that defaulted) that were used to construct the object, and will assume that it matches those particular values for all time. The comparison is done with equal (not eq). The matcher is called inside a without-interrupts.
:checker	The job of the checker is to determine whether the object is safe to allocate. The value is a form or a function, as above. In addition to the parameters, a form here may access the variables object and in-use-p (in the current package). A function receives these as its second and third arguments, after the data structure and before the parameters. If no checker is supplied, the default checker looks only at in-use-p; if the object has been allocated and not freed it is not safe to allocate, otherwise it is. The checker is called inside a without-interrupts.

If these options are used with forms (rather than functions), the forms get compiled into functions as part of the expansion of defresource. The functions, whether user-provided or generated from forms, are given names like (:property resource-name si:resource-constructor); these names are not guaranteed not to change in the future. Most of the options are not used in typical cases. Here is an example: (defresource two-dimensional-array (rows columns) :constructor (make-array (list rows columns))) Suppose the array was usually going to be 100 by 100, and you wanted to preallocate one during loading of the program so that the first time you needed an array you wouldn't have to spend the time to create one. You might simply put (using-resource (foo two-dimensional-array 100 100) ) after your defresource, which would allocate a 100 by 100 array and then immediately free it. Alternatively you could write: (defresource two-dimensional-array (&optional (rows 100) (columns 100)) :constructor (make-array (list rows columns)) :initial-copies 1) Here is an example of how you might use the :matcher option. Suppose you wanted to have a resource of two-dimensional arrays, as above, except that when you allocate one you don't care about the exact size, as long as it is big enough. Furthermore you realize that you are going to have a lot of different sizes and if you always allocated one of exactly the right size, you would allocate a lot of different arrays and would not reuse a pre-existing array very often. So you might write: (defresource sloppy-two-dimensional-array (rows columns) :constructor (make-array (list rows columns)) :matcher (and (≥ (array-dimension-n 1 object) rows) (≥ (array-dimension-n 2 object) columns))) resource-name &rest parameters Allocates an object from the resource specified by resource-name. The various forms and/or functions given as options to defresource, together with any parameters given to allocate-resource, control how a suitable object is found and whether a new one has to be constructed or an old one can be reused. Note that the using-resource special form is usually what you want to use, rather than allocate-resource itself; see below. resource-name resource-object Frees the object resource-object, returning it to the free-object list of the resource specified by resource-name. (variable resource parameters...) body... The body forms are evaluated sequentially with variable bound to an object allocated from the resource named resource, using the given parameters. The parameters (if any) are evaluated, but resource is not. using-resource is often more convenient than calling allocate-resource and deallocate-resource. Furthermore it is careful to free the object when the body is exited, whether it returns normally or via throw. This is done by using unwind-protect; see . Here is an example of the use of resources: (defresource huge-16b-array (&optional (size 1000)) :constructor (make-array size :type 'art-16b)) (defun do-complex-computation (x y) (using-resource (temp-array huge-16b-array) ... ;Within the body, the array can be used. (aset 5 temp-array i) ...)) ;The array is deallocated at the end. resource-name Frees all objects in resource-name. This is like doing deallocate-resource on each one individually. This function is often useful in warm-boot initializations. function resource-name &rest extra-args Calls function on each object created in resource-name. Each time function is called, it receives three fixed arguments, plus whatever extra-args were specified. The three fixed arguments are an object of the resource; t if the object is currently allocated (``in use''); and the resource data structure itself. resource-name Forgets all of the objects being remembered by the resource specified by resource-name. Future calls to allocate-resource will create new objects. This function is useful if something about the resource has been changed incompatibly, such that the old objects are no longer usable. If an object of the resource is in use when clear-resource is called, an error will be signaled when that object is deallocated.

The constructor, initializer, matcher and checker functions receive the internal resource data structure as an argument. This is a named structure array whose elements record the objects both free and allocated, and whose array leader contains sundry other information. This structure should be accessed using the following primitives:

resource-structure index Returns the index'th object remembered by the resource. Both free and allocated objects are remembered. resource-structure index Returns t if the index'th object remembered by the resource has been allocated and not deallocated. Simply defined resources will not reallocate an object in this state. resource-structure index Returns the list of parameters from which the index'th object was originally created. resource-structure Returns the number of objects currently remembered by the resource. This will include all objects ever constructed, unless clear-resource has been used. resource-structure Returns a function, created by defresource, which accepts the supplied parameters as arguments, and returns a complete list of parameter values, including defaults for the optional ones.

When small temporary data structures are allocated so often that they amount to a considerable drain of storage space, an ordinary resource may be unacceptably slow. Here is a simple technique that provides in such cases nearly all the benefit of a resource while costing nearly nothing. The function read uses it to allocate a buffer for reading tokens of input.

(defvar buffer-for-reuse nil) (defsubst get-buffer () (or (do (old) ((%store-conditional (locf buffer-for-reuse) (setq old buffer-for-reuse) nil) old)) (construct-new-buffer)))) (defsubst free-buffer (buffer) (setq buffer-for-reuse buffer))

To allocate a buffer for use, do (get-buffer). To free it when you are done with it, call free-buffer. It is assumed that construct-new-buffer is the function which can create a new buffer when there is none available for reuse.

This technique keeps track of at most one buffer which has been freed and may be reused. It is not effective in this simple form when more than one buffer is needed at any given time by one application. In the case of read, only one token is being read in at any time.

It is safe for more than one process to call read because get-buffer is designed to guarantee that a request cannot get a buffer already handed out and not freed. Likewise, nothing terrible happens if there is an error inside read and read is called recursively within the debugger. The only problem is that multiple buffers will be allocated, which means that some of them will be lost. But the cost of this is minor in the cases where this technique is applicable. For example, if two processes are reading files, process switching will probably happen a few times a second, each time costing one buffer not reused. This is insignificant compared to the storage used up for other purposes by reading large amounts of data.