11. Packages
This chapter documents the package system. It contains the following subsections:
11.1
Rationale for packages11.2
Package Consistency Rules11.3
Package Names11.4
Translating Strings to Symbols11.5 Exporting and Importing Symbols
11.6 Name Conflicts
11.7 Built-in Packages
The
package system allows the programmer to maintain multiple namespaces for symbols. At any given time one and only one package is current; all symbols are referenced relative to the current package. At the core of a package is a hash table of symbols; packages have some additional attributes such as a name, possibly some nicknames, and a list of packages which it uses and is used by. Symbols in other packages than the current package can be referred to using the : and :: syntax. Some symbols do not belong to any package, in which case they are prefixed with #: (that is, poundsign-colon). Symbols which gensym produces will look like this #:G000.It is possible for there to be distinct symbols with the same print name in more than one package. Thus there could be a routine editor::draw, which is part of a package which redraws the screen, and sketch::draw, which redraws the screen for a paint program. If the current package is editor, then editor::draw could be refered to simply as draw. Likewise if sketch is the current package, draw would refer to the similar routine. This simplifies the process of naming routines.
Following is a list of functions which handle packages.
make-package
*package*
in-package
find-package
package-name
package-nicknames
rename-package
package-use-list
package-used-by-list
package-shadowing-symbols
list-all-packages
delete-package
intern
unexport
import
shadowing-import
find-symbol
unintern
export
shadow
use-package
unuse-package
For the functions described here, all optional arguments named
package default to the current value of *package*. Where a function takes an argument that is either a symbol or a list of symbols, an argument of nil is treated as an empty list of symbols. Any argument described as a package name may be either a string or a symbol. If a symbol is supplied, its print name will be age name; if a string is supplied, the user must take care to specify the same capitalization used in the package name, normally all uppercase.
11.1 Rationale for packages
One problem with earlier LISP sysytems is the use of a single name space for all symbols. In large LISP systems, with modules written by many different programmers, accidental name collisions become a serious problem. Common LISP addresses this through the "package system" derived from an earlier package system developed for LISP Machine LISP In addition to preventing name-space conflicts, the package system makes the modular structure of large LISP systems more explicit.
A package is a data structure that establishes a mapping from print names (strings) to symbols. The package thus replaces the oblist or obarray machinery of earlier LISP systems. At any given time one package is current, and this package is used by the LISP reader in translating strings into symbols. The current package is, by definition, the one that is the value of the global variable
*package*. It is possible to refer to symbols in packages other than the current one through the use of package qualifiers in the printed representation of the symbol. For example, foo:bar, when seen by the reader, refers to the symbol whose name is bar in the package whose name is foo. (Actually, this is true only if bar is an external symbol of foo, that is, a symbol that is supposed to be visible outside of foo. A reference to an internal symbol requires the intentionally clumsier syntax foo::bar.)The string-to-symbol mappings available in a given package are divided into two classes, external and internal. We refer to the symbols accessible via these mappings as being external and internal symbols of the package in question, though really it is the mappings that are different and not the symbols themselves. Within a given package, a name refers to one symbol or to one package but not both.
External symbols are part of the package's public interface to other packages. External symbols are supposed to be chosen with some care and are advertised to users of this package. Internal symbols are for internal use only, and these symbols are normally hidden from other packages. Most symbols are created as internal symbols; they become external only if they appear explicitly in an export command for the package.
A symbol may appear in any package. It will always have the same name wherever it appears, but it may be external in some packages and internal in others. On the other hand, the same (string) may refer to different symbols in different packages.
Normally, a symbol that appears in one or more packages will be owned by one particular package, called the home package of the symbol; that package is said to own the symbol. Every symbol has a component called the package cell that is said to be interned. Some symbols are not owned by any package; such a symbol is said to be uninterned, and its package call contains
nil. Packages may be built up in layers. From the point of view of a package's internal and external symbols. However, some of thes mappings may be establised within the package itself, while other mappings are inherited from other packages via the use-package construct. (The mechanisms responsible for this inheritance are described below.) In what follows, we will refer to a symbol as being "accessible" in a package if it can be referred to without a package qualifier when that package is current, regardless of whether the mapping occurs within that package or via inheritance. We will refer to a symbol as being "present" in a package if the mapping is in the package itself and is not inherited from somewhere else. Thus a symbol present in a package is accessible, but an accessible symbol is not necessarily present.A symbol is said to be
interned in a package if it is accessible in that package and also owned (by either that package or some other package). Normally all the symbols accessible in a package will in fact be owned by some package, but the terminology is useful when discussing the pathological case of an accessible but unowned (uninterned) symbol.As a verb, to intern a symbol in a package means to cause the symbol to be interned in the package if it was not already; this process is performed by the function intern. If the symbol was previously unowned, then the package it is being interned in becomes its owner (home package); but if the symbol was previously owned by another package, that other package continues to own the symbol.
To unintern a symbol from the package means to cause it to be not present in the package and, additionally, to cause the symbol to be uninterned if the package was the home package (owner) of the symbol. This process is performed by the function unintern.
11.2 Package Consistency Rules
Package-related bugs can be very subtle and confusing: Things are not what they appear to be. The Common LISP package system is designed with a number of safety features to prevent most of the common bugs that would otherwise occur in normal use. This may seem over-protective, but experience with earlier package systems has shown that such safety features are needed.
In dealing with the package system, it is useful to keep in mind the following consistency rules, which remain in force as long as the value of *package* is not changed by the user:
read-read consistency: Reading the same print name always results in the same (eq) symbol.
print-read consistency: An interned symbol always prints as a sequence of characters that, when read back in, yields the same (eq) symbol.
print-print consistency: If two interned symbols are not eq, then their printed representations will be different sequences of characters.
These consistency rules remain true in spite of any amount of implicit interning caused by typing in LISP forms, loading files, etc. This has the important implication that, as long as the current package is not changed, results in what symbols were typed in when. The rules can only be violated by explicit action: changing the value of *package*, forcing some action by continuing from an error, or calling one of the "dangerous" functions unintern, unexport, shadow, shadowing-import, or unuse-package.
11.3 Package Names
Each package has a name (a string) and perhaps some nicknames. These are assigned when the package is created, though they can be changed later. A package's name should be something long and self-explanatory, like editor; there might be a nickname that is shorter and easier to type, such as ed. There is a single name space for packages. The function find-package translates a package name or nickname into the associated package. The function package-name returns the name of a package. The function rename-package removes a package's current name and nicknames and replaces them with the new ones specified by the user. Package renaming is occasionally useful when, for development purposes, it is desirable to load two versions of a package into the LISP. One can load the first version, rename it, and then load the other version, without getting a lot of name conflicts.
When the LISP reader sees a qualified symbol, it handles the package-name part in the same way as the symbol part with respect to capitalization. Lowercase characters in the package name are converted to corresponding uppercase characters unless preceded by the escape character \ or surrounded by | characters. The lookup done by the find-package function is case-sensitive, like that done for symbols. Note that |Foo|:|Bar| refers to a symbol whose name is Bar in a package whose name is Foo. By contrast, |Foo:Bar| refers to a seven-character symbol that has a colon in its name (as well as two uppercase letters and four lowercase letters) and is interned in the cur show ordinary package names using lowercase letters, even though the name string is internally represented with uppercase letters.
Most of the functions that require a package-name argument from the user accept either a symbol or a string. If a symbol is supplied, its print name will be used; the print name will already have undergone case-conversion by the usual rules. If a string is supplied, it must be so capitalized as to match exactly the string that names the package.
X3J13 voted in January 1989 to clarify that one may use either a package object or a package name (symbol or string) in any one of the following situations:
The :use argument to make-package
The second argument to
intern, find-symbol, unintern, export, unexport, import, shadowing-import, or shadow.The first argument, or a member of the list that is the first argument, to use-package or unuse-package.
The value of the package given to do-symbols, do-external-symbols, or do-all-symbols
A member of the package-list given to with-package-iterator.
Where applicable, Star Sapphire Common LISP supports this expanded usage.
Note that the first argument to make-package must still be a package name and not an actual package; it makes no sense to create an already existing package. Similarly, package nicknames must always be expressed as package names and not as package objects. If find-package is given a package object instead of a name, it simply returns that package.
11.4 Translating Strings to Symbols
The value of the special variable *package* must always be a package object (not a name.) Whatever package object is currently the value of *package* is referred to as the 'current package.'
When the LISP reader has, by parsing, obtained a string of characters thought to name a symbol, that name is looked up in the current package. This lookup may involve looking in other packages whose external symbols are inherited by the current package. If the name is found, the corresponding symbol is returned. If the name is not found (that is, there is no symbol of that name accessible in the current package), a new symbol is created for it and is placed in the current package as an internal symbol. Moreover, the current package becomes the owner (home package) of the symbol, and so the symbol becomes interned in the current package. If the name is later read again while this same package is current, the same symbol will then be found and returned.
Often it is desirable to refer to an external symbol in some package other than the current one. This is done through the use of a qualified name, consisting of a package name, then a colon, then the name of the symbol.
Often is is desirable to refer to an external symbol in some package other than the current one. This is done through the use of a qualified name, consisting of a package name, then a colon, then the name of the symbol. This causes the symbol's name to be looked up in the specified package, rather than in the current one. For example, editor:buffer refers to the external symbol named buffer accessible in the package named editor, regardless of whether there is a symbol named buffer in the current package.
If ther is no package named editor, or if no symbol named buffer is accessible in editor, or if buffer is an internal symbol in editor, the LISP reader will signal a correctable error to ask the user for instructions.
On rare occasions, a user may need to refer to an internal symbol of some package other than the current one. It is illegal to do this with the colon qualifier, since accessing an internal symbol of some other package is usually a mistake. However, this operation is legal if a doubled colon :: is used as the separator in place of the usual single colon. If editor::buffer is seen, the effect is exactly the same as reading buffer with *package* temporarily rebound to the package whose name is editor. This special-purpose qualifier should be used with caution.
The package named keyword contains all keyword symbols used by the LISP system itself and by user-written code. Such symbols must be easily accessible from any package, and name conflicts are not an issue because these symbols are used only as labels and never to carry package-specific values or properties. Because keyword symbols are used so frequently, by a colon, but no package name (for example :foo) is added to (or looked up in) the keyword package as an external symbol. The keyword package is also treated specially in that whenever a symbol is added to the keyword package the symbol is always made external; the symbol is also automatically declared to be a constant (see defconstant) and made to have itself as its value. This is why every keyword evaluates to itself. As a matter of style, keywords should always be accessed using the leading-colon convention; the user should never import or inherit keywords into any other package.
Each symbol contains a package cell that is used to record the home package of the symbol, or nil if the symbol is uninterned. This cell may be accessed by using the function symbol-package. When an interned symbol is printed, if it is a symbol in the keyword package, then it is printed with a preceding colon; otherwise if it is accessible (directly or by inheritance) in the current package, it is printed without any qualification; otherwise, it is printed with the name of the home package but that in fact is accessible in some package. The system does not check for this pathological case, and such symbols will always be printed preceded by #:.
In summary, the following four uses of symbol qualifier
syntax are defined.
foo:bar
When read, looks up
bar among the external symbols of the package named foo. Printed when symbol foo is external in its home package foo and is not accessible in the current package.foo::bar
When read, interns
bar as if foo were the current package. Printed when symbol bar is internal in its home package foo and is not accessible in the current package.:bar
When read, interns
bar as if foo were the current package. Printed when symbol bar is internal in its home package foo and is not accessible in the current package.#:bar
When read, creates a new uninterned symbol named
bar. Printed when the symbol bar is uninterned (has not home package), even in the pathological case that bar is uninterned but nevertheless somehow accessible in the current package.All other uses of colons within names of symbols are not defined by Common LISP, but are reserved for implementation-dependent use; this includes names that end in a colon, contain two or more colons, or consist of just a colon. In Star Sapphire most of these cases are disallowed by the reader.
11.5 Exporting and Importing Symbols
Symbols from one package may be made accessible in another package in two ways.
First, any individual symbol may be added to a package by use of the function import. The form (import editor:buffer) takes the external symbol named buffer in the editor package (this symbol was located when the form was read by the LISP reader) and adds it to the current package as an internal symbol. The symbol is then present in the current package, but if it is already present and external, then the fact that it it external is not changed. After the call to import it is possible to refer to buffer in the importing package without any qualifier. The status of buffer in the package named editor is unchanged, and editor remains the home packge for this symbol. Once imported, a unintern.
If the symbol is already present in the importing package, import has no effect. If a distinct symbol with the name buffer is accessible in the importing (directly or by inheritance), then a correctable error is signaled, as described in section 11.5, because import avoids letting one symbol shadow another.
A symbol is said to be shadowed by another symbol in some package if the first symbol would be accessible by inheritance if not for the presence of the second symbol. To import a symbol without the possibility of getting an error because of shadowing, use the function shadowing-import. This inserts the symbol into the specified package as an internal symbol, regardless of whether another symbol of the same name will be shadowed by this action. If a different symbol of the same name is already present in the package, that symbol will first be uninterned from the package (see unintern). The new symbol is added to the package's shadowing-symbols list. shadowing-import should be used with caution. It changes the state of the package system in such a way that the consistency rules do not hold across the change.
The second mechanism is provided by the function use-package. This causes a package to inherit all of the external symbols of some other package. These they can be referred to without a qualifier while this package is current, but they are not passed along to any other package that uses this package. Note that use-package, unlike import, does not cause any new symbols to be present in the current package but only makes them accessible by inheritance. use-package checks carefully for name conflicts between the newly imported symbols and those already accessible in the importing package. This is described in detail in Section 11.5.
Typically a user, working by default in the user package, will load a number of packages into LISP to provide an augmented working environment, and then call use-package on each of these packages to allow easy access to their external symbols. unuse-package undoes the effects of a previous use-package. The external symbols of the used package are no longer inherited. However, any symbols that have been imported into the using package continue to be present in that package.
There is no way to inherit the internal symbols of another package; to refer to an internal symbol, the user must either make that symbol's home package current, use a qualifier, or import that symbol into the current package.
The distinction between external and internal symbols is a primary means of hiding names to that one program does not tread on the namespace of another.
When intern or some other function wants to look up a symbol in a given package, it first looks for the symbol among the external and internal symbols of the package itself; then it looks through the external symbols of the used packages in some unspecified order. The order does not matter; according to the rules for handling name conflicts (see below), if conflicting symbols appear in two or more packages inherited by package x, a symbol of this name must also appear in x itself as a shadowing symbol. Of course, implementations are free to choose other, more efficient ways of implementing this search, as long as the user-visible behavior is equivalent to what is described here.
The function export takes a symbol that is accessible in some specified package (directly or by inheritance) and makes it an external symbol of that package. If the symbol is already accessible as an external symbol in the package, export has no effect. If the symbol is directly present in the package as an internal symbol, it is simply changed to external status. If it is accessible as an internal symbol via use-package, the symbol is first imported into the package, then exported. (The symbol is then present in the specified package whether or not the package continues to use the package through which the symbol was originally inherited.) If the symbol is not accessible at all in the specified package, a correctable error is signaled that, upon continuing, asks the user whether the symbol should be imported.
The function unexport is provided mainly as a way to undo erroneous calls to export. It works only on symbols directly present in the current package, switching them back to internal status. If unexport is given a symbol already accessible as an internal symbol in the current package, it does nothing; if it is given a symbol not accessible in the package at all,it signals an error.
11.6 Name Conflicts
A fundamental invariant of the package system is that within one package any particular name can refer to at most one symbol. A name conflict is said to occur when there is more than one candidate symbol and it is not obvious which one to choose. If the system does not always choose the same way, the read-read consistency rule would be violated. For example, some programs or data might have been read in under a certain mapping of the name to a symbol. If the mapping changes to a different symbol, and subsequently additional programs or data are read, then the two programs will not access the same symbol even though they use the same name. Even if the system did always choose the same way, a name conflict is likely to result in a mapping from names to symbols different from what was expected by the user, causing programs to execute incorrectly. Therefore, any time a name conflict is about to occur, an error is signaled. The user may continue from the error and tell the package system how to resolve the conflict.
It may be that the same symbol is accessible to a package through more than one path. For example, the symbol might be an external symbol of more than one used package, or the symbol might be directly present in a package and also inherited from another package. In such cases there is no name conflict. The same identical symbol cannot conflict with itself. Name conflicts occur only between distinct symbols with the same name.
The creator of a package can tell the system in advance how to resolve a name conflict through the use of shadowing. Every package has a list of shadowing symbols. A shadowing symbol takes precedence over any other symbol of the same name that would otherwise be accessible to the package. A name conflict involving a shadowing symbol is always resolved in favor of the shadowing symbol, without signaling an error (except for one instance involving import described below). The functions shadow and shadowing-import may be used to declare shadowing symbols.
Name conflicts are detected when they become possible, that is, when the package structure is altered. There is no need to check for name conflicts during every name lookup.
The functions use-package, import, and export check for name conflicts. use-package makes the external symbols of the package being used accessible to the using package; each of these symbols is checked for name conflicts with the symbols already accessible. import adds a single symbol to the internals of a package, checking for a name conflict with an existing symbol either present in the package or accessible to it. import signals a name conflict error even if the conflict is with a shadowing symbol, the rationale being that the user has given two explicit and inconsistent directives, export makes a single symbol accessible to all the packages that use the package from which the symbol is exported. All of these packages are checked for name conflicts: (export s p) does (find-symbol (symbol-name s) q) for each package q in (package-used-by-list p). Note that in the usual case of an export during the initial definition of a package, the result of package-used-by-list will be nil and the name-conflict checking will take negligible time.
The function intern, which is the one used most frequently by the LISP reader for looking up names of symbols, does not need to do any name-conflict checking, because it never creates a new symbol if there is already an accessible symbol with the name given.
shadow
and shadowing-import never signal a name-conflict error because the user, by calling these functions, has specified how any possible conflict is to be resolved. shadow does name-conflict checking to the extent that it checks whether it is directly present in the package or inherited. In the latter case, a new symbol is created to shadow it. shadowing-import does name-conflict checking to the extent that it checks whether a distinct existing symbol with the same name is accessible; if so, it is shadowed by the new symbol, which implies that it must be uninterned if it was directly present in the package.unuse-package, unexport
, and unintern (when the symbol being uninterned is not a shadowing symbol) do not need to do any name-conflict checking because they only remove symbols from a package; they do not make any new symbols accessible.Giving a shadowing symbol to unintern can uncover a name conflict that had previously been resolved by the shadowing. If package A uses packages B and C, A contains a shadowing symbol x, and B and C each contain external symbols named x, then removing the shadowing symbol x from A will reveal a name between b:x and c:x if those two symbols are distinct. In this case unintern will signal an error.
Aborting from a name-conflict error leaves the original symbol accessible. Package functions always signal name-conflict errors before making any change to the package structure. When multiple changes are to be made, however, for example when export is given a list of symbols, it is permissible for the implementation to process each change separately, so that aborting from a name-conflict caused by the second symbol in the list will not unexport the first symbol in the list. However, aborting from a name-conflict error caused by export of a single symbol will not leave that symbol accessible to some packages and inaccessible to others; with respect to each symbol processed, export behaves as if it were an atomic operation.
Continuing from a name-conflict error should offer the user a chance to resolve the name conflict in favor of either of the candidates. The package structure should be altered to reflect the resolution of the name conflict, via shadowing-import, unintern, or unexport.
A name conflict in use-package between a symbol directly present in the using package and an external symbol of the used package may be resolved in favor of the second symbol by uninterning the first symbol from the using package external symbol of the using package, since it will cease to be an external symbol.
A name conflict in use-package between two external symbols inherited by the using package from other packages may be resolved in favor of either symbol by importing it into the using package and making it a shadowing symbol.
A name conflict in export between the symbol being exported and a symbol already present in a package that would inherit the newly exported symbol may be resolved in favor of the exported symbol by uninterning the other one, or in favor of the already-present symbol by making it a shadowing symbol.
A name conflict in export or unintern due to a package inheriting two distinct symbols with the same name from two other packages may be resolved in favor of either symbol by importing it into the using package and making it a shadowing symbol, just as with use-package.
A name conflict in import between the symbol being imported and a symbol inherited from some other package may be resolved in favor of the symbol being imported by making it a shadowing symbol, or in favor of the symbol already accessible by not doing the import. A name conflict in import with a symbol already present in the package may be resolved by uninterning that symbol, or by not doing the import.
Good user-interface style dictates that use-package and export, which can cause many name conflicts simultaneously, first check for all of the name conflicts before presenting any of them to the user. The user may then choose to resolve all of them wholesale or to resolve each of them individually, the latter requiring a lot of interaction but permitting different conflicts to be resolved different ways.
Implementations may offer other ways of resolving name conflicts. For instance, if the symbols that conflict are not being used as objects but only as names for functions, it may be possible to "merge" the two symbols by putting the function definition onto both symbols. References to either symbol for purposes of calling a function would be equivalent. A similar merging operation can be done for variable valuesand for things stored on the property list. In LISP Machine Lisp, for example, one can also forward the value, function, and property cells so that future changes to either symbol will propogate to the other one. Some other implementations are able to do this with value cells but not with property list. Only the user can know whether this way of resolving a name conflict is adequate, because it will work only if the use of two non-eq symbols with the same name will not prevent the correct operation of the program. The value of offering symbol merging as a way of resolving name conflicts is that it can avoid the need to throw away the whole LISP world, correct the package-definition forms that caused the error, and start over from scratch.
11.7 Built-in Packages
The following packages are built into Star Sapphire:
lisp package
the package named lisp contains the primitives of the Common Lisp system. Its external symbols include all of the user-visible functions and global variables that are present in the Common LISP system, such as car, cdr, and *package*. Almost all other packages will want ot use lisp so that these symbols will be accessible without qualification.
user package
The user package is, by default, the current package at the time a Common LISP system starts up. This package uses the lisp package.
keyword package
This package contains all of the keywords used by built-in or user-defined LISP functions. Printed symbol representations that start with a colon are interpreted as referring to symbols in this package, which are always external symbols. All symbols in this package are treated as constants that evaluate to themselves, so that the user can type :foo instead of ':foo.
system package
This package name is reserved to the implementation. Normally this is used to contain names of implementation-dependent system-interface functions. This package uses lisp and has the nickname sys.