Copyright © 2008 Tobias C. Rittweiler <trittweiler at common-lisp.net>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
The software is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.
Parse-Declarations is a Common Lisp library to help writing macros which establish bindings. To be semantically correct, such macros must take user declarations into account, as these may affect the bindings they establish. Yet the ANSI standard of Common Lisp does not provide any operators to work with declarations in a convenient, high-level way.
This library provides such operators. In particular, it includes facilities to
&BODY parameter into body forms, declarations, and documentation string
(see parse-body),
All declaration specifiers defined by the ANSI standard are understood. Furthermore, Parse-Declarations provides ways for the user to customize how unknown declaration specifiers are parsed. In particular, it allows users to
The API.
⇒ result-identifier, result-args, context
The generic function analyze-declaration-specifier is used by parse-declarations to split an arbitrary declaration specifier into semantically-interesting parts. These parts are returned as multiple values.
At the moment, the following three values are returned:
PARSE-DECLARATIONS> (analyze-declaration-specifier 'optimize '((speed 0) debug) nil) => OPTIMIZE => NIL => ((SPEED 0) DEBUG) PARSE-DECLARATIONS> (analyze-declaration-specifier 'type '(fixnum x y z) nil) => TYPE => (X Y Z) => FIXNUM PARSE-DECLARATIONS> (analyze-declaration-specifier 'inline '(f g h) nil) => INLINE => (#'F #'G #'H) => NIL PARSE-DECLARATIONS> (analyze-declaration-specifier '(string 512) '(str1 str2) nil) => TYPE => (STR1 STR2) => (STRING 512)
None.
build-declaration-specifier
parse-declarations
The compilation-env is provided for methods that have to deal with type specifiers. Cf. X3J13 Issue #334.
The default method of analyze-declaration-specifier will construct an unknown declaration specifier whose “context” is the passed declaration-args. This is mostly interesting to know for usages of map-declaration-env.
Users of the library Parse-Declarations can extend the set of understood declaration specifiers
by adding methods to the generic-functions analyze-declaration-specifier and
build-declaration-specifier. These methods should most likely specialize on the first argument
with an EQL specializer.
⇒ result-specifier
The generic function build-declaration-specifier is used by build-declarations to reconstruct a declaration specifier from the parts returned by analyze-declaration-specifier.
It is hence the counterpart of analyze-declaration-specifier: whereas that one disassembles a declaration specifier into parts, build-declaration-specifier assembles the parts back into a declaration specifier.
PARSE-DECLARATIONS> (build-declaration-specifier 'optimize nil '((speed 0) debug)) => (OPTIMIZE (SPEED 0) DEBUG) PARSE-DECLARATIONS> (build-declaration-specifier 'type '(x y z) 'fixnum) => (TYPE FIXNUM X Y Z) PARSE-DECLARATIONS> (build-declaration-specifier 'inline '(#'f #'g #'h) nil) => (INLINE F G H)
None.
analyze-declaration-specifier
build-declarations
build-declaration-specifier can be thought of as being (almost) the inverse function of
analyze-declaration-specifier. That is for an arbitrary declaration specifier in
*SPEC*, the following code should axiomatically result in a declaration specifier that is
equivalent to *SPEC* when interpreted by the Common Lisp implementation:
(multiple-value-call #'build-declaration-specifier
(analyze-declaration-specifier (first *spec*) (rest *spec*) nil))
However, the result may in fact not be EQUAL to the original
specifier, as analyze-declaration-specifier can perform arbitrary normalization:
PARSE-DECLARATIONS> (let ((spec '((string 512) variable)))
(multiple-value-call #'build-declaration-specifier
(analyze-declaration-specifier (first spec) (rest spec) nil)))
=> (TYPE (STRING 512) VARIABLE)
The function build-declarations constructs all the declarations that are stored in each
of the declaration-envs. The symbol tag is used as
the first element of these declarations. If tag is NIL, build-declarations returns a list of declaration specifiers rather than
declarations.
The order of the returned expressions is not specified; in particular, the order doesn't have to be in any relation with the order of declarations that parse-declarations has been invoked with to create one of declaration-envs.
Furthermore, declarations-or-specifiers may share structure with the declaration specifiers that were initially passed to parse-declarations.
PARSE-DECLARATIONS> (defparameter *env* (parse-declarations '((declare (optimize speed))
(declare (fixnum x y))
(declare (inline +)))))
=> *ENV*
PARSE-DECLARATIONS> (build-declarations 'declare *env*)
=> ((DECLARE (INLINE +)) (DECLARE (TYPE FIXNUM X Y)) (DECLARE (OPTIMIZE SPEED)))
PARSE-DECLARATIONS> (build-declarations nil *env*)
=> ((TYPE FIXNUM X Y) (INLINE +) (OPTIMIZE SPEED))
Signals an error of type TYPE-ERROR if tag is not a
symbol, or if any declaration-env is not a declaration-env.
build-declaration-specifier
parse-declarations
Checks that place is a declaration-env with certain properties.
If unknown-allowed is NIL, and declaration-env
contains unknown declaration specifiers, an continuable
error is signalled. If the CONTINUE restart is invoked,
place is set to an declaration-env with all the unknown declaration specifiers
filtered away.
If warn-only is true, warnings instead of errors are signalled, and
check-declaration-env behaves as if the CONTINUE
restart was invoked.
Signals an error of type TYPE-ERROR if
declaration-env is not a declaration-env.
Signals continuable errors, or warnings, under the circumstances described above.
A declaration-env is a container for declaration specifiers.
check-declaration-env
declaration-env-p
declaration-env.affected-variables
declaration-env.policy
parse-declarations
At the discretion of the implementation, either standard-object or structure-object might appear in the class precedence list. Cf. CLHS 4.2.2.
The type declaration-env is opaque, there is no explicit constructor provided. Use parse-declarations instead.
Returns true if declaration-env is a declaration-env, false otherwise.
None.
The function declaration-env.affected-variables returns all binding names that are affected by the declaration specifiers stored in declaration-env. If allowed-decl-ids is given, only the binding names affected by the specifiers starting with one of allowed-decl-ids are returned.
PARSE-DECLARATIONS> (defparameter *env* (parse-declarations '((declare (optimize speed))
(declare (fixnum x y))
(declare (inline +)))))
=> *ENV*
PARSE-DECLARATIONS> (declaration-env.affected-variables *env*)
=> (X Y #'+)
PARSE-DECLARATIONS> (declaration-env.affected-variables *env* '(type))
=> (X Y)
;;; The following returns NIL, as (ANALYZE-DECLARATION-SPECIFIER 'FIXNUM NIL NIL)
;;; returns TYPE as first value.
PARSE-DECLARATIONS> (declaration-env.affected-variables *env* '(fixnum))
=> NIL
Signals an error of type TYPE-ERROR if
declaration-env is not a declaration-env.
analyze-declaration-specifier
declaration-env
declaration-env.policy
The function declaration-env.policy returns the optimize qualities that are stored in declaration-env.
PARSE-DECLARATIONS> (declaration-env.policy (parse-declarations
'((declare (optimize (speed 0) (debug 2)))
(declare (fixnum x y))
(declare (inline +))
(declae (optimize (safety 3))))))
=> ((SAFETY 3) (SPEED 0) (DEBUG 2))
Signals an error of type TYPE-ERROR if
declaration-env is not a declaration-env.
declaration-env
declaration-env.affected-variables
⇒ result-env
:everything, :bound, :free, :unknown,
or a list of normalized declaration identifiers.
:everything, :bound, :free, :unknown,
or a list of normalized declaration identifiers.
The function filter-declaration-env returns a new declaration-env containing a subset of the declaration specifiers in declenv according to a filtering constituted by the given parameters.
include specifies the declaration specifiers to be considered by the filtering. They can be specified either directly via a list of identifiers, or via a keyword representing a certain set of specifiers. These keywords are:
:bound:everything:free:unknownexclude specifies the declaration specifiers to be not considered.
affecting, if given, specifies the set of binding names that the declaration specifiers in the resulting declaration-env must affect. Consequently, only bound declaration specifiers are selected.
not-affecting, if given, specifies the set of binding names that the resulting specifiers must not affect. This possibly includes free declaration specifiers and unknown declaration specifiers
filter-function is called for each declaration specifier in declenv that satisfies the other given parameters. It should return true if this specifier should be included in the resulting declaration-env. The function is invoked with the disassembled parts of this specifier, cf. analyze-declaration-specifier.
PARSE-DECLARATIONS> (defparameter *sample-decls*
'((declare (optimize (speed 3) (safety 0)))
(declare (special *a*) (special *f*))
(declare (inline f))
(declare (author "Tobias C Rittweiler"))
(declare (type integer x y))
(declare (optimize (debug 0)))
(declare (type fixnum z))
(declare ((string 512) output))
(declare (type (vector unsigned-byte 32) chunk))
(declare (quux *a*)) ; assuming QUUX hasn't been defined as type.
(declare (float *f*))
(declare (ftype (function (number) float) f))
))
=> *SAMPLE-DECLS*
PARSE-DECLARATIONS> (defparameter *env* (parse-declarations *sample-decls*))
=> *ENV*
PARSE-DECLARATIONS> (build-declarations 'declare (filter-declaration-env *env* :include :free))
=> ((DECLARE (OPTIMIZE (DEBUG 0))) (DECLARE (OPTIMIZE (SPEED 3) (SAFETY 0))))
PARSE-DECLARATIONS> (build-declarations 'declare (filter-declaration-env *env* :include :unknown))
=> ((DECLARE (QUUX *A*)) (DECLARE (AUTHOR "Tobias C Rittweiler")))
PARSE-DECLARATIONS> (build-declarations 'declare (filter-declaration-env *env* :affecting '(*a*)))
=> ((DECLARE (SPECIAL *A*)))
PARSE-DECLARATIONS> (build-declarations 'declare (filter-declaration-env *env* :affecting '(*f*)))
=> ((DECLARE (TYPE FLOAT *F*)) (DECLARE (SPECIAL *F*)))
PARSE-DECLARATIONS> (build-declarations 'declare (filter-declaration-env *env* :affecting '(#'f)))
=> ((DECLARE (INLINE F)) (DECLARE (FTYPE (FUNCTION (NUMBER) FLOAT) F)))
PARSE-DECLARATIONS> (build-declarations 'declare (filter-declaration-env *env*
:affecting '(#'f)
:exclude '(inline notinline)))
=> ((DECLARE (FTYPE (FUNCTION (NUMBER) FLOAT) F)))
PARSE-DECLARATIONS> (build-declarations 'declare (filter-declaration-env *env* :include '(type)))
=> ((DECLARE (TYPE FLOAT *F*)) (DECLARE (TYPE (VECTOR UNSIGNED-BYTE 32) CHUNK))
(DECLARE (TYPE (STRING 512) OUTPUT)) (DECLARE (TYPE FIXNUM Z))
(DECLARE (TYPE INTEGER X Y)))
Signals an error of type TYPE-ERROR if a passed argument
violates its respective entry in the “Arguments and Values” section.
declaration-env.affected-variables
map-declaration-env
If no &key parameter is given, filter-declaration-env will return a copy of the given
declaration-env. This follows from the default values of the parameters.
The function map-declaration-env maps function over declaration-env in the following way: function is called for each declaration specifier in declaration-env with the parts (cf. analyze-declaration-specifier) of that specifier. function should either return new values for the parts, or return the same values.
If new values are returned for an unknown declaration specifier, the specifier ceases to be considered unknown in result-env.
map-declaration-env cannot be used to filter particular entries of declaration-env away.
Signals an error of type TYPE-ERROR if function is
not a function designator,
or declaration-env is not a declaration-env.
map-declaration-env is provided to let users of the library Parse-Declaration customize the way declaration specifiers are parsed in arbitrary ways that scale even when multiple applications use Parse-Declarations in the same Lisp image.
Users can extend the set of understood declaration specifiers by adding methods on
analyze-declaration-specifier and build-declaration-specifier. These methods ought to
specialize on the declaration identifier with an EQL
specializer. The package machinery will ensure that no conflict will
arise between multiple applications in the same Lisp image.
If users want to customize the behaviour beyond that (for example parsing a standard declaration specifier in a different way), they should locally use map-declaration-env on the result of parse-declarations.
The function merge-declaration-envs returns the union of the declaration specifiers in declaration-env1 and declaration-env2 as a new declaration-env. There is no attempt made at removing duplicates, or at any other kind of normalization.
PARSE-DECLARATIONS> (defparameter *env* (parse-declarations '((declare (optimize speed))
(declare (fixnum x y))
(declare (inline +)))))
=> *ENV*
PARSE-DECLARATIONS> (defparameter *env2* (parse-declarations '((declare (type fixnum y z))
(declare (notinline +))
(declare (optimize (safety 3))))))
=> *ENV2*
PARSE-DECLARATIONS> (build-declarations 'declare (merge-declaration-envs *env* *env2*))
=> ((DECLARE (NOTINLINE +))
(DECLARE (INLINE +))
(DECLARE (TYPE FIXNUM X Y))
(DECLARE (TYPE FIXNUM Y Z))
(DECLARE (OPTIMIZE SPEED))
(DECLARE (OPTIMIZE (SAFETY 3))))
Signals an error of type TYPE-ERROR if
declaration-env1 or declaration-env2 is not a declaration-env.
The function parse-body splits an &BODY argument of an
extended lambda list into its three
distinctive parts which are returned as multiple values:
A documentation string is only parsed if documentation is true. whole is used to indicate the context if an error is signalled, and should hence be a symbol naming the operator parse-body is used to define.
PARSE-DECLARATIONS> (parse-body '("Also sprach Zarathustra.."
(declare (optimize speed))
(declare (author "God"))
(setf *universe* (big-bang))
(frob-light))
:documentation t)
=> ((SETF *UNIVERSE* (BIG-BANG)) (FROB-LIGHT))
=> ((DECLARE (OPTIMIZE SPEED)) (DECLARE (AUTHOR "God")))
=> "Also sprach Zarathustra.."
Signals an error of type SIMPLE-ERROR if
body-and-decls contains more than one documentation string, and documentation is
true.
The symbol PARSE-BODY is not exported from the library Parse-Declarations to
avoid a name conflict with the Alexandria library. However, this symbol is guaranteed to be part of Parse-Declarations, and can
hence be referenced directly, or imported explicitly, if users don't want to use Alexandria.
⇒ declaration-env
The function parse-declarations returns a declaration-env that describes the declaration specifiers in declarations-or-specifiers. This declaration-env can be used to manipulate the declaration specifiers in various high-level ways, and finally to construct actual specifiers again.
If nostrip is true, declarations-or-specifiers should be a list of declaration specifiers rather than declarations.
Signals an error of type TYPE-ERROR if the (implicit)
specifiers in declarations-or-specifiers are not valid declaration specifiers.
analyze-declaration-specifier
build-declarations
declaration-env
filter-declaration-env
map-declaration-env
merge-declaration-envs
parse-body
The set of understood declaration specifiers can be extended by adding methods to the generic functions analyze-declaration-specifier and build-declaration-specifier.
DOAs first example, we provide an implementation of DO. Notice
that CLHS 3.3.4 specifies that the step-forms,
end-test-form, and result-forms of a DO
expression must be evaluated in the scope of the local declarations given.
Thus:
(defmacro do ((&rest bindings) (end-test-form &body result-forms) &body decls-and-body
&environment macro-env)
(let ((loop-tag (gensym "DO-LOOP+")))
(multiple-value-bind (statements decls)
(parse-declarations::parse-body decls-and-body :documentation nil)
`(prog ,(loop for binding in bindings
collect (destructuring-bind (var &optional init step) binding
(declare (ignore step))
`(,var ,init)))
,@(build-declarations 'declare (parse-declarations decls macro-env))
,loop-tag
(when ,end-test-form (return (progn ,@result-forms)))
,@statements
(psetq ,@(loop for binding in bindings
appending (destructuring-bind (var &optional init step) binding
(declare (ignore init))
(when step `(,var ,step)))))
(go ,loop-tag)))))
In this example, all we had to do is to split a &BODY argument up into the real body forms
and the declarations, because, conceptually, we only had to splice in a few new forms before the
actual body forms. The declarations as such aren't touched.
Notice that the expression ,@(build-declarations 'declare (parse-declarations
decls macro-env)) could (and should!) have been written much easier as a simple ,@decls. We
wanted this to be a first and very simple introduction to the basic operators of the
Parse-Declarations library, though.
LET*Next we implement LET* which is a more interesting example
as it's an example of a non-trivial binding construct. A naive implementor would just go off making
a LET* form expand into nested
LET forms. That would, however, be broken with respect to
declarations which must be nested alongside the bindings they affect.
With the :affecting parameter of filter-declaration-env, we can easily extract only
those declarations which affect a given binding. Thus:
(defmacro let* (bindings &body body &environment macro-env)
(flet ((normalize-binding (binding)
(cond ((symbolp binding) `(,binding nil))
((null (cdr binding)) `(,(car binding) nil))
(t binding))))
(multiple-value-bind (real-body decls) (parse-declarations::parse-body body :documentation nil)
(let ((decl-env (parse-declarations decls macro-env)))
(check-declaration-env decl-env :unknown-allowed nil :warn-only t)
(labels ((generate-nested-lets (bindings &optional used-binding-names)
(if (null bindings)
`(locally
,@(build-declarations 'declare
(filter-declaration-env decl-env :include :free)
(filter-declaration-env decl-env :include :bound
:not-affecting used-binding-names))
,@real-body)
(destructuring-bind ((var value) . more-bindings) bindings
`(let ((,var ,value))
,@(build-declarations 'declare
(filter-declaration-env decl-env :affecting `(,var)))
,(generate-nested-lets more-bindings (cons var used-binding-names)))))))
(generate-nested-lets (mapcar #'normalize-binding bindings)))))))
We warn about unknown declaration specifiers, as we don't know which binding such a specifier affects. Hence, we can't know where it is supposed to be located. We could move these into the base case—which may sometimes be exactly the right decision—, but we feel that a user looking at the macroexpansion might be tricked thinking that the right thing is done even though it isn't.
For the base case, we splice in all free declaration specifiers and all bound declaration specifiers which affect bindings not established by the LET* form. These, plus the ignored unknown declaration specifiers and the bound declaration specifiers affecting bindings established by the LET* form, constitute all possible specifiers. So we haven't forgot any.
OPTIMIZE-DECLARATION-ENVAs last example, we show how to change the treatment of standard declaration specifiers.
We want to write a function which takes a list of optimize qualities and which adapts a given declaration-env to these qualities. Existing qualities in that declaration-env are changed, and if no qualities are given, the qualities are added. I.e.:
(defun optimize-declaration-env (declaration-env qualities &aux result-env)
(flet ((ensure-car (thing) (if (consp thing) (car thing) thing)))
(setq result-env
(map-declaration-env
#'(lambda (id args ctx) ; CTX of OPTIMIZE are the qualities.
(if (eq id 'optimize)
(values id args `(,@qualities ,@(set-difference ctx qualities :key #'ensure-car)))
(values id args ctx)))
declaration-env))
(if (declaration-env.policy result-env)
result-env
(merge-declaration-envs result-env (parse-declarations `((optimize ,@qualities)) nil
:nostrip t)))))
CAR is a declaration identifier, and whose CDR are the
declaration arguments. See CLHS 3.3.2,
CLHS 3.3.3.1, and
CLHS declaration specifier.
FUNCTION and the second element is the
function name for a binding in the function
namespace.
analyze-declaration-specifier: analyze-declaration-specifierbuild-declaration-specifier: build-declaration-specifierbuild-declarations: build-declarationscheck-declaration-env: check-declaration-envdeclaration-env: declaration-envdeclaration-env-p: declaration-env-pdeclaration-env.affected-variables: declaration-env.affected-variablesdeclaration-env.policy: declaration-env.policyfilter-declaration-env: filter-declaration-envmap-declaration-env: map-declaration-envmerge-declaration-envs: merge-declaration-envsparse-body: parse-bodyparse-declarations: parse-declarations