5.5 Optimizing Type Translators

Being based on generic functions, the type translation mechanism described above can add a bit of overhead. This is usually not significant, but we nevertheless provide a way of getting rid of the overhead for the cases where it matters.

A good way to understand this issue is to look at the code generated by defcfun. Consider the following example using the previously defined my-string type:

  CFFI> (macroexpand-1 '(defcfun foo my-string (x my-string)))
  ;; (simplified, downcased, etc...)
  (defun foo (x)
    (multiple-value-bind (#:G2019 #:PARAM3149)
        (translate-to-foreign x #<MY-STRING-TYPE {11ED5A79}>)
      (unwind-protect
          (translate-from-foreign
           (foreign-funcall "foo" :pointer #:G2019 :pointer)
           #<MY-STRING-TYPE {11ED5659}>)
        (free-translated-object #:G2019 #<MY-STRING-TYPE {11ED51A79}>
                                #:PARAM3149))))

In order to get rid of those generic function calls, CFFI has another set of extensible generic functions that provide functionality similar to CL's compiler macros: expand-to-foreign-dyn, expand-to-foreign and expand-from-foreign. Here's how one could define a my-boolean with them:

  (define-foreign-type my-boolean-type ()
    ()
    (:actual-type :int)
    (:simple-parser my-boolean))
   
  (defmethod expand-to-foreign (value (type my-boolean-type))
    `(if ,value 1 0))
   
  (defmethod expand-from-foreign (value (type my-boolean-type))
    `(not (zerop ,value)))

And here's what the macroexpansion of a function using this type would look like:

  CFFI> (macroexpand-1 '(defcfun bar my-boolean (x my-boolean)))
  ;; (simplified, downcased, etc...)
  (defun bar (x)
    (let ((#:g3182 (if x 1 0)))
      (not (zerop (foreign-funcall "bar" :int #:g3182 :int)))))

No generic function overhead.

Let's go back to our my-string type. The expansion interface has no equivalent of free-translated-object; you must instead define a method on expand-to-foreign-dyn, the third generic function in this interface. This is especially useful when you can allocate something much more efficiently if you know the object has dynamic extent, as is the case with function calls that don't save the relevant allocated arguments.

This exactly what we need for the my-string type:

  (defmethod expand-from-foreign (form (type my-string-type))
    `(foreign-string-to-lisp ,form))
   
  (defmethod expand-to-foreign-dyn (value var body (type my-string-type))
    (let ((encoding (string-type-encoding type)))
      `(with-foreign-string (,var ,value :encoding ',encoding)
         ,@body)))

So let's look at the macro expansion:

  CFFI> (macroexpand-1 '(defcfun foo my-string (x my-string)))
  ;; (simplified, downcased, etc...)
  (defun foo (x)
    (with-foreign-string (#:G2021 X :encoding ':utf-8)
      (foreign-string-to-lisp
       (foreign-funcall "foo" :pointer #:g2021 :pointer))))

Again, no generic function overhead.

Other details

To short-circuit expansion and use the translate-* functions instead, simply call the next method. Return its result in cases where your method cannot generate an appropriate replacement for it. This analogous to the &whole form mechanism compiler macros provide.

The expand-* methods have precedence over their translate-* counterparts and are guaranteed to be used in defcfun, foreign-funcall, defcvar and defcallback. If you define a method on each of the expand-* generic functions, you are guaranteed to have full control over the expressions generated for type translation in these macros.

They may or may not be used in other CFFI operators that need to translate between Lisp and C data; you may only assume that expand-* methods will probably only be called during Lisp compilation.

expand-to-foreign-dyn has precedence over expand-to-foreign and is only used in defcfun and foreign-funcall, only making sense in those contexts.

Important note: this set of generic functions is called at macroexpansion time. Methods are defined when loaded or evaluated, not compiled. You are responsible for ensuring that your expand-* methods are defined when the foreign-funcall or other forms that use them are compiled. One way to do this is to put the method definitions earlier in the file and inside an appropriate eval-when form; another way is to always load a separate Lisp or FASL file containing your expand-* definitions before compiling files with forms that ought to use them. Otherwise, they will not be found and the runtime translators will be used instead.