\input texinfo   @c -*- Mode: Texinfo; Mode: auto-fill -*-

@c %**start of header
@setfilename cffi.info
@settitle CFFI User Manual
@exampleindent 2

@c @documentencoding utf-8

@c Style notes:
@c
@c * The reference section names and "See Also" list are roman, not
@c   @code.  This is to follow the format of CLHS.
@c
@c * How it looks in HTML is the priority.

@c ============================= Macros =============================
@c The following macros are used throughout this manual.

@macro Function {args}
@defun \args\
@end defun
@end macro

@macro Macro {args}
@defmac \args\
@end defmac
@end macro

@macro Accessor {args}
@deffn {Accessor} \args\
@end deffn
@end macro

@macro GenericFunction {args}
@deffn {Generic Function} \args\
@end deffn
@end macro

@macro ForeignType {args}
@deftp {Foreign Type} \args\
@end deftp
@end macro

@macro Variable {args}
@defvr {Special Variable} \args\
@end defvr
@end macro

@macro Condition {args}
@deftp {Condition Type} \args\
@end deftp
@end macro

@macro cffi
@acronym{CFFI}
@end macro

@macro impnote {text}
@quotation
@strong{Implementor's note:} @emph{\text\}
@end quotation
@end macro

@c Info "requires" that x-refs end in a period or comma, or ) in the
@c case of @pxref.  So the following implements that requirement for
@c the "See also" subheadings that permeate this manual, but only in
@c Info mode.
@ifinfo
@macro seealso {name}
@ref{\name\}.
@end macro
@end ifinfo

@ifnotinfo

@alias seealso = ref

@end ifnotinfo

@c Typeset comments in roman font for the TeX output.
@iftex

@alias lispcmt = r

@end iftex
@ifnottex

@alias lispcmt = asis

@end ifnottex

@c My copy of makeinfo is not generating any HTML for @result{} for
@c some odd reason. (It certainly used to...)
@ifhtml
@macro result
=>
@end macro
@end ifhtml

@c Similar macro to @result. Its purpose is to work around the fact
@c that ⇒ does not work properly inside @lisp.
@ifhtml
@macro res
@html
⇒
@end html
@end macro
@end ifhtml

@ifnothtml
@alias res = result
@end ifnothtml

@c ============================= Macros =============================

@c Show types, functions, and concepts in the same index.
@syncodeindex tp cp
@syncodeindex fn cp

@copying

Copyright @copyright{} 2005 James Bielman <jamesjb at jamesjb.com> @*

Copyright @copyright{} 2005-2010 Lu@'{@dotless{i}}s Oliveira

  <loliveira at common-lisp.net> @*

Copyright @copyright{} 2005-2006 Dan Knapp <danka at accela.net>
Copyright @copyright{} 2005-2006 Emily Backes <lucca at accela.net>

Copyright @copyright{} 2006 Stephen Compall <s11 at member.fsf.org>

@quotation

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
``Software''), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

@sc{The software is provided ``as is'', without warranty of any kind,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose and noninfringement.
In no event shall the authors or copyright holders be liable for any
claim, damages or other liability, whether in an action of contract,
tort or otherwise, arising from, out of or in connection with the
software or the use or other dealings in the software.}

@end quotation
@end copying

@c %**end of header

@dircategory Software development
@direntry
* CFFI Manual: (cffi-manual).           CFFI Manual.
@end direntry

@titlepage
@title CFFI User Manual
@c @subtitle Version X.X
@c @author James Bielman

@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents

@ifnottex

@node Top, Introduction, (dir), (dir)

@top cffi
@insertcopying
@end ifnottex

@menu

* Introduction::                What is CFFI?

* Installation::                

* Implementation Support::      

* Tutorial::                    Interactive intro to using CFFI.
* Wrapper generators::          CFFI forms from munging C source code.

* Foreign Types::               
* Pointers::                    
* Strings::                     
* Variables::                   
* Functions::                   
* Libraries::                   
* Callbacks::                   

* The Groveller::               

* Limitations::                 

* Platform-specific features::  Details about the underlying system.

* Glossary::                    List of CFFI-specific terms and meanings.

* Comprehensive Index::         

@detailmenu

 --- Dictionary ---

Foreign Types

* convert-from-foreign::        Outside interface to backward type translator.
* convert-to-foreign::          Outside interface to forward type translator.

* defbitfield::                 Defines a bitfield.

* defcstruct::                  Defines a C structure type.
* defcunion::                   Defines a C union type.
* defctype::                    Defines a foreign typedef.
* defcenum::                    Defines a C enumeration.
* define-foreign-type::         Defines a foreign type specifier.

* define-parse-method::         Specifies how a type should be parsed.

@c * explain-foreign-slot-value::  <unimplemented>

* foreign-bitfield-symbols::    Returns a list of symbols for a bitfield type.
* foreign-bitfield-value::      Calculates a value for a bitfield type.

* foreign-enum-keyword::        Finds a keyword in an enum type.
* foreign-enum-value::          Finds a value in an enum type.

* foreign-slot-names::          Returns a list of slot names in a foreign struct.
* foreign-slot-offset::         Returns the offset of a slot in a foreign struct.

* foreign-slot-pointer::        Returns a pointer to a slot in a foreign struct.

* foreign-slot-value::          Returns the value of a slot in a foreign struct.
* foreign-type-alignment::      Returns the alignment of a foreign type.
* foreign-type-size::           Returns the size of a foreign type.

* free-converted-object::       Outside interface to typed object deallocators.

* free-translated-object::      Defines how to free a oreign object.
* translate-from-foreign::      Defines a foreign-to-Lisp object translation.
* translate-to-foreign::        Defines a Lisp-to-foreign object translation.

* with-foreign-object::         Allocates a foreign object with dynamic extent.

* with-foreign-objects::        Plural form of @code{with-foreign-object}.

* with-foreign-slots::          Accesses the slots of a foreign structure.

Pointers

* foreign-free::                Deallocates memory.
* foreign-alloc::               Allocates memory.

* foreign-symbol-pointer::      Returns a pointer to a foreign symbol.
* inc-pointer::                 Increments the address held by a pointer.

* incf-pointer::                Increments the pointer address in a place.

* make-pointer::                Returns a pointer to a given address.

* mem-aptr::                    The pointer to an element of an array.

* mem-aref::                    Accesses the value of an index in an array.
* mem-ref::                     Dereferences a pointer.

* null-pointer::                Returns a NULL pointer.
* null-pointer-p::              Tests a pointer for NULL value.

* pointerp::                    Tests whether an object is a pointer or not.

* pointer-address::             Returns the address pointed to by a pointer.

* pointer-eq::                  Tests if two pointers point to the same address.

* with-foreign-pointer::        Allocates memory with dynamic extent.

Strings

* *default-foreign-encoding*::  Default encoding for the string types.

* foreign-string-alloc::        Converts a Lisp string to a foreign string.
* foreign-string-free::         Deallocates memory used by a foreign string.
* foreign-string-to-lisp::      Converts a foreign string to a Lisp string.
* lisp-string-to-foreign::      Copies a Lisp string into a foreign string.
* with-foreign-string::         Allocates a foreign string with dynamic extent.

* with-foreign-strings::        Plural form of @code{with-foreign-string}.

* with-foreign-pointer-as-string::  Similar to CL's with-output-to-string.

Variables

* defcvar::                     Defines a C global variable.

* get-var-pointer::             Returns a pointer to a defined global variable.

Functions

* defcfun::                     Defines a foreign function.
* foreign-funcall::             Performs a call to a foreign function.

* foreign-funcall-pointer::     Performs a call through a foreign pointer.

* translate-camelcase-name::    Converts a camelCase foreign name to/from a Lisp name.
* translate-name-from-foreign::  Converts a foreign name to a Lisp name.
* translate-name-to-foreign::   Converts a Lisp name to a foreign name.
* translate-underscore-separated-name::  Converts an underscore_separated foreign name to/from a Lisp name.

Libraries

* close-foreign-library::       Closes a foreign library.

* *darwin-framework-directories*::  Search path for Darwin frameworks.
* define-foreign-library::      Explain how to load a foreign library.
* *foreign-library-directories*::  Search path for shared libraries.
* load-foreign-library::        Load a foreign library.
* load-foreign-library-error::  Signalled on failure of its namesake.

@c * reload-foreign-libraries::    Reload foreign libraries.

* use-foreign-library::         Load a foreign library when needed.

Callbacks

* callback::                    Returns a pointer to a defined callback.
* defcallback::                 Defines a Lisp callback.
* get-callback::                Returns a pointer to a defined callback.

@end detailmenu

@end menu

@c ===================================================================
@c CHAPTER: Introduction

@node Introduction, Installation, Top, Top

@chapter Introduction

@cffi{} is the Common Foreign Function Interface for @acronym{ANSI}
Common Lisp systems.  By @dfn{foreign function} we mean a function
written in another programming language and having different data and
calling conventions than Common Lisp, namely, C.  @cffi{} allows you
to call foreign functions and access foreign variables, all without
leaving the Lisp image.

We consider this manual ever a work in progress.  If you have
difficulty with anything @cffi{}-specific presented in the manual,
please contact @email{cffi-devel@@common-lisp.net,the developers} with
details.

@heading Motivation

@xref{Tutorial-Comparison,, What makes Lisp different}, for
an argument in favor of @acronym{FFI} in general.

@cffi{}'s primary role in any image is to mediate between Lisp
developers and the widely varying @acronym{FFI}s present in the
various Lisp implementations it supports.  With @cffi{}, you can
define foreign function interfaces while still maintaining portability
between implementations.  It is not the first Common Lisp package with
this objective; however, it is meant to be a more malleable framework
than similar packages.


@heading Design Philosophy

@itemize
@item
Pointers do not carry around type information. Instead, type
information is supplied when pointers are dereferenced.

@item
A type safe pointer interface can be developed on top of an
untyped one.  It is difficult to do the opposite.

@item
Functions are better than macros.  When a macro could be used
for performance, use a compiler-macro instead.
@end itemize

@c ===================================================================
@c CHAPTER: Installation

@node Installation, Implementation Support, Introduction, Top

@chapter Installation

@cffi{} can be obtained through one of the following means available
through its @uref{http://common-lisp.net/project/cffi/,,website}:

@itemize
@item
@uref{http://common-lisp.net/project/cffi/releases/?M=D,,official release
tarballs}

@item

@uref{http://common-lisp.net/gitweb?p=projects/cffi/cffi.git,,git

repository}

@c snapshots have been disabled as of
@c @item
@c @uref{http://common-lisp.net/project/cffi/tarballs/?M=D,,nightly-generated
@c snapshots}

@end itemize

In addition, you will need to obtain and install the following
dependencies:

@itemize
@item
@uref{http://common-lisp.net/project/babel/,,Babel}, a charset
encoding/decoding library.

@item
@uref{http://common-lisp.net/project/alexandria/,,Alexandria}, a
collection of portable public-domain utilities.

@item
@uref{http://www.cliki.net/trivial-features,,trivial-features}, a

portability layer that ensures consistent @code{*features*} across

multiple Common Lisp implementations.

@end itemize

Furthermore, if you wish to run the testsuite,
@uref{http://www.cliki.net/rt,,RT} is required.

You may find mechanisms such as
@uref{http://common-lisp.net/project/clbuild/,,clbuild} (recommended)
or @uref{http://www.cliki.net/ASDF-Install,,ASDF-Install} (not as
recommendable) helpful in getting and managing @cffi{} and its
dependencies.

@c ===================================================================

@c CHAPTER: Implementation Support

@node Implementation Support, Tutorial, Installation, Top

@chapter Implementation Support

@cffi{} supports various free and commercial Lisp implementations:
Allegro CL, Corman CL, @sc{clisp}, @acronym{CMUCL}, @acronym{ECL},

LispWorks, Clozure CL, @acronym{SBCL} and the Scieneer CL.

In general, you should work with the latest versions of each
implementation since those will usually be tested against recent
versions of CFFI more often and might include necessary features or
bug fixes. Reasonable patches for compatibility with earlier versions
are welcome nevertheless.

@section Limitations

Some features are not supported in all implementations.
@c TODO: describe these features here.
@c       flat-namespace too

@subheading Allegro CL

@itemize

@item

Does not support the @code{:long-long} type natively.
@item
Unicode support is limited to the Basic Multilingual Plane (16-bit
code points).

@end itemize

@subheading CMUCL

@itemize
@item
No Unicode support. (8-bit code points)
@end itemize

@subheading Corman CL

@itemize

@item

Does not support @code{foreign-funcall}.

@end itemize

@subheading @acronym{ECL}

@itemize

@item
On platforms where ECL's dynamic FFI is not supported (ie. when
@code{:dffi} is not present in @code{*features*}),
@code{cffi:load-foreign-library} does not work and you must use ECL's
own @code{ffi:load-foreign-library} with a constant string argument.

@item
Does not support the @code{:long-long} type natively.
@item
Unicode support is not enabled by default.

@end itemize

@subheading Lispworks

@itemize

@item

Does not completely support the @code{:long-long} type natively in
32-bit platforms.

@item
Unicode support is limited to the Basic Multilingual Plane (16-bit
code points).

@end itemize

@subheading @acronym{SBCL}

@itemize

@item

Not all platforms support callbacks.

@end itemize

@c ===================================================================
@c CHAPTER: An Introduction to Foreign Interfaces and CFFI

@c This macro is merely a marker that I don't think I'll use after
@c all.
@macro tutorialsource {text}
@c \text\
@end macro

@c because I don't want to type this over and over
@macro clikicffi
http://www.cliki.net/CFFI
@end macro
@c TeX puts spurious newlines in when you use the above macro
@c in @examples &c.  So it is expanded below in some places.

@node Tutorial, Wrapper generators, Implementation Support, Top

@chapter An Introduction to Foreign Interfaces and @acronym{CFFI}

@c Above, I don't use the cffi macro because it breaks TeX.

@cindex tutorial, @cffi{}
Users of many popular languages bearing semantic similarity to Lisp,
such as Perl and Python, are accustomed to having access to popular C
libraries, such as @acronym{GTK}, by way of ``bindings''.  In Lisp, we
do something similar, but take a fundamentally different approach.
This tutorial first explains this difference, then explains how you
can use @cffi{}, a powerful system for calling out to C and C++ and
access C data from many Common Lisp implementations.

@cindex foreign functions and data
The concept can be generalized to other languages; at the time of
writing, only @cffi{}'s C support is fairly complete, but C++
support is being worked on.  Therefore, we will interchangeably refer
to @dfn{foreign functions} and @dfn{foreign data}, and ``C functions''
and ``C data''.  At no time will the word ``foreign'' carry its usual,
non-programming meaning.

This tutorial expects you to have a working understanding of both
Common Lisp and C, including the Common Lisp macro system.

@menu
* Tutorial-Comparison::         Why FFI?
* Tutorial-Getting a URL::      An FFI use case.
* Tutorial-Loading::            Load libcurl.so.
* Tutorial-Initializing::       Call a function in libcurl.so.
* Tutorial-easy_setopt::        An advanced libcurl function.
* Tutorial-Abstraction::        Why breaking it is necessary.
* Tutorial-Lisp easy_setopt::   Semi-Lispy option interface.
* Tutorial-Memory::             In C, you collect the garbage.
* Tutorial-Callbacks::          Make useful C function pointers.
* Tutorial-Completion::         Minimal get-url functionality.
* Tutorial-Types::              Defining new foreign types.
* Tutorial-Conclusion::         What's next?
@end menu

@node Tutorial-Comparison, Tutorial-Getting a URL, Tutorial, Tutorial

@section What makes Lisp different

The following sums up how bindings to foreign libraries are usually
implemented in other languages, then in Common Lisp:

@table @asis
@item Perl, Python, Java, other one-implementation languages
@cindex @acronym{SWIG}
@cindex Perl
@cindex Python
Bindings are implemented as shared objects written in C.  In some
cases, the C code is generated by a tool, such as @acronym{SWIG}, but
the result is the same: a new C library that manually translates
between the language implementation's objects, such as @code{PyObject}
in Python, and whatever C object is called for, often using C
functions provided by the implementation.  It also translates between
the calling conventions of the language and C.

@item Common Lisp
@cindex @acronym{SLIME}
Bindings are written in Lisp.  They can be created at-will by Lisp
programs.  Lisp programmers can write new bindings and add them to the
image, using a listener such as @acronym{SLIME}, as easily as with
regular Lisp definitions.  The only foreign library to load is the one
being wrapped---the one with the pure C interface; no C or other
non-Lisp compilation is required.
@end table

@cindex advantages of @acronym{FFI}
@cindex benefits of @acronym{FFI}
We believe the advantages of the Common Lisp approach far outweigh any
disadvantages.  Incremental development with a listener can be as
productive for C binding development as it is with other Lisp
development.  Keeping it ``in the [Lisp] family'', as it were, makes
it much easier for you and other Lisp programmers to load and use the
bindings.  Common Lisp implementations such as @acronym{CMUCL}, freed
from having to provide a C interface to their own objects, are thus
freed to be implemented in another language (as @acronym{CMUCL} is)
while still allowing programmers to call foreign functions.

@cindex minimal bindings
Perhaps the greatest advantage is that using an @acronym{FFI} doesn't
obligate you to become a professional binding developer.  Writers of
bindings for other languages usually end up maintaining or failing to
maintain complete bindings to the foreign library.  Using an
@acronym{FFI}, however, means if you only need one or two functions,
you can write bindings for only those functions, and be assured that
you can just as easily add to the bindings if need be.

@cindex C abstractions
@cindex abstractions in C
The removal of the C compiler, or C interpretation of any kind,
creates the main disadvantage: some of C's ``abstractions'' are not
available, violating information encapsulation.  For example,
@code{struct}s that must be passed on the stack, or used as return
values, without corresponding functional abstractions to create and
manage the @code{struct}s, must be declared explicitly in Lisp.  This
is fine for structs whose contents are ``public'', but is not so
pleasant when a struct is supposed to be ``opaque'' by convention,
even though it is not so defined.@footnote{Admittedly, this is an
advanced issue, and we encourage you to leave this text until you are
more familiar with how @cffi{} works.}

Without an abstraction to create the struct, Lisp needs to be able to
lay out the struct in memory, so must know its internal details.

@cindex workaround for C
In these cases, you can create a minimal C library to provide the
missing abstractions, without destroying all the advantages of the
Common Lisp approach discussed above.  In the case of @code{struct}s,
you can write simple, pure C functions that tell you how many bytes a
struct requires or allocate new structs, read and write fields of the
struct, or whatever operations are supposed to be
public.@footnote{This does not apply to structs whose contents are
intended to be part of the public library interface.  In those cases,
a pure Lisp struct definition is always preferred.  In fact, many
prefer to stay in Lisp and break the encapsulation anyway, placing the
burden of correct library interface definition on the library.}

@impnote{cffi-grovel, a project not yet part of @cffi{}, automates
this and other processes.}

Another disadvantage appears when you would rather use the foreign
language than Lisp.  However, someone who prefers C to Lisp is not a
likely candidate for developing a Lisp interface to a C library.

@node Tutorial-Getting a URL, Tutorial-Loading, Tutorial-Comparison, Tutorial

@section Getting a @acronym{URL}

@cindex c@acronym{URL}
The widely available @code{libcurl} is a library for downloading files
over protocols like @acronym{HTTP}.  We will use @code{libcurl} with
@cffi{} to download a web page.

Please note that there are many other ways to download files from the
web, not least the @sc{cl-curl} project to provide bindings to
@code{libcurl} via a similar @acronym{FFI}.@footnote{Specifically,
@acronym{UFFI}, an older @acronym{FFI} that takes a somewhat different
approach compared to @cffi{}.  I believe that these days (December
2005) @cffi{} is more portable and actively developed, though not as
mature yet.  Consensus in the free @sc{unix} Common Lisp community
seems to be that @cffi{} is preferred for new development, though
@acronym{UFFI} will likely go on for quite some time as many projects
already use it.  @cffi{} includes the @code{UFFI-COMPAT} package for
complete compatibility with @acronym{UFFI}.}

@uref{http://curl.haxx.se/libcurl/c/libcurl-tutorial.html,,libcurl-tutorial(3)}
is a tutorial for @code{libcurl} programming in C.  We will follow
that to develop a binding to download a file.  We will also use
@file{curl.h}, @file{easy.h}, and the @command{man} pages for the
@code{libcurl} function, all available in the @samp{curl-dev} package
or equivalent for your system, or in the c@acronym{URL} source code
package.  If you have the development package, the headers should be
installed in @file{/usr/include/curl/}, and the @command{man} pages
may be accessed through your favorite @command{man} facility.

@node Tutorial-Loading, Tutorial-Initializing, Tutorial-Getting a URL, Tutorial

@section Loading foreign libraries

@cindex loading @cffi{}
@cindex requiring @cffi{}
First of all, we will create a package to work in.  You can save these
forms in a file, or just send them to the listener as they are.  If
creating bindings for an @acronym{ASDF} package of yours, you will
want to add @code{:cffi} to the @code{:depends-on} list in your
@file{.asd} file.  Otherwise, just use the @code{asdf:oos} function to
load @cffi{}.

@tutorialsource{Initialization}
@lisp
(asdf:oos 'asdf:load-op :cffi)

;;; @lispcmt{Nothing special about the "CFFI-USER" package.  We're just}
;;; @lispcmt{using it as a substitute for your own CL package.}

(defpackage :cffi-user
  (:use :common-lisp :cffi))

(in-package :cffi-user)

(define-foreign-library libcurl
  (:unix (:or "libcurl.so.3" "libcurl.so"))
  (t (:default "libcurl")))

(use-foreign-library libcurl)
@end lisp

@cindex foreign library load
@cindex library, foreign
Using @code{define-foreign-library} and @code{use-foreign-library}, we
have loaded @code{libcurl} into Lisp, much as the linker does when you
start a C program, or @code{common-lisp:load} does with a Lisp source
file or @acronym{FASL} file.  We special-cased for @sc{unix} machines
to always load a particular version, the one this tutorial was tested
with; for those who don't care, the @code{define-foreign-library}
clause @code{(t (:default "libcurl"))} should be satisfactory, and
will adapt to various operating systems.

@node Tutorial-Initializing, Tutorial-easy_setopt, Tutorial-Loading, Tutorial

@section Initializing @code{libcurl}

@cindex function definition
After the introductory matter, the tutorial goes on to present the
first function you should use.

@example
CURLcode curl_global_init(long flags);
@end example

@noindent
Let's pick this apart into appropriate Lisp code:

@tutorialsource{First CURLcode}
@lisp

;;; @lispcmt{A CURLcode is the universal error code.  curl/curl.h says}
;;; @lispcmt{no return code will ever be removed, and new ones will be}
;;; @lispcmt{added to the end.}

(defctype curl-code :int)

;;; @lispcmt{Initialize libcurl with FLAGS.}

(defcfun "curl_global_init" curl-code
  (flags :long))
@end lisp

@impnote{By default, CFFI assumes the UNIX viewpoint that there is one
C symbol namespace, containing all symbols in all loaded objects.
This is not so on Windows and Darwin, but we emulate UNIX's behaviour
there.  @ref{defcfun} for more details.}

Note the parallels with the original C declaration.  We've defined
@code{curl-code} as a wrapping type for @code{:int}; right now, it
only marks it as special, but later we will do something more
interesting with it.  The point is that we don't have to do it yet.

@cindex calling foreign functions
Looking at @file{curl.h}, @code{CURL_GLOBAL_NOTHING}, a possible value
for @code{flags} above, is defined as @samp{0}.  So we can now call
the function:

@example
@sc{cffi-user>} (curl-global-init 0)
@result{} 0
@end example

@cindex looks like it worked
Looking at @file{curl.h} again, @code{0} means @code{CURLE_OK}, so it
looks like the call succeeded.  Note that @cffi{} converted the
function name to a Lisp-friendly name.  You can specify your own name
if you want; use @code{("curl_global_init" @var{your-name-here})} as
the @var{name} argument to @code{defcfun}.

The tutorial goes on to have us allocate a handle.  For good measure,
we should also include the deallocator.  Let's look at these
functions:

@example
CURL *curl_easy_init( );
void curl_easy_cleanup(CURL *handle);
@end example

Advanced users may want to define special pointer types; we will
explore this possibility later.  For now, just treat every pointer as
the same:

@tutorialsource{curl_easy handles}
@lisp
(defcfun "curl_easy_init" :pointer)

(defcfun "curl_easy_cleanup" :void
  (easy-handle :pointer))
@end lisp

Now we can continue with the tutorial:

@example
@sc{cffi-user>} (defparameter *easy-handle* (curl-easy-init))
@result{} *EASY-HANDLE*
@sc{cffi-user>} *easy-handle*
@result{} #<FOREIGN-ADDRESS #x09844EE0>
@end example

@cindex pointers in Lisp
Note the print representation of a pointer.  It changes depending on
what Lisp you are using, but that doesn't make any difference to
@cffi{}.

@node Tutorial-easy_setopt, Tutorial-Abstraction, Tutorial-Initializing, Tutorial

@section Setting download options

The @code{libcurl} tutorial says we'll want to set many options before
performing any download actions.  This is done through
@code{curl_easy_setopt}:

@c That is literally ..., not an ellipsis.

@example

CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);

@end example

@cindex varargs
@cindex foreign arguments
We've introduced a new twist: variable arguments.  There is no obvious
translation to the @code{defcfun} form, particularly as there are four
possible argument types.  Because of the way C works, we could define
four wrappers around @code{curl_easy_setopt}, one for each type; in
this case, however, we'll use the general-purpose macro
@code{foreign-funcall} to call this function.

@cindex enumeration, C
To make things easier on ourselves, we'll create an enumeration of the
kinds of options we want to set.  The @code{enum CURLoption} isn't the
most straightforward, but reading the @code{CINIT} C macro definition
should be enlightening.

@tutorialsource{CURLoption enumeration}
@lisp
(defmacro define-curl-options (name type-offsets &rest enum-args)

  "As with CFFI:DEFCENUM, except each of ENUM-ARGS is as follows:

    (NAME TYPE NUMBER)

Where the arguments are as they are with the CINIT macro defined
in curl.h, except NAME is a keyword.

TYPE-OFFSETS is a plist of TYPEs to their integer offsets, as
defined by the CURLOPTTYPE_LONG et al constants in curl.h."
  (flet ((enumerated-value (type offset)
           (+ (getf type-offsets type) offset)))
    `(progn
       (defcenum ,name
         ,@@(loop for (name type number) in enum-args
              collect (list name (enumerated-value type number))))

       ',name)))                ;@lispcmt{for REPL users' sanity}

(define-curl-options curl-option
    (long 0 objectpoint 10000 functionpoint 20000 off-t 30000)
  (:noprogress long 43)
  (:nosignal long 99)
  (:errorbuffer objectpoint 10)
  (:url objectpoint 2))
@end lisp

With some well-placed Emacs @code{query-replace-regexp}s, you could
probably similarly define the entire @code{CURLoption} enumeration.  I
have selected to transcribe a few that we will use in this tutorial.

If you're having trouble following the macrology, just macroexpand the
@code{curl-option} definition, or see the following macroexpansion,
conveniently downcased and reformatted:

@tutorialsource{DEFINE-CURL-OPTIONS macroexpansion}
@lisp
(progn
  (defcenum curl-option
    (:noprogress 43)
    (:nosignal 99)
    (:errorbuffer 10010)
    (:url 10002))
  'curl-option)
@end lisp

@noindent
That seems more than reasonable.  You may notice that we only use the
@var{type} to compute the real enumeration offset; we will also need
the type information later.

First, however, let's make sure a simple call to the foreign function
works:

@example
@sc{cffi-user>} (foreign-funcall "curl_easy_setopt"
               :pointer *easy-handle*
               curl-option :nosignal :long 1 curl-code)
@result{} 0
@end example

@code{foreign-funcall}, despite its surface simplicity, can be used to
call any C function.  Its first argument is a string, naming the
function to be called.  Next, for each argument, we pass the name of
the C type, which is the same as in @code{defcfun}, followed by a Lisp
object representing the data to be passed as the argument.  The final
argument is the return type, for which we use the @code{curl-code}
type defined earlier.

@code{defcfun} just puts a convenient fa@,cade on
@code{foreign-funcall}.@footnote{This isn't entirely true; some Lisps
don't support @code{foreign-funcall}, so @code{defcfun} is implemented
without it.  @code{defcfun} may also perform optimizations that
@code{foreign-funcall} cannot.}  Our earlier call to
@code{curl-global-init} could have been written as follows:

@example
@sc{cffi-user>} (foreign-funcall "curl_global_init" :long 0
                            curl-code)
@result{} 0
@end example

Before we continue, we will take a look at what @cffi{} can and can't
do, and why this is so.

@node Tutorial-Abstraction, Tutorial-Lisp easy_setopt, Tutorial-easy_setopt, Tutorial

@section Breaking the abstraction

@cindex breaking the abstraction
@cindex abstraction breaking
In @ref{Tutorial-Comparison,, What makes Lisp different}, we mentioned
that writing an @acronym{FFI} sometimes requires depending on
information not provided as part of the interface.  The easy option
@code{CURLOPT_WRITEDATA}, which we will not provide as part of the
Lisp interface, illustrates this issue.

Strictly speaking, the @code{curl-option} enumeration is not
necessary; we could have used @code{:int 99} instead of
@code{curl-option :nosignal} in our call to @code{curl_easy_setopt}
above.  We defined it anyway, in part to hide the fact that we are
breaking the abstraction that the C @code{enum} provides.  If the
c@acronym{URL} developers decide to change those numbers later, we
must change the Lisp enumeration, because enumeration values are not
provided in the compiled C library, @code{libcurl.so.3}.

@cffi{} works because the most useful things in C libraries ---
non-static functions and non-static variables --- are included
accessibly in @code{libcurl.so.3}.  A C compiler that violated this
would be considered a worthless compiler.

The other thing @code{define-curl-options} does is give the ``type''
of the third argument passed to @code{curl_easy_setopt}.  Using this
information, we can tell that the @code{:nosignal} option should
accept a long integer argument.  We can implicitly assume @code{t}
@equiv{} 1 and @code{nil} @equiv{} 0, as it is in C, which takes care
of the fact that @code{CURLOPT_NOSIGNAL} is really asking for a
boolean.

The ``type'' of @code{CURLOPT_WRITEDATA} is @code{objectpoint}.
However, it is really looking for a @code{FILE*}.
@code{CURLOPT_ERRORBUFFER} is looking for a @code{char*}, so there is
no obvious @cffi{} type but @code{:pointer}.

The first thing to note is that nowhere in the C interface includes
this information; it can only be found in the manual.  We could
disjoin these clearly different types ourselves, by splitting
@code{objectpoint} into @code{filepoint} and @code{charpoint}, but we
are still breaking the abstraction, because we have to augment the
entire enumeration form with this additional
information.@footnote{Another possibility is to allow the caller to
specify the desired C type of the third argument.  This is essentially
what happens in a call to the function written in C.}

@cindex streams and C

@cindex @sc{file}* and streams

The second is that the @code{CURLOPT_WRITEDATA} argument is completely
incompatible with the desired Lisp data, a
stream.@footnote{@xref{Other Kinds of Streams,,, libc, GNU C Library
Reference}, for a @acronym{GNU}-only way to extend the @code{FILE*}
type.  You could use this to convert Lisp streams to the needed C
data.  This would be quite involved and far outside the scope of this
tutorial.}  It is probably acceptable if we are controlling every file
we might want to use as this argument, in which case we can just call
the foreign function @code{fopen}.  Regardless, though, we can't write
to arbitrary streams, which is exactly what we want to do for this
application.

Finally, note that the @code{curl_easy_setopt} interface itself is a
hack, intended to work around some of the drawbacks of C.  The
definition of @code{Curl_setopt}, while long, is far less cluttered
than the equivalent disjoint-function set would be; in addition,
setting a new option in an old @code{libcurl} can generate a run-time
error rather than breaking the compile.  Lisp can just as concisely
generate functions as compare values, and the ``undefined function''
error is just as useful as any explicit error we could define here
might be.

@node Tutorial-Lisp easy_setopt, Tutorial-Memory, Tutorial-Abstraction, Tutorial

@section Option functions in Lisp

We could use @code{foreign-funcall} directly every time we wanted to
call @code{curl_easy_setopt}.  However, we can encapsulate some of the
necessary information with the following.

@lisp

;;; @lispcmt{We will use this type later in a more creative way.  For}

;;; @lispcmt{now, just consider it a marker that this isn't just any}
;;; @lispcmt{pointer.}