-*- mode: text -*-

(we are really in muse mode; but most people don't have it readily
available and we aren't forcing the use of Emacs/muse.   M-x muse-mode
makes reading easier IMHO).


* Interfacing Common Lisp and R

This library considers multiple approaches for connecting Common Lisp
and R, particularly for embedding R within CL.   CLSR also attempts to
do the reverse.  RCL is a third approach, which provides more
porcelain around the plumbing.  This will be commented on later.


* Quick start

** Requirements

You will need the following libraries available:

1. ASDF   (system definition facility, for loading packages)
2. CFFI   (Common foreign function interface)
3. RCLG   (this library)


1. If using SBCL, add rclg.asd to the ASDF systems path.

2. See  rclg-demo.lisp  for getting started.  It has incantations for:
   a. compiling and loading cffi
   b. compiling and loading rclg




* Past and possibly present "Issues"

** From the file formerly known as NOTES/06032006.rif (date contained)

1. In the current version of cffi, the variable names get wrapped in
   asterisks, so

(defcvar "R_CStackLimit"  :unsigned-long)  ;; :unsigned long
(defcvar "R_SignalHandlers" :unsigned-long) ;; :unsigned long

   make variables named *R-CSTACKLIMIT* and *R-SIGNALHANDLERS*

2. In "rclg-demo.lisp", I wanted to mention that rnb doesn't mean "no
   blocking", it means "no backconverting".  It's not that it doesn't
   protect --- it does all the normal protection, it just doesn't
   convert the result back from R to Lisp.  It's to avoid pushing big
   objects back and forth through the pipe needlessly.  In some sense,
   rnb is what you'd use to do an R assignment, but you get a CL name
   for whatever it is rather than an R name.

3. At this point, it seems like setting *R-SIGNALHANDLERS* to 0 is
   preventing an immediate seg fault.  So that's good.

4. We are still getting errors related to the stack, and I can't
   actually *find* anything.  Whenever I try to find a function, I'm
   getting back "Unbound value".  I don't know for sure that this is
   related to the stack problems.  The equivalent C program does find
   things.  I've put the C and Lisp programs in subdirectory
   simple-test.  I'm not sure whether want to be using Rf_findVar or
   Rf_findFun.

5. I'm not sure 100% I'm handling the stack correctly.  On the mailing
   list, it suggests setting R_CStackLimit = -1.  However, CFFI
   believes that *R-CSTACKLIMIT* is an unsigned long, and won't let me
   set it to -1.  I'm seting it to the two's complement value, which
   is 429476295.  I think this is OK, but tell me if it isn't.

6. It is worth noting that Rf_initEmbeddedR *will* change the value of
   R_CStackLimit.  So it is important to set signalhandlers to 0
   BEFORE starting R, and then probably set StackLimit afterwards.
   I've played a bit with going into the R source and turning off the
   stuff in the initialization that sets it to some value, but that
   produced an infinite loop of stack checks.

7. I had to make a bunch of changes so that all the built-in file
   constants match my directory structure.  I think it's set up so you
   only have to change the path in one place now (the defvar of
   *R-HOME-STR* in rclg-load).

** From the file formerly known as src/NOTES



1.R SEXP

rclg-types:sexptype
  "Gets the sexptype of an robj.  WARNING: ASSUMES THAT THE TYPE
IS STORED IN THE LOW ORDER 5 BITS OF THE SXPINFO-STRUCT, AND THAT
IT CAN BE EXTRACTED VIA A 'mod 32' OPERATION!  MAY NOT BE PORTABLE."

2. NAs


(defvar *r-NA-internal* -2147483648) ;;  PLATFORM SPECIFIC HACK!!!


3. SBCL-specific hacks

rclg-convert:sequence-to-robj is sbcl-specific!  Should think about
removing rclg-helpers for more portability, if it's fast enough.

Consolidate with-gensyms somewhere?

We only get r-names and r-dims back at the "toplevel" call to r.
Should rnb protect?  Don't think we're using poss-sexp for anything...

with-r-traps is SBCL specific.  
Multiprocessing stuff (with mutex) is SBCL specific.

In R, memory.c contains allocVector.  Looks like it *should* be doable
to directly pass vectors around (non-portably, of course).


4. FILES:

rclg-access.lisp : low-level defuns for data transport between R SEXPs
                   and CL. 
rclg-types.lisp : structures and objects for R internal types
rclg-control.lisp : primary interface to R evaluator
rclg-load.lisp : initialize environment and load libraries.
rclg-init.lisp : initialize and maintain the R evaluator process. 


* Comparison of Implementation and Design of RCLG, CLSR.

** Tools to initialize R

1. rclg-init : start-rclg update-R
               start-rclg-update-thread stop-rclg-update-thread 
	       with-R-traps with-r-mutex

2. rclg-load : load-r-libraries  (FFI initializer, not FFI <-> CL specifier.

3. clsr-loader: uffi-load-r-library (package: clsr)

4. clsr : instantiates an R process.

** Tools to handle R data structures and SEXPs in CL

1. rclg-access  : r-setcar, r-car, r-cdr
                  (R SEXP content <-> CL data)

2. rclg-convert : convert-to-r, convert-from-r, *r-na*, r-nil, r-bound
                  (data type and data conversion)

3. rclg-types : SEXP data structure information.  exports types, print method.

4. clsr-rref : High level R SEXP data structures and mappings

5. clsr-sxp : SEXP data structures and mappings

** Mappings:  Name (Function/variable), object registry

1. rclg-control : r (internal: rname-to-robj, rname-to-rfun

2. clsr-objects : tracks created R objects  in CL (reference counter).



** R evaluation

1. rclg-control : r (converts results),
                  rnb (uneval'd R object, unprotected)

1. rclg-parse-objects : tools to handle string commands.


** internal FFI

1. rclg-foreigns : maps libR to CL

** CL tools (some R, some general)

1. rclg-utils : with-gensyms, over-column-major-indices, to-list, to-vector