/[cmucl]/src/BUILDING
ViewVC logotype

Diff of /src/BUILDING

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.1 by toy, Fri Oct 24 16:55:36 2003 UTC revision 1.7 by rtoy, Tue Jan 24 17:27:20 2006 UTC
# Line 51  Setting up a build environment Line 51  Setting up a build environment
51    
52  2.) Fetch the sources and put them into the base directory  2.) Fetch the sources and put them into the base directory
53    
54      tar xzf /tmp/cmucl-18d.source.tar.gz      tar xzf /tmp/cmucl-source.tar.gz
55    
56      or, if you want to use the CVS sources directly:      or, if you want to use the CVS sources directly:
57    
58      export CVSROOT=:pserver:anonymous@cvs2.cons.org:/home/anoncvs/CVS-cmucl      export CVSROOT=:pserver:anonymous@common-lisp.net:/project/cmucl/cvsroot
59      cvs login  (password is `anonymous')      cvs login  (password is `anonymous')
60      cvs co src      cvs co src
61    
# Line 68  Setting up a build environment Line 68  Setting up a build environment
68    
69  That's it, you are now ready to build CMU CL.  That's it, you are now ready to build CMU CL.
70    
71    A quick guide for simple builds
72    -------------------------------
73    
74    We recommend that you read all of this document, but in case you don't
75    want to do that and in case you know, somehow, that the version of
76    CMUCL you are building from will build the sources you have, here is a
77    quick guide.
78    
79    a) Simple builds
80    
81       Use this to build from a version of CMUCL that is very close to the
82       sources you are trying to build now:
83    
84       src/tools/build.sh -C "" -o "<name-of-old-lisp> <options-to-lisp>"
85    
86       This will build CMUCL 3 times, each time with the result of the
87       previous build.  The last time, the additional libraries like CLX,
88       CLM, and Hemlock are built.  The final result will be in the
89       directory build-4.
90    
91       This script basically runs create-target.sh, build-world.sh,
92       load-world.sh three times.  See below for descriptions of these
93       scripts.
94    
95    b) Slightly more complicated builds
96    
97       For slightly more complicated builds, you may need to use some
98       bootstrap files.  See below for more information about these
99       bootstrap files.
100    
101       For these, you can use this:
102    
103       src/tools/build.sh -C "" -o "<old-lisp>" -B boot1.lisp -B boot2.lisp
104    
105       The bootstrap files listed with the -B option (as many as needed)
106       are loaded in order, so be sure to get them right.
107    
108       As in a) above, three builds are done, and the result is in the
109       directory build-4.
110    
111    c) More complicated builds
112    
113       If you have more complicated builds, this script probably will not
114       work, and definitely does not handle cross-compiles.  In this case,
115       you will have to invoke the individual scripts by hand, as
116       described below.
117    
118    How do you know which of the three options above apply?  The easiest
119    way is to look in src/bootfiles/<version>/* for boot files.  If the
120    file date of a boot file is later than the version of CMUCL you are
121    building from, then you need to use b) or c) above.  You may need to
122    read the bootfiles for additional instructions, if any.
123    
124    If there are no bootfiles, then you can use a) above.
125    
126    The build.sh script supports other options, and src/tools/build.sh -?
127    will give a quick summary.  Read src/tools/build.sh for more
128    information.
129    
130  A general outline of the build process  A general outline of the build process
131  --------------------------------------  --------------------------------------
# Line 155  d) Populating the kernel.core, and dumpi Line 213  d) Populating the kernel.core, and dumpi
213     were compiled in phase a) are loaded into it, and a new lisp.core     were compiled in phase a) are loaded into it, and a new lisp.core
214     file is dumped out.     file is dumped out.
215    
216    We're not quite done yet.  This produces just a basic lisp.core.
217    To complete the build so that you something similar to what the
218    releases of CMUCL do, there are a few more steps:
219    
220    e) Build the utilities like Gray streams, simple streams, CLX, CLM,
221       and Hemlock.  Use the src/tools/build-utils.sh script for this, as
222       described below
223    
224    f) Create tarfiles using the src/tools/make-dist.sh script, as
225       explained below.
226    
227    With these tarfiles, you can install them anywhere.  The contents of
228    the tarfiles will be the same as the snapshots and releases of CMUCL.
229    
230  When cross-compiling, there is additional phase at the beginning, and  When cross-compiling, there is additional phase at the beginning, and
231  some of the phases happen with different hosts/platforms.  The initial  some of the phases happen with different hosts/platforms.  The initial
232  phase is setting up and compiling the cross-compilation backend, using  phase is setting up and compiling the cross-compilation backend, using
# Line 173  the scripts included with this little te Line 245  the scripts included with this little te
245  Overview of the included build scripts  Overview of the included build scripts
246  --------------------------------------  --------------------------------------
247    
248  * create-target.sh target-directory [lisp-variant [motif-variant]]  * src/tools/create-target.sh target-directory [lisp-variant [motif-variant]]
249    
250  This script creates a new target directory, which is a shadow of the  This script creates a new target directory, which is a shadow of the
251  source directory, that will contain all the files that are created by  source directory, that will contain all the files that are created by
# Line 211  sample setenv.lisp includes a set of fea Line 283  sample setenv.lisp includes a set of fea
283  intended configuration.  Note also that some adding or removing some  intended configuration.  Note also that some adding or removing some
284  features may require a cross-compile instead of a normal compile.  features may require a cross-compile instead of a normal compile.
285    
286  * clean-target.sh [-l] target-directory [more dirs]  * src/tools/clean-target.sh [-l] target-directory [more dirs]
287    
288  Cleans the given target directory, so that all created files will be  Cleans the given target directory, so that all created files will be
289  removed.  This is useful to force recompilation.  If the -l flag is  removed.  This is useful to force recompilation.  If the -l flag is
# Line 219  given, then the C runtime is also remove Line 291  given, then the C runtime is also remove
291  executable, any lisp cores, all object files, lisp.nm, internals.h,  executable, any lisp cores, all object files, lisp.nm, internals.h,
292  and the config file.  and the config file.
293    
294  * build-world.sh target-directory [build-binary] [build-flags...]  * src/tools/build-world.sh target-directory [build-binary] [build-flags...]
295    
296  Starts a complete world build for the given target, using the lisp  Starts a complete world build for the given target, using the lisp
297  binary/core specified as a build host.  The recompilation step will  binary/core specified as a build host.  The recompilation step will
# Line 230  step of the world build will inform you Line 302  step of the world build will inform you
302  you'll have to use the rebuild-lisp.sh script, and then restart the  you'll have to use the rebuild-lisp.sh script, and then restart the
303  world build process with build-world.sh  world build process with build-world.sh
304    
305  * rebuild-lisp.sh target-directory  * src/tools/rebuild-lisp.sh target-directory
306    
307  This script will force a complete recompilation of the C runtime code  This script will force a complete recompilation of the C runtime code
308  of CMU CL (aka the lisp executable).  Doing this will necessitate  of CMU CL (aka the lisp executable).  Doing this will necessitate
309  building a new kernel.core file, using build-world.sh.  building a new kernel.core file, using build-world.sh.
310    
311  * load-world.sh target-directory version  * src/tools/load-world.sh target-directory version
312    
313  This will finish the CMU CL rebuilding process, by loading the  This will finish the CMU CL rebuilding process, by loading the
314  remaining compiled files generated in the world build process into the  remaining compiled files generated in the world build process into the
# Line 251  ISO8601 format is often a good idea, e.g Line 323  ISO8601 format is often a good idea, e.g
323  binary that is based on sources current on the 6th May, 2002, which is  binary that is based on sources current on the 6th May, 2002, which is
324  post the 18d release.  post the 18d release.
325    
326  * build-utils.sh target-directory  * src/tools/build-utils.sh target-directory
327    
328  This script will build auxiliary libraries packaged with CMU CL,  This script will build auxiliary libraries packaged with CMU CL,
329  including CLX, CMUCL/Motif, the Motif debugger, inspector, and control  including CLX, CMUCL/Motif, the Motif debugger, inspector, and control
330  panel, and the Hemlock editor.  It will use the lisp executable and  panel, and the Hemlock editor.  It will use the lisp executable and
331  core of the given target.  core of the given target.
332    
333  * make-dist.sh [-bg] [-G group] [-O owner] target-directory version arch os  * src/tools/make-dist.sh [-bg] [-G group] [-O owner] target-directory version arch os
334    
335  This script creates both main and extra distribution tarballs from the  This script creates both main and extra distribution tarballs from the
336  given target directory, using the make-main-dist.sh and  given target directory, using the make-main-dist.sh and
337  make-extra-dist.sh scripts.  You can select the compression method via  make-extra-dist.sh scripts.  The result will be two tar files.  One
338  the -b (bzip2) and -g (gzip) flags.  The default is gzip compression.  contains the main distribution including the runtime and lisp.core
339  When making the distribution, you can also select the group and owner  with PCL (CLOS); the second contains the extra libraries such as
340  of the files via the -G and -O options.  Gray-streams, simple-streams, CLX, CLM, and Hemlock.
341    
342    Some options that are available:
343    
344      -b           Use bzip2 compression
345      -g           Use gzip compression
346      -G group     Group to use
347      -O owner     Owner to use
348    
349    If you specify both -b and -g, you will get two sets of tarfiles.  The
350    -G and -O options will attempt to set the owner and group of the files
351    when building the tarfiles.  This way, when you extract the tarfiles,
352    the owner and group will be set as specified.  You may need to be root
353    to do this because many Unix systems don't normally let you change the
354    owner and group of a file.
355    
356    The remaining arguments used to create the name of the tarfiles.  The
357    names will have the form:
358    
359       cmucl-<version>-<arch>-<os>.tar.bz2
360       cmucl-<version>-<arch>-<os>.extras.tar.bz2
361    
362  * make-main-dist.sh target-directory version arch os  Of course, the "bz2" will be "gz" if you specified gzip compression
363    instead of bzip.
364    
365    * /src/tools/make-main-dist.sh target-directory version arch os
366    
367    This is script is not normally invoked by the user; make-dist will do
368    it appropriately.
369    
370  This script creates a main distribution tarball (both in gzipped and  This script creates a main distribution tarball (both in gzipped and
371  bzipped variants) from the given target directory.  This will include  bzipped variants) from the given target directory.  This will include
# Line 277  simple streams. Line 375  simple streams.
375    
376  This is intended to be run from make-dist.sh.  This is intended to be run from make-dist.sh.
377    
378  * make-extra-dist.sh target-directory version arch os  * src/tools/make-extra-dist.sh target-directory version arch os
379    
380    This is script is not normally invoked by the user; make-dist will do
381    it appropriately.
382    
383  This script creates an extra distribution tarball (both in gzipped and  This script creates an extra distribution tarball (both in gzipped and
384  bzipped variants) from the given target directory.  This will include  bzipped variants) from the given target directory.  This will include
# Line 402  a) Bootfiles Line 503  a) Bootfiles
503     in the order they are numbered.  Be sure to remove the bootstrap     in the order they are numbered.  Be sure to remove the bootstrap
504     file once it is no longer needed.     file once it is no longer needed.
505    
506       Alternatively, the bootstrap file can just "load" the individual
507       bootfiles as needed.
508    
509  b) Cross-compiling  b) Cross-compiling
510    
# Line 440  that is up-to-date with the sources.  Ot Line 543  that is up-to-date with the sources.  Ot
543  add a bootstrap file to get any bootfiles to make your lisp  add a bootstrap file to get any bootfiles to make your lisp
544  up-to-date with the current sources.  up-to-date with the current sources.
545    
546  Create a cross-compiler directory to hold the cross-compiler and a  1.  Select a directory for the cross-compiler and compiled target:
547  target directory to hold the result:  
548            Create a cross-compiler directory to hold the cross-compiler
549            and a target directory to hold the result:
550    
551                   src/tools/create-target.sh xcross
552                   src/tools/create-target.sh xtarget
553    
554    2.  Adjust cross-compilation script
555    
556            Copy the src/tools/cross-scripts/cross-sparc-sparc.lisp to
557            xtarget/cross.lisp.  Edit it appropriately.  In this case, it
558            should look something like:
559    
560                (c::new-backend "SPARC"
561                   ;; Features to add here
562                   '(:sparc :sparc-v8
563                     :complex-fp-vops
564                     :linkage-table
565                     :gencgc
566                     :stack-checking
567                     :relative-package-names
568                     :conservative-float-type
569                     :hash-new :random-mt19937
570                     :cmu :cmu19 :cmu19a
571                     )
572                   ;; Features to remove from current *features* here
573                   '(:sparc-v9 :sparc-v7 :x86 :x86-bootstrap :alpha :osf1 :mips
574                     :propagate-fun-type :propagate-float-type :constrain-float-type
575                     :openbsd :freebsd :glibc2 :linux :pentium
576                     :long-float :new-random :small))
577    
578                (setf *features* (remove :sparc-v9 *features*))
579                (pushnew :sparc-v8 *features*)
580    
581            It's important to add frob *features* here as well as in the
582            new-backend.  If you don't adjust *features*, they won't be
583            set appropriately in the result.
584    
585    3.  Build the cross compiler and target
586            Now compile the result:
587    
588                src/tools/cross-build-world.sh xtarget xcross xtarget/cross.lisp [v9 binary]
589    
590    4.  Rebuild the lisp files:
591    
592            When this finishes, you need to compile the C code:
593    
594                    src/tools/rebuild-lisp.sh xtarget
595    
596            At this point, you may want to run cross-build-world.sh again
597            to generate a new kernel.core.  It shouldn't build anything;
598            just loads everything and creates a kernel.core.
599    
600    5.  Build the world:
601    
602            With the new kernel.core, we need to create a lisp.core:
603    
604                    src/tools/load-world.sh xtarget "new lisp"
605    
606         src/tools/create-target.sh xcross          Test the result with
        src/tools/create-target.sh xtarget  
607    
608  Copy the src/tools/cross-scripts/cross-sparc-sparc.lisp to                  xtarget/lisp/lisp -noinit
 xtarget/cross.lisp.  Edit it appropriately.  In this case, it should  
 look something like:  
609    
610      (c::new-backend "SPARC"  However, this lisp will be missing some functionality like PCL.  You
611         ;; Features to add here  probably now want to use the compiler to rebuild everything once
612         '(:sparc :sparc-v8  again.  Just follow the directions for a normal build, and use
613           :complex-fp-vops  xtarget/lisp/lisp as your compiler.  Be sure to use create-target.sh
614           :linkage-table  to create a new directory where the result can go.
          :gencgc  
          :stack-checking  
          :relative-package-names  
          :conservative-float-type  
          :hash-new :random-mt19937  
          :cmu :cmu19 :cmu19a  
          )  
        ;; Features to remove from current *features* here  
        '(:sparc-v9 :sparc-v7 :x86 :x86-bootstrap :alpha :osf1 :mips  
          :propagate-fun-type :propagate-float-type :constrain-float-type  
          :openbsd :freebsd :glibc2 :linux :pentium  
          :long-float :new-random :small))  
615    
616      (setf *features* (remove :sparc-v9 *features*))  Cross-Platform Cross-Compile
617      (pushnew :sparc-v8 *features*)  ----------------------------
618    
619  It's important to add frob *features* here as well as in the  A cross-platform cross-compile is very similar to a normal
620  new-backend.  If you don't adjust *features*, they won't be set  cross-compile, and the basic steps are the same.  For the sake of
621  appropriately in the result.  concreteness, assume we are on ppc/darwin and want to cross-compile
622    to x86/linux.
623    
624  Now compile the result:  To simplify things, we assume that both platforms have access to the
625    same file system, via NFS or something else.
626    
627          src/tools/cross-build-world.sh xtarget xcross xtarget/cross.lisp [v9 binary]  1. As above, we need to create directories for the cross-compiler and
628       compiled target.  We assume we are on ppc/darwin.  So, when running
629       create-target.sh we need to specify the target:
630    
631  When this finishes, you need to compile the C code:          src/tools/create-target.sh x86-cross x86
632            src/tools/create-target.sh x86-target x86
633    
634          src/tools/rebuild-lisp.sh xtarget  2. Adjust the cross-compilation script.  An example for ppc/darwin to
635       x86/linux is in src/tools/cross-scripts/cross-ppc-x86.lisp.
636    
637  At this point, you may want to run cross-build-world.sh again to  3. Build the cross compiler and target, as above, using the specified
638  generate a new kernel.core.  It shouldn't build anything;  just loads     cross-compile script:
 everything and creates a kernel.core.  
639    
640  With the new kernel.core, we need to create a lisp.core:          src/tools/cross-build-world.sh x86-target x86-cross cross.lisp [ppc binary]
641    
642          src/tools/load-world.sh xtarget "new lisp"     where cross.lisp is the cross-compile script from 2) above.
643    
644  Test the result with  4. Everything has now been compiled for the x86/linux target.  We need
645       to compile the C code for x86 and create a lisp.core from the
646       kernel.core.  This is where it's useful to have both platforms be
647       able to access the same file system.  If not, you will need to copy
648       all of the generated files from ppc/darwin to x86/linux.  Basically
649       everything in xtarget needs to be copied.
650    
651          xtarget/lisp/lisp -noinit     Note carefully that you may have to edit lisp/internals.h and/or
652       lisp/internals.inc to have the correct features.  This is a known
653       bug in the generation of these files during cross-compilation.
654    
655    
656  You may now want to use this cross-compiled lisp to rebuild itself, to  5. Now run load-world.sh to create the desired lisp.core from lisp and
657  make sure you haven't messed up.  You can do a normal build as     kernel.core.  As above, PCL has not been compiled, so select
658  described above, except your build-lisp is xtarget/lisp/lisp.     restart 3 (return nil from pclload) to create lisp.core
659    
660    At this point, you will have a shiny new lisp on the new platform.
661    Since it's missing PCL, you will need to do at least one normal build
662    to get PCL included.  This is also a good check to see if everything
663    was compiled properly.  A full set of builds via build.sh might be
664    good at this point too.

Legend:
Removed from v.1.1  
changed lines
  Added in v.1.7

  ViewVC Help
Powered by ViewVC 1.1.5