/[cmucl]/src/BUILDING
ViewVC logotype

Diff of /src/BUILDING

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.1 by toy, Fri Oct 24 16:55:36 2003 UTC revision 1.24 by rtoy, Wed May 11 14:20:27 2011 UTC
# Line 51  Setting up a build environment Line 51  Setting up a build environment
51    
52  2.) Fetch the sources and put them into the base directory  2.) Fetch the sources and put them into the base directory
53    
54      tar xzf /tmp/cmucl-18d.source.tar.gz      tar xzf /tmp/cmucl-source.tar.gz
55    
56      or, if you want to use the CVS sources directly:      or, if you want to use the CVS sources directly:
57    
58      export CVSROOT=:pserver:anonymous@cvs2.cons.org:/home/anoncvs/CVS-cmucl      export CVSROOT=:pserver:anonymous@common-lisp.net:/project/cmucl/cvsroot
59      cvs login  (password is `anonymous')      cvs login  (password is `anonymous')
60      cvs co src      cvs co src
61    
# Line 68  Setting up a build environment Line 68  Setting up a build environment
68    
69  That's it, you are now ready to build CMU CL.  That's it, you are now ready to build CMU CL.
70    
71    A quick guide for simple builds
72    -------------------------------
73    
74    We recommend that you read all of this document, but in case you don't
75    want to do that and in case you know, somehow, that the version of
76    CMUCL you are building from will build the sources you have, here is a
77    quick guide.
78    
79    a) Simple builds
80    
81       Use this to build from a version of CMUCL that is very close to the
82       sources you are trying to build now:
83    
84       src/tools/build.sh -C "" -o "<name-of-old-lisp> <options-to-lisp>"
85    
86       This will build CMUCL 3 times, each time with the result of the
87       previous build.  The last time, the additional libraries like CLX,
88       CLM, and Hemlock are built.  The final result will be in the
89       directory build-4.
90    
91       This script basically runs create-target.sh, build-world.sh,
92       load-world.sh three times.  See below for descriptions of these
93       scripts.
94    
95    b) Slightly more complicated builds
96    
97       For slightly more complicated builds, you may need to use some
98       bootstrap files.  See below for more information about these
99       bootstrap files.
100    
101       For these, you can use this:
102    
103       src/tools/build.sh -C "" -o "<old-lisp>" -B boot1.lisp -B boot2.lisp
104    
105       The bootstrap files listed with the -B option (as many as needed)
106       are loaded in order, so be sure to get them right.
107    
108       As in a) above, three builds are done, and the result is in the
109       directory build-4.
110    
111    c) More complicated builds
112    
113       If you have more complicated builds, this script probably will not
114       work, and definitely does not handle cross-compiles.  In this case,
115       you will have to invoke the individual scripts by hand, as
116       described below.
117    
118    How do you know which of the three options above apply?  The easiest
119    way is to look in src/bootfiles/<version>/* for boot files.  If the
120    file date of a boot file is later than the version of CMUCL you are
121    building from, then you need to use b) or c) above.  You may need to
122    read the bootfiles for additional instructions, if any.
123    
124    If there are no bootfiles, then you can use a) above.
125    
126    The build.sh script supports other options, and src/tools/build.sh -?
127    will give a quick summary.  Read src/tools/build.sh for more
128    information.
129    
130  A general outline of the build process  A general outline of the build process
131  --------------------------------------  --------------------------------------
# Line 155  d) Populating the kernel.core, and dumpi Line 213  d) Populating the kernel.core, and dumpi
213     were compiled in phase a) are loaded into it, and a new lisp.core     were compiled in phase a) are loaded into it, and a new lisp.core
214     file is dumped out.     file is dumped out.
215    
216    We're not quite done yet.  This produces just a basic lisp.core.
217    To complete the build so that you something similar to what the
218    releases of CMUCL do, there are a few more steps:
219    
220    e) Build the utilities like Gray streams, simple streams, CLX, CLM,
221       and Hemlock.  Use the src/tools/build-utils.sh script for this, as
222       described below
223    
224    f) Create tarfiles using the src/tools/make-dist.sh script, as
225       explained below.
226    
227    With these tarfiles, you can install them anywhere.  The contents of
228    the tarfiles will be the same as the snapshots and releases of CMUCL.
229    
230  When cross-compiling, there is additional phase at the beginning, and  When cross-compiling, there is additional phase at the beginning, and
231  some of the phases happen with different hosts/platforms.  The initial  some of the phases happen with different hosts/platforms.  The initial
232  phase is setting up and compiling the cross-compilation backend, using  phase is setting up and compiling the cross-compilation backend, using
# Line 173  the scripts included with this little te Line 245  the scripts included with this little te
245  Overview of the included build scripts  Overview of the included build scripts
246  --------------------------------------  --------------------------------------
247    
248  * create-target.sh target-directory [lisp-variant [motif-variant]]  * src/tools/build.sh [-123obvuBCU?]
249    
250    This is the main build script.  It essentially calls the other build
251    scripts described below in the proper sequence to build cmucl from an
252    existing binary of cmucl.
253    
254    * src/tools/create-target.sh target-directory [lisp-variant [motif-variant]]
255    
256  This script creates a new target directory, which is a shadow of the  This script creates a new target directory, which is a shadow of the
257  source directory, that will contain all the files that are created by  source directory, that will contain all the files that are created by
# Line 211  sample setenv.lisp includes a set of fea Line 289  sample setenv.lisp includes a set of fea
289  intended configuration.  Note also that some adding or removing some  intended configuration.  Note also that some adding or removing some
290  features may require a cross-compile instead of a normal compile.  features may require a cross-compile instead of a normal compile.
291    
292  * clean-target.sh [-l] target-directory [more dirs]  * src/tools/clean-target.sh [-l] target-directory [more dirs]
293    
294  Cleans the given target directory, so that all created files will be  Cleans the given target directory, so that all created files will be
295  removed.  This is useful to force recompilation.  If the -l flag is  removed.  This is useful to force recompilation.  If the -l flag is
# Line 219  given, then the C runtime is also remove Line 297  given, then the C runtime is also remove
297  executable, any lisp cores, all object files, lisp.nm, internals.h,  executable, any lisp cores, all object files, lisp.nm, internals.h,
298  and the config file.  and the config file.
299    
300  * build-world.sh target-directory [build-binary] [build-flags...]  * src/tools/build-world.sh target-directory [build-binary] [build-flags...]
301    
302  Starts a complete world build for the given target, using the lisp  Starts a complete world build for the given target, using the lisp
303  binary/core specified as a build host.  The recompilation step will  binary/core specified as a build host.  The recompilation step will
# Line 230  step of the world build will inform you Line 308  step of the world build will inform you
308  you'll have to use the rebuild-lisp.sh script, and then restart the  you'll have to use the rebuild-lisp.sh script, and then restart the
309  world build process with build-world.sh  world build process with build-world.sh
310    
311  * rebuild-lisp.sh target-directory  * src/tools/rebuild-lisp.sh target-directory
312    
313  This script will force a complete recompilation of the C runtime code  This script will force a complete recompilation of the C runtime code
314  of CMU CL (aka the lisp executable).  Doing this will necessitate  of CMU CL (aka the lisp executable).  Doing this will necessitate
315  building a new kernel.core file, using build-world.sh.  building a new kernel.core file, using build-world.sh.
316    
317  * load-world.sh target-directory version  * src/tools/load-world.sh target-directory version
318    
319  This will finish the CMU CL rebuilding process, by loading the  This will finish the CMU CL rebuilding process, by loading the
320  remaining compiled files generated in the world build process into the  remaining compiled files generated in the world build process into the
# Line 251  ISO8601 format is often a good idea, e.g Line 329  ISO8601 format is often a good idea, e.g
329  binary that is based on sources current on the 6th May, 2002, which is  binary that is based on sources current on the 6th May, 2002, which is
330  post the 18d release.  post the 18d release.
331    
332  * build-utils.sh target-directory  * src/tools/build-utils.sh target-directory
333    
334  This script will build auxiliary libraries packaged with CMU CL,  This script will build auxiliary libraries packaged with CMU CL,
335  including CLX, CMUCL/Motif, the Motif debugger, inspector, and control  including CLX, CMUCL/Motif, the Motif debugger, inspector, and control
336  panel, and the Hemlock editor.  It will use the lisp executable and  panel, and the Hemlock editor.  It will use the lisp executable and
337  core of the given target.  core of the given target.
338    
339  * make-dist.sh [-bg] [-G group] [-O owner] target-directory version arch os  * src/tools/make-dist.sh [-bg] [-G group] [-O owner] target-directory version arch os
340    
341  This script creates both main and extra distribution tarballs from the  This script creates both main and extra distribution tarballs from the
342  given target directory, using the make-main-dist.sh and  given target directory, using the make-main-dist.sh and
343  make-extra-dist.sh scripts.  You can select the compression method via  make-extra-dist.sh scripts.  The result will be two tar files.  One
344  the -b (bzip2) and -g (gzip) flags.  The default is gzip compression.  contains the main distribution including the runtime and lisp.core
345  When making the distribution, you can also select the group and owner  with PCL (CLOS); the second contains the extra libraries such as
346  of the files via the -G and -O options.  Gray-streams, simple-streams, CLX, CLM, and Hemlock.
347    
348    Some options that are available:
349    
350      -b           Use bzip2 compression
351      -g           Use gzip compression
352      -G group     Group to use
353      -O owner     Owner to use
354    
355    If you specify both -b and -g, you will get two sets of tarfiles.  The
356    -G and -O options will attempt to set the owner and group of the files
357    when building the tarfiles.  This way, when you extract the tarfiles,
358    the owner and group will be set as specified.  You may need to be root
359    to do this because many Unix systems don't normally let you change the
360    owner and group of a file.
361    
362    The remaining arguments used to create the name of the tarfiles.  The
363    names will have the form:
364    
365       cmucl-<version>-<arch>-<os>.tar.bz2
366       cmucl-<version>-<arch>-<os>.extras.tar.bz2
367    
368    Of course, the "bz2" will be "gz" if you specified gzip compression
369    instead of bzip.
370    
371    * /src/tools/make-main-dist.sh target-directory version arch os
372    
373  * make-main-dist.sh target-directory version arch os  This is script is not normally invoked by the user; make-dist will do
374    it appropriately.
375    
376  This script creates a main distribution tarball (both in gzipped and  This script creates a main distribution tarball (both in gzipped and
377  bzipped variants) from the given target directory.  This will include  bzipped variants) from the given target directory.  This will include
# Line 277  simple streams. Line 381  simple streams.
381    
382  This is intended to be run from make-dist.sh.  This is intended to be run from make-dist.sh.
383    
384  * make-extra-dist.sh target-directory version arch os  * src/tools/make-extra-dist.sh target-directory version arch os
385    
386    This is script is not normally invoked by the user; make-dist will do
387    it appropriately.
388    
389  This script creates an extra distribution tarball (both in gzipped and  This script creates an extra distribution tarball (both in gzipped and
390  bzipped variants) from the given target directory.  This will include  bzipped variants) from the given target directory.  This will include
# Line 402  a) Bootfiles Line 509  a) Bootfiles
509     in the order they are numbered.  Be sure to remove the bootstrap     in the order they are numbered.  Be sure to remove the bootstrap
510     file once it is no longer needed.     file once it is no longer needed.
511    
512       Alternatively, the bootstrap file can just "load" the individual
513       bootfiles as needed.
514    
515  b) Cross-compiling  b) Cross-compiling
516    
# Line 440  that is up-to-date with the sources.  Ot Line 549  that is up-to-date with the sources.  Ot
549  add a bootstrap file to get any bootfiles to make your lisp  add a bootstrap file to get any bootfiles to make your lisp
550  up-to-date with the current sources.  up-to-date with the current sources.
551    
552  Create a cross-compiler directory to hold the cross-compiler and a  1.  Select a directory for the cross-compiler and compiled target:
553  target directory to hold the result:  
554            Create a cross-compiler directory to hold the cross-compiler
555            and a target directory to hold the result:
556    
557                   src/tools/create-target.sh xcross
558                   src/tools/create-target.sh xtarget
559    
560    2.  Adjust cross-compilation script
561    
562            Copy the src/tools/cross-scripts/cross-sparc-sparc.lisp to
563            xtarget/cross.lisp.  Edit it appropriately.  In this case, it
564            should look something like:
565    
566                (c::new-backend "SPARC"
567                   ;; Features to add here
568                   '(:sparc :sparc-v8
569                     :complex-fp-vops
570                     :linkage-table
571                     :gencgc
572                     :stack-checking
573                     :relative-package-names
574                     :conservative-float-type
575                     :hash-new :random-mt19937
576                     :cmu :cmu19 :cmu19a
577                     )
578                   ;; Features to remove from current *features* here
579                   '(:sparc-v9 :sparc-v7 :x86 :x86-bootstrap :alpha :osf1 :mips
580                     :propagate-fun-type :propagate-float-type :constrain-float-type
581                     :openbsd :freebsd :glibc2 :linux :pentium
582                     :long-float :new-random :small))
583    
584                (setf *features* (remove :sparc-v9 *features*))
585                (pushnew :sparc-v8 *features*)
586    
587            It's important to add frob *features* here as well as in the
588            new-backend.  If you don't adjust *features*, they won't be
589            set appropriately in the result.
590    
591    3.  Build the cross compiler and target
592            Now compile the result:
593    
594                src/tools/cross-build-world.sh xtarget xcross xtarget/cross.lisp [v9 binary]
595    
596    4.  Rebuild the lisp files:
597    
598            When this finishes, you need to compile the C code:
599    
600                    src/tools/rebuild-lisp.sh xtarget
601    
602            At this point, you may want to run cross-build-world.sh again
603            to generate a new kernel.core.  It shouldn't build anything;
604            just loads everything and creates a kernel.core.
605    
606    5.  Build the world:
607    
608            With the new kernel.core, we need to create a lisp.core:
609    
610                    src/tools/load-world.sh xtarget "new lisp"
611    
612            Test the result with
613    
614                    xtarget/lisp/lisp -noinit
615    
616    However, this lisp will be missing some functionality like PCL.  You
617    probably now want to use the compiler to rebuild everything once
618    again.  Just follow the directions for a normal build, and use
619    xtarget/lisp/lisp as your compiler.  Be sure to use create-target.sh
620    to create a new directory where the result can go.
621    
622    Cross-Platform Cross-Compile
623    ----------------------------
624    
625    A cross-platform cross-compile is very similar to a normal
626    cross-compile, and the basic steps are the same.  For the sake of
627    concreteness, assume we are on ppc/darwin and want to cross-compile
628    to x86/linux.
629    
630         src/tools/create-target.sh xcross  To simplify things, we assume that both platforms have access to the
631         src/tools/create-target.sh xtarget  same file system, via NFS or something else.
632    
633  Copy the src/tools/cross-scripts/cross-sparc-sparc.lisp to  1. As above, we need to create directories for the cross-compiler and
634  xtarget/cross.lisp.  Edit it appropriately.  In this case, it should     compiled target.  We assume we are on ppc/darwin.  So, when running
635  look something like:     create-target.sh we need to specify the target:
636    
637      (c::new-backend "SPARC"          src/tools/create-target.sh x86-cross x86
638         ;; Features to add here          src/tools/create-target.sh x86-target x86
        '(:sparc :sparc-v8  
          :complex-fp-vops  
          :linkage-table  
          :gencgc  
          :stack-checking  
          :relative-package-names  
          :conservative-float-type  
          :hash-new :random-mt19937  
          :cmu :cmu19 :cmu19a  
          )  
        ;; Features to remove from current *features* here  
        '(:sparc-v9 :sparc-v7 :x86 :x86-bootstrap :alpha :osf1 :mips  
          :propagate-fun-type :propagate-float-type :constrain-float-type  
          :openbsd :freebsd :glibc2 :linux :pentium  
          :long-float :new-random :small))  
639    
640      (setf *features* (remove :sparc-v9 *features*))  2. Adjust the cross-compilation script.  An example for ppc/darwin to
641      (pushnew :sparc-v8 *features*)     x86/linux is in src/tools/cross-scripts/cross-ppc-x86.lisp.
642    
643  It's important to add frob *features* here as well as in the  3. Build the cross compiler and target, as above, using the specified
644  new-backend.  If you don't adjust *features*, they won't be set     cross-compile script:
 appropriately in the result.  
645    
646  Now compile the result:          src/tools/cross-build-world.sh x86-target x86-cross cross.lisp [ppc binary]
647    
648          src/tools/cross-build-world.sh xtarget xcross xtarget/cross.lisp [v9 binary]     where cross.lisp is the cross-compile script from 2) above.
649    
650  When this finishes, you need to compile the C code:  4. Everything has now been compiled for the x86/linux target.  We need
651       to compile the C code for x86 and create a lisp.core from the
652       kernel.core.  This is where it's useful to have both platforms be
653       able to access the same file system.  If not, you will need to copy
654       all of the generated files from ppc/darwin to x86/linux.  Basically
655       everything in xtarget needs to be copied.
656    
657          src/tools/rebuild-lisp.sh xtarget     Note carefully that you may have to edit lisp/internals.h and/or
658       lisp/internals.inc to have the correct features.  This is a known
659       bug in the generation of these files during cross-compilation.
660    
661  At this point, you may want to run cross-build-world.sh again to     Compile the lisp code:
 generate a new kernel.core.  It shouldn't build anything;  just loads  
 everything and creates a kernel.core.  
662    
663  With the new kernel.core, we need to create a lisp.core:          src/tools/rebuild-lisp.sh x86-target
664    
665          src/tools/load-world.sh xtarget "new lisp"  5. Now run load-world.sh to create the desired lisp.core from lisp and
666       kernel.core.  As above, PCL has not been compiled, so select
667       restart 3 (return nil from pclload) to create lisp.core
668    
669  Test the result with          src/tools/load-world.sh x86-target "new x86"
670    
671          xtarget/lisp/lisp -noinit  At this point, you will have a shiny new lisp on the new platform.
672    Since it's missing PCL, you will need to do at least one normal build
673    to get PCL included.  This is also a good check to see if everything
674    was compiled properly.  A full set of builds via build.sh might be
675    good at this point too.
676    
677  You may now want to use this cross-compiled lisp to rebuild itself, to  Some of the details for each command may have changed;  You can get
678  make sure you haven't messed up.  You can do a normal build as  help for each command by using the -h argument.
 described above, except your build-lisp is xtarget/lisp/lisp.  
679    
680    In particular steps 3, 4, and 5 can be combined into one by using the
681    -c, -r, and -l options for cross-build-world.sh.  The -c option cleans
682    out the targe and cross directories; -r does step 4; and -l does step
683    5.

Legend:
Removed from v.1.1  
changed lines
  Added in v.1.24

  ViewVC Help
Powered by ViewVC 1.1.5