[asdf-devel] A dream for ASDF2

[Faré]

Dear Daniel,

On 2 February 2010 23:01,  <nuntius at strider2.example.org> wrote:
> Tobias C. Rittweiler wrote:
> Re: [asdf-devel] Is this necessary in this form? Re:  ASDF 1.501
>> Isn't asdf-output-locations somewhat like a CL pendant to `make install'
>> if you squint your eyes?
>>
>> Shouldn't that be an integral part of ASDF?
>
> Exactly yes and absolutely no.  Not necessarily in that order.
>
> My issue is that there is a great similarity; this similarity is driven by
> a need inherent in the use of libraries; but ASDF (due to history) has
> certain things absolutely backwards.
>
> There are many users that a build system must support.
> - developers do incremental compiles
> - maintainer creates a release (builds sources, docs, etc. into a self-contained archive)
> - maintainer checks that the release actually builds, cleans, installs, etc.
> - user or system integrator configures the release for local needs
> - the release is compiled (often into a separate directory or on a separate computer)
> - the release is tested (before install)
> - the release is installed (often to a read-only space, as root, or into a
> temporary directory that will be archived for distribution)
> - the release is loaded (often without sources)
> - the release is uninstalled
>
> ASDF needs a coherent story for the person doing each of these jobs.
> Instead, the current model lumps most everything into a single "load-op",
> and there is no principled way to perform or configure each of the above
> phases separately.
>
No. Most of these use cases are outside the scope of ASDF.
Instead of trying to retrofit them in the current ASDF design,
I'd rather we incorporate them into a proper replacement for ASDF, e.g. XCVB.

> My biggest concern with the recent changes is that they build a big
> scaffold that institutionalizes a design in need of fundamental changes.
> Some parts of .asd files may need incompatible changes; but I think most
> uses are completely upgradeable.  However, I do not think the recent
> ASDF proposals will be.
>
>
> I would like the ASDF architecture to look something like
> - bootstrap a few (self-contained) core utils using a special install sequence (simple COMPILE, test, and install scripts)
> - ASDF uses LOAD to pull these in; as each loads, more functionality becomes available (aka SBCL boostrap).
> - All other packages can now be built with the full ASDF.
> - (optional) ASDF may upgrade itself without the bootstrap sequence.
>
This was already suggested before by James Anderson, and rejected
on the grounds that
 - If you squint your eyes, ASDF is already like that: asdf.lisp is the core,
   and you load ASDF extension with (asdf:load-system :cffi) or some such.
 - There is precious little to split out of asdf.lisp, and it is a lot of
   engineering to break it apart ASDF without breaking anything,
   for no gain whatsoever - actually a loss as users of implementations
   that don't bundle ASDF now have to maintain a directory instead
   of a single file, and bug reports become harder.

> ASDF features I would like:
>
> - Standard way to pass configuration options to ASDF for building a
> specific system.  For example, ASDF should let me configure which
> compression backend cl-pdf uses, and ASDF should write these choices to
> file for me.  This should be a replayable config file that can be modified
> for reconfiguration, or read in for compilation and install.  I should be
> able to have several builds at once, each with a different set of choices.
>
Probably won't happen. If you come up with a plan and implement it, great.
I'd rather you implement it for XCVB, though. Currently, the way it works is,
you push stuff into *features* in some initialization script before you
compile and load your system.

> - Standard options for each system.  For example, a popular one might
> allow ASDF to autorecompile fasls when the sources change.  Another would
> be an install prefix.  Another is to modify the installed system name
> (e.g. add version info; asdf-1.5 would load the ASDF package, version
> 1.5).  Another to enable the system config cache (below).
>
I don't understand the first suggestion. We already do that to some extent
and won't be doing more of it. Second suggestion is not applicable, as
installation is beyond the scope of ASDF. Third solution is also beyond
the scope of ASDF; use debian build tools or something to manage versioning.

> - Generic configuration cache, independent of use.  In other words, a
> simple configuration format where I can set key/value pairs.  Some of
> which may happen to control ASDF source locations, others may control ASDF
> output locations, yet others may be defined by end-user ASDF files.  This
> cache should have three levels, one for the sysadmin, one for the user,
> and one for inside the system configuration area (usually the source
> tree).
> Compare the autoconf site configuration mechanism.
> http://www.gnu.org/software/autoconf/manual/html_node/Site-Configuration.html
>
Sounds nice, but we have nothing to configure right now.
Sorry to be redundant, but what about you contribute to XCVB?

> - ASDF compile/install spits out a system.fasl.  Semantically, loading
> this causes the entire library to be loaded.  On some CL's, it might
> simply be the concatenation of the compiled source files; on others, it
> might LOAD them all in order.
>
What's wrong with (asdf:load-system :system) instead of
(load "/one/more/path/to/configure/system.magicfaslextension") ?
Distribution is out of the scope of ASDF.
If you want to distribute proprietary software as ASDF-managed FASL,
you could I suppose create empty (or eval-when erroring) backdated
source files so ASDF may load the FASL without trying to recompile the file.
Or just distribute a precompiled image. Or concatenate FASLs for your
implementation. Or write an extension to ASDF that does this.

> The above is a long-enough list for today.  I think it covers most of the
> bases.  These things aren't problems I'm creating; they are lessons I have
> gleaned from GNU autotools and other build systems.  Whenever ASDF takes a
> shortcut, we have issues.
>
Agreed. But ASDF probably isn't going to address any of these issues,
unless you do the hard work - and even then I'd rather you work on XCVB.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
A cuddle a day keeps the shrink away