This ensures that all pathnames for all .proto files we load get put
in PROTO-IMPL::*ALL-SCHEMAS*, not just those we import.

Don't override file extension for imports; assume the string that was
used to form the pathname is accurate. This makes base-path and
proto-file the same, so fold them both into proto-file.

Drop bogus checks for type "proto".

* DO-PROCESS-IMPORT is now responsible for deciding whether or not to
  process an import. It does so by searching for the .proto to be
  imported, and upon finding it, checks to see if we alredy have a
  schema for that pathname. Additionally, it now returns the pathname.

* PROCESS-IMPORTS and PROCESS-IMPORTS-FROM-FILE now defer the work of
  figuring out whether or not an import has been done to
  DO-PROCESS-IMPORT. PROCESS-IMPORTS uses the return value of
  DO-PROCESS-IMPORT to update the PROTO-IMPORTED-SCHEMAS of the schema
  being processed.

New functions:

* ASDF::PROTOBUF-MANGLE-NAME: Takes a pathname, returns a string to be
  used as a filename that represents the original path.
  e.g., #P"/foo/bar/baz.quux" becomes "foo-bar-baz-quux"

* ASDF::PROTOBUF-LISPIZE-PATHNAME: Takes a pathname, returns a new
  pathname with the name mangled as described above, and with a lisp
  type.

 - Use 'defparameter' instead of 'defvar' where appropriate.
 - Fix 'defvar' doc strings to distinguish between globals and "thread locals".
 - Avoid using 'nconc'. introduce a new 'appendf' macro instead.
 - Add a comment lamenting the fact that exporting something like 'proto-options'
   also exports the writer '(setf proto-options)'. Fixed in Dylan.

This prevents geodata.proto's FASL from being clobbered by
geodata.lisp's FASL. Before both were geodata.fasl, now the former is
geodata.proto.fasl.

This is generally a good idea.

which shall fix the issue that causes of these actions never being operation-done-p.
However this does NOT handle recursive import dependencies,
which requires further work.

Tested: QRes doesn't always recompile the proto and all dependees anymore.
Reviewer: asedeno

* Previously, if a field in a .proto file referenced a message in
  another proto file using a different lisp package, the cl-protobufs
  library would silently fail to serialize the field.  A similar
  problem would occur if a message defined later in the file used the
  lisp_name option to override the name generated by cl-protobufs.
  This change fixes these issues and others.
* Add conditions that are signaled when encountering an undefined
  type.
* Delay assigning lisp classes/types to fields and methods until all
  possible forward references have been parsed.
   * This allows the class slot to be unbound, so check for that case
     in print-object methods.
   * Add a test for forward references to messages that override the
     lisp name.
   * Add a test for references to messages and enums defined in
     another proto file with a different lisp package.
   * Change color-wheel-stability test, because it used "string" as
     the input type for an rpc, which seems to be disallowed (although
     I haven't found this documented anywhere).
* Signal errors during parsing for undefined types.
   * Add a test for these errors.  Add assert-error macro to qtest.
* Signal a condition if we are unable to find the definition for a
  field's type during serialization, deserialization, determining an
  object's serialized size, printing text format, parsing text format,
  or generating code for one of the above.
* Remove logic in find-qualified-name that indirects through lisp
  packages.  Proto packages and lisp packages do not necessarily map
  1-to-1.
* Always use the schema's lisp package for any symbols generated when
  parsing proto files.
* When generating lisp code using write-schema-as, set the package to
  the package used in the generated file, so that ~s will print the
  package prefix in the correct circumstances.
* Remove broken proto1 "streams" parsing ("returns" comes before
  "streams" in every example I've found); replace with proto2 syntax.
* In process-imports, the call to find-schema using a pathname was not
  giving the expected result.  Sidestep this issue by using the same
  logic to find the schema as is used earlier in the function.

Testing: [trunk r567444] precheckin --parallel 11  PASSED
Reviewers: dcrawford, asedeno

Rationale: there is a problem in quicklisp due to recent asdf package changes,
that requires urgent fix to cl-protobufs,
so I'm updating these for the sake of issuing
a cl-protobufs fix to quicklisp ASAP.
I require these updates for some further changes I'm making to QUUX, anyway,
so it's just a matter of timing.



git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@567483 f8382938-511b-0410-9cdd-bb47b084005c

package, if it has been loaded.

Passes 'precheckin' with the new Protobufs unit tests in place.
CL-Stubby tests pass in Google3 environment.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@551156 f8382938-511b-0410-9cdd-bb47b084005c

git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@551150 f8382938-511b-0410-9cdd-bb47b084005c

git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@551149 f8382938-511b-0410-9cdd-bb47b084005c

Tested: [trunk r550170] precheckin --full+ --parallel 5  PASSED
Reviewed: swm


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@551126 f8382938-511b-0410-9cdd-bb47b084005c

so that 'import' just plain works everywhere.

Passes 'precheckin' with the new Protobufs unit tests in place.
CL-Stubby tests pass in Google3 environment.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@551048 f8382938-511b-0410-9cdd-bb47b084005c

both in the QRes environment and the Google3 environment.

Passes 'precheckin' with the new Protobufs unit tests in place.
Fortuneseeker server gets simpler.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@549101 f8382938-511b-0410-9cdd-bb47b084005c

git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@548806 f8382938-511b-0410-9cdd-bb47b084005c

Tested: doesn't break the build; isn't currently used by our tests.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@548440 f8382938-511b-0410-9cdd-bb47b084005c

some things that needed to be addressed:
 - We need to do a more thorough search of namespaces so that
   qualified references to imported messages, etc, work properly.
   It's still not 100%, but it covers most of what I can find.
   - Name mappers now do their best to preserve qualified names.
   - 'find-xxx' methods search the qualified names.
 - Fix the importation process to cooperate better with ASDF, i.e.,
   don't bother re-importing things that have already compiled
   as part of an ASDF compile/load operation.
 - When generating .lisp from .proto, if no package has been
   declared, do it. And fill it with nice exports, too. This
   won't be exactly right, but it'll make it easier to create
   a proper package declaration, if you so desire.
 - Do a better job of generating client- and server-side stubs.

Passes 'precheckin' with the new Protobufs unit tests in place.
And more to the point, the Fortuneseeker server now works as a
Stubby client using my new Google3 http2 and rpc2 libraries.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@548386 f8382938-511b-0410-9cdd-bb47b084005c

 - Use 'defgeneric' with ':method' for things that are meant to
   have just a single method.
 - Lose some unreachable code in the parser.
 - Weak hash tables in SBCL need to use ':weakness :value'.
 - The wire-level stuff can use some more type declarations.
 - Fix a package problem in one of the examples.

Passes 'precheckin' with the new Protobufs unit tests in place.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@547515 f8382938-511b-0410-9cdd-bb47b084005c

git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@547277 f8382938-511b-0410-9cdd-bb47b084005c

 - More sophisticated and easier to use ASDF support.
 - Add search paths to the :protobuf-file ASDF module

Passes 'precheckin' with the new Protobufs unit tests in place.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@546487 f8382938-511b-0410-9cdd-bb47b084005c

 - 'reinitialize-slot' didn't quite work.
 - Add a geodata example that uses vectors for repeated fields,
   which uncovered a bug in the optimized deserializers.
 - Importing the geodata CLOS classes revealed a bug in default
   handling when the default is provided only in 'defclass'.
 - Fix the knock-on bug in deserialization and the optimized
   (de)serialization caused the above.
 - Add tighter types in 'decode-uint32' and 'decode-uint64'.

Passes 'precheckin'. Even with the new unit tests in place.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@545865 f8382938-511b-0410-9cdd-bb47b084005c

 - Don't generate warnings for anonymous enums, they're harmless
   and ubiquitous.
 - 'member' types where are all the members are keywords, is a symbol,
   which got converted to a Protobufs 'string'. Wrong.
 - If a field has an enum type, the (Lisp) default value should be
   treated as a keyword.
 - When parsing a .proto file or using the 'define-xxx' macros, any
   options that are handled specially should be trimmed from the
   options list so that they don't get printed twice.
 - Add 'remove-options' for the above.
 - Make the options printer be more type-aware.
 - Clean up examples.lisp
 - 'schemas-equal' can ignore the schema name/class if they're null.

Passes 'precheckin'. Even with the new unit tests in place.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@544737 f8382938-511b-0410-9cdd-bb47b084005c

 - 'define-proto' -> 'define-schema'
 - 'protobuf' (class name) -> 'protobuf-schema'
 - 'find-protobuf' -> 'find-schema'
 - 'parse-protobuf-from-file' -> 'parse-schema-from-file'
 - 'parse-protobuf-from-stream' -> 'parse-schema-from-file'
 - 'write-protobuf' -> 'write-schema'
 - 'write-protobuf-as' -> 'write-schema-as'
 - 'ensure-all-protobufs' -> 'ensure-all-protobufs'
 - 'ensure-protobuf' -> 'ensure-protobuf'
 - 'protobuf-upgradable' -> 'schema-upgradable'
 - 'protobufs-equal' -> 'schemas-equal'
 - 'generate-protobuf-schema-for-classes' -> 'generate-schema-for-classes'
 - 'write-protobuf-schema-for-classes' -> 'write-schema-for-classes'

Update the Quake Protobufs schema generator to use the new names.

The Protobufs documentation doesn't say so, but enums can be packed.
 - Update the wire-level protocol to (de)serialize packed enums.
 - Update the serialization code to use the new wire functions.

Passes 'precheckin'.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@544299 f8382938-511b-0410-9cdd-bb47b084005c

edge case problems, but it's good enough to continue the Stubby work.
The critical .proto files (descriptor.proto, etc) all appear to work.

More precise mapping of (mainly integer) types from .lisp to .proto files
so that we can a truly exact model once we compile the .lisp file. It may
not make any difference, but better safe than sorry.
 - Introduce Lisp type 'int32', 'int64', 'uint32', etc
 - Make Lisp->Protobufs recognize the new Lisp types
 - Make Protobufs schema printers recognize the new types

Fix up the handling of imports and namespaces.
 - Always use truenames to record schemas
 - Fix typo in 'find-enum' that caused it never to search other schemas
 - Record the schema in *all-protobufs* at exactly the right time,
   and simplify 'make-load-form' methods

A few tweaks to (de)serialization.
 - Make the bodies for empty messages not generate compiler warnings
 - Don't make ':packed t' be the default, the user has to say so
 - The user documentation doesn't say so, but booleans can be packed.

Passes 'precheckin'.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@544222 f8382938-511b-0410-9cdd-bb47b084005c