diff --git a/TODO b/TODO new file mode 100644 index 0000000000000000000000000000000000000000..44b496f2e554d3b255e8ff13a0139dd724b28a8e --- /dev/null +++ b/TODO @@ -0,0 +1,210 @@ +TO DO + +- Address the remaining "---" issues + +- Get the rest of the unit tests working + - Ensure we can parse/print descriptor.proto, unittests.proto, etc + - unittests.proto completely fails when 'optimize = SPEED' + - Ensure we can (de)serialize "golden" data against unittests.proto + - Get the rest of Bob Brown's tests working +[ - Convert examples into a test suite ] +[ - Implement 'protobufs-equal' predicate ] + +- Get "namespaces" (i.e., nested classes) fully working +[ - Messages and enums should have a 'fully-qualified-names' slot ] +[ that records both symbols and strings, and in the correct package ] +[ - Printers need to print qualified names, as needed ] +[ - 'find-message' and 'find-enum' need to search namespaces ] +[ if the quick compare against the name/class fails ] + - 'message Foo { message Bar { ... } ... }' + should (maybe!) produce a Lisp class defaultly named 'foo.bar' + - Add a keyword arg, :class-name, that overrides this convention; + in .proto files this should be called lisp_class + - Also, add :slot-name for slots and :type-name for enums + (called lisp_slot and lisp_type in .proto files) + - Or maybe not, since 'foo.bar' isn't very Lispy; + for the most part, packages provide enough room to avoid name clashes + +- Get 'merge-from-message' fully working + - See the place in 'deserialize-object' that needs to merge, too + - Maybe this isn't so useful in Lisp, where I'd prefer to encourage + a more function style + +- Lose 'proto-class', replace it with 'proto-type'; + its type is (or null symbol) + 'proto-type' should always be the name from the Lisp side +[ All 'define-xxx' macros should have, as their first argument, ] +[ a type (not a "name") ] +[ In .proto files, there should be an 'option lisp_name="pkg:name"' ] +[ which can be used to override the 'proto->class-name' default ] +[ for messages and enums ] + +- Refactor 'protobuf-field' so the names are clearer: + - 'proto-value' should be the Lisp name of the slot + (this is already the case) + - Add new 'proto-field' slot, + which will hold the Protobufs name of the field + (this used to be in 'proto-name') + - Lose the existing 'proto-type' slot, + then rename the 'proto-class' slot to 'proto-type'; + this will hold the Lisp name of the slot type + - 'proto-name' will hold the Protobufs name of the field type + (this used to hold the Protobufs name of the field) +[ - &key name should give the Protobufs name for the field ] +[ - 'option lisp_name="pkg:name"' should give the Lisp name for the slot ] + +[- Need search paths in the ASDF .proto module ] + +[- Make 'import' really work ] +[ - *all-protobufs* should record keyed by pathname, too ] +[ - 'import' should not bother loading if the import is present, ] +[ and should record the imported schema when it's done ] +[ - 'find-message' and 'find-enum' should search imports, too ] + +[- Implement 'vector-of' in addition to 'list-of' ] + +[- Do we need 'has-extension' and 'clear-extension'? ] +[ If so, just generate methods for them in 'define-extend' ] + +[- 'package'/'lisp_package' should create the package, if necessary. ] + +[- Don't generate 'import "net/proto2/proto/descriptor.proto"' and ] +[ 'extend proto2.MessageOptions { ..}' lines unless we need to include ] +[ one of the lisp_* options in the generated model ] + +[- Handle 'import' ] +[ It should probably just compile/load the file when the import ] +[ is seen ] +[ - The Lisp macros should be able to import both .lisp and .proto files ] +[ - The .proto parser should only be able to import .proto files ] + +[- Handle 'extension' and 'extends' ] +[ - 'define-extension' just declares the extension within a message ] +[ and adds it to 'proto-extensions' for the message ] +[ - Easiest is to add an "extension-p" flag to 'protobuf-message' ] +[ - Define new 'define-extends' macro ] +[ which *copies* all slots from the message being extended, ] +[ then just adds the extension fields as (flagged) ordinary fields ] +[ with reader/writer function (see below) ] +[ - Factor out field-processing code from 'define-message' ] +[ - If we do it this way, no other special magic is needed ] +[ Serialization, e.g., should just work ] +[ Scoping should also just work ] +[ - We will need 'proto-writer' as an analog to 'proto-reader' ] +[ - Get deserialization working; ] +[ Need to call the 'proto-writer' function if its around ] +[ instead of just using (setf (slot-value ...) ...) ] +[ - Don't forget to check that the "extends" indexes match ] +[ what was declared by 'define-extension' ] +[ - And don't allow overlapping extension indexes ] +[ - Generate new getter and setter methods for each extended slot: ] +[ (let ((an-int 10)) ] +[ (declare (type (or integer null) an-int)) ] +[ (defmethod an-int ((object extendable-foo)) ] +[ an-int) ] +[ (defmethod (setf an-int) (val (object extendable-foo)) ] +[ (declare (type (or integer null) val) ] +[ (setq an-int val)))) ] +[ - Handle "extends" in the parser, too ] + +[- Support "deprecated" groups. Sigh. ] +[ - Need 'proto:define-group', which is like 'define-message' ] +[ - Need to parse 'group' things ] +[ - "option group Locator = 1 { ... }" ] +[ means create a submessage (and class!) called "Locator" ] +[ whose field is named "locator" (note lower case) ] +[ then just treat it like a message ] +[ - Need a printer for groups, too ] +[ - Make 'proto-extension-p' be 'proto-message-type', ] +[ which is one of :message, :extends or :group ] +[ - (De)serialization of groups uses a start and end tag, ] +[ and the end tag counts as part of the group length ] +[ - Serialization uses the start-group wire type, ] +[ then serializes the message, ] +[ then writes the end-group tag ] +[ - Deserialization works the usual way, ] +[ except that it stops at a matching end-group tag ] +[ (i.e., has the same field number as the start tag) ] +[ - Add new 'end-tag' argument to 'serialize-object' and ] +[ the 'serialize' internal function, it's the expected ] +[ end tag value to indicate end-of-group; if the default ] +[ value for this is 0, then the (i= tag 0) test can be ] +[ changed to (i= tag end-tag) (sweet!) ] + +[- We need both 'package' and 'lisp-package' for protobufs ] +[ 'package' gets set from the 'package' line in .proto files ] +[ 'lisp-package' gets set from a new 'option lisp_package="pkg"' ] +[ 'define-protobuf' needs both :package and :proto/:lisp-package ] + +[- Speed up 'list-of' by shadowing the Quux version, ] +[ and don't do any type-checking (!) ] +[ Having done this, we can probably remove the hack of using map:map, ] +[ which should make deserialization much faster ] + +[- 'proto-name' should always be the name from the Protobufs side; ] +[ its type is (or null string) ] +[ All 'define-xxx' macros should have '&key name' which gives the Protobufs ] +[ name, which can be used to override the 'class-name->proto' default ] + +[- 'class-override' should be called 'alias-for'; ] +[ its type is (or null symbol) ] +[ 'define-message' and 'define-enum' both get '&key alias-for' ] + +[- 'find-message' and 'find-enum' should only look at (the new) 'proto-type', ] +[ and never (the new) 'proto-alias-for' ] + +[- 'deserialize-object' uses the alias when it calls 'make-instance', ] +[ or if there is no alias, 'proto-type' ] + +[- 'serialize-object' and 'object-size' should take both 'object' and 'type' ] +[ args, and should not call 'class-of' (and do call it "type") ] +[ The 'cl' (and 'class') locals in '(de)serialize-object' and 'object-size' ] +[ should instead be called 'type' ] +[ In .proto files, there should be 'option lisp_alias="pkg:name"' ] +[ that gets used to fill in 'proto-alias-for' ] + +[- 'serialize-object' and 'object-size' never use the alias; ] +[ instead all the "dispatching" should be done of off 'proto-type', ] +[ Ditto, 'print-text-format' ] + +[- Deserialization should stop if it sees a #x00 in the position where ] +[ it's expecting a tag, since no tag can be #x00 (fields start at 1) ] + +[- The "name" of a protobuf should be the camel-cased file name ] +[ The "type" of a protobuf should just be the file name as a keyword symbol ] + +[- RPCs should have 'input-name', 'input-type', 'output-name' and 'output-type' ] +[ The "name" ones are for the Protobufs side ] +[ The "types" ones are for the Lisp side; we look up on "type" only ] +[ In the Lisp, the RPC types can be either 'type' or '(type :name )' ] +[ In .proto files, there should be an 'option lisp_name="pkg:name"' ] +[ which can be used to override the 'proto->class-name' default ] +[ for the name of the RPC function ] + +[- Finishing eliminating 'protobuf' argument from the API ] +[ - Lose it in 'serialize-object-to-stream' and 'deserialize-object-from-stream'] +[ - Rename it to 'trace' in '(de)serialize-object' and 'object-size', ] +[ make it be the last supplied argument ] +[ - Fix up the '(de)serialize-object' and 'object-size' generators, too ] +[ - Apply the same fix to 'print-text-format' ] +[ - Then make the compatibility API actuallty work ] + +[- Implement 'parse-text-format' ] + +[- Renamings ] +[ 'define-proto' -> 'define-schema' ] +[ 'protobuf' (class name) -> 'protobuf-schema' ] +[ 'find-protobuf' -> 'find-schema' ] +[ *all-protobufs* -> *all-schemas* ] +[ 'parse-protobuf-from-file' -> 'parse-schema-from-file' ] +[ 'parse-protobuf-from-stream' -> 'parse-schema-from-file' ] +[ 'write-protobuf' -> 'write-schema' ] +[ 'write-protobuf-as' -> 'write-schema-as' ] +[ 'ensure-all-protobufs' -> 'ensure-all-protobufs' ] +[ 'ensure-protobuf' -> 'ensure-protobuf' ] +[ 'protobuf-upgradable' -> 'schema-upgradable' ] +[ 'protobufs-equal' -> 'schemas-equal' ] +[ 'generate-protobuf-schema-for-classes' -> 'generate-schema-for-classes' ] +[ 'write-protobuf-schema-for-classes' -> 'write-schema-for-classes' ] + +[- Write documentation ]