Newer
Older
TO DO
- Address the remaining "---" issues
- Get the rest of the unit tests working
- Ensure we can parse/print descriptor.proto, unittests.proto, etc
- unittests.proto completely fails when 'optimize = SPEED'
- Ensure we can (de)serialize "golden" data against unittests.proto
- Get the rest of Bob Brown's tests working
[ - Convert examples into a test suite ]
[ - Implement 'protobufs-equal' predicate ]
- Get "namespaces" (i.e., nested classes) fully working
[ - Messages and enums should have a 'fully-qualified-names' slot ]
[ that records both symbols and strings, and in the correct package ]
[ - Printers need to print qualified names, as needed ]
[ - 'find-message' and 'find-enum' need to search namespaces ]
[ if the quick compare against the name/class fails ]
- 'message Foo { message Bar { ... } ... }'
should not produce a Lisp class defaultly named 'foo.bar', but...
- Add a keyword arg, :class-name, that overrides this convention;
in .proto files this should be called lisp_class
- Also, add :slot-name for slots and :type-name for enums
(called lisp_slot and lisp_type in .proto files)
- Rationale: 'foo.bar' isn't very Lispy;
for the most part, packages provide enough namespaces to avoid clashes
- Get 'merge-from-message' fully working
- See the place in 'deserialize-object' that needs to merge, too
- Maybe this isn't so useful in Lisp, where I'd prefer to encourage
a more function style
- Lose 'proto-class', replace it with 'proto-type';
its type is (or null symbol)
'proto-type' should always be the name from the Lisp side
[ All 'define-xxx' macros should have, as their first argument, ]
[ a type (not a "name") ]
[ In .proto files, there should be an 'option lisp_name="pkg:name"' ]
[ which can be used to override the 'proto->class-name' default ]
[ for messages and enums ]
- Refactor 'protobuf-field' so the names are clearer:
- 'proto-value' should be the Lisp name of the slot
(this is already the case)
- Add new 'proto-field' slot,
which will hold the Protobufs name of the field
(this used to be in 'proto-name')
- Lose the existing 'proto-type' slot,
then rename the 'proto-class' slot to 'proto-type';
this will hold the Lisp name of the slot type
- 'proto-name' will hold the Protobufs name of the field type
(this used to hold the Protobufs name of the field)
[ - &key name should give the Protobufs name for the field ]
[ - 'option lisp_name="pkg:name"' should give the Lisp name for the slot ]
- Refactor 'define-message'/'define-extend'/'define-group'
to avoid so much duplicated code
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
[- Need search paths in the ASDF .proto module ]
[- Make 'import' really work ]
[ - *all-protobufs* should record keyed by pathname, too ]
[ - 'import' should not bother loading if the import is present, ]
[ and should record the imported schema when it's done ]
[ - 'find-message' and 'find-enum' should search imports, too ]
[- Implement 'vector-of' in addition to 'list-of' ]
[- Do we need 'has-extension' and 'clear-extension'? ]
[ If so, just generate methods for them in 'define-extend' ]
[- 'package'/'lisp_package' should create the package, if necessary. ]
[- Don't generate 'import "net/proto2/proto/descriptor.proto"' and ]
[ 'extend proto2.MessageOptions { ..}' lines unless we need to include ]
[ one of the lisp_* options in the generated model ]
[- Handle 'import' ]
[ It should probably just compile/load the file when the import ]
[ is seen ]
[ - The Lisp macros should be able to import both .lisp and .proto files ]
[ - The .proto parser should only be able to import .proto files ]
[- Handle 'extension' and 'extends' ]
[ - 'define-extension' just declares the extension within a message ]
[ and adds it to 'proto-extensions' for the message ]
[ - Easiest is to add an "extension-p" flag to 'protobuf-message' ]
[ - Define new 'define-extends' macro ]
[ which *copies* all slots from the message being extended, ]
[ then just adds the extension fields as (flagged) ordinary fields ]
[ with reader/writer function (see below) ]
[ - Factor out field-processing code from 'define-message' ]
[ - If we do it this way, no other special magic is needed ]
[ Serialization, e.g., should just work ]
[ Scoping should also just work ]
[ - We will need 'proto-writer' as an analog to 'proto-reader' ]
[ - Get deserialization working; ]
[ Need to call the 'proto-writer' function if its around ]
[ instead of just using (setf (slot-value ...) ...) ]
[ - Don't forget to check that the "extends" indexes match ]
[ what was declared by 'define-extension' ]
[ - And don't allow overlapping extension indexes ]
[ - Generate new getter and setter methods for each extended slot: ]
[ (let ((an-int 10)) ]
[ (declare (type (or integer null) an-int)) ]
[ (defmethod an-int ((object extendable-foo)) ]
[ an-int) ]
[ (defmethod (setf an-int) (val (object extendable-foo)) ]
[ (declare (type (or integer null) val) ]
[ (setq an-int val)))) ]
[ - Handle "extends" in the parser, too ]
[- Support "deprecated" groups. Sigh. ]
[ - Need 'proto:define-group', which is like 'define-message' ]
[ - Need to parse 'group' things ]
[ - "option group Locator = 1 { ... }" ]
[ means create a submessage (and class!) called "Locator" ]
[ whose field is named "locator" (note lower case) ]
[ then just treat it like a message ]
[ - Need a printer for groups, too ]
[ - Make 'proto-extension-p' be 'proto-message-type', ]
[ which is one of :message, :extends or :group ]
[ - (De)serialization of groups uses a start and end tag, ]
[ and the end tag counts as part of the group length ]
[ - Serialization uses the start-group wire type, ]
[ then serializes the message, ]
[ then writes the end-group tag ]
[ - Deserialization works the usual way, ]
[ except that it stops at a matching end-group tag ]
[ (i.e., has the same field number as the start tag) ]
[ - Add new 'end-tag' argument to 'serialize-object' and ]
[ the 'serialize' internal function, it's the expected ]
[ end tag value to indicate end-of-group; if the default ]
[ value for this is 0, then the (i= tag 0) test can be ]
[ changed to (i= tag end-tag) (sweet!) ]
[- We need both 'package' and 'lisp-package' for protobufs ]
[ 'package' gets set from the 'package' line in .proto files ]
[ 'lisp-package' gets set from a new 'option lisp_package="pkg"' ]
[ 'define-protobuf' needs both :package and :proto/:lisp-package ]
[- Speed up 'list-of' by shadowing the Quux version, ]
[ and don't do any type-checking (!) ]
[ Having done this, we can probably remove the hack of using map:map, ]
[ which should make deserialization much faster ]
[- 'proto-name' should always be the name from the Protobufs side; ]
[ its type is (or null string) ]
[ All 'define-xxx' macros should have '&key name' which gives the Protobufs ]
[ name, which can be used to override the 'class-name->proto' default ]
[- 'class-override' should be called 'alias-for'; ]
[ its type is (or null symbol) ]
[ 'define-message' and 'define-enum' both get '&key alias-for' ]
[- 'find-message' and 'find-enum' should only look at (the new) 'proto-type', ]
[ and never (the new) 'proto-alias-for' ]
[- 'deserialize-object' uses the alias when it calls 'make-instance', ]
[ or if there is no alias, 'proto-type' ]
[- 'serialize-object' and 'object-size' should take both 'object' and 'type' ]
[ args, and should not call 'class-of' (and do call it "type") ]
[ The 'cl' (and 'class') locals in '(de)serialize-object' and 'object-size' ]
[ should instead be called 'type' ]
[ In .proto files, there should be 'option lisp_alias="pkg:name"' ]
[ that gets used to fill in 'proto-alias-for' ]
[- 'serialize-object' and 'object-size' never use the alias; ]
[ instead all the "dispatching" should be done of off 'proto-type', ]
[ Ditto, 'print-text-format' ]
[- Deserialization should stop if it sees a #x00 in the position where ]
[ it's expecting a tag, since no tag can be #x00 (fields start at 1) ]
[- The "name" of a protobuf should be the camel-cased file name ]
[ The "type" of a protobuf should just be the file name as a keyword symbol ]
[- RPCs should have 'input-name', 'input-type', 'output-name' and 'output-type' ]
[ The "name" ones are for the Protobufs side ]
[ The "types" ones are for the Lisp side; we look up on "type" only ]
[ In the Lisp, the RPC types can be either 'type' or '(type :name <name>)' ]
[ In .proto files, there should be an 'option lisp_name="pkg:name"' ]
[ which can be used to override the 'proto->class-name' default ]
[ for the name of the RPC function ]
[- Finishing eliminating 'protobuf' argument from the API ]
[ - Lose it in 'serialize-object-to-stream' and 'deserialize-object-from-stream']
[ - Rename it to 'trace' in '(de)serialize-object' and 'object-size', ]
[ make it be the last supplied argument ]
[ - Fix up the '(de)serialize-object' and 'object-size' generators, too ]
[ - Apply the same fix to 'print-text-format' ]
[ - Then make the compatibility API actuallty work ]
[- Implement 'parse-text-format' ]
[- Renamings ]
[ 'define-proto' -> 'define-schema' ]
[ 'protobuf' (class name) -> 'protobuf-schema' ]
[ 'find-protobuf' -> 'find-schema' ]
[ *all-protobufs* -> *all-schemas* ]
[ 'parse-protobuf-from-file' -> 'parse-schema-from-file' ]
[ 'parse-protobuf-from-stream' -> 'parse-schema-from-file' ]
[ 'write-protobuf' -> 'write-schema' ]
[ 'write-protobuf-as' -> 'write-schema-as' ]
[ 'ensure-all-protobufs' -> 'ensure-all-protobufs' ]
[ 'ensure-protobuf' -> 'ensure-protobuf' ]
[ 'protobuf-upgradable' -> 'schema-upgradable' ]
[ 'protobufs-equal' -> 'schemas-equal' ]
[ 'generate-protobuf-schema-for-classes' -> 'generate-schema-for-classes' ]
[ 'write-protobuf-schema-for-classes' -> 'write-schema-for-classes' ]
[- Write documentation ]