Skip to content
  1. May 27, 2013
  3. Apr 17, 2013
  5. Apr 16, 2013
  7. Mar 03, 2013
  9. Mar 01, 2013
  11. Feb 28, 2013
  13. Feb 26, 2013
  15. Jan 02, 2013
  17. Dec 17, 2012
  19. Nov 07, 2012
    • Francois-Rene Rideau's avatar
      Improve schema generation performance on large schemas · ae041468
      Francois-Rene Rideau authored 
      Avoid SUBTYPEP during macroexpansion, it can be very expensive on SBCL.
Actually recognize LIST-OF:LIST-OF constructs.
In DEFINE-SCHEMA, don't nest the DEFMETHODs inside the LET,
this also can cause SBCL's control flow analysis to blow up.

Work done with Steven Spitz for QPX.

Tested: (asdf:test-system :cl-protobufs)
      ae041468
  21. Sep 14, 2012
    • Sergey Vasilyev's avatar
      Tsunami framework + use tsunami in customer profiles · 48aa0eee
      Sergey Vasilyev authored 
      
 Testing : precheckin --full (both against Oracle and against megastore for customers)
 Reviewer: eschwartz

 JTB impact: n
 Ops impact: n
 Security impact: n

 Change to config                        : n
 Change to XML schema                    : Y (XMLREQ bug 120820)
 Change to DB schema                     : Y (DBREQ bug 119801)
 Change to transport (timeouts, headers) : n
 Any change (or new use) of OAQs         : n
 Change to inter-component transactions  : sort of (use megastore in development-only environments)
 Depends on any other checkin / bug      : n

 Tests that will verify : tsunami and megastore unit tests + existing customer profile tests

Tsunami is a new layer that sits between business logic code and the
persistence logic. The main manifestation of this layer is
define-domain-class macro that defines an interface to a domain
class. You can then define how to perist in various implementations,
e.g. using define-quake-db-class-bindings for quake and
define-proto-binding & define-megastore-proto-binding for megastore.

Tsunami framework is still a work in progress, and there is a bunch of
outstanding tasks/issues.

As the first application, I refactored customer profile code to use
the tsunami framework. This enables storing of customer profiles in
megastore (or other data stores in the future), while everything else
is stored in Oracle.

Here is the list of other miscellaneous/harder-to-understand changes:

- I changed @id attribute of ContactInfo objects to use the actual
  numeric ID, rather than passing it through encode-record-locator (in
  qres-dev schema only). This required changing XML schema type from
  typeBasicLocatorString to typeBasicID.

- Drop quake/relationships.lisp. Use a more straightforward way to
  associated ADO agencies to customer profiles.

- Customer usernames are now stored as lowercase strings (this means
  that I don't need a separate field in megastore protobuf to
  implement case-insensitive index).

- agency-id permission attribute is now the agency-key (serialized to
  string), not the agency's record ID. This way you don't need to load
  agency to do permission check when all you have is the agency-key

- moved agency-key from core/agencies.lisp to
  lisp/core/agency-types.lisp, so that agency-key type can be used in
  permissions

- add when* test handler that takes a random Lisp expression (as
  opposed to the existing when that takes a variable name and looks at
  whether it is empty)

- remove cc-deleted-error -- there is no such thing as deleting
  cc-info

- refactored cc-information, so that it is used as a value object (ie
  a mostly-immutable object with no identity) in the application. When
  saving to Oracle database, it is saved in the central
  tbl_cc_information table with appropriately populated ownership
  columns (pointing to the customer, journey,
  accountable_doc_container, etc)

- drop unneeded tbl_cc_transaction columns that were caching cc_info
  values


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@562275 f8382938-511b-0410-9cdd-bb47b084005c
      48aa0eee
  23. Sep 10, 2012
    • Scott McKay's avatar
      Add a more readable syntax for enum and field indices · 58c795d8
      Scott McKay authored 
      Testing : precheckin --full --strict-errors
Reviewer: Fare (please)

JTB impact: No
Ops impact: No

Change to config                        : No
Change to XML schema                    : No
Change to DB schema                     : No
Change to transport (timeouts, headers) : No
Any change (or new use) of OAQs         : No
Change to inter-component transactions  : No
Depends on any other checkin / bug      : No

Tests that will verify:

I extended the CL-Protobufs tests

Description:

Add a more readable syntax for enum and field indices.

For define-enum, it was (name value).
  Allow (name :index value).

For define-message, it was ((name index) ...).
  Allow (name :index index ...).
  Complain if both forms appear in the same field.



git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@561136 f8382938-511b-0410-9cdd-bb47b084005c
      58c795d8
  25. Sep 04, 2012
    • Scott McKay's avatar
      Add stub support for streaming RPC APIs. · 5293ada2
      Scott McKay authored 
      Testing : precheckin --full --strict-errors
Reviewer: Fare (please)

JTB impact: No
Ops impact: No

Change to config                        : No
Change to XML schema                    : No
Change to DB schema                     : No
Change to transport (timeouts, headers) : No
Any change (or new use) of OAQs         : No
Change to inter-component transactions  : No
Depends on any other checkin / bug      : No

Tests that will verify:

I extended the CL-Protobufs tests

Description:

Add support for a 'streams' type in Protobufs RPC methods.
 - Add a few slots to 'protobufs-method'
 - Add '&key streams' to the method arglists in 'define-service'
 - Extend the parser to handle "streams" in addition to the
   'streams_type' option
 - Extend the printer
 - Change a test to use a 'streams' type
 - While we're in the neighborhood, allow an optional '=>'
   between the input and output methods in 'define-service';
   it makes it a bit more readable
 - Update the documentation



git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@560147 f8382938-511b-0410-9cdd-bb47b084005c
      5293ada2
  27. Aug 31, 2012
    • Scott McKay's avatar
      Add type aliases to CL-Protobufs · a732dcb3
      Scott McKay authored 
      Testing : precheckin --full --strict-errors
Reviewer: Fare (please)

JTB impact: No
Ops impact: No

Change to config                        : No
Change to XML schema                    : No
Change to DB schema                     : No
Change to transport (timeouts, headers) : No
Any change (or new use) of OAQs         : No
Change to inter-component transactions  : No
Depends on any other checkin / bug      : No

Tests that will verify:

The CL-Protobufs tests

Description:

Add type aliases to CL-Protobufs. It's a Lisp-only
feature that allows you define a new Protobufs type in
terms of a Lisp type, a serializer and a deserializer.
 - Add a new model class, 'protobuf-type-alias'.
 - Extend schemas and messages so that they can hold
   a set of type aliases; add 'find-type-alias'.
 - Add 'define-type-alias'.
 - Factor out 'lisp-type-to-protobuf-type' from
   'clos-type-to-protobuf-type', I need it for the
   'define-type-alias' macro.
 - Fix the schema printer to print 'define-type-alias'
   for .lisp schemas, and to add a comment describing
   the type alias in .proto schemas.
 - Extend the wire format (de)serializer to know
   about type aliases; it just (de)serializes the
   Lisp type as the Protobufs primitive type using
   the provided (de)serializer functions. There's
   zero cost to this if you don't use aliases.
 - Update the optimized serialization generation.
 - Similarly extend the text format (de)serializer
   to know about type aliases.
 - Change an example to use a type alias for 'date'.
 - Update the documentation.



git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@559861 f8382938-511b-0410-9cdd-bb47b084005c
      a732dcb3
  29. Aug 28, 2012
  31. Aug 27, 2012
  33. Aug 24, 2012
  35. Jul 03, 2012
  37. Jun 28, 2012
  39. Jun 27, 2012
  41. Jun 26, 2012
  43. Jun 20, 2012
    • Scott McKay's avatar
      Random things discovered while working on Stubby support... · 5ae2da97
      Scott McKay authored 
       - Make the 'find-xxx' support searching "relative to" another namespace.
 - There was a bug in the non-optimized deserializer when deserializing
   a repeated slot into a vector; create a stretchy vector on demand
   if it's needed.
 - 'define-extends' should wrap 'eval-when' around the generated 'defsetf'
   forms so that they are visible at compile time.
 - Fix a formatting bug in the export list in the Lisp printer.
 - Straighten of the ASDF declaration for the tests.
 - Add the Protobufs test suite to QRes, to keep things honest.

Passes 'precheckin' with the new Protobufs unit tests in place.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/libs/cl-protobufs@550053 f8382938-511b-0410-9cdd-bb47b084005c
      5ae2da97
  45. Jun 18, 2012
  47. Jun 14, 2012
  49. Jun 13, 2012
  51. Jun 11, 2012
  53. Jun 05, 2012
  55. May 25, 2012
  57. May 23, 2012
  59. May 22, 2012
    • Scott McKay's avatar
      Add more stuff to the test suite, and fix what it uncovered. · b6317f29
      Scott McKay authored 
       - Behavior of ':default xxx'/[default=xxx] was not stable,
   so add an "empty default" marker and respect it during
   parsing and printing.
 - 'find-option' now returns a third "found-p" value, useful
   for the better handling of defaults.
 - Don't print slot readers/writers if they have the same name
   as what would be (silently) generated by the macrology.
 - Don't print 'import descriptor.proto' lines if there are
   no 'lisp_xxx' options in the generated code.
 - Add 'clear-field' to the API; add the functions it needs.
 - Be more careful about how :conc-name interacts with the
   generation of readers/writers.
 - When parsing from a .proto file, force the conc-name to
   be "" unless requested explicitly not to. This has the
   effect of generating readers/writers for all the fields
   of imported Protobufs messages, which I think is probably
   the desired behavior.
 - Add 'make-byte-vector' for convenience.

Passes 'precheckin'. Even with the new unit tests in place.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@545177 f8382938-511b-0410-9cdd-bb47b084005c
      b6317f29
  61. May 18, 2012
    • Scott McKay's avatar
      Now that Protobufs has a test suite, it found a few things to fix. · 0d1611ab
      Scott McKay authored 
       - Don't generate warnings for anonymous enums, they're harmless
   and ubiquitous.
 - 'member' types where are all the members are keywords, is a symbol,
   which got converted to a Protobufs 'string'. Wrong.
 - If a field has an enum type, the (Lisp) default value should be
   treated as a keyword.
 - When parsing a .proto file or using the 'define-xxx' macros, any
   options that are handled specially should be trimmed from the
   options list so that they don't get printed twice.
 - Add 'remove-options' for the above.
 - Make the options printer be more type-aware.
 - Clean up examples.lisp
 - 'schemas-equal' can ignore the schema name/class if they're null.

Passes 'precheckin'. Even with the new unit tests in place.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@544737 f8382938-511b-0410-9cdd-bb47b084005c
      0d1611ab
  63. May 16, 2012
    • Scott McKay's avatar
      At Sergey's request, make some of the names better: · a31c828e
      Scott McKay authored 
       - 'define-proto' -> 'define-schema'
 - 'protobuf' (class name) -> 'protobuf-schema'
 - 'find-protobuf' -> 'find-schema'
 - 'parse-protobuf-from-file' -> 'parse-schema-from-file'
 - 'parse-protobuf-from-stream' -> 'parse-schema-from-file'
 - 'write-protobuf' -> 'write-schema'
 - 'write-protobuf-as' -> 'write-schema-as'
 - 'ensure-all-protobufs' -> 'ensure-all-protobufs'
 - 'ensure-protobuf' -> 'ensure-protobuf'
 - 'protobuf-upgradable' -> 'schema-upgradable'
 - 'protobufs-equal' -> 'schemas-equal'
 - 'generate-protobuf-schema-for-classes' -> 'generate-schema-for-classes'
 - 'write-protobuf-schema-for-classes' -> 'write-schema-for-classes'

Update the Quake Protobufs schema generator to use the new names.

The Protobufs documentation doesn't say so, but enums can be packed.
 - Update the wire-level protocol to (de)serialize packed enums.
 - Update the serialization code to use the new wire functions.

Passes 'precheckin'.


git-svn-id: http://svn.internal.itasoftware.com/svn/ita/trunk/qres/lisp/quux/protobufs@544299 f8382938-511b-0410-9cdd-bb47b084005c
      a31c828e