ViewVC logotype

Contents of /elephant/TODO

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.83 - (hide annotations)
Sat Apr 28 02:31:06 2007 UTC (6 years, 11 months ago) by ieslick
Branch: MAIN
Changes since 1.82: +0 -0 lines
Cleaning up root directory files; map-index performance enhancement, index api cleanup, ensure transaction fix, alpha quality documentation draft
1 ieslick 1.18
2 ieslick 1.61 Last updated: February 25th, 2007
3 ieslick 1.10
4 ieslick 1.61 Ongoing release plan notes.
6     (Note: use of this file is deprecated after 0.6.1, see Trac site at
7     http://trac.common-lisp.net/elephant)
8 ieslick 1.22
9 ieslick 1.30 0.6.1 - performance, safety and portability
10     --------------------------------------------
11 ieslick 1.22
13 ieslick 1.45
14 ieslick 1.79 Bugs:
15     - Fix awkward serializer API
16     - Support for asdf-install?
17 ieslick 1.80 - problem with other .asd files in root
18 ieslick 1.79
19 ieslick 1.58 Lisp Support:
20     - Validate OpenMCL pre-1.1 on Mac OS X
21     - Validate OpenMCL 1.1 and/or 64-bit on Mac OS X?
22     - Verify db_deadlock for other lisps (launch and kill background program I/F)
24 ieslick 1.25 Documentation:
25 ieslick 1.77 - Update install, build and test procedures
26 ieslick 1.75 - Upgrade, migration and other system level issues
27 ieslick 1.58 - Performance and design issues
28     - More notes about transaction performance
29     - Serious discussion of threading implications
30     - Add notes about with/ensure-transaction usage (abort & commit behavior on exit)
31     - Add notes about deadlock-detect
32     - Add notes about checkpoint (null in SQL?)
33 ieslick 1.23
34     0.6.1 - Features COMPLETED to date
35     ----------------------------------
36 ieslick 1.34
37 ieslick 1.76 POST BETA CHECKINS:
39     Bugs:
40     x Fix duplicate opening of CLSQL db bug that caused errors in SQLite
41     x Fix for persistent-object inheritance calculation when inheriting from standard classes
42     x Remove problematic asserts in lisp-compare-eq make equal for strings instead of eq
43     x Fix map over nils bug in map-index and get-instances-by-value; clarify map-index interface
45     Tweaks:
46     x Add bounds to map-btree
47     x Change wipe-class-indexing so it can be called without the class object being created
48     x add-to-root, etc now uses :sc instead of :store-controller for brevity
49     x Remove hard coded paths in ele-clsql
51 ieslick 1.55 POST ALPHA CHECKINS:
53     Major Bugs:
54     x Derived indices fail to re-connect after reopening a database under :class synchronization policy (Ian)
55 ieslick 1.56 x Package translation to properly upgrade databases where packages were renamed (Robert)
56 ieslick 1.57 x Fix a bug where slot-makunbound on a persistent object failed to remove secondary index references
57 ieslick 1.63 for class and slot indices. Made a test to validate this. (Ian)
58     x Fixed a bug in string serialization for char-code > #x7F (Henrik, Ties)
59 ieslick 1.69 x Fixed migrate bug where oid->oid map was reset on every migrate call, yikes! (Ian)
60 ieslick 1.55
61     Minor Bugs:
62 ieslick 1.67 x Fix class index slot option test
63 ieslick 1.55 x Enable with-transactions to properly process forms returning multiple values (Ian)
64     x Fixed typos in SQL backend (Ian/Robert/Henrik)
65     x Fixed build bug for linux (Henrik)
66     x Fixed error condition while opening SQLITE3 in SQL backend (Robert)
67 ieslick 1.62 x Fixed idempotence problem in PREPARE-BDB test (Ian)
68     x Review and document or address all NOTE comments in the code (Ian)
69     x Review SBCL string serialization performance (false alarm, comparable to allegro)
70 ieslick 1.63 x Fixed a missing package export: translate-and-intern-symbol from elephant (Ties)
71 ieslick 1.55
72     Feature tweaking:
73 ieslick 1.69 x Migration: Improve support for nested persistent objects inside lists, arrays and hash tables (Ian)
74     x Migration: Test embedded object support
75 ieslick 1.63 x Enabled 8-bit encoding of char-codes between #x7F and #xFF; enabled by earlier rewrite of memutil (Ian)
76 ieslick 1.55 x Orthogonal feature addition: map-index, map-class and map-instances to avoid consing (Ian)
77 ieslick 1.61 x Tests to validate new map interfaces on top of existing tests (Ian)
78     x Added support and tests for serializing structure objects on all supported platforms (Ian)
79 ieslick 1.66 x Fixed cygwin-ming32 -mno-cygwin build for Windows for Lispworks and Allegro (Frank, Ian)
80 ieslick 1.68 x Migration: Validate that migrate can use either O(c) or O(n*c) where c approaches 0 memory for large DBs
81     Uses O(n*c) where C is the cost to store a persistent object reference and old oid reference
82 ieslick 1.57
84 ieslick 1.54
85 ieslick 1.50 February 13-17th, 2007:
86 ieslick 1.54 x Allow checkpoint of BDB via db-bdb::checkpoint
87 ieslick 1.52 x Fix BDB migration 0.6.0->0.6.1, improve nested transaction/store interactions
88 ieslick 1.51 x Legacy conversions issue for SQL (and BDB?) due to package rename (both)
89 ieslick 1.50 x Figure out how to compact a specific btree and/or key-range using optimize-storage.
90     Probably need to update keyword part of the API
91 ieslick 1.48 x User choice to run db_deadlock when opening a bdb backend? Requires path to
92     functions and ability to launch shell command. Closing the store stops the sub-process.
93     x Resolve duplicate ordering issues (punting to future release; documentation fix)
94     x Trace all paths to db-put or db-delete and ensure that there is a check or a
95     default ensure-transaction around the primitive components - write a document
96     clarifying transaction design & assumptions in the backend
99 ieslick 1.47 February 9th, 2007
100     x Improve SQL secondary cursor performance (Robert)
101     x Fix *dbconnection-spec* to support multiple controllers for multiple threads
102     for CLSQL backend (Robert)
103     x Fix cur-del2 failure under SBCL (robert to reproduce and fix)
105 ieslick 1.42 February 3rd, 2007 checkins:
106     x Finished char -> unsigned char for buffer streams to solve SBCL type problems
107     x Finished new serializer-initialization and open-controller protocol to handle
108     subtle issues in database metadata and the user of the serializer
110 ieslick 1.40 Feburary 2nd, 2007 checkins:
111 ieslick 1.48 x Punted duplicate sorting to 0.7.0
112 ieslick 1.41 x Support locks in serializer for all systems
113     x Provide support for fast and slow critical sections by lisps: buffer-streams,
114     circularity-arrays/hashes, shared controller side-effects... (see email)
115 ieslick 1.40 x Check for manual & automatic transactions running concurrently
116     x Modify *current-transaction* to be null on default, allowing backends to choose the default format (vs. +NULL-VOID+)
117     x Update BDB backend to properly provide result
118     x Roll deprecation of *auto-commit* through code base so leaf functions stop referring to it; modify berkeley-db to not refer to auto-commit except where it's appropriate (open commands)
119     x Ensure serialization is thread-safe and reasonably efficient
120     x Resourced-byte-spec should be per-thread (or removed - ok to cons during bignum serialization)
121     x Allow elephant threads to appropriately bind dynamic variables?
122     x Thread safety for all global vars
123     x Thread safe API option for user-managed store-controller?
124     x Thread safe API for transactions
125     x Ported to and tested with BDB 4.5
126     x Full 64-bit support (arrays, native 64-bit fixnums, etc)
127     x Mark fixnums appropriately: 32-bit lisps can decode 64-bit fixnums as bignums (two 32-bit entities)
128     x char vs. uint8 in buffer-stream to read-out (See Marco e-mail)
130     January 22, 2007 checkins:
131 ieslick 1.34 x Modularize serializers for easy upgrade
132     x MCL 1.1 unicode support; clean up other lisp support for unicode
133 ieslick 1.31 x Simplify user-specific configuration parameters using config.sexp and my-config.sexp
134 ieslick 1.34 x Ensure thread safety in buffer-stream allocation!
136 ieslick 1.40 January 2007 checkins; minor fixes
137 ieslick 1.35 x Think through default *store-controller* vs. explicit parameter passing referencing all over the APIs
138     (Enable explicit passing everywhere, maintain *store-controller* defaults. This makes multi-threading
139     support simpler. Users can pass the store controller or rely on a global *store-controller*)
140 ieslick 1.34 x New build interface; all-lisp compilation (sans win32), (help from elephant-devel)
141     x Verify that operations such as indexing are thread safe
142     x Diffs for lisp-controlled DB checkpointing (by Gabor Melis)
143     x Improved optimization options to be more user controlled (Pierre Thierry)
144 ieslick 1.32 x Investigated gensym warnings in berkeley-db.lisp (caused by an FFI macro, no harm in it)
145     x Remove warnings in libberkeley-db.c
146 ieslick 1.31
147 ieslick 1.32 Prior to December 2006
148 ieslick 1.23 x BDB overwrite of values makes DB grow
149     [So far I can only find that it grows on the 2nd write, but not after that...artifact of
150     page allocation or caching of memory pools?]
151     x FEATURE: Investigate BDB record size; it's 2x larger than expected?
152     [Ditto above]
153 ieslick 1.31 x Update to support BDB 4.4/4.5
154 ieslick 1.25 x Add ability from within lisp to reclaim DB space after deleting btree key-value pairs
155 ieslick 1.27 x Should we delete slot-values in the db when redefining classes, currently those values
156     stay around - probably indefinitely unless we GC (no, we'll resolve this with a
157     stop-and-copy GC - need to make migration bookkeeping more efficient)
158 ieslick 1.28 x MCL type-declaration compatibility
159 ieslick 1.30 x Remove sleepycat name. Change sleepycat to db-bdb to reflect oracle ownership and avoid
160     confusion for new users
161 ieslick 1.23
162 ieslick 1.22 ========================================================
163     ========================================================
165 ieslick 1.23 June 20, 2006:
167     Features completed in 0.6.0:
168     -----------------------------------
169     x Add a class-indexing class option to the metaclass so we can maintain class instances
170     index without any secondary indices or indexed slots
171     x Upgrade solution to 0.6.0, DB properties & version tag for future upgrades
172     x Validate migration
173     x Documentation update
174     x Indexing tutorial and tutorial review
175     x On class change, new slots should have their initform values pushed
176 ieslick 1.19 into the slot value as if the slot was being created the first time
177     (currently this doesn't happen) [fixed in 0.6.0-rc1]
179 ieslick 1.23
180     Feb. 4, 2006:
181 rread 1.9
182     As of 0.5.0, we have seem to have a stable suite on
183     ACL, SBCL, and OpenMCL.
185     The things I would most like to see improved are:
187     1) Robustness around error handling.
188     2) Ian Eslick has written some great indexing stuff in a branch
189     that I intend to merge in after this release.
190     3) The SQL serializer could be made more efficiently very
191     easily, greatly enhancing the speed of that as a back-end.
193     In general, Elephant is very usable and modestly robust,
194     but heavier use, better documentation, and a good example
195     application would help it a lot.
197 ieslick 1.23 Merge in the todos from the source and the NOTES!
198 rread 1.9
200 ieslick 1.23 October 19, 2005:
201 rread 1.8
202     The SQL back-end stuff has only been tested with
203     Postgress and SBCL.
205     Using SQLite and mysql would really expand the
206     usage of the system, I assume.
208     The database-stuff is fairly slow since it
209     does normal serialization and then Base64 encoding.
210     This is very safe and simple, but costs us a lot of bytes
211     to and from the database; a better serializer would
212     make things MUCH faster.
215 ieslick 1.23 Notes from original developers at time of handoff:
216     (minus stuff recorded elsewhwere)
217     --------------------------------------------------
218 blee 1.4 understand the profiler / timer, tweak performance of CLOS
219     stuff
220 blee 1.1
221 blee 1.5 tweak performance of transactions! dynamic-extent in CMUCL
222     / SBCL.
223 blee 1.1
224 blee 1.6 equality joins have to be done on the lisp side:
225 blee 1.5 end-of-table is not the same as end-of-btree.
226 blee 1.1
227     GC (need cursors)
229     Lispworks stuff (fli:replace-foreign-array...)
231 blee 1.6 bignum fix: OpenMCL: check that ldb is non-consing (i think
232     it is), look at %ldb-fixnum-from-bignum. profile
233     %bignum-ref on CMUCL / SBCL.
234 blee 1.1
235 blee 1.5 serialize lambdas, closures, packages.....this is hard!
236 blee 1.1
237 blee 1.6 openmcl lispy pointer arithmetic (profile sap-alien, etc).
238     profile CMUCL / SBCL sap arithmetic.
239 blee 1.1
240 blee 1.5 CMUCL, SBCL, Allegro? (NOT OpenMCL) can directly pass memory
241     like foreign arrays. Use these instead of foreign arrays?
243     byte-ordering? nah.....
245     incorporate requirements from ICU license
247 blee 1.6 cursor-put : move the cursor after insert.
249 blee 1.7 change :transient flag to an allocation type (fix CMUCL!)

  ViewVC Help
Powered by ViewVC 1.1.5