projects/xcvb/babel.git
3 years agoAdded support for specifying endiannes for utf-16 and utf-32. Also fixed some utf... master
Dmitry Ignatiev [Thu, 4 Nov 2010 13:33:36 +0000] 
Added support for specifying endiannes for utf-16 and utf-32. Also fixed some utf-32 bugs.

Detailed discussion: http://thread.gmane.org/gmane.lisp.babel.devel/56

3 years agomove the test suite over to hu.dwim.stefil
attila.lendvai [Thu, 14 Oct 2010 22:20:47 +0000] 
move the test suite over to hu.dwim.stefil

3 years agoadded defgeneric for get-output-stream-sequence to avoid a warning
attila.lendvai [Thu, 24 Jun 2010 14:40:38 +0000] 
added defgeneric for get-output-stream-sequence to avoid a warning

4 years agorenamed +sub+ to +default-substitution-code-point+
attila.lendvai [Wed, 21 Oct 2009 13:42:02 +0000] 
renamed +sub+ to +default-substitution-code-point+

4 years agoAdd Japanese encodings: EUC-JP and CP932
SANO Masatoshi [Tue, 29 Dec 2009 21:15:05 +0000] 
Add Japanese encodings: EUC-JP and CP932

4 years agobabel-streams: implement STREAM-LINE-COLUMN for VECTOR-OUTPUT-STREAM
Luis Oliveira [Sat, 5 Dec 2009 17:29:53 +0000] 
babel-streams: implement STREAM-LINE-COLUMN for VECTOR-OUTPUT-STREAM

Patch courtesy of Ariel Badichi.

4 years agoMore OCCFD.
Luis Oliveira [Sat, 5 Dec 2009 17:27:58 +0000] 
More OCCFD.

4 years agoObsessive-compulsive code formatting disorder.
Luis Oliveira [Thu, 3 Dec 2009 23:31:27 +0000] 
Obsessive-compulsive code formatting disorder.

4 years agoAdd AMBIGUOUS-ENCODING-P method for symbols.
Luis Oliveira [Thu, 3 Dec 2009 22:46:00 +0000] 
Add AMBIGUOUS-ENCODING-P method for symbols.

4 years agoFix return values of unibyte encoders and decoders.
Luis Oliveira [Thu, 3 Dec 2009 22:12:13 +0000] 
Fix return values of unibyte encoders and decoders.

Initial patch courtesy of Ariel Badichi.

4 years agoAdd new predicate AMBIGUOUS-ENCODING-P.
Luis Oliveira [Thu, 3 Dec 2009 22:10:23 +0000] 
Add new predicate AMBIGUOUS-ENCODING-P.

4 years agoSmall tweak to BOM-VECTOR
Luis Oliveira [Mon, 1 Jun 2009 21:59:56 +0000] 
Small tweak to BOM-VECTOR

Reported by Tobias Rittweiler: "Compiling BOM-VECTOR emits a
'type assertion too complex to check' note. The reason is that
(THE FOO (QUUX)) is actually of type (VALUES FOO &REST T) which
SBCL, at least for the moment, cannot cope well with."

4 years agoadded with-input-from-sequence to babel-streams
attila.lendvai [Fri, 15 May 2009 18:48:29 +0000] 
added with-input-from-sequence to babel-streams

5 years agoVarious improvements to the test suite.
Luis Oliveira [Sat, 25 Apr 2009 17:02:09 +0000] 
Various improvements to the test suite.

- Rename RUN-TESTS to RUN and export it from BABEL-TESTS.
- New test CODE-CHAR-NILNESS checks whether we can encode all
  code points below BABEL:UNICODE-CHAR-CODE-LIMIT.
- TEST-UNICODE-ROUNDTRIP: don't try to encode non-characters.
- UTF8-ILLEGAL-CODE-POSITIONS: new test adapted from MGK's UTF8
  test suite.
- Disable a few tests for CCL: UTF-8B.* and CODE-CHAR-NILNESS.

5 years agoHonor *READ-SUPPRESS* in our #\ reader.
Luis Oliveira [Sat, 25 Apr 2009 16:57:39 +0000] 
Honor *READ-SUPPRESS* in our #\ reader.

- Add regression test: SHARP-BACKSLASH.3.

5 years agoFix the documentation Makefile.
Luis Oliveira [Sat, 25 Apr 2009 16:51:20 +0000] 
Fix the documentation Makefile.

5 years agoNew encoding: CP1251.
Luis Oliveira [Mon, 16 Mar 2009 23:43:47 +0000] 
New encoding: CP1251.

Patch courtesy of Andrey Moskvitin.

5 years agobah, do that previous fix nicer
attila.lendvai [Mon, 8 Dec 2008 17:36:43 +0000] 
bah, do that previous fix nicer

5 years agosmall smartness to with-output-to-sequence, does not bind *standard-output* & co...
attila.lendvai [Sun, 7 Dec 2008 23:20:06 +0000] 
small smartness to with-output-to-sequence, does not bind *standard-output* & co. to NIL

5 years agoadded note to NOTES about enforcing endianness
attila.lendvai [Tue, 25 Nov 2008 19:39:21 +0000] 
added note to NOTES about enforcing endianness

5 years agofix (string-to-octets simple-base-string :encoding :utf-16)
attila.lendvai [Mon, 24 Nov 2008 20:54:55 +0000] 
fix (string-to-octets simple-base-string :encoding :utf-16)

5 years agoadded make-in-memory-input-stream
attila.lendvai [Thu, 16 Oct 2008 17:27:11 +0000] 
added make-in-memory-input-stream

5 years agoBe more conservative with base-string optimizations: only enable it on lisps that...
attila.lendvai [Fri, 26 Sep 2008 20:39:49 +0000] 
Be more conservative with base-string optimizations: only enable it on lisps that support it (SBCL for now)

5 years agoAdd support for the Scieneer CL
Luis Oliveira [Fri, 26 Sep 2008 20:05:28 +0000] 
Add support for the Scieneer CL

Patch courtesy of Douglas Crosher.

5 years agodisable simple-base-string optimization for clisp and ccl.
attila.lendvai [Fri, 26 Sep 2008 12:37:42 +0000] 
disable simple-base-string optimization for clisp and ccl.

5 years agofixed-width encodings don't need to define an octet counter anymore
attila.lendvai [Sat, 20 Sep 2008 15:15:35 +0000] 
fixed-width encodings don't need to define an octet counter anymore

5 years agosmall refactor around instantiate-concrete-mappings
attila.lendvai [Sat, 20 Sep 2008 15:15:29 +0000] 
small refactor around instantiate-concrete-mappings

5 years agodrop the use of in-root-suite in the tests
attila.lendvai [Fri, 19 Sep 2008 17:18:53 +0000] 
drop the use of in-root-suite in the tests

5 years agotype annotate and inline use-bom
attila.lendvai [Tue, 9 Sep 2008 19:34:18 +0000] 
type annotate and inline use-bom

5 years agouse named-lambda in enc-*
attila.lendvai [Tue, 9 Sep 2008 17:37:51 +0000] 
use named-lambda in enc-*

5 years agouse define-constant instead of special variables in enc-ebcdic.lisp
attila.lendvai [Tue, 9 Sep 2008 17:36:53 +0000] 
use define-constant instead of special variables in enc-ebcdic.lisp

5 years agorenames and named-lambda for code clarity. abstract/concrete mappings don't inherit...
attila.lendvai [Tue, 9 Sep 2008 12:50:47 +0000] 
renames and named-lambda for code clarity. abstract/concrete mappings don't inherit anymore, use a -factory prefix where approriate

5 years agosome local renames for more code clarity
attila.lendvai [Mon, 8 Sep 2008 13:43:47 +0000] 
some local renames for more code clarity

5 years agooptimize string-to-octets for simple-base-string
attila.lendvai [Tue, 24 Jun 2008 15:28:35 +0000] 
optimize string-to-octets for simple-base-string

5 years agocomments
attila.lendvai [Sun, 7 Sep 2008 18:30:59 +0000] 
comments

5 years agosupport external-format as the encoding parameter in bom-vector (needed for the strea...
attila.lendvai [Wed, 27 Aug 2008 22:08:38 +0000] 
support external-format as the encoding parameter in bom-vector (needed for the stream code).

5 years agoTAG 0.3.0
Luis Oliveira [Tue, 29 Jul 2008 03:21:41 +0000] 
TAG 0.3.0

5 years agoupdate babel.asd for version 0.3.0
Luis Oliveira [Tue, 29 Jul 2008 03:21:37 +0000] 
update babel.asd for version 0.3.0

5 years agomisc cleanups
Luis Oliveira [Tue, 29 Jul 2008 03:21:01 +0000] 
misc cleanups

5 years agoAssume big-endianness by default in the UTF-32 decoder
Luis Oliveira [Mon, 28 Jul 2008 22:38:19 +0000] 
Assume big-endianness by default in the UTF-32 decoder

- added regression test.

5 years agomake lookup-mapping inlined and change it not to error when called with a concrete...
attila.lendvai [Tue, 24 Jun 2008 15:08:57 +0000] 
make lookup-mapping inlined and change it not to error when called with a concrete-mapping

5 years agoimplement :USE-BOM in STRING-TO-OCTETS
Luis Oliveira [Tue, 24 Jun 2008 15:18:07 +0000] 
implement :USE-BOM in STRING-TO-OCTETS

5 years ago80-column freak changes
Luis Oliveira [Tue, 24 Jun 2008 15:13:23 +0000] 
80-column freak changes

5 years agoPort babel-tests to Stefil
Luis Oliveira [Tue, 24 Jun 2008 15:10:44 +0000] 
Port babel-tests to Stefil

- Uses DEFSTEST a lot. Should probably get rid of that.

5 years agoenable the sharp-backslash-syntax in the tests to make test-op work
attila.lendvai [Tue, 24 Jun 2008 14:00:30 +0000] 
enable the sharp-backslash-syntax in the tests to make test-op work

5 years agoTAG 0.2.0
Luis Oliveira [Mon, 9 Jun 2008 01:29:36 +0000] 
TAG 0.2.0

5 years agoupdate babel.asd for version 0.2.0
Luis Oliveira [Mon, 9 Jun 2008 01:29:29 +0000] 
update babel.asd for version 0.2.0

5 years agoadded release.sh script
Luis Oliveira [Mon, 9 Jun 2008 01:28:43 +0000] 
added release.sh script

5 years agobah, fix my previous initial-buffer-size patch for streams
attila.lendvai [Sun, 25 May 2008 10:42:08 +0000] 
bah, fix my previous initial-buffer-size patch for streams

5 years agoadded a :initial-buffer-size to with-output-to-sequence and co.
attila.lendvai [Sun, 25 May 2008 08:13:51 +0000] 
added a :initial-buffer-size to with-output-to-sequence and co.

5 years agostream's :element-type works with deftype'd (unsigned-byte 8)
attila.lendvai [Wed, 21 May 2008 17:08:15 +0000] 
stream's :element-type works with deftype'd (unsigned-byte 8)

6 years agofactored out lookup-string-vector-mapping
attila.lendvai [Fri, 18 Apr 2008 14:29:43 +0000] 
factored out lookup-string-vector-mapping

5 years ago#\u reader changes, don't enable it when loaded, only by explicit user request
attila.lendvai [Thu, 8 May 2008 11:22:55 +0000] 
#\u reader changes, don't enable it when loaded, only by explicit user request

5 years agomake sure the external-format slot of the in-memory stream is an external-format...
attila.lendvai [Tue, 6 May 2008 17:59:01 +0000] 
make sure the external-format slot of the in-memory stream is an external-format instance.

also fix make-in-memory-output-stream accepting '(unsigned-byte 8).

5 years agodrop :as-list and add a :return-as keyword arg to get-output-stream-sequence and...
attila.lendvai [Tue, 6 May 2008 17:58:52 +0000] 
drop :as-list and add a :return-as keyword arg to get-output-stream-sequence and with-output-to-sequence

5 years agooptimize in-memory stream, make it accept plain vectors, too, not only ub8 vectors
attila.lendvai [Tue, 6 May 2008 17:31:31 +0000] 
optimize in-memory stream, make it accept plain vectors, too, not only ub8 vectors

5 years agowith-output-to-sequence accepts an :external-format and properly parses its body...
attila.lendvai [Tue, 6 May 2008 16:41:13 +0000] 
with-output-to-sequence accepts an :external-format and properly parses its body for declarations

5 years agoAdd a test idea to NOTES
Luis Oliveira [Mon, 5 May 2008 20:13:19 +0000] 
Add a test idea to NOTES

5 years agoadded a THE to fire up the make-array transform in string-to-octets
attila.lendvai [Mon, 5 May 2008 19:56:44 +0000] 
added a THE to fire up the make-array transform in string-to-octets

5 years agoadded :external-format to make-in-memory-output-stream
attila.lendvai [Mon, 5 May 2008 18:14:39 +0000] 
added :external-format to make-in-memory-output-stream

6 years agosmall notes in streams.lisp about status/todo
attila.lendvai [Tue, 22 Apr 2008 18:02:46 +0000] 
small notes in streams.lisp about status/todo

6 years agoTAG 0.1.0
Luis Oliveira [Tue, 22 Apr 2008 14:29:48 +0000] 
TAG 0.1.0

6 years agofirst scratch of stream support.
attila.lendvai [Fri, 18 Apr 2008 14:14:48 +0000] 
first scratch of stream support.

in-memory bivalent vector-buffered streams with file-position and with-output-to-sequence (hope)fully works.

6 years agoECL doesn't like nor need fix-sharp-backslash.lisp
Luis Oliveira [Sun, 6 Apr 2008 12:41:00 +0000] 
ECL doesn't like nor need fix-sharp-backslash.lisp

6 years agobabel-tests.asd fixups
Luis Oliveira [Sun, 6 Apr 2008 12:40:10 +0000] 
babel-tests.asd fixups

6 years agoMove LOOKUP-MAPPING to external-formats.lisp and make it accept EXTERNAL-FORMATs...
Stelian Ionescu [Sun, 24 Feb 2008 17:06:31 +0000] 
Move LOOKUP-MAPPING to external-formats.lisp and make it accept EXTERNAL-FORMATs too.

6 years agoFix concatenate-string-to-octets.
Luis Oliveira [Sun, 24 Feb 2008 16:37:10 +0000] 
Fix concatenate-string-to-octets.

- Was calling a nonexistent function.
- Added a note about endianness to its unit test.

6 years agoFix run-babel-tests function.
Luis Oliveira [Sun, 24 Feb 2008 16:36:32 +0000] 
Fix run-babel-tests function.

6 years agoIndent previous patch to 80 columns. :-)
Luis Oliveira [Sun, 24 Feb 2008 13:14:20 +0000] 
Indent previous patch to 80 columns. :-)

6 years agoAdded concatenate-strings-to-octets
attila.lendvai [Sat, 23 Feb 2008 22:36:35 +0000] 
Added concatenate-strings-to-octets

6 years agoAdded declaims to make it possible to locally inline string-to-octets and friends...
attila.lendvai [Sat, 23 Feb 2008 22:06:59 +0000] 
Added declaims to make it possible to locally inline string-to-octets and friends upon explicit request

6 years agofix the return values of the encoders, the upcoming concatenate-strings-to-octets...
attila.lendvai [Sat, 23 Feb 2008 22:35:00 +0000] 
fix the return values of the encoders, the upcoming concatenate-strings-to-octets patch/test relies on it

6 years agoDefine EOL-STYLE type and enforce it.
Luis Oliveira [Wed, 12 Dec 2007 21:55:34 +0000] 
Define EOL-STYLE type and enforce it.

- MAKE-EXTERNAL-FORMAT now takes EOL-STYLE as a keyword argument
  instead of an optional one.

6 years agoAdd some documentation
Luis Oliveira [Wed, 12 Dec 2007 21:52:23 +0000] 
Add some documentation

6 years agoAddressing minor style obsessions.
Luis Oliveira [Mon, 10 Dec 2007 17:53:15 +0000] 
Addressing minor style obsessions.

6 years agoExport EXTERNAL-FORMAT from the Babel package.
Luis Oliveira [Mon, 10 Dec 2007 17:52:50 +0000] 
Export EXTERNAL-FORMAT from the Babel package.

6 years agoFix/add TEST-OP
Luis Oliveira [Mon, 10 Dec 2007 17:38:42 +0000] 
Fix/add TEST-OP

6 years agoFixed function test-file(used by ICONV-TEST): always take the truename of a .asd...
Stelian Ionescu [Tue, 4 Dec 2007 00:52:31 +0000] 
Fixed function test-file(used by ICONV-TEST): always take the truename of a .asd file.

6 years agoFixed ISO-8859-11 table in *iso-8859-tables*, iso-8859-decode-check now passes.
Stelian Ionescu [Tue, 4 Dec 2007 00:47:13 +0000] 
Fixed ISO-8859-11 table in *iso-8859-tables*, iso-8859-decode-check now passes.

6 years agoFix :iso-8859-11 encoding
Stelian Ionescu [Tue, 4 Dec 2007 00:34:42 +0000] 
Fix :iso-8859-11 encoding

6 years agoReplace FIXNUM declarations with UB8 in ISO-8859 encodings.
Stelian Ionescu [Mon, 3 Dec 2007 22:48:06 +0000] 
Replace FIXNUM declarations with UB8 in ISO-8859 encodings.

6 years agoMisc. cleanups in the ISO-8859 encodings
Stelian Ionescu [Mon, 3 Dec 2007 22:39:01 +0000] 
Misc. cleanups in the ISO-8859 encodings

6 years agoFix :iso-8859-10 encoder
Stelian Ionescu [Mon, 3 Dec 2007 18:25:57 +0000] 
Fix :iso-8859-10 encoder

6 years agoFix :iso-8859-5 encoder
Stelian Ionescu [Mon, 3 Dec 2007 18:09:14 +0000] 
Fix :iso-8859-5 encoder

6 years agoFix :iso-8859-4 encoder
Stelian Ionescu [Mon, 3 Dec 2007 17:55:31 +0000] 
Fix :iso-8859-4 encoder

6 years agoFix test-8bit-roundtrip to ignore undefined code points
Stelian Ionescu [Mon, 3 Dec 2007 17:39:24 +0000] 
Fix test-8bit-roundtrip to ignore undefined code points

6 years agoENSURE-EXTERNAL-FORMAT can now handle arguments of the form '(<external-format> ...
Stelian Ionescu [Tue, 23 Oct 2007 20:34:50 +0000] 
ENSURE-EXTERNAL-FORMAT can now handle arguments of the form '(<external-format> :eol-style <eol-style>)

6 years agoInitialize errorp from *suppress-character-coding-errors* to keep the global policy...
attila.lendvai [Thu, 23 Aug 2007 02:13:23 +0000] 
Initialize errorp from *suppress-character-coding-errors* to keep the global policy when not overridden

6 years agoFix docstring typo in src/string.lisp
Luis Oliveira [Thu, 23 Aug 2007 12:07:15 +0000] 
Fix docstring typo in src/string.lisp

6 years agoNew function: EXTERNAL-FORMAT-EQUAL
Luis Oliveira [Thu, 23 Aug 2007 12:06:55 +0000] 
New function: EXTERNAL-FORMAT-EQUAL

6 years agoAdd note to NOTES.
Luis Oliveira [Thu, 23 Aug 2007 12:06:35 +0000] 
Add note to NOTES.

6 years agoFix ` placement in WITH-SIMPLE-VECTOR
Luis Oliveira [Thu, 23 Aug 2007 12:05:29 +0000] 
Fix ` placement in WITH-SIMPLE-VECTOR

6 years agoAdd :RE endianness to UB-GET.
Luis Oliveira [Mon, 13 Aug 2007 22:49:44 +0000] 
Add :RE endianness to UB-GET.

6 years agoHandle reverse endian and character out-of-range in UTF-32.
Luis Oliveira [Mon, 13 Aug 2007 21:16:47 +0000] 
Handle reverse endian and character out-of-range in UTF-32.

- Add regression tests.

6 years agoNew tests
Luis Oliveira [Mon, 13 Aug 2007 19:45:52 +0000] 
New tests

- Added tests adapted from SBCL's tests/octets.pure.lisp
  (which discovered all of those UTF-8 bugs)
- Added tests against flexi-streams' ISO-8859-* tables.

6 years agoAdd debugging helper: RECOMPILE-MAPPINGS.
Luis Oliveira [Mon, 13 Aug 2007 19:45:27 +0000] 
Add debugging helper: RECOMPILE-MAPPINGS.

6 years agoFix handling of leading BOM in UTF-32.
Luis Oliveira [Mon, 13 Aug 2007 19:44:56 +0000] 
Fix handling of leading BOM in UTF-32.

6 years agoFix silly ISO-8859-11 bug.
Luis Oliveira [Mon, 13 Aug 2007 19:44:05 +0000] 
Fix silly ISO-8859-11 bug.

- Decoder code was defined as encoder and vice-versa.

6 years agoHandle overlong UTF-8 sequences properly.
Luis Oliveira [Mon, 13 Aug 2007 17:58:30 +0000] 
Handle overlong UTF-8 sequences properly.

- Also, defined more specific decoding error conditions:
  CHARACTER-OUT-OF-RANGE, INVALID-UTF8-START-BYTE,
  INVALID-UTF8-CONTINUATION-BYTE and OVERLONG-UTF8-SEQUENCE.

6 years agoHandle erroneous UTF-8 continuation bytes properly.
Luis Oliveira [Sun, 12 Aug 2007 20:29:55 +0000] 
Handle erroneous UTF-8 continuation bytes properly.