Subject: typo: gapOffset != AE00

From: Roman Czyborra <[email-masked]>
Date: Tue, 4 Aug 1998 15:14:36 +0200 (CEST)

http://www.unicode.org/unicode/reports/tr6.html seems to contain a
typo in Table X-3: the gapOffset should be defined as AC00 instead of
AE00 because otherwise you get 68 * 80 + AE00 == E200 != E000. 

Please apply the following patch:

*** tr6.html	1998/01/28 08:17:06	6.10
--- tr6.html	1998/08/04 13:04:48
***************
*** 397,403 ****
    </tr>
    <tr>
      <td>68..A7 </td>
!     <td>x*80+AE00 </td>
      <td>half-blocks from U+E000 to U+FF80 </td>
    </tr>
    <tr>
--- 397,403 ----
    </tr>
    <tr>
      <td>68..A7 </td>
!     <td>x*80+AC00 </td>
      <td>half-blocks from U+E000 to U+FF80 </td>
    </tr>
    <tr>

Cheers, Roman http://czyborra.com/

---

From: Roman Czyborra <[email-masked]>
Date: Tue, 4 Aug 1998 16:04:33 +0200 (CEST)

I think that anding a hbyte with 1F is better than & 1FFF:

*** tr6.html	1998/08/04 13:04:48	6.11
--- tr6.html	1998/08/04 13:46:58
***************
*** 449,455 ****
  The window index <em>n</em> is given by the top 3 bits of hbyte. The window offset is
  calculated from the remaining thirteen bits of hbyte and lbyte as follows: </p>
  
! <p><em>offset</em> = 10000 + (80 * ((hbyte &amp; 1FFF) * 100 + lbyte)) </p>
  
  <p>where &amp; is the bitwise AND operator and all values are in hexadecimal notation.
  After an extended Window is defined each subsequent byte in the range 80 to FF represents
--- 449,455 ----
  The window index <em>n</em> is given by the top 3 bits of hbyte. The window offset is
  calculated from the remaining thirteen bits of hbyte and lbyte as follows: </p>
  
! <p><em>offset</em> = 10000 + (80 * ((hbyte &amp; 1F) * 100 + lbyte)) </p>
  
  <p>where &amp; is the bitwise AND operator and all values are in hexadecimal notation.
  After an extended Window is defined each subsequent byte in the range 80 to FF represents

---

From: Roman Czyborra <[email-masked]>
Date: Tue, 4 Aug 1998 17:06:14 +0200 (CEST)

According to table X-5 of http://www.unicode.org/unicode/reports/tr6.html
Russian should be using the default SC2 instead of SC7:

*** tr6.html	1998/08/04 13:46:58	6.12
--- tr6.html	1998/08/04 14:56:16
***************
*** 808,815 ****
  
  <h3>Russian</h3>
  
! <p>Russian can use the default position of window 7. The first byte of the compressed data
! is the tag SC7.</p>
  
  <p>Unicode values (6 characters):</p>
  
--- 808,815 ----
  
  <h3>Russian</h3>
  
! <p>Russian can use the default position of window 2. The first byte of the compressed data
! is the tag SC2.</p>
  
  <p>Unicode values (6 characters):</p>
  
***************
*** 817,823 ****
  
  <p>Compressed (7 bytes): </p>
  
! <pre>17 9C BE C1 BA B2 B0</pre>
  
  <h3>All Features</h3>
  
--- 817,823 ----
  
  <p>Compressed (7 bytes): </p>
  
! <pre>12 9C BE C1 BA B2 B0</pre>
  
  <h3>All Features</h3>

---

From: Roman Czyborra <[email-masked]>
Date: Fri, 14 Aug 1998 13:38:21 +0200 (CEST)

> Russian should be using the default SC2 instead of SC7:

The third example in http://www.unicode.org/unicode/reports/tr6.html
is also wrong because

=41=DF=11=DF=03=01=03=DF=19=01=DF=0E=DF=DF=0F=01=DF=F0=F0=00=F1=FF=FF=FF

expands to

<U0041><U00DF><U015F><U0101><U045F><U00DF><UDFDF><U01DF><UF000><UDBFF><UDFFF>

with your sample implementation java CompressMain /expand /dev/stdin
and to

<U0041><U00DF><U011F><U0101><U045F><U00DF><UDFDF><U01DF><UF000><U0010FFFF>

with my own implementation but not to the 

<U0041><U00DF>       <U0401><U015F><U00DF><UDFDF><U01DF><UF000><UDBFF><UDFFF>

mentioned in your technical report.  Manually, I get

=41	pass as <U0041>
=DF	dynamicOffset[0] + DF - 80 = 0080 + 5F = <U00DF>
=11	SC1 change to activewindow = 1 (locking shift)
=DF     dynamicOffset[1] + DF - 80 = 00C0 + 5F = <U011F>
=03	SQ2 single quote 2
=01	staticOffset[2] + 01 = 0100 + 01 = <U0101>
=03	SQ2 single quote 2
=DF	dynamicOffset[2] + DF - 80 = 0400 + 5F = <U045F>
=19	SD1 define window 1
=01	dynamicOffset[1] := winOffset [1] = 0080
=DF	dynamicOffset[1] + DF - 80 = 0080 + 5F = <U00DF>
=0E	SQU single quote Unicode
=DF=DF	<UDFDF>
=0F	SCU change to Unicode mode
=01=DF	<U01DF>
=F0	UQU quote Unicode
=F0=00	<UF000>
=F1	UDX define extended
=FF=FF	dynamicOffset[7] := 10000 + FFF80 = 10F80
=FF	dynamicOffset[7] + FF - 80 = 10F80 + 7F = <U0010FFFF>

which is what I implemented.  I think that the 

      static final int initialDynamicOffset[] =
      {
          0x0080, // Latin-1
!         0x0100, // Latin Extended A

in ftp://ftp.unicode.org/Public/PROGRAMS/SCSU/SCSU.java has to be
brought in sync with table X-5 Default Positions for Dynamically
Positioned Windows:

      static final int initialDynamicOffset[] =
      {
          0x0080, // Latin-1
!         0x00C0, // combined partial Latin-1/-A

Greetings, Roman http://czyborra.com/

---

From: Roman Czyborra <[email-masked]>
Date: Sat, 15 Aug 1998 18:54:06 +0200 (CEST)

I have just added the joining of UTF-16 surrogates to my
implementation at http://czyborra.com/scsu/ and noticed another oddity

> =0F	SCU change to Unicode mode

<UDFDF> is an unpaired low-surrogate.  Unicode conformance requirement
C4 says that a process shall not interpret an unpaired high- or
low-surrogate as an abstract character.  What else are we supposed to
do with it?  My implementation outputs <U000103DF> now as if there had
been a <UD800>.