[dvblast-devel] biTStream string decodings vs. ISO6934

[Ondřej Caletka]

Hi,

I'm just trying to write some piece of code using biTStream and found
some issues with functions for encoding and deconding DVB text strings.

The ETSI EN 300 468 says: If the first byte of the text field has a
value in the range "0x20" to "0xFF" then this and all subsequent bytes
in the text item are coded using the default character coding table
(table 00 - Latin alphabet) of figure A.1.

And a comment below figure A.1: This table is a superset of ISO/IEC 6937
with addition of the Euro symbol.

However, when biTStream sees first byte in range 0x20-0xff it qualifies
the encoding as ISO 8859-1:

file dvb/si/strings.h
  71     i_first = (*pp_string)[0];
  72
  73     if (!i_first) return NULL;
  74     if (i_first >= 0x20) return "ISO_8859-1";

I know that there are some broadcasters, mostly in the Western Europe,
transmitting ISO 8859-1 texts without correct prefix 0x10 0x00 0x01 so
this quirk is necessary in that case. However, there should be a
possibility to turn that quirk off for countries where ISO 6937 is used.

In reverse, when creating a new DVB string using dvb_string_set, it
looks like that it is encoding ISO 8859-9, which is output without any
prefix, instead of ISO 6937:

 105     if (!strcmp(psz_encoding, "ISO_8859-9")) {
 106         *pi_out_length = i_length;
 107         return (uint8_t *)strdup((const char *)p_string);
 108     }

This looks very odd to me. Why ISO 8859-9? I would expect 8859-1, as in
the decoding function.

Thanks in advance for some explanation.

All the Best,
Ondřej Caletka