[vlc-devel] UTF-8 support

[Rémi Denis-Courmont]

	Hi,

r11988 introduces 4 new APIs :

FromLocale() takes a C string in OS locale's charset and returns a C 
string in UTF-8. Invalid character sequences are replaced with question 
marks (that should obviously never happen if the locale is 
ISO-8859-something).

ToLocale() takes a C string in UTF-8 and returns a C string in locale's 
charset. Invalid character sequences are replaced with question marks;  
if that happens, a FromLocale() or EnsureUTF8() call is probably 
missing somewhere.

LocaleFree() frees memory allocated by FromLocale() or ToLocale().

It should be noted that all of these are no-ops if the locale is UTF-8. 
DO NOT assume that LocaleFree() is free(), otherwise you'll crash if 
the locale is UTF-8. DO NOT assume that LocaleFree() is no-op otherwise 
you'll leak memory if the locale is not UTF-8.

Finally, EnsureUTF8() replaces all invalid characters sequences in a 
supposedly UTF-8 string with question marks. It should be used when a 
string is supposed to be UTF-8 encoded, but needs sanity checking (e.g. 
obtained from a file, from the network, etc). It should be called 
before passing the string to some other VLC components.

The idea now is that every string that is potentially passed among 
different plugins/components should be UTF-8 encoded.

In particular, putting a non-UTF8 character string in some psz_whatever 
parameter of a public structure is now to be considered a bug. It 
should also be assumed that every thing in the playlist is UTF8 
encoded.


TODO :
* optimize the APIs,
* add lots of EnsureUTF8() where appropriate
* all interface plugins that need to work in the user's locale should be 
reviewed and (From|To)Locale() should be used as appropriate.

-- 
Rémi Denis-Courmont
http://www.simphalempin.com/home/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20050804/f9a7a53b/attachment.sig>