JavaScript Strings are “finite ordered sequence of zero or more 16-bit unsigned integer values.” Usually these integer values are UTF-16 code units. The UTF-16 encoding uses one 16-bit unit for Unicode characters from U+0000 to U+FFFF, and two units for characters from U+10000 to U+10FFFF. Unfortunately all the usual String functions length, charAt, charCodeAt, are …
Tag Archives: Unicode
Java and File Names With Invalid UTF-8
On Unix systems – Linux and OS X included – file names can be arbitrary binary data with very few limitations. This means that in order to make sense of the name a character encoding must be used. Recently UTF-8 has become the default encoding on many systems, but sometimes you have to deal with …
The Ultimate Guide To UTF-8 and MySQL
How character encodings work in MySQL seem to continue baffle people, at least based on the number of questions posted on Stackoverflow. Read on if you have an application that still outputs funny text like “señor” or “se�or” where “señor” is expected. The truth about text in MySQL Strings in MySQL consist of an encoding …
Fixing mangled characters in MySQL
Some 30000 UTF-8 encoded posts in a popular Spanish forum had been stored in a column defined as latin1 with the usual consequences: accented characters like á é ó ñ got mangled into á é ó ñ, also known as mojibake. To complicate matters more, the more recent content was properly encoded in latin1, so a …