From UTF-16 to UTF-8… in JavaScript

JavaScript Strings are “finite ordered sequence of zero or more 16-bit unsigned integer values.” Usually these integer values are UTF-16 code units. The UTF-16 encoding uses one 16-bit unit for Unicode characters from U+0000 to U+FFFF, and two units for characters from U+10000 to U+10FFFF. Unfortunately all the usual String functions length, charAt, charCodeAt, are …

The Ultimate Guide To UTF-8 and MySQL

How character encodings work in MySQL seem to continue baffle people, at least based on the number of questions posted on Stackoverflow. Read on if you have an application that still outputs funny text like “señor” or “se�or” where “señor” is expected. The truth about text in MySQL Strings in MySQL consist of an encoding …

Fixing mangled characters in MySQL

Some 30000 UTF-8 encoded posts in a popular Spanish forum had been stored in a column defined as latin1 with the usual consequences: accented characters like á é ó ñ got mangled into á é ó ñ, also known as mojibake. To complicate matters more, the more recent content was properly encoded in latin1, so a …