So, you need to set a locale.

sudo locale-gen en_US.UTF-8  
sudo dpkg-reconfigure locales

If the install has been configured to use another locale, such as en_GB (you'll see it in the message), use that instead.