[Django]-Unicode issue with makemessages –all Django 1.6.2 Python 3.3

by

9👍

✅

This threw me off as everything was fine with Py2, but not Py3, so I assumed that’s where the problem would be.

The issue was in part because I’m using Docker, and running makemessages from within the container which didn’t have the locale set to anything in particular for bash.

Things I tried:

  • Saving files with UTF-8 (with and without BOM)
  • Ensuring I had UTF-8 in the header of the .po files
  • Creating a new blank slate project
  • Re-creating all .po files using Py3 (as they were originally created with Py2)

The top exception was being thrown in subprocess.py on this line 847: 

def _translate_newlines(self, data, encoding):
    data = data.decode(encoding)
    return data.replace("\r\n", "\n").replace("\r", "\n")

The encoding being passed in was ANSI_X3.4-1968, which was weird as I had saved the files as UTF-8 etc (it was being set as ANSI… due to my bash session not have a locale set specifically).

Answer
In my Docker container I had no locale settings set in the terminal, so they were: 

# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

These were my available locales (missing my specific locale, en_US.UTF-8, but as long as it’s UTF-8 I’m okay): 

# locale -a
C
C.UTF-8
POSIX

Placed this in ~/.bashrc: 

export LC_ALL=C.UTF-8
export LANG=C.UTF-8
export LANGUAGE=C.UTF-8

And now I get UTF-8 as the content type within subprocess.py, and everything works with Py3/Django1.6 =)

long story short, I was caught off guard that Django/subprocess.py is using the environment locale, and not the encoding of the file / or the header Content-Type.

👤Justin

Leave a comment