[Django]-How does one make logging color in Django/Google App Engine?

Django already has support for color output through the ‘DJANGO_COLORS‘ environment variable used for example when running the built in development server. Some person has noticed this and created a plug-and-play solution https://github.com/tiliv/django-colors-formatter; with that package on the project’s python path my logging settings.py is as follow:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse'
        }
    },
    'formatters': {
        'verbose': {
            '()': 'djangocolors_formatter.DjangoColorsFormatter', # colored output
            'format': '%(levelname)s %(name)s %(asctime)s %(module)s %(process)d %(thread)d %(pathname)s@%(lineno)s: %(message)s'
        },
        'simple': {
            '()': 'djangocolors_formatter.DjangoColorsFormatter', # colored output
            'format': '%(levelname)s %(name)s %(filename)s@%(lineno)s: %(message)s'
        },
    },
     # omitting the handler 'level' setting so that all messages are passed and we do level filtering in 'loggers'
    'handlers': {
        'null': {
            'class':'django.utils.log.NullHandler',
        },
        'console':{
            'class':'logging.StreamHandler',
            'formatter': 'simple',
        },
        'mail_admins': {
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler',
            'formatter': 'verbose'
        }
    },
    'loggers': {
        '': { 
            'handlers': ['mail_admins', 'console'],
            'level': 'WARNING',
        },
    }
}

Sample console logging output using django-colors-formatter:

Although this is not quite what the OP wanted (each line coloured by level), I wanted to share a nice alternative for log output colouring called rich – a fantastic library for various rich text stuff to display in the terminal, authored by @will-mcgugan.

Simple example

Activating rich colouring for Django logs is easy: use rich.logging.RichHandler instead of logging.StreamHandler, e.g.

# project/settings.py

...

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'rich.logging.RichHandler',  # <-- this
        },
    },
    'root': {
        'handlers': ['console'],
        'level': 'INFO',
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
            'propagate': False,
        },
    },
}

(this is the modified example from Django logging docs). This will produce terminal logs like these:

Changing output format

Format customizations are done as usual by passing the handler arguments. E.g. to turn on rich tracebacks and hide the timestamp column:

LOGGING = {
    ...
    'handlers': {
        'console': {
            'class': 'rich.logging.RichHandler',
            'rich_tracebacks': True,
            'show_time': False,
        },
    },
}

will yield

Changing output colours

Changing the colouring is possible, but not via logging config, as multiple colours and styles are applied to each line; you must provide a customized theme. An example that changes colouring of INFO label from the default blue to bold magenta:

import rich
import rich.theme

my_theme = rich.theme.Theme({
    'logging.level.info': 'bold magenta',
})
rich.reconfigure(theme=my_theme)

LOGGING = {
    ...  # no changes here
}

For more details, see Styles documentation. To inspect available theme keys and default values, issue

$ python -m rich.theme

and look for keys prefixed with log. or logging..

Outro

Note that rich is so much more than just coloured logging; go check it out:

$ python -m pip install rich
$ python -m rich

Specifically for the logging use case, check out the output of

$ python -m rich.logging

to see more rendering examples than in the screenshots.

I also wanted color output for the dev_appserver. I found the solutions here a little OTT (all I wanted was to make my logging.error() calls stand out. I ended up monkeypatching the logging module by dropping this in my main.py as a quick solution:

# monkey patch logger to dump ERRORs in red
import os
if os.environ['SERVER_SOFTWARE'].find('Development') >= 0:
    import logging
    old_error = logging.error
    def red_error(msg,*args,**kwargs):
        old_error("\033[22;31m%s\033[0;0m" % msg, *args, **kwargs)
    logging.error = red_error

This will only for on ANSI-color terminals.

Here is a sample formater:

class Formatter(logging.Formatter) :
    _level_colors  = {
      "DEBUG": "\033[22;32m", "INFO": "\033[01;34m",
      "WARNING": "\033[22;35m", "ERROR": "\033[22;31m",
      "CRITICAL": "\033[01;31m"
     };    

    def format(self, record):
        if(Formatter._level_colors.has_key(record.levelname)):
            record.levelname = "%s%s\033[0;0m" % \
                            (Formatter._level_colors[record.levelname],
                             record.levelname)
        record.name = "\033[37m\033[1m%s\033[0;0m" % record.name
        return logging.Formatter.format(self, record)    

You need to configure it, for example:

...
[formatters]
keys=console_formatter
...
[handler_console_handler]
class=StreamHandler
formatter=console_formatter
args=(sys.stdout,)

I used the coloredlogs package. Unlike DJANGO_COLORS, it’s not specific to Django commands, and unlike django-colors-formatter, it’s actively maintained.

I added a single line to my logging config, and now I get configurable coloured logs.

logging.config.dictConfig({
    ...
    'formatters': {
        'console': {
            # This line right here:
            "()": "coloredlogs.ColoredFormatter",
            'format': '%(asctime)s %(levelname)s [%(name)s:%(lineno)s] %(message)s',
        },
    },
    ...

The reset codes mentioned in the answer you linked to will work on a console in the local development server (but will likely take some tweaking – you’ll have to chain it with the existing App Engine logging handler), but won’t work in production, since in production log entries are output to an HTML page in your admin console.

You can, however, filter by log level in the admin console.

I don’t believe that you should create a logger subclass just for this – airmind’s answer is fine as far as creating a specialised Formatter and specifying its use on a StreamHandler. But there’s no need for a logger subclass. In fact airmind’s use of a logger class adds a handler to every logger created, which is not what you want.

The solution airmind gave only works for terminals which support ANSI escape sequences – are you sure that your console does support them?

Install colorlog

(don’t forget to put colorlog in INSTALLED_APPS)

create ‘colored’ in your formatter

 'colored': {
        '()': 'colorlog.ColoredFormatter',
        'format': "%(log_color)s %(levelname)-8s %(asctime)s %(module)s %(reset)s %(blue)s%(message)s",
    }

full example of logger in settings.py

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
    'colored': {
        '()': 'colorlog.ColoredFormatter',
        'format': "%(log_color)s %(levelname)-8s %(asctime)s %(module)s %(reset)s %(blue)s%(message)s",
    }
},
'handlers': {
    'console': {
        'class': 'logging.StreamHandler',
        'formatter': 'colored'
    },
},
'root': {
    'handlers': ['console'],
    'level': 'WARNING',
},
'loggers': {
    'django': {
        'handlers': ['console'],
        'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
        'propagate': False,
     },
  },
}

Here is my solution for Django with colorlog. It just colors simple django messages as they are. You only need to put it in your settings.py.

pip install colorlog

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'special': {
            '()': 'colorlog.ColoredFormatter',
            'format': '%(log_color)s[%(asctime)s] %(message)s'
        }
    },
    'filters': {
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        },
    },
    'handlers': {
        'console': {
            'level': 'INFO',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
            'formatter': 'special'
        }
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'propagate': True,
        }
    }
}

Screen