[Django]-How to run Django units test with default and unmanaged database?

by

5👍

✅

We use a setup with managed and unmanaged tables in the same database, which might also work for your use case:

We have a script to generate the test database from two dumps: test_structure.sql an test_fixtures.sql. The former contains the structure of the database at a certain point in time, including all unmanaged tables. The latter contains any data you might need in the unmanaged tables during testing, and the contents of the django_migrations table. We dump test_fixtures.sql using a generated list of COPY (SELECT * FROM {table}) TO STDOUT; statements, for example: COPY (SELECT * FROM obs_00.django_migrations) TO STDOUT WITH NULL '--null--';.

The output from psql -c {copy_statement} is transformed to INSERT statements using a function like this:

def csv2sqlinsert(table_name, data):
    """
    Convert TSV output of  COPY (SELECT * FROM {table}) TO STDOUT
    to                     INSERT INTO {table} VALUES (), ()...();
    """

    def is_int(val):
        try:
            return "{}".format(int(val)) == val
        except ValueError:
            return False

    def column(data):
        if data == "--null--":
            return "null"
        elif is_int(data):
            return data
        else:
            return "'{}'".format(data.replace("'", "''"))  # escape quotes

    rows = [row.split("\t") for row in data.decode().split("\n") if len(row) > 1]

    if len(rows) == 0:
        return f"-- no data for {table_name}\n"

    data = ",\n".join("({})".format(",".join(column(col) for col in row)) for row in rows)

    ret = ""
    ret += f"-- {table_name} ({len(rows)} rows)\n"
    ret += f"INSERT INTO {table_name} VALUES\n{data};\n"

    return ret

In reality this function is more complicated, also simplifying our postgis geometries and truncating large text fields to save space.

creating the test db

Define the test db name in settings_test.py:

DATABASES["default"].update({
    "NAME": "django_test_db",
    "TEST": {"NAME": "django_test_db",},
})

With the two files above, (re)creating the test database looks like this:

dropdb django_test_db
createdb django_test_db
psql -d django_test_db -f test_structure.sql
psql -d django_test_db < test_fixtures.sql

We now have the state of the database at the moment of the dump. Because there might be new migrations, we let django migrate:

./manage.py migrate --settings=settings_test

Running the tests

Now we can run the tests using ./manage.py test --settings=settings_test. Because recreating the database every test run might take a considerable amount of time, adding --keepdb will save you a lot of time waiting for the test database restore procedure.

We’ve amended manage.py like this so we cannot forget:

#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
    if len(sys.argv) > 1 and sys.argv[1] == "test":
        os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings_test")
        cmd = sys.argv + ["--keepdb"]
    else:
        os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
        cmd = sys.argv

    from django.core.management import execute_from_command_line
    execute_from_command_line(cmd)
👤Jieter

Leave a comment