[Django]-Mocking Django Storages Model ImageField backend S3

by

21👍

✅

I just had this problem too. I got much faster tests by using dj-inmemorystorage.

The quick way of setting this up is by creating a test_settings.py in the same folder as your settings:

from settings import *
DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'

…and calling ./manage.py test --settings=project.test_settings to run the tests.

My preferred way is to set up a custom test runner:

In project/test_runner.py:

from django.conf import settings
from django.test.runner import DiscoverRunner

class FastTestRunner(DiscoverRunner):
    def setup_test_environment(self):
        super(FastTestRunner, self).setup_test_environment()
        # Don't write files
        settings.DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
        # Bonus: Use a faster password hasher. This REALLY helps.
        settings.PASSWORD_HASHERS = (
            'django.contrib.auth.hashers.MD5PasswordHasher',
        )

Note: This also sets the PASSWORD_HASHER, because it significantly improves User creation time. This should NOT be set in production.

In project/settings.py:

TEST_RUNNER = 'project.test_runner.FastTestRunner'

The requirements:

pip install dj-inmemorystorage

UPDATE: changed from django-inmemorystorage to dj-inmemorystorage.

UPDATE 2: Removed django-discover-runner, as it’s now the default test runner in django, and fixed the link to the PASSWORD_HASHER related blog post.

👤meshy

5👍

I also use S3Boto but for testing, I prefer having custom settings which include using the file system storage. You can have your custom settings declared in a file which you can then import and use in your test cases. Even so, you can mock the file storage so that the files are not actually written to disk.

Here’s a sample test_settings.py

# myproject/myproject/test_settings.py

from django.test import override_settings

common_settings = override_settings(
    DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
    PASSWORD_HASHERS=(
        'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
    ),
)

Usage:


from django.test import TestCase

from myproject.test_settings import common_settings

@common_settings
class MyTestCase(TestCase):
    """Tests go here"""

On mocking the file system storage you can check out my answer here on SO.

👤omushpapa

4👍

Just ran into this as well so I thought I’d put my solution up. My solution uses Mock

import mock
from django.core.files.storage import FileSystemStorage
from django.test import TestCase

class ATestCase(TestCase):
    def setUp(self):
        # Stuff Happens

    def tearDown(self):
        # more Stuff

    @mock.patch('storages.backends.s3boto.S3BotoStorage', FileSystemStorage)
    def test_file_stuff(self):
        self.assertMagicPonies(True)

Some gotchas – make sure you have a sane MEDIA_ROOT setup in the settings. as of django 1.4, you can’t use the testing context manager to override MEDIA_ROOT, so you need a separate settings config for it (https://code.djangoproject.com/ticket/17787) This was fixed in 1.6. Also, make sure your upload_to works in normal filesystem, or you will get permission errors.

👤yarbelk

1👍

I know this is a rather old thread, but Django 4.2 now offers a much simpler way with the InMemoryStorage. You just have to override the settings to declare it, and all media file accesses will be mocked in memory:

@override_settings(STORAGES={
    "default": {
        "BACKEND": "django.core.files.storage.memory.InMemoryStorage",
    },
})
class StorageTest(TestCase):
    """Any upload test will only use memory here"""
    ...
👤Serge Ballesta

0👍

I would propose to use the standard Django Storage for testing, where you can a define custom path for storage and cleanup that path in your test suite once your done. Both the storage and the path can be set in the settings and overwritten for testing.

👤schacki

Leave a comment