[Django]-How to tell Django, that memcached running with item-size larger than default?

7👍

From Maximum size of object that can be saved in memcached with memcache.py:

There are two entries about that in the memcached FAQ :

  • What is the maximum data size you can store? Why are items limited to
    1 megabyte in size? The answer to the first one is (quoting, emphasis
    mine) :

  • The maximum size of a value you can store in memcached is 1 megabyte.
    If your data is larger, consider clientside compression or splitting
    the value up into multiple keys.

So I’m guessing your 11MB file is quite too big to fit in one
memcached entry.

If you do really want to cache larger objects, you will have to subclass Django’s MemcachedCache as it doesn’t allow you to pass in options:

self._client = self._lib.Client(self._servers, pickleProtocol=pickle.HIGHEST_PROTOCOL)

Example subclass implementation:

from django.core.cache.backends.memcached import MemcachedCache

class LargeMemcachedCache(MemcachedCache):
    "Memcached cache for large objects"

    @property
    def _cache(self):
        if getattr(self, '_client', None) is None:
            self._client = self._lib.Client(self._servers, 
                           pickleProtocol=pickle.HIGHEST_PROTOCOL, 
                           server_max_value_length = 1024*1024*10)
        return self._client
👤Bouke

3👍

In more recent Django versions you don’t need to subclass the cache class, you can instead specify the arguments passed to the cache class’s constructor in OPTIONS:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
        'KEY_FUNCTION': 'some.path.to.a.function',
        'OPTIONS': {
            'server_max_value_length': 1024 * 1024 * 10
        }
    }
}

See docs.

Note that you must also increase the size in memcached itself by adding the following line:

-I 10m

to /etc/memcached.conf and restarting it:

sudo service memcached restart

Leave a comment