[Django]-Django Cache: Use Memcached and fallback to FileSystem

by

8👍

✅

It sounds like you want Memcached to behave like normal RAM and page to disk when it is full. It doesn’t by default, but you can imitate it by writing your own cache backend as @AlexanderAfanasiev mentioned. The implementation would be something along these lines:

First, define three caches:

CACHES = {
    'default': {
        'BACKEND': 'myapp.cache.DoubleCache',
    },
    'memcached': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
    },
    'filecache': {
        'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
        'LOCATION': '/foo/bar',
    }
}

Then, in myapp/cache.py:

from django.core.cache.backends.base import BaseCache
from django.core.cache import get_cache, cache
mem_cache = get_cache('memcached')
file_cache = get_cache('filecache')

class DoubleCache(BaseCache):

    def get(self, key, default=None, version=None):
        result = mem_cache.get(key, default=default, version=version)
        if result:
            return result
        else:
            return file_cache.get(key, default=default, version=version)

    def set(self, key, value, timeout=None, version=None, client=None, _add_only=False):
        memcache_result = mem_cache.set(key, value, timeout=timeout, version=version, client=client, _add_only=_add_only)
        file_result = file_cache.set(key, value, timeout=timeout, version=version, client=client, _add_only=_add_only)
        return memcache_result

This will always store values in both caches. It will retrieve values from Memcached, and if it misses, will try the file_cache. That means that Memcached can manage its own roll-off and only the oldest hits will have to fall back to file_cache. This is what you’d want.

Of course, you’ll also have to implement the rest of the cache functions like delete(), get_many(), etc. Hopefully this gets you on the right path.

👤krimkus

-1👍

Exploit TCP/IP. With minor effort this logic can be extended to create nice multicache backend with fallbacks.

import socket
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

try:
    socket.connect(('127.0.0.1', 11211))
    socket.close()

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': '127.0.0.1:11211',
        }
    }
except:
    CACHES = { 
        'default': {
            'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
            'LOCATION': '/srv/django_cache',
        }
    }
👤Sandeep

Leave a comment