Patching Django’s Memcached cache.set() Method

Django’s memcached set method looks innocuous enough:

    def set(self, key, value, timeout=0, version=None):
        key = self.make_key(key, version=version)
        self._cache.set(key, value, self._get_memcache_timeout(timeout))

However it suppresses the return value of the underlying call to memcached’s set command. In most cases this is fine – calls to memcached usually never fail. However, in those rare cases where something goes wrong, it makes debugging far more difficult. I recently had one of those “This cannot happen” moments, which ultimately turned out to be downstream of a silent failed cache.set() call. It would be much nicer if the set call either threw an exception or returned an error if the set failed. (I suggested this to the django maintainers a while back but was summarily rejected.)

Here are two ways to work around this. The first is to patch the set method. Just put this somewhere in your code:

def patched_memcached_set(self, key, value, timeout=0, version=None):
    key = self.make_key(key, version=version)
    return self._cache.set(key, value, self._get_memcache_timeout(timeout))

CacheClass.set = patched_memcached_set

Now you can check the return value of cache.set() – True means success, False means failure.

from django.core.cache import cache
import logging

if not cache.set(key, value):
    logging.error('Failed to set key/val %s/%s in memcache!', key, value)
    # Recovery code

The second way is to use your own cache backend.

class MemcachedCache(BaseMemcachedCache):
    """A (slightly hacked) implementation of a cache binding using python-memcached"""
    def __init__(self, server, params):
        import memcache

        super(MemcachedCache, self).__init__(server, params,
                                             library=memcache,
                                             value_not_found_exception=ValueError)

    # Return the value of set so we can tell if the set was successful or not.
    def set(self, key, value, timeout=0, version=None):
        key = self.make_key(key, version=version)
        return self._cache.set(key, value, self._get_memcache_timeout(timeout))

This time you’ll need to do one more thing – install the cache backend in your settings.py file:

CACHES = {
    'default': {
        #'BACKEND' : 'django.core.cache.backends.memcached.MemcachedCache',
        'BACKEND' : 'apps.utils.memcached.MemcachedCache',
        'LOCATION': [
            '10.1.1.101:11211',
            '10.1.1.102:11211',
            '10.1.1.103:11211',
        ],
    }
}

Easy enough. Now, as before, you can check the return value of cache.set() to see if your set actually succeeded.

I strongly recommend doing one of these options if you’re running django and memcached. It’ll save you a ton of time debugging when you hit the rare case of cache.set() failing.