Nginx caching with django

For a small django site I run, I'd like to add nginx caching in front of my django server. Here's the setup I ended up with.

The pages I want to cache are available for both anonymous and logged in users. The majority of traffic is from anonymous users and since they all get the same contents I'd like to serve it from the cache. Contents for logged in users should never be cached.

Because we show different content to logged in users, and therefore check e.g. request.user.is_authenticated, the SessionMiddleware correctly sets a Vary: cookie header.

As an extra spanner in the works, we use google analytics which adds its own cookies. These aren't used by the django server but means that even anonymous visitors submit cookies with their request.

By default, nginx respects the vary header and adds the cookies to the cache key, effectively giving everyone their own cache entry which is useless.

The fix is to ignore the Vary header, and instead bypass the cache for anyone with a sessionid cookie.

The nginx blog had a nice guide on how to configure caching where i got most of the settings from, though it unhelpfully doesn't mention the need for a proxy_cache_valid entry.

Final nginx config:

proxy_cache_path /tmp/nginx-cache levels=1:2 keys_zone=my_cache:10m max_size=1g inactive=60m use_temp_path=off;

server {
    ...
    proxy_cache my_cache;
    proxy_cache_valid 200 10m;
    # ignore "Vary: cookie" from django and instead use
    # cache_bypass for the sessionid cookie
    proxy_ignore_headers Vary;
    proxy_cache_bypass $cookie_sessionid;
    # add a custom header with cache status, for debugging
    add_header X-Cache-Status $upstream_cache_status;
}

Result:

$ curl -s -i  -b _utm=foo https://site | head
HTTP/1.1 200 OK
...
X-Cache-Status: HIT

$ curl -s -i  -b sessionid=bar https://site | head
HTTP/1.1 200 OK
...
X-Cache-Status: BYPASS

Other articles

Watch out for Mock(spec)

Thu 28 January 2016

tags: development python

Despite mocks being a bit of a smell, they are handy and I do find myself using them from time to time. As a cheap way to get some extra checks, I try to always make use of autospec=True in my mock.patch() calls.

Sometimes instead of patch I …
read more
Strace to the rescue

Sat 18 October 2014

tags: development python

Recently I was debugging a strange error when using the eventlet support in the new the new coverage 4.0 alpha. The issue manifested itself as a network connection problem: Name or service not known. This was confusing, since the host it was trying to connect to was localhost. How …
read more
Custom redirects in the django admin

Sun 20 January 2013

tags: django python

On one of my sites i have a bunch of different models for various content resources, e.g. articles, videos and podcasts. For admins I wanted to add an "edit this item" link to the menu when browsing the site. Pretty easy: check permissions and add a link to the …
read more
Dynamically creating django models for m2m relationships
Sun 09 September 2012

tags: django python

Occasionally when using a django ManyToManyField I find myself wanting to query the join table directly. Consider for example the models below:
```
class Category(models.Model):
    name = models.TextField()

class Entry(models.Model):
    data = models.TextField()
    categories = models.ManyToManyField(Category)
```
Suppose I have an entry, and want to find entries …
read more
Automating save-build-refresh on os x

Sun 15 April 2012

tags: osx pelican python

This blog is powered by Pelican. When I first started out, I was composing in vim, switching over to a terminal to build, and finally to my browser to preview the result. Clearly this becomes tedious pretty quickly.

I was already familiar with tools to watch folders for changes and …
read more
Introspecting django urls for fun and profit

Sat 14 April 2012

tags: django python

One of the projects I work on has a fairly large and old code base, and is unfortunately somewhat lacking in the test suite department. We are working to improve this, but in the medium term we are stuck with a fair amount of manual testing.

As part of some …
read more
Show number of queries in runserver output

Sat 03 March 2012

tags: django python

Sometimes I'd like to know how many database queries were used to generate a particular view, but can't be bothered with the full weight of the (otherwise excellent) django-debug-toolbar.

The toolbar does a lot of logging and slows down rendering. Also, by default I want 'INTERCEPT_REDIRECTS': False, but then I …
read more