[Solved]-How to use django-compressor behind load balancer?

by

4👍

✅

If you want to have identical cache files you must be sure that you have identical input on both servers.

You should check:

  • if code in {% compress %}...{% endcompress %} is identical on both servers (if you deploying to both servers at once it should be)
  • if all your .css/.js files are identical on both servers (if you deploying to both servers at once it should be)
  • if mtime (modify time) of your .css/.js files is identical on both servers (your deployment script may affect those and set current date)

If all of this requirements ale satisfied, generated files should be identical (content and names).

You can check mtime using “stat” unix command.

Answers to your questions:

  • Purpose of the cache in django-compressor is to reduce reads from file system.
  • Generated file with combined code is stored only on filesystem.

Edit:

I have checked it on one of my websites behind load balancer. I have different file names for .css files, but they are identical for .js.

For .css files I use preprocessor (http://lesscss.org/), so it affects mtime.

Edit (after topic developed):

What is in the cache?

Due to documentation django-compressor stores in the cache two different things:

  • mtime of cached files (rechecked every COMPRESS_MTIME_DELAY seconds)

  • full generated code ie.:

    <link rel=”stylesheet” href=”http://cdn.inprl.pl/CACHE/css/117f97d818b8.css” type=”text/css”>

Due to following cache usage django-compressor reduces number of reads to filesystem to 0. This is essential for page speed, because reading from memory is hundreds times faster than reading from filesystem. Also filesystem is very often bottleneck.

How it is stored in the cache?

django-compress is storing code in the cache using generated key. Key is generated from:

  • code in {% compress %}...{% endcompress %}
  • mtime of files mentioned in {% compress %}...{% endcompress %}

So those must be the same on all servers if you want to have consistent responses.

PS.

Please check constrains (like mtime) on your server and post here information if they match.

I will be fixing the same problem on my site probably next week, I will post additional details then.

👤Tomasz Wysocki

12👍

In the develop branch there is a new option to change the css hashing method.
https://github.com/jezdez/django_compressor

See line 61 in filters/css_default.py

The settings I’m using:

COMPRESS_ENABLED = True
COMPRESS_OFFLINE = False
COMPRESS_STORAGE = 'compressor.storage.GzipCompressorFileStorage'
COMPRESS_CSS_HASHING_METHOD = 'hash' # not using mtime since it differs between servers.

There is no option like this for js files since their hash key is never generated using mtime anyway.

This works perfectly behind my loadbalancer.

When this is written, the following is the latest commit in the develop branch: https://github.com/jezdez/django_compressor/commit/d48bc5f45d5a55b0f826eb605ccf09a6bf33fcb9

👤demux

0👍

What you should do is to put all your compressed files on a Storage outside your computing instances that are behind the load balancer. For example, use Amazon S3 to store all your files on another subdomain than the rest of your application.

So http://myapp.com is pointing to your load balancer and http://s3.myapp.com is pointing to your storage, such as Amazon S3. You will not have to worry about the storing multiple different versions on different instances.

Here you can find a complete guide of how to setup Amazon S3, Gzip Compression and django-compressor with Django.

👤Marcus Lind

Leave a comment