[Fixed]-Unit testing elastic search inside Django app

by

3👍

✅

You can write some basic integration tests that are actually calling elasticsearch and then cover remaining related methods inside views, models etc. with unit tests. This way you can test everything without having to mock elasticsearch, and discover possible errors/behavior that you wouldn’t otherwise.

We are using django haystack (https://github.com/django-haystack/django-haystack) which provides a unified api for search backends including elasticsearch and also the following management commands:

  • build_solr_schema
  • clear_index
  • haystack_info
  • rebuild_index
  • update_index

You can wrap the above inside your base integration test class to manage search indexes. E.g.:

from django.core.management import call_command
from django.test import TestCase
from model_mommy import mommy


class IntegrationTestCase(TestCase):
    def rebuild_index(self):
        call_command('rebuild_index', verbosity=0, interactive=False)

class IntegrationTestUsers(IntegrationTestCase):
    def test_search_users_in_elasticsearch(self):
        user = mommy.make(User, first_name='John', last_name='Smith')
        user = mommy.make(User, first_name='Andy', last_name='Smith')
        user = mommy.make(User, first_name='Jane', last_name='Smith')
        self.rebuild_index()

        # Search api and verify results e.g. /api/users/?last_name=smith
👤fips

2👍

My personal opinion is that tests that call the real Elasticsearch will generate more value and confidence than tests that mock it out. And it’s very doable to setup test infrastructure in Django so that these tests are isolated:

from example.elasticsearch import search_media


def index_test_fixtures(es, index_name, data):
    created = es.index(index=index_name, body=data)
    assert created["result"] == "created"
    es.indices.refresh(index_name)


class TestElasticsearch:
    def test_elasticsearch(self, elasticsearch):
        index_test_fixtures(
            elasticsearch,
            "movie",
            {
                "slug": "episode-5",
                "title": "Star Wars: Episode V - The Empire Strikes Back",
                "description": "After the Rebels are brutally overpowered by the Empire on the ice planet Hoth, Luke Skywalker begins Jedi training with Yoda, while his friends are pursued by Darth Vader and a bounty hunter named Boba Fett all over the galaxy.",
            },
        )

        results = search_media("Star Wars")
        assert results[0]["slug"] == "episode-5"

I wrote a bit about how to setup all the configuration (docker-compose, pytest, elasticsearch, etc) in my personal blog: https://yanglinzhao.com/posts/test-elasticsearch-in-django. There’s also a working example demo https://github.com/yanglinz/django-pytest-elasticsearch-example if that helps.

👤Yanglin Zhao

Leave a comment