[Django]-Prevent django admin from running SELECT COUNT(*) on the list form

Django 1.8 lets you disable this by setting show_full_result_count = False.

https://docs.djangoproject.com/en/stable/ref/contrib/admin/#django.contrib.admin.ModelAdmin.show_full_result_count

I found Nova’s answer very helpful, but i use postgres. I modified it slightly to work for postgres with some slight alterations to handle table namespaces, and slightly different “detect postgres” logic.

Here’s the pg version.

class ApproxCountPgQuerySet(models.query.QuerySet):
  """approximate unconstrained count(*) with reltuples from pg_class"""

  def count(self):
      if self._result_cache is not None and not self._iter:
          return len(self._result_cache)

      if hasattr(connections[self.db].client.connection, 'pg_version'):
          query = self.query
          if (not query.where and query.high_mark is None and query.low_mark == 0 and
              not query.select and not query.group_by and not query.having and not query.distinct):
              # If query has no constraints, we would be simply doing
              # "SELECT COUNT(*) FROM foo". Monkey patch so the we get an approximation instead.
              parts = [p.strip('"') for p in self.model._meta.db_table.split('.')]
              cursor = connections[self.db].cursor()
              if len(parts) == 1:
                  cursor.execute("select reltuples::bigint FROM pg_class WHERE relname = %s", parts)
              else:
                  cursor.execute("select reltuples::bigint FROM pg_class c JOIN pg_namespace n on (c.relnamespace = n.oid) WHERE n.nspname = %s AND c.relname = %s", parts)
          return cursor.fetchall()[0][0]
      return self.query.get_count(using=self.db)

The Nova’s solution (ApproxCountQuerySet) works great, however in newer versions of Django queryset method got replaced with get_queryset, so it now should be:

class MyAdmin(admin.ModelAdmin):

    def get_queryset(self, request):
        qs = super(MyAdmin, self).get_queryset(request)
        return qs._clone(klass=ApproxCountQuerySet)

If this is a serious problem you may have to take Drastic Actions™.

Looking at the code for a 1.3.1 install, I see that the admin code is using the paginator returned by get_paginator(). The default paginator class appears to be in django/core/paginator.py. That class has a private value called _count which is set in Paginator._get_count() (line 120 in my copy). This in turn is used to set a property of the Paginator class called count. I think that _get_count() is your target. Now the stage is set.

You have a couple of options:

1. Directly modify the source. I do not recommend this, but since you seem to be stuck at 1.2.7 you may find that it is the most expedient. Remember to document this change! Future maintainers (including possibly yourself) will thank you for the heads up.

2. Monkeypatch the class. This is better than direct modification because a) if you don’t like the change you just comment out the monkeypatch, and b) it is more likely to work with future versions of Django. I have a monkeypatch going back over 4 years because they still have not fixed a bug in the template variable _resolve_lookup() code that doesn’t recognize callables at the top level of evaluation, only at lower levels. Although the patch (which wraps the method of a class) was written against 0.97-pre, it still works at 1.3.1.

I did not spend the time to figure out exactly what changes you would have to make for your problem, but it might be along the lines of adding a _approx_count member to appropriate classes class META and then testing to see if that attr exists. If it does and is None then you do the sql.count() and set it. You might also need to reset it if you are on (or near) the last page of the list. Contact me if you need a little more help on this; my email is in my profile.

It is possible to change the default paginator used by the admin class. Here’s one that caches the result for a short period of time: https://gist.github.com/e4c5/6852723

I managed to create a custom paginator that shows the current page numbe, a next button and a show full count link. It allows for the use of the original paginator if needed.

The trick used is to take per_page + 1 elements from db in order to see if we have more elements and then provide a fake count.

Let’s say that we want the the third page and the page has 25 elements => We want object_list[50:75]. When calling Paginator.count the queryset will be evaluated for object_list[50:76](note that we take 75+1 elements) and then return either the count as 76 if we got 25+1 elements from db or 50 + the number of elements received if we didn’t received 26 elements.

TL;DR:
I’ve created a mixin for the ModelAdmin:

from django.core.paginator import Paginator
from django.utils.functional import cached_property


class FastCountPaginator(Paginator):
    """A faster paginator implementation than the Paginator. Paginator is slow
    mainly because QuerySet.count() is expensive on large queries.

    The idea is to use the requested page to generate a 'fake' count. In
    order to see if the page is the final one  it queries n+1 elements
    from db then reports the count as page_number * per_page + received_elements.
    """

    use_fast_pagination = True

    def __init__(self, page_number, *args, **kwargs):
        self.page_number = page_number
        super(FastCountPaginator, self).__init__(*args, **kwargs)

    @cached_property
    def count(self):
        # Populate the object list when count is called. As this is a cached property,
        # it will be called only once per instance
        return self.populate_object_list()

    def page(self, page_number):
        """Return a Page object for the given 1-based page number."""
        page_number = self.validate_number(page_number)
        return self._get_page(self.object_list, page_number, self)

    def populate_object_list(self):
        # converts queryset object_list to a list and return the number of elements until there
        # the trick is to get per_page elements + 1 in order to see if the next page exists.
        bottom = self.page_number * self.per_page
        # get one more object than needed to see if we should show next page
        top = bottom + self.per_page + 1
        object_list = list(self.object_list[bottom:top])
        # not the last page
        if len(object_list) == self.per_page + 1:
            object_list = object_list[:-1]
        else:
            top = bottom + len(object_list)
        self.object_list = object_list
        return top


class ModelAdminFastPaginationMixin:
    show_full_result_count = False  # prevents root_queryset.count() call

    def changelist_view(self, request, extra_context=None):
        # strip count_all query parameter from the request before it is processed
        # this allows all links to be generated like this parameter was not present and without raising errors
        request.GET = request.GET.copy()
        request.GET.paginator_count_all = request.GET.pop('count_all', False)

        return super().changelist_view(request, extra_context)

    def get_paginator(self, request, queryset, per_page, orphans=0, allow_empty_first_page=True):
        # use the normal paginator if we want to count all the ads
        if hasattr(request.GET, 'paginator_count_all') and request.GET.paginator_count_all:
            return Paginator(queryset, per_page, orphans, allow_empty_first_page)
        page = self._validate_page_number(request.GET.get('p', '0'))
        return FastCountPaginator(page, queryset, per_page, orphans, allow_empty_first_page)

    def _validate_page_number(self, number):
        # taken from Paginator.validate_number and adjusted
        try:
            if isinstance(number, float) and not number.is_integer():
                raise ValueError
            number = int(number)
        except (TypeError, ValueError):
            return 0
        if number < 1:
            number = 0
        return number

The pagination.html template:

{% if cl and cl.paginator and cl.paginator.use_fast_pagination %}
    {# Fast paginator with only next button and show the total number of results#}
    {% load admin_list %}
    {% load i18n %}
    {% load admin_templatetags %}
    <p class="paginator">
        {% if pagination_required %}
            {% for i in page_range %}
                {% if forloop.last %}
                    {% fast_paginator_number cl i 'Next' %}
                {% else %}
                    {% fast_paginator_number cl i %}
                {% endif %}
            {% endfor %}
        {% endif %}
        {% show_count_all_link cl "showall" %}
    </p>
{% else %}
    {#  use the default pagination template if we are not using the FastPaginator  #}
    {% include "admin/pagination.html" %}
{% endif %}

and templatetags used:

from django import template
from django.contrib.admin.views.main import PAGE_VAR
from django.utils.html import format_html
from django.utils.safestring import mark_safe

register = template.Library()

DOT = '.'


@register.simple_tag
def fast_paginator_number(cl, i, text_display=None):
    """Generate an individual page index link in a paginated list.

    Allows to change the link text by setting text_display
    """
    if i == DOT:
        return '… '
    elif i == cl.page_num:
        return format_html('<span class="this-page">{}</span> ', i + 1)
    else:
        return format_html(
            '<a href="{}"{}>{}</a> ',
            cl.get_query_string({PAGE_VAR: i}),
            mark_safe(' class="end"' if i == cl.paginator.num_pages - 1 else ''),
            text_display if text_display else i + 1,
        )


@register.simple_tag
def show_count_all_link(cl, css_class='', text_display='Show the total number of results'):
    """Generate a button that toggles between FastPaginator and the normal
    Paginator."""
    return format_html(
        '<a href="{}"{}>{}</a> ',
        cl.get_query_string({PAGE_VAR: cl.page_num, 'count_all': True}),
        mark_safe(f' class="{css_class}"' if css_class else ''),
        text_display,
    )

You can use it this way:

class MyVeryLargeModelAdmin(ModelAdminFastPaginationMixin, admin.ModelAdmin):
# ...

Or an even simpler version that does not show the Next button and Show the total number of results :

from django.core.paginator import Paginator
from django.utils.functional import cached_property


class FastCountPaginator(Paginator):
    """A faster paginator implementation than the Paginator. Paginator is slow
    mainly because QuerySet.count() is expensive on large queries.

    The idea is to use the requested page to generate a 'fake' count. In
    order to see if the page is the final one  it queries n+1 elements
    from db then reports the count as page_number * per_page + received_elements.
    """

    use_fast_pagination = True

    def __init__(self, page_number, *args, **kwargs):
        self.page_number = page_number
        super(FastCountPaginator, self).__init__(*args, **kwargs)

    @cached_property
    def count(self):
        # Populate the object list when count is called. As this is a cached property,
        # it will be called only once per instance
        return self.populate_object_list()

    def page(self, page_number):
        """Return a Page object for the given 1-based page number."""
        page_number = self.validate_number(page_number)
        return self._get_page(self.object_list, page_number, self)

    def populate_object_list(self):
        # converts queryset object_list to a list and return the number of elements until there
        # the trick is to get per_page elements + 1 in order to see if the next page exists.
        bottom = self.page_number * self.per_page
        # get one more object than needed to see if we should show next page
        top = bottom + self.per_page + 1
        object_list = list(self.object_list[bottom:top])
        # not the last page
        if len(object_list) == self.per_page + 1:
            object_list = object_list[:-1]
        else:
            top = bottom + len(object_list)
        self.object_list = object_list
        return top


class ModelAdminFastPaginationMixin:
    show_full_result_count = False  # prevents root_queryset.count() call

    def get_paginator(self, request, queryset, per_page, orphans=0, allow_empty_first_page=True):
        page = self._validate_page_number(request.GET.get('p', '0'))
        return FastCountPaginator(page, queryset, per_page, orphans, allow_empty_first_page)

    def _validate_page_number(self, number):
        # taken from Paginator.validate_number and adjusted
        try:
            if isinstance(number, float) and not number.is_integer():
                raise ValueError
            number = int(number)
        except (TypeError, ValueError):
            return 0
        if number < 1:
            number = 0
        return number