[Fixed]-Django Prefetch with custom queryset which uses managers method

by

9👍

I haven’t tested this directly, but you should not invoke a method or filter again in the loop, as prefetch_related has already attached the data. So either of these should work:

pizzas = Pizza.objects.prefetch_related(
    Prefetch('toppings', queryset=Topping.objects.filter(is_vegetarian=True))
)
for pizza in pizzas:
    print(pizza.toppings.all()) # uses prefetched queryset

or

pizzas = Pizza.objects.prefetch_related(
    Prefetch('toppings', queryset=Topping.objects.filter_vegetarian(),
             to_attr="veg_toppings"))
for pizza in pizzas:
    print(pizza.toppings.veg_toppings)

Your examples do not work because they invoke another queryset, and this cannot be compared to the prefetched one to determine if it would be same.

It also says so in the docs:

The prefetch_related('toppings') implied pizza.toppings.all(), but pizza.toppings.filter() is a new and different query. The prefetched cache can’t help here; in fact it hurts performance, since you have done a database query that you haven’t used.

and

Using to_attr is recommended when filtering down the prefetch result as it is less ambiguous than storing a filtered result in the related manager’s cache.

👤webjunkie

1👍

This implementation:

class ToppingManager(models.Manager):
    def filter_vegetarian(self):
        return self.filter(is_vegetarian=True)

Looks non-standard. docs look like they do a safer method of modifying the super-class method for this sort of lazy-eval stuff. If I rewrite your method in that style, it would look like:

class ToppingManager(models.Manager):
    def filter_vegetarian(self):
        return super(ToppingManager, self).get_queryset().filter(is_vegetarian=True)

You wouldn’t strictly need the super() here, but safer to use it because you should know that you want to start with the models.Manager get_queryset method.

Doing a brief test of this in my own environment, I find that it works feeding into Prefetch without triggering queries on each item. I do not have any reason to believe this would not work for the problem here.

However, I’m also inclined to believe that specifying to_attr in webjunkie’s answer may also be necessary.

👤AlanSE

Leave a comment