[Django]-Django multiple forms with modelchoicefield -> too many queries

by

1👍

✅

I subclassed ChoiceField as suggested by GwynBleidD and it works sufficiently for now. 

class ListModelChoiceField(forms.ChoiceField):
    """
    special field using list instead of queryset as choices
    """
    def __init__(self, model, *args, **kwargs):
        self.model = model
        super(ListModelChoiceField, self).__init__(*args, **kwargs)

    def to_python(self, value):

        if value in self.empty_values:
            return None
        try:
            value = self.model.objects.get(id=value)
        except self.model.DoesNotExist:
            raise ValidationError(self.error_messages['invalid_choice'], code='invalid_choice')
        return value


    def valid_value(self, value):
        "Check to see if the provided value is a valid choice"

        if any(value.id == int(choice[0]) for choice in self.choices):
            return True
        return False
👤Dalbenn

3👍

ModelChoiceField is an subclass of ChoiceField in which “normal” choices are replaced with iterator that will iterate through provided queryset. Also there is customized ‘to_python’ method that will return actual object instead of it’s pk. Unfortunately that iterator will reset queryset and hit database once again for each choice field, even if they are sharing queryset

What you need to do is subclass ChoiceField and mimic behaviour of ModelChoiceField with one difference: it will take static choices list instead of queryset. That choices list you will build in your view once for all fields (or forms).

👤GwynBleidD

3👍

A maybe less invasive hack, using an overload of Django’s FormSets and keeping the base form untouched (i.e. keeping the ModelChoiceFields with their dynamic queryset):

from django import forms

class OptimFormSet( forms.BaseFormSet ):
    """
    FormSet with minimized number of SQL queries for ModelChoiceFields
    """

    def __init__( self, *args, modelchoicefields_qs=None, **kwargs ):
        """
        Overload the ModelChoiceField querysets by a common queryset per
        field, with dummy .all() and .iterator() methods to avoid multiple
        queries when filling the (repeated) choices fields.

        Parameters
        ----------

        modelchoicefields_qs :          dict
            Dictionary of modelchoicefield querysets. If ``None``, the
            modelchoicefields are identified internally

        """

        # Init the formset
        super( OptimFormSet, self ).__init__( *args, **kwargs )

        if modelchoicefields_qs is None and len( self.forms ) > 0:
            # Store querysets of modelchoicefields
            modelchoicefields_qs = {}
            first_form = self.forms[0]
            for key in first_form.fields:
                if isinstance( first_form.fields[key], forms.ModelChoiceField ):
                    modelchoicefields_qs[key] = first_form.fields[key].queryset

        # Django calls .queryset.all() before iterating over the queried objects
        # to render the select boxes. This clones the querysets and multiplies
        # the queries for nothing.
        # Hence, overload the querysets' .all() method to avoid cloning querysets
        # in ModelChoiceField. Simply return the queryset itself with a lambda function.
        # Django also calls .queryset.iterator() as an optimization which
        # doesn't make sense for formsets. Hence, overload .iterator as well.
        if modelchoicefields_qs:
            for qs in modelchoicefields_qs.values():
                qs.all = lambda local_qs=qs: local_qs  # use a default value of qs to pass from late to immediate binding (so that the last qs is not used for all lambda's)
                qs.iterator = qs.all

            # Apply the common (non-cloning) querysets to all the forms
            for form in self.forms:
                for key in modelchoicefields_qs:
                    form.fields[key].queryset = modelchoicefields_qs[key]

In your view, you then call:

formset_class = forms.formset_factory( form=MyBaseForm, formset=OptimFormSet )
formset = formset_class()

And then render your template with the formset as described in Django’s doc.

Note that on form validation, you will still have 1 query per ModelChoiceField instance, but limited to a single primary key value each time. That is also the case with the accepted answer. To avoid that, the to_python method should use the existing queryset, which would make the hack even hackier.

This works at least for Django 1.11.

👤jxrossel

Leave a comment