[Fixed]-Django breaking long lookup names on queries

pep8 says:

The preferred way of wrapping long lines is by using Python’s implied line continuation inside parentheses, brackets and braces.

That’s what you’ve done, so I think what you’ve got is the most pythonic (or, at least, most pep8ic) way of doing it.

EDIT (A simplified more attractive answer is here. The original detailed answer is below under line.)

I wrote a module django_dot_filter.py that helps to write query filters more naturally and readable. An expression starts with symbol V and names separated by dots:

from django_dot_filter import V

QuerySet.filter(V.myfk.child.onetoone.another.manytomany
                .relation.monster.relationship
                .mycustomlookup == ':P')

I read it like “this unknown Variable” with fields… therefore I use the letter V. That class is really only a symbol that can be followed by dots, methods, operators etc., everything separated by . instead of __.

Standard readable relationship operators like <, <=, == or != are supported, also parentheses and boolean operators &, |, ~.

Queryset.filter((V.some_related.my_field >= 10)
                | ~V.field_x.startswith('Y') & (V.date_field.year() == 2017)
                & V.price.range(10, 100))

Every lookup can be written the classic way like an attribute V.date_field.year == 2017 or like a method V.date_field.year() == 2017. Many lookups are much more readable as a method with an argument V.my_field.regex(r'^[abc]') instead of my_field__regex=value. It is much more readable for me to see a convention that .date() is a lookup method, but .date is a field.

It is no magic. Only a method with arguments or a relationship operator is every times the last part of lookup. A method without arguments is only a symbol that it is a lookup. Something with a value follows always. Expressions are compiled to Q expressions, including boolean expressions. They can be easily reused in similar projects, saved to variables etc., while exclude(..) conditions instead of missing != operator are less reusable.

(No unsupported features are currently known. Some tests have been written. If I get enough feedback, it can a package. It is a little more verbose than the classic good name=value, suitable for simple cases.

A different answer, if you like readable filters with possible long chains of related fields even if they are complicated.

I wrote a simple module django_dot_filter.py today, that allows to use dot syntax for fields on related models and use operators ==, !=, <, <=, >, >= for conditions. It can use bitwise operators ~ | & as boolean operators similarly like Q objects use, but due to operators priority, the comparision must be enclosed in parentheses. It is inspired by syntax used in SQLAlchemy and Pandas.

doc string:

class V(...):
    """
    Syntax suger for more readable queryset filters with "." instead "__"

    The name "V" can be understand like "variable", because a shortcut for
    "field" is occupied yet.
    The syntax is very similar to SQLAlchemy or Pandas.
    Operators < <= == != >= > are supperted in filters.

    >>> from django_dot_filter import V
    >>>
    >>> qs = Product.objects.filter(V.category.name == 'books',
    >>>                             V.name >= 'B', V.name < 'F',
    >>>                             (V.price < 15) | (V.date_created != today),
    >>>                             ~V.option.in_(['ABC', 'XYZ'])
    >>>                             )

    This is the same as

    >>> qs = Product.objects.filter(category__name='books',
    >>>                             name__gte='B', name__lt='F',
    >>>                             Q(price__lt=15) | ~Q(date_created=today),
    >>>                             ~Q(option__in=['ABC', 'XYZ'])
    >>>                             )
    """

(The class “V” automatically creates a new instance if used with dot. All elements are compiled to Q expression after relational operator or after .in_(iterable) method and the instance is deleted again.)

Some examples from tests

    #       this is V. syntax         compiled Q syntax
    test_eq(V.a.b.c == 1,             Q(a__b__c=1))
    test_eq(V.a == 1,                 Q(a=1))
    test_eq(V.a != 1,                 ~Q(a=1))
    test_eq(V.a < 2,                  Q(a__lt=2))
    test_eq(V.a <= 3,                 Q(a__lte=3))
    test_eq(V.a > 'abc',              Q(a__gt='abc'))
    test_eq(V.a >= 3.14,              Q(a__gte=3.14))
    test_eq((V.a == 1) & (V.b == 2),  Q(a=1) & Q(b=2))
    test_eq((V.a == 1) | (V.b == 2),  Q(a=1) | Q(b=2))
    test_eq((V.a == 1) | ~(V.b == 2), Q(a=1) | ~Q(b=2))
    # method "in_(..)" is used because the word "in" is reserved.
    test_eq(V.first_name.in_([1, 2]), Q(first_name__in=[1, 2]))
    test_eq(~V.a.in_(('Tim', 'Joe')), ~Q(a__in=('Tim', 'Joe')))

    # this should be eventually improved to support all lookup
    # functions automatically e.g. by ".contains('abc')" instead of "=="
    test_eq(V.a.contains == 'abc',    Q(a__contains='abc'))

It’s a little joke inspired by your question, but it works. I remember some old discussion of (core developers? vague memories) that the syntax name__operator=value wouldn’t be used again if Django be a new project. It is very concise, but less readable. It is too late to have two official syntaxes.

Django project repository itself configures the max-line-length to 119 characters in .editorconfig and setup.cfg (see the highlighted line in both links). All modern code checkers (pycodestyle, pylint, pyflakes, pep8) and editors understand this configuration and accept it instead of 79 ch.

Reflection: I also prefer to write 79 ch mostly, because it is usually better readable, but in your case a line 119 chars long is definitely more readable than splitting a name of variable to short strings by **{...}. Very short lines are important if Python is used in a bare terminal e.g in a Linux installer script. For Django you usually have a much better local pseudo-terminal or SSH terminal. Github supports 119 chars in every view. Maybe graphic tools used to solve merge conflicts side-by-side could require to scroll horizontally on some monitor. On the other side, a merge or diff tools automatic can fail more frequently because repeated sequences of the same lines are created only by breaking line due to 79 ch rule.

I think the answer depends on how often this kind of things occur.
If you tend to use such keys everywhere across your code a tiny wrapper similar to
the solutions suggested in the answers might be helpful.

If it’s a one-time case (okay, not a one-time but a rare case) I would leave it as is and just mark it with # noqa
indicator to make linters and your code reviewers happy otherwise you’d just considerably hamper readability because it’s not obvious that you’re doing all these tricks just to shorten your key length

BTW google code style suggests a couple of exclusions out of the 79-columns rule
https://google.github.io/styleguide/pyguide.html?showone=Line_length#Line_length (long import statements and URLs in comments) so any rules should be followed wisely

Late to the party (stumbled over this looking for something else) but you can write a function returning a Django Q object to remove ugly long filter/exclude definitions from mainline code.

def q_monster(value):
    return Q(
      **{
        'myfk__child__onetoone__another' 
        '__manytomany__relation__monster' 
        '__relationship__mycustomlookup': value
      }
    )

and the mainline then looks like

from my_Q import q_monster
...
    Queryset.filter( q_monster( ':P'))

An alternative is to package the ugly code as a method of what is probably the only Model it could possibly be applied to, and return the whole filtered queryset. The mainline code would then look like

yukky_objects = MyModel.monster_filter(':P')

and the detail is encapsulated in the nether reaches of the model, where only folks needing to know the ugly details will ever need to look. ( I can’t think of any reason you couldn’t put it in an abstract model base class, either).