[Answered ]-Django Order QuerySet based on ManyToManyField value

You can use the Sum function [Django docs] with a Conditional Expression [Django docs] to annotate a value according to which you would order:

from django.db.models import Case, Sum, Value, When

Business.objects.annotate(
    order_value=Sum(
        Case(
            When(upvote__gender='Male', then=Value(1)),
            When(upvote__gender='Female', then=Value(-1)),
            default=Value(0)
        )
    )
).order_by('order_value')

The above query would give you Business objects with more upvotes by females first and males later, you can reverse the order by writing .order_by('-order_value') instead.