[Answered ]-Improving Django performance with 350000+ regs and complex query

by

2đź‘Ť

âś…

Arthur’s right, the len(stock) isn’t the most efficient way to do that. You could go further along the “easier to ask for forgiveness than permission” route with something like this inside the inner loop:

key = u'%s%s' % (prod.id, place.id)
try:
    stock = stocks.filter(product=prod, place=place, date=date_at)[0]
    quantity = stock.quantity
except IndexError:
    try:
        stock = stocks.filter(product=prod, place=place).order_by('-date')[0]
        quantity = stock.quantity
    except IndexError:
        quantity = 0
stock_values[key] = quantity

I’m not sure how much that would improve it compared to just changing the length check, though I think this should at least restrict it to two queries with LIMIT 1 on them (see Limiting QuerySets).

Mind you, this is still performing a lot of database hits since you could run through that loop almost 50000 times. Optimize how you’re looping and you’re in a better position still.

👤eric

0đź‘Ť

maybe the trick is in that len() method!

follow docs from:

Note: Don’t use len() on QuerySets if all you want to do is determine
the number of records in the set. It’s much more efficient to handle a
count at the database level, using SQL’s SELECT COUNT(*), and Django
provides a count() method for precisely this reason. See count()
below.

So try changing the len to count(), and see if it makes faster!

👤Arthur Neves

Leave a comment