[Django]-Django Queryset Iterator for Ordered queryset

7đź‘Ť

âś…

Here is mine, with sort function.

By the way the iterator you are using has a “forever loop” when in process the queryset items are modified: deleted or added, even one item.

And below iterator has no useless query on last_pk

def queryset_iterator(queryset, chunksize=10000, key=None):
    key = [key] if isinstance(key, str) else (key or ['pk'])
    counter = 0
    count = chunksize
    while count == chunksize:
        offset = counter - counter % chunksize
        count = 0
        for item in queryset.all().order_by(*key)[offset:offset + chunksize]:
            count += 1
            yield item
        counter += count
        gc.collect()

Leave a comment