[Answered ]-Django query limit and streamingHttpResponse

by

2👍

✅

You can limit queries quite easily in Django, using Python’s array-splicing notation. You can use this on any query. Here are some examples:

# Will return only the first 1000 objects in the query
queryset = MyModel.objects.filter(some_field=some_value)[:1000]  

# Will return only the first 500 objects
queryset = MyModel.objects.all()[:500] 

# Will return the first 1000 objects of the filtered query ordered by the objects pk
queryset = MyModel.objects.filter(some_field=some_value).order_by('pk')[:1000] 

# Will return the 25th through 50th object
queryset = MyModel.objects.all()[25:50] 

# Will return all objects but the first 10
queryset = MyModel.objects.all()[10:]

Django is smart about this, and will only query what is needed and then stop, rather than gathering the entire query and then chopping off the rest.

See the documentation here for more details.

👤Johndt

0👍

You can use the below method. It will iterate over a Django Queryset ordered by the primary key

def queryset_iterator(queryset, chunksize=1000):
    pk = 0
    last_pk = queryset.order_by("-pk")[0].pk
    queryset = queryset.order_by("pk")
    while pk < last_pk:
        for row in queryset.filter(pk__gt=pk)[:chunksize]:
            pk = row.pk
            yield row
        gc.collect()
👤TheSohan

Leave a comment