[Answer]-Django – Getting/Saving large objects takes a lot of time

1👍

✅

Defining lists by appending repeadedly is very slow. Use list comprehensions or even just the list() constructor.

In python you should not join a list of strings using for loops and +=, you should use join().

But that is not the primary bottleneck here. You have a lot of objects.get()s which each takes a database roundtrip. If you didn’t have milions of rows in the mapt table, you should probably just make a dictionary mapping mapt primary keys to mapt objects.

Had you defined your foreign keys as foreign keys the django orm could help you do much of this in like five queries in total. That is, instead of SomeModel.objects.get(id=some_instance.some_fk_id) you can do some_instance.some_fk (which will only hit the databse the first time you do it for each instance). You can then even get rid of the foreign key query if some_instance had been initialized as some_instance = SomeOtherModel.objects.select_related('some_fk').get(id=id_of_some_instance).

Perhaps changing the models without changing the database will work.

Leave a comment