[Django]-MemoryError with django

2đź‘Ť

âś…

The function may not be, but if your realing with such a huge database, the sqlite library may be consuming it in order to update the table. If you need to store such huge amounts of data you may have to move away from SQLite to a fully featured database server such as MySQL.

👤Geoffrey

7đź‘Ť

from
https://docs.djangoproject.com/en/dev/faq/models/#why-is-django-leaking-memory

Django isn’t known to leak memory. If you find your Django processes are allocating more and more memory, with no sign of releasing it, check to make sure your DEBUG setting is set to False. If DEBUG is True, then Django saves a copy of every SQL statement it has executed.

(The queries are saved in django.db.connection.queries. See How can I see the raw SQL queries Django is running?.)

To fix the problem, set DEBUG to False.

If you need to clear the query list manually at any point in your functions, just call reset_queries(), like this:

from django import db
db.reset_queries()
👤errx

4đź‘Ť

SQLite is an in-memory database. It’s stored on your hard drive for posterity, but it’s loaded in entirety into memory to interact with it. That means it’s inherently limited by the memory resources of your system. It was never designed to be a full-fledged database solution, but rather a speedy database to handle data for which larger database solutions like PostgreSQL and MySQL would be overkill. It’s also great for development environments, where it’s easy to use, usually pre-installed, and perfectly sufficient for the amount of data used in development and testing. As a production database, though, you’d be insane to use it.

However, even if you’re just in development, if you’re dealing with “millions of entries”, you’ve officially outgrown SQLite.

👤Chris Pratt

Leave a comment