[Django]-Updating django models with multiprocessing pool locks up database

6👍

Your question includes the terms multiprocessing and multithread, but it’s important to understand that these are different ways of achieving concurrency.

Django has built-in support for multithreading, and will create a new database connection for each thread. If you switch from multiprocessing to multithreading your problem should be solved.

In multiprocessing, the entire process is forked and the new process will have the same database connection as the old one. That results in the problems you’re seeing, where, for example, you try to open a new transaction when one has already been opened on the same database connection by another process.

If you truly need multiprocessing instead of multithreading, there are probably solutions. For example, this answer suggests simply closing the database connections, forcing Django to create fresh ones.

Leave a comment