[Fixed]-Nose test case fails to set up with "Transaction managed block ended with pending COMMIT/ROLLBACK"

1👍

Well, it took me a lot of debugging, but I figured out the problem…

Our 10,000 tests take a long, long time to run, unless we run them in parallel processes. So we use a tool that splits up the nose tests into 20 parallel processes and runs the tests in groups. (This still takes roughly 20 minutes to complete, but it’s better than nearly two hours.)

We use the FastFixtureTestCase, which extends the TransactionTestCase. At the start of each test case, FastFixtureTestCase calls django.test.testcases: connections_support_transactions(). That function loops over all the DATABASES connections and calls supports_transactions on each one. My mistake was assuming that supports_transactions should be an inherently safe operation. It’s not.

supports_transactions does the following things:

  1. Creates a new table
  2. Commits
  3. Inserts a value into that table
  4. Rolls back
  5. Selects the number of rows in that table
  6. Drops the table
  7. Commits
  8. Returns True if the number of rows in the table was 0 (meaning the rollback succeeded, so transactions must be supported).

This is not safe. This is very dangerous. No two processes or servers can run this at the same time against the same database. If two or more processes run this function at the same time, at best, one will return True and the other(s) will raise an exception. At worst, all will raise an exception.

In my case, because we have so many test cases, most of the time the processes were avoiding executing connections_support_transactions at the same time, but when they did, it resulted in a small handful of random failures that were different each time.

One possible solution is to use SimpleTestCase instead of FastFixtureTestCase, as @kmmbvnr pointed out. However, this is not an option for us, as our entire test infrastructure depends on what FastFixtureTestCase does to the rest of our databases. So, instead, I overrode supports_transactions just for the static, shared database with the following line of code, and the errors went away:

connections['geo'].features.supports_transactions = True

Leave a comment