[Django]-Making sharding simple with Django

Mike Clarke recently gave a talk at PyPgDay on how Disqus shards their users with Django and PostgreSQL. He wrote up a blog post on how they do it.

Several strategies can be employed when sharding Postgres databases. At Disqus, we chose to shard based on table name. Where as the original table name as generated by Django might be comments_post, our sharding tools will rewrite the SQL to query a table comments_post_X, where X is the shard ID calculated based on a consistent hashing scheme. All these tables live in a single schema, on a single database instance.

In addition, they released some code as part of a sample application demonstrating how they shard.

You really don’t want to be in the position of asking this question. If you are sharding by user id then you probably don’t want to search by name.

If you are sharding your database then it’s not going to be invisible to your application and will probably end up requiring schema alterations.

You might find SkyTools useful – read up on PL/Proxy. It’s how Skype shard their databases.

it is better to use professional sharding middleware, for example: Apache ShardingSphere.

The project contains 2 productions, ShardingSphere-JDBC for java driver, and ShardingSphere-Proxy for all programing languages. It can support python and Django as well.