1๐
I suggest following way:
Create models to store your tree structure and files for example:
class Node(TreeModel): parent # foreign key to Node class File(Model): node # foreign key to Node name # name of file path # path to the file on disk for example
After that move your files in one or few directories (read this How many files can I put in a directory?)
also you can rename them (for example by using hash from files).
Update the model File to put there new paths to your files.
Having done this you are able to easy show files and build path to files etc.
For the model Node use [django-mptt][1] (there are other solutions for django, google it) to get an efficient API to manage a Tree-like model.
You can also create your own Django Storage Backend (or find there are many solutions on the Internet).
Updated
You can add new files by using django admin. You should use amazon s3 django storage backend http://django-storages.readthedocs.org/en/latest/backends/amazon-S3.html.
Change:
class File(Model): node # foreign key to Node name # name of file file # django models.FileField
In this case you have not to update index.