[Django]-Google application engine, maximum number of static files?

5👍

Welcome to Stack Overflow!

One of the limitations in App Engine is that you cannot write directly to the filesystem from your app. Static files would be things like HTML, CSS, javascript and images that are global to your application, and get uploaded manually when you deploy. They are uploaded to and served from different servers than the ones that handle dynamic content.

Since you can’t write to the filesystem from your app, files uploaded by users must be saved to the datastore as blobs. These are not considered static files. As others have mentioned, you can use S3 or the Blobstore API, however both of these require billing. With the free quotas, each entity can be up to 1MB, and each HTTP request and response can be up to 10MB. Using standard entities with a BlobProperty, you can easily store and serve dynamically uploaded files up to 1MB, or 10MB if you want to get fancy and store your blob in slices across multiple entities.

2👍

There is a new service called the BlobStore that let’s you store binary data in the database. Also you may want to look into Amazon S3 as storage for the data. If user’s are uploading images they cannot be stored as static files. Static files are files included in your GAE project like html and png/jpg/gif files.

0👍

As others have mentioned, for more dynamic content such as user-uploaded files, those should go into the datastore as blobs, or if they’re larger, as Blobstore objects (max size 2GB).

3000 static files is somewhat reasonable unless you have a lot of static assets (such as images, HTML, CSS, and JS files). for Python source however, you have another workaround, and that is to throw all your .py files into a single ZIP so they don’t hit that count so badly. here’s an article that describes how to do this:

Using Django 1.0 on App Engine with Zipimport

Just be aware that this article talks about how to bundle Django’s source with App Engine; however, that’s unnecessary unless you’re doing 1.3 or are using a fork. App Engine systems already have 0.96 or 1.2.5 available to you for free.

UPDATE (Mar 2011): In SDK 1.4.3, the App Engine team has released the Files API which allows you to read/write files/data programmatically with Blobstore. This applies to both Python and Java. More info can be found in the corresponding blogpost. In addition to Blobstore, the public roadmap shows a future feature integrating in Google Storage access.

UPDATE (Sep 2011): In SDK 1.5.4, the App Engine team removed the Blobstore filesize limitation from 2GB to allow files of unlimited size. You pay per GB of storage however.

UPDATE (Oct 2011): In SDK 1.5.5, the App Engine team extended the maximum number of files from 3000 to 10000, which is a great boost for users. Furthermore, the maximum individual filesize was bumped up from 10MB to 32MB. Another storage related improvement is that users can now write to Google’s Cloud Storage directly from their App Engine app.

👤wescpy

Leave a comment