[Django]-Saving a Pillow file to S3 with boto

by

2👍

✅

Note that the solution below is for Python 2.

models.py

class Image(models.Model):
    caption = models.CharField(max_length=100, null=True, blank=True)
    width = models.IntegerField(default=0, blank=True)
    height = models.IntegerField(default=0, blank=True)
    image = models.ImageField(width_field='width', height_field='height',blank=True)

tasks.py

from celery import shared_task
import os
from django.core.files.storage import default_storage as storage
from django.conf import settings
import mimetypes
import cStringIO
from PIL import Image as PillowImage
import boto
from .models import Image

@shared_task
def create_thumbnails(pk):
    try:
        image = Image.objects.get(pk=pk)
    except Image.ObjectDoesNotExist:
        pass
    try:
        thumbnail_size = (450,200)
        filename, ext = os.path.splitext(image.image.name)
        filename = filename +'_thumbnail' +ext
        existing_file = storage.open(image.image.name, 'r')
        im = PillowImage.open(existing_file)
        im = im.resize(thumbnail_size, PillowImage.ANTIALIAS)
        memory_file = cStringIO.StringIO()
        mime = mimetypes.guess_type(filename)[0]
        plain_ext = mime.split('/')[1]
        im.save(memory_file, plain_ext)
        
        conn  = boto.connect_s3(settings.AWS_ACCESS_KEY_ID, settings.AWS_SECRET_ACCESS_KEY)
        bucket = conn.get_bucket( 'yourbucketname', validate=False)
        k = bucket.new_key('media/' +filename)
        k.set_metadata('Content-Type', mime)
        k.set_contents_from_string(memory_file.getvalue())
        k.set_acl("public-read")
        memory_file.close()
    except Exception as error:
        print("cannot create thumbnail for ", filename, 'error ', error)
👤RunLoop

10👍

This was super helpful and I used it to find a way to write images to s3 from django without using boto directly.

Basically PIL’s save() method doesn’t work with s3, but the default_storage.write() method does. The key was to use the default_storage.write() method to write binary data directly from the StringIO memory file like this:

file_to_write.write(memory_file.getvalue())

Here’s the code I ran in the django shell (python manage.py shell) to test this:

>>> from django.core.files.storage import default_storage as storage
>>> from PIL import Image
>>> import StringIO
>>> i = storage.open('ImageToCreate.jpg','w+')
>>> m = storage.open('ImageAlreadyOnS3.jpg','r')
>>> im = Image.open(m)
>>> im = im.resize((640,360),3)
>>> sfile = StringIO.StringIO() #cStringIO works too
>>> im.save(sfile, format="JPEG")
>>> i.write(sfile.getvalue())
>>> i.close()
>>> m.close()

It works in my views.py as well.

I found it useful since it works both on the remote s3 storage and my local development environment (a folder on my laptop).

👤Jeremy S.

4👍

Note that with Python3, you might want to use BytesIO:

from io import BytesIO

# 'image' is a PIL image object.

imageBuffer = BytesIO()
image.save(imageBuffer, format=imageType)

imageFile = default_storage.open(imageFileName, 'wb')
imageFile.write(imageBuffer.getvalue())
imageFile.flush()
imageFile.close()
👤rotten

2👍

I had the same issue. 

default_storage.write() # It didn't work. it was not saving anything to s3

The @RunLoop answer was quite complicated and different then what I wanted to do with Django so, I did this and it worked. 

    import StringIO
    from PIL import Image

First, read the file uploaded

    image = request.FILES['image'].read()  #atleast in my case

create a file like object so that we can use Image to read it

    image_file = StringIO.StringIO(image)
    thumbnail_image = Image.open(image_file)

resize the image to desired size

    resized_thumbnail_image = thumbnail_image.resize((200, 200), Image.ANTIALIAS)

create another file like object or inmemory file, so that we can write the Image instance to it and get string value – which should be passed to default storage

    resized_thumbnail_image_file = StringIO.StringIO()
    resized_thumbnail_image.save(resized_thumbnail_image_file, 'JPEG',quality=90)

    default_storage.save(save_path, ContentFile(resized_thumbnail_image_file.getvalue()))

My first detailed answer, hope it helps.

Stack:
python 2.7
Django 1.8

👤Nagashayan

0👍

A slightly more understandable, workable example than rotten‘s answer

from io import BytesIO
from PIL import Image
from django.core.files.base import ContentFile
from django.core.files.storage import default_storage
from django.contrib.staticfiles.storage import staticfiles_storage

img = Image.new('RGB', (1920, 1080), color = 'red') # <-- use this
#img = Image.open(r'C:\Users\you\Pictures\profile.jpg') # <-- or a file on your computer

buffer = BytesIO()
img.save(buffer, format='JPEG')

fp = '{path}{name}'.format(path='adminz/profile/', name='default.jpg')
f = default_storage.open(fp, 'wb')
f.write(buffer.getvalue())
f.close()

I’m using Django Storages which is why I needed to figure this out. Extra unused imports I included because you’ll often find a reason to know them when dealing with this subject.

Another example of something I used in a project:

image = Image.open(photo) # <- open
cropped_image = image.crop((x, y, w + x, h + y)) # <- do something
with BytesIO() as f:
    cropped_image.save(f, format=settings.THUMBNAIL_FORMAT, quality=95) # <- save to buffer
    img = f.getvalue()

post.featured_image.save(photo.name,
                         content=ContentFile(img))
👤Jarad

Leave a comment