[Fixed]-Django: RunSQL: use PostgreSQL COPY command

by

7👍

✅

The psycopg2 driver exposes copy_to and copy_from methods that can be used to implement the same behavior as the psql client; the key is to use the RunPython operation instead of the RunSQL operation.

You’ll need:

  • A function in your migration to open your file and interact with the copy methods
  • A RunPython operation in your migration’s operations list to invoke the function

Example, using Django 1.8.4, Python 2.7.10, psycopg2 2.6.1 – 

from django.db import models, migrations

def forwards(apps, schema_editor):
    with open('./path/to/file.csv', 'r') as infile:
        with schema_editor.connection.cursor() as stmt:
            #for finer control, use copy_expert instead
            stmt.copy_from(infile, 'your_table', sep=',')

class Migration(migrations.Migration):

    dependencies = [
    ]

    operations = [
        #this runs your function
        migrations.RunPython(forwards)
    ]

Some notes:

  • The file object passed to copy is essentially STDIN in the statement.
  • The copy command can be picky about columns; using copy_expert you can control all the same options as the command: format, headers, delimiter, etc.

For more info on the copy_* methods, check the psycopg2 docs: http://initd.org/psycopg/docs/cursor.html

👤bimsapi

Leave a comment