189👍
How to migrate using south.
Lets say we got two apps: common and specific:
myproject/
|-- common
| |-- migrations
| | |-- 0001_initial.py
| | `-- 0002_create_cat.py
| `-- models.py
`-- specific
|-- migrations
| |-- 0001_initial.py
| `-- 0002_create_dog.py
`-- models.py
Now we want to move model common.models.cat to specific app (precisely to specific.models.cat).
First make the changes in the source code and then run:
$ python manage.py schemamigration specific create_cat --auto
+ Added model 'specific.cat'
$ python manage.py schemamigration common drop_cat --auto
- Deleted model 'common.cat'
myproject/
|-- common
| |-- migrations
| | |-- 0001_initial.py
| | |-- 0002_create_cat.py
| | `-- 0003_drop_cat.py
| `-- models.py
`-- specific
|-- migrations
| |-- 0001_initial.py
| |-- 0002_create_dog.py
| `-- 0003_create_cat.py
`-- models.py
Now we need to edit both migration files:
#0003_create_cat: replace existing forward and backward code
#to use just one sentence:
def forwards(self, orm):
db.rename_table('common_cat', 'specific_cat')
if not db.dry_run:
# For permissions to work properly after migrating
orm['contenttypes.contenttype'].objects.filter(
app_label='common',
model='cat',
).update(app_label='specific')
def backwards(self, orm):
db.rename_table('specific_cat', 'common_cat')
if not db.dry_run:
# For permissions to work properly after migrating
orm['contenttypes.contenttype'].objects.filter(
app_label='specific',
model='cat',
).update(app_label='common')
#0003_drop_cat:replace existing forward and backward code
#to use just one sentence; add dependency:
depends_on = (
('specific', '0003_create_cat'),
)
def forwards(self, orm):
pass
def backwards(self, orm):
pass
Now both apps migrations are aware of the change and life sucks just a little less 🙂
Setting this relationship between migrations is key of success.
Now if you do:
python manage.py migrate common
> specific: 0003_create_cat
> common: 0003_drop_cat
will do both migration, and
python manage.py migrate specific 0002_create_dog
< common: 0003_drop_cat
< specific: 0003_create_cat
will migrate things down.
Notice that for upgrading of schema I used common app and for downgrading, I used specific app. That’s because how the dependency here works.
35👍
To build on Potr Czachur‘s answer, situations that involve ForeignKeys are more complicated and should be handled slightly differently.
(The following example builds on the common
and specific
apps referred to the in the current answer).
# common/models.py
class Cat(models.Model):
# ...
class Toy(models.Model):
belongs_to = models.ForeignKey(Cat)
# ...
would then change to
# common/models.py
from specific.models import Cat
class Toy(models.Model):
belongs_to = models.ForeignKey(Cat)
# ...
# specific/models.py
class Cat(models.Model):
# ...
Running
./manage.py schemamigration common --auto
./manage.py schemamigration specific --auto # or --initial
would generate the following the migrations (I’m intentionally ignoring Django ContentType changes—see previously referenced answer for how to handle that):
# common/migrations/0009_auto__del_cat.py
class Migration(SchemaMigration):
def forwards(self, orm):
db.delete_table('common_cat')
db.alter_column('common_toy', 'belongs_to_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['specific.Cat']))
def backwards(self, orm):
db.create_table('common_cat', (
# ...
))
db.alter_column('common_toy', 'belongs_to_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['common.Cat']))
# specific/migrations/0004_auto__add_cat.py
class Migration(SchemaMigration):
def forwards(self, orm):
db.create_table('specific_cat', (
# ...
))
def backwards(self, orm):
db.delete_table('specific_cat')
As you can see, the FK must be altered to reference the new table. We need to add a dependency so that we know the order in which the migrations will be applied (and thus that the table will exist before we try to add a FK to it) but we also need to make sure rolling backwards works too because the dependency applies in the reverse direction.
# common/migrations/0009_auto__del_cat.py
class Migration(SchemaMigration):
depends_on = (
('specific', '0004_auto__add_cat'),
)
def forwards(self, orm):
db.alter_column('common_toy', 'belongs_to_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['specific.Cat']))
def backwards(self, orm):
db.rename_table('specific_cat', 'common_cat')
db.alter_column('common_toy', 'belongs_to_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['common.Cat']))
# specific/migrations/0004_auto__add_cat.py
class Migration(SchemaMigration):
def forwards(self, orm):
db.rename_table('common_cat', 'specific_cat')
def backwards(self, orm):
pass
Per the South documentation, depends_on
will ensure that 0004_auto__add_cat
runs before 0009_auto__del_cat
when migrating forwards but in the opposite order when migrating backwards. If we left db.rename_table('specific_cat', 'common_cat')
in the specific
rollback, the common
rollback would fail when trying to migrate the ForeignKey because the table referenced table wouldn’t exist.
Hopefully this is closer to a “real world” situation than the existing solutions and someone will find this helpful. Cheers!
- [Django]-Django: list all reverse relations of a model
- [Django]-CORS error while consuming calling REST API with React
- [Django]-Setting Django up to use MySQL
13👍
Models aren’t very tightly coupled to apps, so moving is fairly simple. Django uses the app name in the name of the database table, so if you want to move your app you can either rename the database table via an SQL ALTER TABLE
statement, or – even simpler – just use the db_table
parameter in your model’s Meta
class to refer to the old name.
If you’ve used ContentTypes or generic relations anywhere in your code so far, you will probably want to rename the app_label
of the contenttype pointing at the model that’s moving, so that existing relations are preserved.
Of course, if you don’t have any data at all to preserve, the easiest thing to do is to drop the database tables completely and run ./manage.py syncdb
again.
- [Django]-Django 1.5 – How to use variables inside static tag
- [Django]-PyCharm: DJANGO_SETTINGS_MODULE is undefined
- [Django]-How can I save my secret keys and password securely in my version control system?
4👍
Here’s one more fix to Potr’s excellent solution. Add the following to specific/0003_create_cat
depends_on = (
('common', '0002_create_cat'),
)
Unless this dependency is set South will not guarantee that the common_cat
table exists at the time when specific/0003_create_cat is run, throwing an django.db.utils.OperationalError: no such table: common_cat
error at you.
South runs migrations in lexicographical order unless dependency is explicitly set. Since common
comes before specific
all the common
‘s migrations would get run before table renaming, so it probably wouldn’t reproduce in the original example shown by Potr. But if you rename common
to app2
and specific
to app1
you will run into this problem.
- [Django]-Django edit user profile
- [Django]-How to get the name of current app within a template?
- [Django]-Django + Ajax
4👍
The process I’ve currently settled on since I’ve been back here a few times and decided to formalise it.
This was originally built on
Potr Czachur’s answer
and Matt Briançon’s answer,
using South 0.8.4
Step 1. Discover child foreign key relationships
# Caution: This finds OneToOneField and ForeignKey.
# I don't know if this finds all the ways of specifying ManyToManyField.
# Hopefully Django or South throw errors if you have a situation like that.
>>> Cat._meta.get_all_related_objects()
[<RelatedObject: common:toy related to cat>,
<RelatedObject: identity:microchip related to cat>]
So in this extended case, we have discovered another related model like:
# Inside the "identity" app...
class Microchip(models.Model):
# In reality we'd probably want a ForeignKey, but to show the OneToOneField
identifies = models.OneToOneField(Cat)
...
Step 2. Create migrations
# Create the "new"-ly renamed model
# Yes I'm changing the model name in my refactoring too.
python manage.py schemamigration specific create_kittycat --auto
# Drop the old model
python manage.py schemamigration common drop_cat --auto
# Update downstream apps, so South thinks their ForeignKey(s) are correct.
# Can skip models like Toy if the app is already covered
python manage.py schemamigration identity update_microchip_fk --auto
Step 3. Source control: Commit changes so far.
Makes it a more repeatable process if you run into merge conflicts like team mates writing migrations on the updated apps.
Step 4. Add dependencies between the migrations.
Basically create_kittycat
depends on the current state of everything, and everything then depends on create_kittycat
.
# create_kittycat
class Migration(SchemaMigration):
depends_on = (
# Original model location
('common', 'the_one_before_drop_cat'),
# Foreign keys to models not in original location
('identity', 'the_one_before_update_microchip_fk'),
)
...
# drop_cat
class Migration(SchemaMigration):
depends_on = (
('specific', 'create_kittycat'),
)
...
# update_microchip_fk
class Migration(SchemaMigration):
depends_on = (
('specific', 'create_kittycat'),
)
...
Step 5. The table rename change we want to make.
# create_kittycat
class Migration(SchemaMigration):
...
# Hopefully for create_kittycat you only need to change the following
# 4 strings to go forward cleanly... backwards will need a bit more work.
old_app = 'common'
old_model = 'cat'
new_app = 'specific'
new_model = 'kittycat'
# You may also wish to update the ContentType.name,
# personally, I don't know what its for and
# haven't seen any side effects from skipping it.
def forwards(self, orm):
db.rename_table(
'%s_%s' % (self.old_app, self.old_model),
'%s_%s' % (self.new_app, self.new_model),
)
if not db.dry_run:
# For permissions, GenericForeignKeys, etc to work properly after migrating.
orm['contenttypes.contenttype'].objects.filter(
app_label=self.old_app,
model=self.old_model,
).update(
app_label=self.new_app,
model=self.new_model,
)
# Going forwards, should be no problem just updating child foreign keys
# with the --auto in the other new South migrations
def backwards(self, orm):
db.rename_table(
'%s_%s' % (self.new_app, self.new_model),
'%s_%s' % (self.old_app, self.old_model),
)
if not db.dry_run:
# For permissions, GenericForeignKeys, etc to work properly after migrating.
orm['contenttypes.contenttype'].objects.filter(
app_label=self.new_app,
model=self.new_model,
).update(
app_label=self.old_app,
model=self.old_model,
)
# Going backwards, you probably should copy the ForeignKey
# db.alter_column() changes from the other new migrations in here
# so they run in the correct order.
#
# Test it! See Step 6 for more details if you need to go backwards.
db.alter_column('common_toy', 'belongs_to_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['common.Cat']))
db.alter_column('identity_microchip', 'identifies_id', self.gf('django.db.models.fields.related.OneToOneField')(to=orm['common.Cat']))
# drop_cat
class Migration(SchemaMigration):
...
def forwards(self, orm):
# Remove the db.delete_table(), if you don't at Step 7 you'll likely get
# "django.db.utils.ProgrammingError: table "common_cat" does not exist"
# Leave existing db.alter_column() statements here
db.alter_column('common_toy', 'belongs_to_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['specific.KittyCat']))
def backwards(self, orm):
# Copy/paste the auto-generated db.alter_column()
# into the create_kittycat migration if you need backwards to work.
pass
# update_microchip_fk
class Migration(SchemaMigration):
...
def forwards(self, orm):
# Leave existing db.alter_column() statements here
db.alter_column('identity_microchip', 'identifies_id', self.gf('django.db.models.fields.related.OneToOneField')(to=orm['specific.KittyCat']))
def backwards(self, orm):
# Copy/paste the auto-generated db.alter_column()
# into the create_kittycat migration if you need backwards to work.
pass
Step 6. Only if you need backwards() to work AND get a KeyError running backwards.
# the_one_before_create_kittycat
class Migration(SchemaMigration):
# You many also need to add more models to South's FakeORM if you run into
# more KeyErrors, the trade-off chosen was to make going forward as easy as
# possible, as that's what you'll probably want to do once in QA and once in
# production, rather than running the following many times:
#
# python manage.py migrate specific <the_one_before_create_kittycat>
models = {
...
# Copied from 'identity' app, 'update_microchip_fk' migration
u'identity.microchip': {
'Meta': {'object_name': 'Microchip'},
u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
'identifies': ('django.db.models.fields.related.OneToOneField', [], {to=orm['specific.KittyCat']})
},
...
}
Step 7. Test it – what works for me may not be enough for your real life situation 🙂
python manage.py migrate
# If you need backwards to work
python manage.py migrate specific <the_one_before_create_kittycat>
- [Django]-"<Message: title>" needs to have a value for field "id" before this many-to-many relationship can be used.
- [Django]-How to set and get cookies in Django?
- [Django]-A field with precision 10, scale 2 must round to an absolute value less than 10^8
3👍
So using the the original response from @Potr above did not work
for me on South 0.8.1 and Django 1.5.1. I am posting what did
work for me below in the hope that it is helpful to others.
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
db.rename_table('common_cat', 'specific_cat')
if not db.dry_run:
db.execute(
"update django_content_type set app_label = 'specific' where "
" app_label = 'common' and model = 'cat';")
def backwards(self, orm):
db.rename_table('specific_cat', 'common_cat')
db.execute(
"update django_content_type set app_label = 'common' where "
" app_label = 'specific' and model = 'cat';")
- [Django]-Django Rest Framework: Dynamically return subset of fields
- [Django]-How to dynamically compose an OR query filter in Django?
- [Django]-How to add multiple objects to ManyToMany relationship at once in Django ?
1👍
I’m going to give a more explicit version of one of the things Daniel Roseman suggested in his answer…
If you just change the db_table
Meta attribute of the model you have moved to point to the existing table name (instead of the new name Django would give it if you dropped and did a syncdb
) then you can avoid complicated South migrations. eg:
Original:
# app1/models.py
class MyModel(models.Model):
...
After moving:
# app2/models.py
class MyModel(models.Model):
class Meta:
db_table = "app1_mymodel"
Now you just need to do a data migration to update the app_label
for MyModel
in the django_content_type
table and you should be good to go…
Run ./manage.py datamigration django update_content_type
then edit the file that South creates for you:
def forwards(self, orm):
moved = orm.ContentType.objects.get(app_label='app1', model='mymodel')
moved.app_label = 'app2'
moved.save()
def backwards(self, orm):
moved = orm.ContentType.objects.get(app_label='app2', model='mymodel')
moved.app_label = 'app1'
moved.save()
- [Django]-How to run own daemon processes with Django?
- [Django]-Django admin and MongoDB, possible at all?
- [Django]-Github issues api 401, why? (django)