[Django]-How do I refresh the values on an object in Django?

Finally, in Django 1.8, we have a specific method to do this. It’s called refresh_from_db and it’s a new method of the class django.db.models.Model.

An example of usage:

def update_result(self):
    obj = MyModel.objects.create(val=1)
    MyModel.objects.filter(pk=obj.pk).update(val=F('val') + 1)
    # At this point obj.val is still 1, but the value in the database
    # was updated to 2. The object's updated value needs to be reloaded
    # from the database.
    obj.refresh_from_db()

If your version of Django is less than 1.8 but you want to have this functionality, modify your model to inherit from RefreshableModel:

from django.db import models
from django.db.models.constants import LOOKUP_SEP
from django.db.models.query_utils import DeferredAttribute

class RefreshableModel(models.Model):

    class Meta:
        abstract = True

    def get_deferred_fields(self):
        """
        Returns a set containing names of deferred fields for this instance.
        """
        return {
            f.attname for f in self._meta.concrete_fields
            if isinstance(self.__class__.__dict__.get(f.attname), DeferredAttribute)
        }

    def refresh_from_db(self, using=None, fields=None, **kwargs):
        """
        Reloads field values from the database.
        By default, the reloading happens from the database this instance was
        loaded from, or by the read router if this instance wasn't loaded from
        any database. The using parameter will override the default.
        Fields can be used to specify which fields to reload. The fields
        should be an iterable of field attnames. If fields is None, then
        all non-deferred fields are reloaded.
        When accessing deferred fields of an instance, the deferred loading
        of the field will call this method.
        """
        if fields is not None:
            if len(fields) == 0:
                return
            if any(LOOKUP_SEP in f for f in fields):
                raise ValueError(
                    'Found "%s" in fields argument. Relations and transforms '
                    'are not allowed in fields.' % LOOKUP_SEP)

        db = using if using is not None else self._state.db
        if self._deferred:
            non_deferred_model = self._meta.proxy_for_model
        else:
            non_deferred_model = self.__class__
        db_instance_qs = non_deferred_model._default_manager.using(db).filter(pk=self.pk)

        # Use provided fields, if not set then reload all non-deferred fields.
        if fields is not None:
            fields = list(fields)
            db_instance_qs = db_instance_qs.only(*fields)
        elif self._deferred:
            deferred_fields = self.get_deferred_fields()
            fields = [f.attname for f in self._meta.concrete_fields
                      if f.attname not in deferred_fields]
            db_instance_qs = db_instance_qs.only(*fields)

        db_instance = db_instance_qs.get()
        non_loaded_fields = db_instance.get_deferred_fields()
        for field in self._meta.concrete_fields:
            if field.attname in non_loaded_fields:
                # This field wasn't refreshed - skip ahead.
                continue
            setattr(self, field.attname, getattr(db_instance, field.attname))
            # Throw away stale foreign key references.
            if field.rel and field.get_cache_name() in self.__dict__:
                rel_instance = getattr(self, field.get_cache_name())
                local_val = getattr(db_instance, field.attname)
                related_val = None if rel_instance is None else getattr(rel_instance, field.related_field.attname)
                if local_val != related_val:
                    del self.__dict__[field.get_cache_name()]
        self._state.db = db_instance._state.db

class MyModel(RefreshableModel):
    # Your Model implementation
    pass

obj = MyModel.objects.create(val=1)
obj.refresh_from_db()

I assume you must need to do this from within the class itself, or you would just do something like:

def refresh(obj):
    """ Reload an object from the database """
    return obj.__class__._default_manager.get(pk=obj.pk)

But doing that internally and replacing self gets ugly…

Hmm. It seems to me that you can never be sure that your any foo.counter is actually up to date… And this is true of any kind of model object, not just these kinds of counters…

Let’s say you have the following code:

    f1 = Foo.objects.get()[0]
    f2 = Foo.objects.get()[0]  #probably somewhere else!
    f1.increment() #let's assume this acidly increments counter both in db and in f1
    f2.counter # is wrong

At the end of this, f2.counter will now be wrong.

Why is refreshing the values so important – why not just can get back a new instance whenever needed?

    f1 = Foo.objects.get()[0]
    #stuff
    f1 = Foo.objects.get(pk=f1.id)

But if you really need to you could create a refresh method yourself… like you indicated in your question but you need to skip related fields, so you could just specify the lists of fieldnames that you want to iterate over (rather than _meta.get_all_fieldnames). Or you could iterate over Foo._meta.fields it will give you Field objects, and you can just check on the class of the field — I think if they are instances of django.db.fields.field.related.RelatedField then you skip them. You could if you wanted then speed this up by doing this only on loading your module and storing this list in your model class (use a class_prepared signal)

I see why you’re using SELECT ... FOR UPDATE, but once you’ve issued this, you should still be interacting with self.

For example, try this instead:

@transaction.commit_on_success
def increment(self):
    Foo.objects.raw("SELECT id from fooapp_foo WHERE id = %s FOR UPDATE", [self.id])[0]
    self.counter += 1
    self.save()

The row is locked, but now the interaction is taking place on the in-memory instance, so changes remain in sync.

You can use Django’s F expressions to do this.

To show an example, I’ll use this model:

# models.py
from django.db import models
class Something(models.Model):
    x = models.IntegerField()

Then, you can do something like this:

    from models import Something
    from django.db.models import F

    blah = Something.objects.create(x=3)
    print blah.x # 3

    # set property x to itself plus one atomically
    blah.x = F('x') + 1
    blah.save()

    # reload the object back from the DB
    blah = Something.objects.get(pk=blah.pk)
    print blah.x # 4

Quick, ugly and untested:

from django.db.models.fields.related import RelatedField

for field in self.__class__._meta.fields:
    if not isinstance(field, RelatedField):
        setattr(self, field.attname, getattr(offer, field)) 

though I think you could do this using some other _meta approach over the isinstance() call.

Obviously, we both know this should be avoided if possible. Maybe a better approach would be to futz with the internal model state?

EDIT – Is the Django 1.4 SELECT FOR UPDATE support going to solve this?

I have some long running processes which are running in parallel. Once the calculations are done, I want to update the values and save the model, but I don’t want the entire process to tie up a transaction. So my strategy is something like

model = Model.objects.get(pk=pk)

# [ do a bunch of stuff here]

# get a fresh model with possibly updated values
with transaction.commit_on_success():
    model = model.__class__.objects.get(pk=model.pk)
    model.field1 = results
    model.save()

This combines the best of the two answers above, and adds up-to-date django syntax:

Get fresh data and guarantee* it stays fresh for your transaction:

def refresh_and_lock(obj):
    """ Return an fresh copy with a lock."""
    return obj.__class__._default_manager.select_for_update().get(pk=obj.pk)

This will only work if everything that changes the object goes through select_for_update. Other processes that get the object without a lock will hang at save() instead of get(), and stomp on the change right after the first transaction commits.

I had a similar need and, whilst you cannot effectively refresh the existing object without potentially tampering its integrity, you can still enforce best practice at implementation time. For what I am concerned, I earmark the object as stale and make that prevent any further access to it, as illustrated in the below example:

class MyModelManager(Manager):
    def get_the_token(self, my_obj):

        # you need to get that before marking the object stale :-)
        pk = my_obj.pk

        # I still want to do the update so long a pool_size > 0
        row_count = self.filter(pk=pk, pool_size__gt=0).update(pool_size=F('pool_size')-1)
        if row_count == 0:
            # the pool has been emptied in the meantime, deal with it
            raise Whatever

        # after this row, one cannot ask anything to the record
        my_obj._stale = True

        # here you're returning an up-to-date instance of the record
        return self.get(pk=pk)


class MyModel(Model):
    pool_size = IntegerField()

    objects = MyModelManager()

    def __getattribute__(self, name):
        try:
            # checking if the object is marked as stale
            is_stale = super(MyModel, self).__getattribute__('_stale'):

            # well, it is probably...
            if is_stale: raise IAmStale("you should have done obj = obj.get_token()")
        except AttributeError:
            pass

        # it is not stale...
        return super(MyModel, self).__getattribute__(name)

    def get_token(self):
        # since it's about an operation on the DB rather than on the object,
        # we'd rather do that at the manager level
        # any better way out there to get the manager from an instance?
        # self._meta.concrete_model.objects ?
        self.__class__.objects.get_the_token(self, my_obj)

(written on the fly, forgive any possible typos 🙂 )

You can use

refresh_from_db()

Eg:

obj.refresh_from_db()

https://docs.djangoproject.com/en/1.11/ref/models/instances/#refreshing-objects-from-database

I have been using a method like this, because the new builtin refresh_from_db does not refresh children that have had their attributes changed, frequently causing problems. This clears the cache of any foreign keys.

def super_refresh_from_db(self):
    """ refresh_from_db only reloads local values and any deferred objects whose id has changed.
    If the related object has itself changed, we miss that.  This attempts to kind of get that back. """
    self.refresh_from_db()

    db = self._state.db
    db_instance_qs = self.__class__._default_manager.using(db).filter(pk=self.pk)

    db_instance = db_instance_qs.get()
    non_loaded_fields = db_instance.get_deferred_fields()
    for field in self._meta.concrete_fields:
        if field.attname in non_loaded_fields:
            # This field wasn't refreshed - skip ahead.
            continue

        if field.is_relation and field.get_cache_name() in self.__dict__:
            del self.__dict__[field.get_cache_name()]

I had the same issue and this worked for me:

obj.reload()