[Answered ]-L[:] performance problem

The code that you are showing would run in the 0.1 microseconds that it would take to raise an exception:

org_list = getList()
list = orgList[:]#this line cause cpu problems.

orgList should be org_list. Show the minimal code that actually reproduces the problem.

Also, that kills the list built-in function. Don’t do that.

Update Another thought: A common response to “My Django app runs slowly!” is “turn off the debug flag” … evidently it doesn’t free up memory in debug mode.

Update2 about “””I found out that when you slice a list. it actually works as a view to that list and just call original list methods to get actual item.””” Where did you get that idea? That can’t be the case with a plain old list! Have you redefined list somewhere?

In your getList function:

(1) put in print type(list)

(2) replace result = list() with result = [] and see whether the problem goes away.

list = org_list[:] makes a copy of org_list. Usually you see something like this when you need to modify the list but you want to keep the original around as well.

The code you are showing me doesn’t seem like it actually modifies org_list, and if doSomethingelse(list[0]) doesn’t modify list, I don’t see why it’s actually being copied in the first place. Even if it does modify list, as long as org_list isn’t needed after those few lines of code, you could probably get away with just using org_list and not doing the slice copy.

I would like to know:

• what is the type of total_list ? Please do print type(total_list)

• what are the types of the elements of total_list ?

• have you an idea of the sizes of these elements ?

• what is the number of elements in the list returned by getList() ? 20-30 ?

Remarks:

-giving the keyword “list” as a name for a list isn’t a good idea (in list = orgList[:] )

-your getList function can be written :

def getList():
    #query db and return a list
    total_list = Model.objects.all()
    return [ item for item in total_list if item.attr1 ]

unfortunately actual code is property of the company I worked for so I cannot give it exactly like it is but here is some part of it (variable names are changed,but argument of mainTree method was really named list.)

    mainObj = get_object_or_404(MainObj, code_name=my_code_name)
    if (not mainObj.is_now()) and (not request.user.is_staff):
        return HttpResponseRedirect("/")

    main_tree = mainObj.main_tree()
    objects = main_tree[:] # !!!
    if objects.__len__() == 1 and not objects[0].has_special_page:
        my_code_name = objects[0].code_name

...
...
...

mainObj.main_tree() is the following

    def main_tree(self):
        def append_obj_recursive(list, obj):
            list.append(obj)
            for c in self.obj_set.filter(parent_obj=obj):
                append_obj_recursive(list, c)
            return list

        root_obj = self.get_or_create_root_obj();
        top_level_objs = self.obj_set.filter(parent_obj=root_obj).order_by("weight")
        objs = []

        for tlc in top_level_objs:
            append_obj_recursive(objs, tlc)
        return objs

This is a really weird problem. slicing should not cause such a bizare problem. If I can find an answer I will post it here too.