[Answered ]-JQuery, Showing Percentage of Completion of a Heavy Page Generated by Django

1👍

As a rule, a django view should never take too much time.

If you have heavy computations to do, you should delegate it to some backend, such as Celeri. The workflow should be something along those lines:

you client triggers the computations.
something on the backend does the heavy lifing.
when result is ready, the client retrieves it.

For instance:

1.a) Client sends a POST request to /api/crunching_job/
1.b) Servers queues the job to Celeri and replies with a job id, say 42.
2.a) Client polls /api/crunching_job/42/ to get current status, say every second.
2.b) In the meantime, the celeri job is worked upon on the backend. It may even be another server than the web server.
2.c) When it is done, the celeri job makes the result available to the web server…
2.d) …which the the client will know because the status in /api/crunching_job/42/ will change.
3.a) The client can now retrieve the result in /api/crunching_job/42/result/
3.b) When the result is no longer needed, the server gets rid of it. You decide when that is. It may expire after some time, or you may have the client tell being sending a DELETE request to /api/crunching_job/42/.

Hope it helps, I cannot go much more into details unfortunately as I don’t know Celeri.

👤spectras

[Answered ]-How to make the Django light weight server loads quickly

1👍

Finally I found a solution:

The logic is using Django SessionStore in order to save the percentage of the progress for each session and request this percentage every second or so to update the progress bar.

Imagine we have two hyperlinks, the former one linking to a webpage and the latter one linking to download a csv file. Both of them require processing the datasets to generate the results that takes a fair amount of time. That’s why they have ProcessRequired class.

<a class="ProcessRequired" href="/results?version=1" TITLE="Results of study 1.">Study 1 Results</a>
<a class="ProcessRequired csvDataset" href="/csvresults?version=1" TITLE="Study 1 Processed Dataset">Study 1 Processed Dataset</a>

Having csvDataset class means by clicking the hyperlink, a csv file is going to be downloaded.

To show the percentage of the progress, one can use PACE, NProgress.js, define a custom progress bar, or just show the progress percentage somewhere in the webpage. In my code, I have used NProgress.js together with a percentage number in the middle of the screen.

Here is how I have defined the percentage number:

<style>
.pace {
  -webkit-pointer-events: none;
  pointer-events: none;

  -webkit-user-select: none;
  -moz-user-select: none;
  user-select: none;

  z-index: 2000;
  position: fixed;
  margin: auto;
  top: 0;
  left: 0;
  right: 0;
  bottom: 0;
  height: 130px;
  width: 250px;
  font-size:100px;
  color: #ff8000;

}
.pace.pace-inactive {
  display: none;
}
</style>

<div class="pace pace-inactive">
  <div id="pace">0%</div>
</div>

In urls.py, I have defined four types of urls and their corresponding views:

url(r'^(?i)results', 'MyProject.views.results', name='results'),
url(r'^(?i)csvresults', 'MyProject.views.csvresults', name='csvresults'),
url(r'^(?i)csvdownload', 'MyProject.views.csvdownload', name='csvdownload'),
url(r'^(?i)progressPage', 'MyProject.views.progressPage', name='progressPage'),

Accordingly, I have four view methods:
1- To initialize the SessionStore and progressPercent value in the session and to respond the value and the session_key to the client:

def progressPage(request):

    if request.method == 'GET':
        if 'key' in request.GET:
            session_key = request.GET['key']

            s = None;
            if session_key == "":
                s = SessionStore()
                s['progressPercent'] = 0.0
                s.save()
            else:
                s = SessionStore(session_key=session_key)
                # If progress is complete, restart it.
                if s['progressPercent'] == 1:
                    s['progressPercent'] = 0.0
                    s.save()

            return HttpResponse(json.dumps({'progressPercent':s['progressPercent'], 'key':s.session_key, }), content_type="application/json")

2- To analyze the data and generate the report webpage:

def results(request):
    if request.method == 'GET':
        if 'version' in request.GET and request.GET['version'] != '' and 'key' in request.GET and request.GET['key'] != '':
            version = num(request.GET['version'])
            session_key = request.GET['key']

            if session_key != "":
                s = SessionStore(session_key=session_key)
                s['progressPercent'] == 0.0
                s.save()

            # Define a variable to store the number of iterations of the analysis process.
            iterNum = 0.00
            for each iteration in analysis:
               iterNum += 1.00
               ...
               # Do the analysis for this iteration.
               ...
               if session_key != "":
                  s['progressPercent'] = str(iterNum / float(len(analysis)))
                  s.save()

            return render_to_response("Results.html", { Dictionary of Parameters }, context_instance=RequestContext(request))

3- To analyze the data and generate the csv dataset file:

def csvresults(request):
    if request.method == 'GET':
        if 'version' in request.GET and request.GET['version'] != '' and 'key' in request.GET and request.GET['key'] != '':
            version = num(request.GET['version'])
            session_key = request.GET['key']

            if session_key != "":
                s = SessionStore(session_key=session_key)
                s['progressPercent'] == 0.0
                s['csvResponse'] = ""
                s.save()

            iterNum = 0.00
            for each iteration in analysis:
               iterNum += 1.00
               ...
               # Do the analysis for this iteration.
               ...
               if session_key != "":
                  s['progressPercent'] = str(iterNum / float(len(analysis)))
                  s['csvResponse'] += "Content of the generated csv file."
                  s.save()

            return HttpResponse(json.dumps({}), content_type="application/json")

4- To download the generated csv dataset file:

def csvdownload(request):
    if request.method == 'GET':
        if 'version' in request.GET and request.GET['version'] != '' and 'key' in request.GET and request.GET['key'] != '':
            version = num(request.GET['version'])
            session_key = request.GET['key']

            if session_key != "":
                s = SessionStore(session_key=session_key)
                s['progressPercent'] == 0.0
                s['csvResponse'] = ""
                s.save()

                result = HttpResponse(s['csvResponse'], content_type='text/csv')
                result['Content-Disposition'] = 'attachment; filename="csvFile.csv"'

                return result

In my JQuery code, I have handled the click event on all the links with class “ProcessRequired” as follows:

<script>
   $("a.ProcessRequired").click(function ( event ) {

      // Stop the link from opening the link.
      event.preventDefault();

      // Retrieve the href of the link.
      var href = $(this).attr('href');

      // Configure NProgress.
      NProgress.configure({ showSpinner: false, speed: 40 });

      // Define a variable to save the session_key.
      var key = ''

      // Check if the link is for downloading a csv file.
      var isCSVDatasetRequest = $(this).hasClass("csvDataset");

      // Send a request to initialize the session.
      var jqxhr = $.get( "/progressPage", {'key':key} )
          .done(function( data ) {
            NProgress.start();
            $(".pace").removeClass( "pace-inactive" );

            // When the session generated, save it's key to be used in the following requests.
            key = data.key;

            var jqxhr_main = $.get( href, {'key':key} )
              .done(function( data ) {
                clearInterval ( intervalID );
                NProgress.done();
                $(".pace").addClass( "pace-inactive" );
                if (!isCSVDatasetRequest) {
                  document.open();
                  document.write( data );
                  document.close();
                }
                else {
                  document.location.href = '/csvdownload' + href.substring(10,href.length) + '&key=' + key;
                }
              });
          });

        // Every one minute retrieve the percentage of completion.
        var intervalID = setInterval(function(){
          var jqxhr = $.get( "/progressPage", {'key':key} )
            .done(function( data ) {
              progressP = Math.floor(data.progressPercent * 100);
              NProgress.set(progressP / 100.00);
              $("#pace").html(progressP + "%");
              key = data.key;
            });
          }
        , 1000);
      });
    });

</script>

I’ll appreciate it if you provide me with a better solution.

👤1man

Source:stackexchange.com

Leave a comment Cancel reply