[Django]-Converting pandas dataframe to json is slow

2👍

The fastest way to do something is usually to avoid doing it, so maybe you could just save the generated json to a data.json file in your app/static directory, moving your current code to a custom management command that you execute as part of your deployment process.

Custom management commands are python scripts that can be executed from the command line using ./manage.py <yourcommandname> .... This is documented here: https://docs.djangoproject.com/en/2.0/howto/custom-management-commands/

In this case you’re command’s handle method would be responsible for converting the csv to json (using the code that’s currently in your view) and storing it to a data.json file in your app/static folder. Then your view would just have to json.load() this data.json file and serve it.

Then all you have to do is to make sure this command is called whenever you update your csv file. This can be done manually if you have no deployment script (just don’t forget to document it in your deployment procedure documentation), or automated in your deployment script to make sure you won’t have stale data.

Leave a comment