Optimizing Django: Some Quick Lessons
While working on a recent project, Luke Plant decided to try django-fiber, a lightweight CMS, because of its improved frontend editing and the ease of sharing content between pages over django-cms. But he noticed that it was requiring large numbers of queries, some pulling back tons of data, just to render fairly simple pages so he decided to create a "query count reduction branch" and was kind enough to share his results and some further thoughts.
If you're not familiar with Django-fiber, check out this video:
Now, let's look at Luke's results:
I used the example django-fiber project for testing my changes (as well as my own project), and tried them out on both the home page and a more deeply nested page.
-- Luke Plant
What he found was that the home page queries for an anonymous user dropped from 30 to 15, and from 103 to 28 for a staff user. For the deeply nested page, he found that queries for an anonymous user dropped from 64 to 16 and from 150 to 31 for a staff user. So by using his query count reduction branch, Luke was able to reduce queries for anonymous users by an average factor of 3 and by an average factor of 4 for staff users.
Luke is quick to point out that a lot of unnecessary work was done just to reach this point in his evaluation, and offers some advice for those working on similar projects:
- Understand how to optimize DB access for Django.
- Avoid queries that will duplicate information you already have.
- Ensure efficient DB filtering when searching for special values.
- Use the django-debug-toolbar from the get go and make sure to check it regularly.
- Consider big O scaling issues early in development to avoid future problems with schema design and third-party compatibility.
- Use tests created with assertNumQueries so you will know when performance regressions are affecting scaling.
So if you're working with Django-Fiber and find it's running too slow, try Luke's query count reduction branch method and be sure to follow his advice.