Chris has been married to the love of his life, Danielle, for over four years now and they recently had their first child, their daughter Helena. Chris was an expert wrangler of developer content for DZone, then spent some time in the marketing department for DZone and AnswerHub, where he taught himself to code html. Now he handles production duties for DZone, so he's responsible for all those emails you get from DZone! BTW, if you're looking for someone to bore you to death with endless football knowledge, Chris is your go-to-guy! Chris is a DZone employee and has posted 320 posts at DZone. You can read more from them at their website. View Full User Profile

Optimizing Django: Some Quick Lessons

  • submit to reddit

While working on a recent project, Luke Plant decided to try django-fiber, a lightweight CMS, because of its improved frontend editing and the ease of sharing content between pages over django-cms.  But he noticed that it was requiring large numbers of queries, some pulling back tons of data, just to render fairly simple pages so he decided to create a "query count reduction branch" and was kind enough to share his results and some further thoughts.

If you're not familiar with Django-fiber, check out this video:

Now, let's look at Luke's results:

I used the example django-fiber project for testing my changes (as well as my own project), and tried them out on both the home page and a more deeply nested page.

-- Luke Plant

What he found was that the home page queries for an anonymous user dropped from 30 to 15, and from 103 to 28 for a staff user.  For the deeply nested page, he found that queries for an anonymous user dropped from 64 to 16 and from 150 to 31 for a staff user.  So by using his query count reduction branch, Luke was able to reduce queries for anonymous users by an average factor of 3 and by an average factor of 4 for staff users.

Luke is quick to point out that a lot of unnecessary work was done just to reach this point in his evaluation, and offers some advice for those working on similar projects:

  • Understand how to optimize DB access for Django.
  • Avoid queries that will duplicate information you already have.
  • Ensure efficient DB filtering when searching for special values.
  • Use the django-debug-toolbar from the get go and make sure to check it regularly.
  • Consider big O scaling issues early in development to avoid future problems with schema design and third-party compatibility.
  • Use tests created with assertNumQueries so you will know when performance regressions are affecting scaling.

So if you're working with Django-Fiber and find it's running too slow, try Luke's query count reduction branch method and be sure to follow his advice.