Django Performance

A Guide To Django Performance Tips For Optimization & Testing

By Harikrishna Kundariya Python 3 weeks ago 14 min read 27 Views

In web-application development, performance optimization is an endless yet quintessential task. This not only helps in increasing the number of users but also helps in increasing the usability of the applications. Today, we will discuss Django Performance Tips.

Django is based on the Python Web Framework. Django is an easy to go solution for developing, releasing, and maintaining a program. It is used by several big houses such as Google, Instagram, Youtube, NASA, etc.

Django is based on MVT (Model-View-Template) architecture. Django’s data can be handled by “model”, whereas “view” helps to see how the website or web application would look to the user, and the template helps in designing of the website by showing us the wide range of layouts which are present in Django.

Django, along with python, provides you with high-performance web application but you can further optimize the Django Performance based web application by implementing these tips:

1. Caching

Typically, if you are going to the same value repetitively, it might be a good idea to save it so that every time a client requests, we do not have to recompute it again and again.

This caching of data is a powerful technique that can provide you huge benefits while serving a high number of requests.

High Performance Django has a strong caching framework in which you can save the dynamic content so that you don’t have to compute the data again and again. It gives you granular options to either cache the whole page or some selected fragments or some specific output.

You can use the cached-property decorator on a function so that next time the function is called, you will get the saved value instead of recomputing the function again 

2. Laziness

Laziness and Caching go hand-in-hand. As Caching prevents the computation of the values, again and again, Laziness helps you by executing a certain function only when the value is required.

Laziness avoids computation in the first place and it is done only when it is required. This helps since the value that was computed earlier might not be even used in the future and we will save this computation.

The operations in High Performance Django are lazy. For instance, QuerySets are lazy. QuerySets will not fetch data when you create, pass, or combine them. Only when the values in QuerySets are being used, those values will be fetched from the database.

One more way you can optimize the performance is by forcing delayed executions of fetching data from the database. Certain operations force the QuerySets to prematurely fetch the data. You should try to avoid such operations.

You can also add keep_lazy () decorator to the function. This decorator forces Django to evaluate the function only when the values from those are required. Typically you should use this decorator for functions that are computation extensive.

3. Database Optimization

One of the main factors slowing the performance of the web applications is the requests done to the database. So, the database operations and queries have to be molded for a function to work optimally.

With SQL, you need to take care of certain aspect of database queries such as:

  • Database triggers,
  • stored views,
  • functions,
  • and stored procedures.

The database operation can be optimized if you fetch all the data required in one query. QuerySets are lazy and fetch only required data which may cause multiple fetch operations on the database as the function gets executed.

You can see from the below code snippet that when we get the authors list again from the database, it fetches the data from the cache.

>> entry = Entry.objects.get (id=1)

>> entry.authors.all()  #query performed

>> entry.authors.all() #query performed again

Another way to optimize the database is to get only the required fields. Typically, Django gets all the fields from the table irrespective of the size of the table.

But, if you require only a few fields and it brings all, then the effort of bringing the other irrelevant fields is wasted. This can be done by using defer ( ) and only( ).

Defer command tells what all fields should not be fetched whereas only () command specifies the field which should be fetched. 

4. HTTP Performance

Django provides middleware such as ConditionalGetMiddleware and GzipMiddleware which you can use to enhance the performance of the requests.

The ConditionalGetMiddleware enhances the performance by conditionally sending GET responses. It adds Etag and Last-Modifier header to the requests.

Next time it gets a request, if the header value is If-None-Match or If-Modified-Since, it will not process the GET request and send HttpResponseNotModified response.

The GzipMiddleware zips all the responses from the browsers that understand gzip compression. You should add this middleware once the response is completely formed and all headers are added.

Yet, GzipMiddleware is considered a security threat currently by the security experts. So, make sure you use it with caution.

5. Sessions

Cached sessions can be stored in order to optimize performance. The cached Session will help to bring down the number of times a system tries to reach the database. The cached Session will save the information in itself.

Django supports the session cached by using SessionMiddleware. You can add the middleware by adding “django.contrib.sessions.middleware.SessionMiddleware” in your default settings.py.

High Performance by default stores your session in the database, but you can change it to store the data either on the file system or in the cache.

You can set the SESSION_ENGINE field to “django.contrib.sessions.backends.cache” in order to save the sessions in the cache.

You can also save the sessions in cookies. The session saved in the cookies is encrypted using the SECRET_KEY and you should make sure attackers should not get access to SECRET_KEY otherwise they may read the session data and execute code remotely.

6. Code Optimization

It is important that you optimize your domain logic code so that it does not take a lot of time to execute.

There are three prominent ways to lift the efficiency of the code:

  • Upgrading/Replacing the third-party packages
  • Simplifying the code
  • Restructuring the code

To upgrade or replace the third-party packages you can give a quick glance at the code and find the third-party packages that come with a lot of baggage for any simple function.

Then you can remove or replace them as per requirement. Otherwise, you can also use a Python profiler to do this task.

Another way is to simplify the code: In Django, the rest framework is really multifaceted and has a lot of innovative features. But for simpler purposes, you can write your own serializer and save your time.

Also, you can use python profiler for not only third-party packages but also your own code. You can restructure the part of the code which is eating a lot of CPU and memory in order to optimize the Django Performance.

Below is a very simple example where we are profiling regex call

import cProfile

import re

cProfile.run('re.compile("foo|bar")')

Once you run the above application, you get the output as below. It shows how many functions calls were executed and how many of them are primitive.

197 function calls (192 primitive calls) in 0.002 seconds

Ordered by: standard name

ncalls  tottime  percall  cumtime  percall filename:lineno(function)

     1    0.000    0.000    0.001    0.001 <string>:1(<module>)

     1    0.000    0.000    0.001    0.001 re.py:212(compile)

     1    0.000    0.000    0.001    0.001 re.py:268(_compile)

     1    0.000    0.000    0.000    0.000 sre_compile.py:172(_compile_charset)

     1    0.000    0.000    0.000    0.000 sre_compile.py:201(_optimize_charset)

     4    0.000    0.000    0.000    0.000 sre_compile.py:25(_identityfunction)

   3/1    0.000    0.000    0.000    0.000 sre_compile.py:33(_compile)

7. Cached Template Loading

The Cache Template loader of Django is another weapon that can be used for optimization. You can enable Cache Template Loader and it will save all of your time and bandwidth to bring the same templates now and then.

For instance, some templates are frequently used and if you enable Cache Template Loader, those templates will be saved.

You can always get these templates directly from the memory, saving you a round trip to the file system. This can help in significantly reducing the response time and making the application performance better. So, Django Performance Testing is important. 

8. Specific Functions

Since QuerySets lazily loads the data, Django added two functions – select_related and prefetch_related() which can be used to execute the query and get better results.

select_related()

This function will return additional related objects based on foreign keys when the query is executed. It is a single more complex query, which if executed, gives foreign objects. If we had used traditional select objects, we might have to query the database again to get those objects.

prefetch_related()

This function is also similar to select_related() and prefetches the foreign objects. select_related() is limited to single-value relationships but prefetch_related() performs a separate lookup for each relationship and fetches more data. 

9. Using Newer Version

Upgradation is obvious in development. It is always thought that the newer the better and efficient it is.

Yet, one should not always run behind the newer versions. When you try to get the latest version of Python or Django make sure you have your system optimized.

Also, you should not keep your hopes high from the latest versions as sometimes it can be the other way around. So, being more cautious while updating the newer version is the key.

10. Make Use Of line_profiler

You can use line_profiler if you don’t know which line of code is eating up the time. You can install the line_profiler package using pip.

Once you suspect a function that is causing the delay, you need to add IPython Debugger just before the function is called. Once the function is hit, you need to load the line profiler extension in the debugger and execute the function.

The line profiler tells which lines take how much time to execute. Thus you can fix those lines and have better performance.

For example, imagine a function that returns a name of books from the database.

def get_books_by_library_id():

    libraries = Library.objects.all()

    result = {}

    for library in libraries:

        books_in_library = library.books.all()

        result[library.id] = list(books_in_library)

    return result

This slow downs the process. So to find the root cause of this problem, you can make use of IPython debugger as shown below:

from django.http import HttpResponse

from test_app.helpers import get_books_by_library_id

def books_by_library_id_view(request):

    from IPython import embed; embed()

    books_by_library_id = get_books_by_library_id()

    ...

    return HttpResponse(response)

When you use this function, you will be able to know the exact time that CPU spends in solving this code. This is as shown below:

Python-Line-Profiler

11. Turn On SQL Logging

Another way to find out the cause behind the delay of execution is by enabling SQL Logging.

By adding these lines of code in settings.py, you can print all the SQL queries executing in your code. This also helps in Django Performance Testing.

This may sound like too much but it will give you meaningful insight about which all queries are getting executed and how much time each query is taking.

It can also help you in finding if the query is repeated.

# settings.py

LOGGING = {

    'version': 1,

    'filters': {

        'require_debug_true': {

            '()': 'django.utils.log.RequireDebugTrue',

        }

     },

     'handlers': {

        'console': {

            'level': 'DEBUG',

            'filters': ['require_debug_true'],

            'class': 'logging.StreamHandler',

         }

    },

    'loggers': {

        'django.db.backends': {

            'level': 'DEBUG',

            'handlers': ['console'],

        }

    }

}

One of the best ways to do this would be to put a debugger at the starting and end of the code, which you think is taking a lot of time. Then, slowly move further and find the queries which are slow and repeated.

12. Avoid Queries in Loops

There are a few instances where we execute a query to get some data and then again execute the query to get the related data.

Imagine you want to fetch all countries of the world, so you execute a query for it.  Later you realize you also need their capitals, so you would have to query again.

So in order to solve this problem of executing queries in the loop, Django introduced functions like select_related() and prefetch_related() in the QuerySets API.

It can be used to fetch all the related data in one trip to the database. As we can see from the below code snippet, it fetches the related also.

def get_books_by_library_id():

    libraries = Library.objects.all()

    result = {}

    for library in libraries:

        books_in_library = library.books.all()

        result[library.id] = list(books_in_library)

    return result

This creates a performance problem. Instead of this, use the following coding snippet:

def get_books_by_library_id_one_query():

    books = Book.objects.all()

    result = defaultdict(list)    

    for book in books:

        result[book.library_id].append(book)

    return result

This  will generate only 1 SQL query and it will reflect in the Django Performance, without a pinch of a doubt.!

13. Use Memcached Server

Django provides a very powerful and efficient cache solution called Memcached.  It was initially developed for livejournal.com and later it was open-sourced.

Few prominent websites like Facebook and Wikipedia use Memcached to reduce database operations and fetching data from the cache.

You have to set your backend to use Memcached by setting BACKEND to ‘django.core.cache.backends.memcached.MemcachedCache’.

Memcached runs as a daemon and it is allocated specific memory on the RAM, so all the operations performed by Memcached are directly interacting with memory and they never fetch data from database or filesystem,

Another major advantage of Memcached is it can run on multiple servers and treats all the cache on these servers as a single cache. As a result of this, duplicate values will not be saved.

However, you cannot completely rely on Memcached for data storage. This feature can only be used for short-term data storage.

Read also: Things To Consider While Hiring A Python Developer

14. Make Use of Elasticsearch

Django has a very efficient paginator. It works well but as the length of pages increases, you might have to fetch a lot of data from the database and it slows down the applications.

An efficient way is to use Elasticsearch where you store your data in indexes and once the index is fetched, you can fetch the data in the database for that particular index. This helps us still have high performance when we have many pages.

15. Reduce the Number of Queries

One of the most overlooked factors when considering performance is the query execution speed and the way queries are written. Typically, when we develop applications, the amount of data present in the database is less and we are not able to see the performance hit.

But when your application is used by multiple users, the database can grow in size and the query execution can cause slowness. Also since we use Django ORM, which is a wrapper, we do not get a picture of how the query is being executed.

Django offers a tool – Debug Toolbar which has a SQL panel in which you can see the execution of queries and the time taken by each of them.

SQL Query

Conclusion

To put it all together, these are the sure-shot expert ways to optimize the Django Performance. But as you try to implement the ways, you should analyze the situation from your end, the database and your requirements, and choose the most appropriate way.

Time is money. This optimization has proved that. Hence, the time that you take and the efforts that you put it have to be well-thought of. Just as an accelerated vehicle uses more fuel, you should also consider when and how to accelerate the system.

Remember, the acceleration should not add to the time and effort but help you to optimize the function. We hope this article will be of great help to any Python Web Development Company. Thank You.!

FAQ

There are some tips and tricks through which you can easily speed up the performance of a Django app. They are as listed below:

Caching
Laziness
Using Elasticsearch
Database Optimization
Code Optimization
Cache Template Loading

The simple answer to this question would be YES. Django is a free and open-source python framework. It helps you develop websites in a faster and efficient way. So, speed is not an issue when it comes to Django.

Django is neither Front-End or Back-End. It is a framework based on the python programming language that provides you with a collection of libraries to create high-quality web applications.

Oh.! YES. Django is way better than PHP especially when you talk of speed and performance. PHP is generally good for the short term. But, if you’re developing something that needs to be scalable, Django is fantastic.

Experts recommend that you should always use the latest version that is available as it has all the fixes for bug and security issues. So, in the case of Django, you should utilize Python 3.5 at least.

Django Performance

Disclaimer: We at eSparkBiz Technologies have created this blog with all the consideration and utmost care. We always strive for excellence in each of our blog posts and for that purpose, we ensure that all the information written in the blog is complete, correct, comprehensible, accurate and up-to-date. However, we can’t always guarantee that the information written in the blog correct, accurate or up-to-date. Therefore, we always advise our valuable readers not to take any kind of decisions based on the information as well as the views shared by our authors. The readers should always conduct an in-depth research before making the final decision. In addition to these, all the logos, 3rd part trademarks and screenshots of websites & mobile apps are the property of the individual owners. We’re not associated with any of them.

Harikrishna Kundariya

Harikrishna Kundariya

Harikrishna Kundariya, a marketer, developer, app lover, technology savvy, designer, co-founder, Director of eSparkBiz @Mobile App Development Company where you can Hire Mobile App Developer. His 8+ experience enables him to provide digital solutions to new start-ups based on app development.

Search

Follow us

Subscribe Newsletter

Call us Today

telephone

Got any Questions?
call us Today!

+1 408 850 1488 [email protected]

Our Guides

Related Post

Why Python For Startup Is A Perfect Combination?

Why Python For Startup Is A Perfect Combination?

Do you know the top-notch brands of contemporary times were started during the great recession of 2008-2009? Airbnb, Uber, Pinterest, Whatsapp, and Slack are among…

1 Share
The Ultimate Guide To Find & Hire Django Developer

The Ultimate Guide To Find & Hire Django Developer

Django is a python framework for building web-based applications. It is built with a toolbox approach where all important functionalities required to build a web…

1 Share
Top 40 Python Web Frameworks that Every Developer Should Know

Top 40 Python Web Frameworks that Every Developer Should Know

Python is an exceptional programming language next to JavaScript, which is popular and widely used by developers across. Today, we will talk about the Python…

2 Shares
Share via
Copy link