How to Improve Django 3.1 View Performance with Async Support

January 2nd 2021

Django 3.1 was recently released and along with it came support for asynchronous views.

0 reactions

To anyone that uses Django and works with lots of data from lots of different Web sources, this is a big deal. Supporting asynchronous views means cleaner code, a different way to think about things, and most importantly, the ability to dramatically improve performance of applications.

0 reactions

But let’s back up a bit.

0 reactions

If you’re unfamiliar with the term “views”, don’t worry, it’s an easy concept to understand.

0 reactions

Views are key components of applications built-in with the Django framework. At their very simplest, views are Python functions or classes that take a web request and produce a web response. Prior to Django 3.1, views had to run with Python threads. Now, views can run in an asynchronous event loop without Python threads. This means Python’s
asyncio library can be used inside of Django views.

0 reactions

According to the documentation:

The main benefits are the ability to service hundreds of connections without using Python threads.

0 reactions

There are other benefits as well including the use of Python’s asyncio.gather.

0 reactions

Let’s say you have a view that makes four API calls. Even in a best-case scenario, if each call takes only a second, it’s a total of four seconds if executed synchronously. And that’s the best-case scenario.

0 reactions

We can cut down on that time frame significantly and improve the situation overall by using Pythons concurrent.futures library.

This makes it possible to make the four API calls in the previous example concurrently meaning the view could take roughly one second in total if using four workers with the ThreadPoolExecutor. By all accounts, the practice cuts down on the time needed and improves the calls.

0 reactions

That’s important in a world where seconds matter and making someone
wait around for an application to load can cause frustration.

0 reactions

A Real-World Example: Tracking Montana's Yellowstone River

To illustrate how asynchronous views improve performance, I created an example project to display statistical data from the United States Geological Survey (USGS).

0 reactions

The project makes six API calls to the USGS to collect data about six access points on the Yellowstone River in my home state of Montana. This data includes the volume of water moving at each access point at the time, known as discharge rate, as well as the gage height, or stage, which is the surface level of the water relative to its streambed.

0 reactions

Example 1: The Synchronous Method

Code:

0 reactions

def get_river_flow_and_height(site_id):
  """
  Synchronous method to get river data from USGS
  """
  response = requests.get(f'https://waterservices.usgs.gov/nwis/iv/?format=json&sites={site_id}&parameterCd=00060,00065&siteStatus=all')
  data = parse_flow_and_height_from_json(response.json())
  return data

def dashboard_v1(request):
  """
  Synchronous view that loads data one at a time
  """
  start_time = time.time()

  river_data = []

  for site_id in SITES.keys():
      river_data.append((SITES[site_id], get_river_flow_and_height(site_id)))

  return render(request, 'rivers/dashboard.html', {
      'river_data': river_data,
      'version': 'v1',
      'load_time': time.time() - start_time,
  })

Result:

0 reactions

The data loads and takes almost four seconds. For the purposes of this post, that’ll be our baseline. Let’s see if we can improve that situation.

0 reactions

Example 2: A Concurrent View Loading Some Data Simultaneously

def dashboard_v2(request):
  """
  Concurrent view that loads some data simultaneously
  """
  start_time = time.time()

  river_data = []

  with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
      for site_id, data in zip(SITES.keys(), executor.map(get_river_flow_and_height, SITES.keys())):
          river_data.append((SITES[site_id], data))

  return render(request, 'rivers/dashboard.html', {
      'river_data': river_data,
      'version': 'v2',
      'load_time': time.time() - start_time,
  })

Result:

0 reactions

Now we’re down to roughly 1.5 seconds and that’s a big improvement. Let’s see what happens when we leverage asynchronous views.

0 reactions

Example 3: The Asynchronous Method

async def get_river_flow_and_height_async(site_id):
  """
  Asynchronous method to get river data from USGS
  """
  async with httpx.AsyncClient() as client:
      response = await client.get(f'https://waterservices.usgs.gov/nwis/iv/?format=json&sites={site_id}&parameterCd=00060,00065&siteStatus=all')
      data = parse_flow_and_height_from_json(response.json())
  return data


async def dashboard_v3(request):
  """
  Asynchronous view that loads data using asyncio
  """
  start_time = time.time()

  river_data = []

  datas = await asyncio.gather(*[get_river_flow_and_height_async(site_id) for site_id in SITES.keys()])
  for site_id, data in zip(SITES.keys(), datas):
      river_data.append((SITES[site_id], data))

  return render(request, 'rivers/dashboard.html', {
      'river_data': river_data,
      'version': 'v3',
      'load_time': time.time() - start_time,
  })

Result:

0 reactions

Wow, we got results back in just under a second. That’s roughly a three-second improvement on the original method.

0 reactions

This example shows pretty clearly how asynchronous views can be leveraged to drastically improve performance.

This is just a small example of the performance improvements we see on a daily basis with the projects we work on at NextLink Labs.

0 reactions

View the full project on GitLab: https://gitlab.com/nextlink/example-django-async-rivers

0 reactions

Previously published at https://nextlinklabs.com/insights/django-async-views-improves-API-calls

0 reactions