Django Unboxed – Awesome Boilerplate by Systango

Last Updated on: July 10, 2020

Django Unboxed – Awesome Boilerplate by Systango

Django in itself is a good framework with great features and can be used for many applications straightaway. We at Systango have used that in its core best many a times. But over time we have realised that we can make Django great with some additions.

Hence we designed our very own Boilerplate – “Django Unboxed”. This boilerplate was designed to cater the basic challenges and aid a developer with general prerequisites that are needed often by any web developer. In modern era of web development where people are generally aiming for REST based APIs and docker ready applications. We have boxed all such exciting features in this very boilerplate.

This can be cloned from: Github 

Here are highlights of packages that we have boxed in this boilerplate –

  • Django Rest-framework + jwt authentication
  • Docker + Docker Compose
  • Mocked Unit Test Cases
  • Logging (Database + Requests)
  • Data visualisation plugin(Kibana)
  • Celery
  • Swagger

Let me now take you through each addition that we have done in this boilerplate and how they are configured.

Django Rest-framework(DRF)
Django Rest-framework (DRF) is a library which works with standard Django models to build a flexible and powerful REST APIs. We have used this as most of the work is nowadays API based. With introduction of powerful web front-ends like React, Angular etc. APIs are becoming quintessential in web development and DRF is a powerful tool for doing that with its slick architecture and design, it becomes an obvious addition to the boilerplate.

Usage of django rest-framework:

1)Install django rest framework using:
   pip install djangorestframework

2)Update your settings.py file to add:

INSTALLED_APPS = (
	...
    'rest_framework',
)
REST_FRAMEWORK = {
   'DEFAULT_AUTHENTICATION_CLASSES': (
       'need_done.lib.authentication.CsrfExemptSessionAuthentication',
   ),
   'DEFAULT_PERMISSION_CLASSES': ( 'rest_framework.permissions.IsAuthenticated',),
}

3)Create serializers.py file and add serializer:

class UserSerializer(serializers.ModelSerializer):
    class Meta:
           model = User
           fields = ['first_name', 'username', 'email', 'last_name']

4)Create views.py file and add viewset:

class UserViewSet(viewsets.ModelViewSet):
    queryset = User.objects.all()
    serializer_class = UserSerializer

5)Create urls.py file and add urls in it:
url(r’^list_users$’, UserViewSet.as_view(), name=’list-users’)

JWT (JSON Web Token)
JSON Web Token (JWT) defines a way for securely transmitting information between parties as a JSON object. JWT is digitally signed using JSON Web Signature (JWS) and/or encrypted using JSON Web Encryption (JWE). This has become a standard in the industry and most of the web apps are using, hence added in boilerplate.

Steps to use JWT:
1)Install django rest framework jwt using:
Pip install djangorestframework-jwt

2)Update settings.py file to add:

JWT_AUTH = {
    'JWT_VERIFY_EXPIRATION': True,
    'JWT_EXPIRATION_DELTA': datetime.timedelta(days=1),
    'JWT_ALLOW_REFRESH': False,
    'JWT_REFRESH_EXPIRATION_DELTA': datetime.timedelta(days=7),
    'JWT_AUTH_HEADER_PREFIX': 'token',
}

3)Add this in views.py file:
   from rest_framework_jwt.serializers import JSONWebTokenSerializer
   from rest_framework_jwt.views import JSONWebTokenAPIView

class LoginView(JSONWebTokenAPIView):
    serializer_class = JSONWebTokenSerializer
    
    __doc__ = "Log In API for user which returns token"

    def post(self, request, format=None):
        try:
            serializer = self.serializer_class(data=request.data)
            serializer.is_valid(raise_exception=True):
            serialized_data = serializer.validate(request.data)
            user = User.objects.get(email=request.data.get('email'))
            return Response({
                'status': True,
                'token': serialized_data['token'],
            })
        except Exception as e:
            raise e

4)Update app’s urls.py file to add url for LoginView:
    url(r’^login$’, LoginView.as_view(), name=’login’)

Docker
Docker is an open platform for building, shipping and running distributed applications. It gives programmers, development teams and operations engineers the common toolbox they need to take advantage of the distributed and networked nature of modern applications.

Docker Compose:
Compose is a tool for defining and running multi-container Docker applications. Docker Compose will assist you in specifying how you want the containers to be built and connected, using a single command. In order to user docker compose you must install it first.

Dockerize your django project:
1)Create a Dockerfile: Dockerfile will have a set of instructions on how Docker will build a container image for your application. Dockerfile should be saved by the name Dockerfile without any extension.

Dockerfile
FROM python:3

ENV PYTHONUNBUFFERED 1

RUN mkdir /code

WORKDIR /code

ADD requirements.txt /code/

RUN pip install -r requirements.txt

ADD . /code/

2)Create a docker-compose.yml file in your project directory:
   docker-compose.yml
    version: ‘3’
    services:
        django:
            build:
               context: .
               dockerfile: Dockerfile (path to your Dockerfile)
               command: bash -c “python manage.py makemigrations && python manage.py migrate && python manage.py runserver 0.0.0.0:8000
               restart: always
               ports:
               – “0.0.0.0:8000:8000”

3) To run docker use command:
docker-compose build: is to install the requirements and run migrations and migrate them if any, then
docker-compose up: this will start the server.

Mocked Unit Test Cases
This was a challenging piece of work. As the default Django behaviour does not provides a pure mockable environment. It generates a test database on starting of test cases and then deletes that post completion. We have provided examples as well by which one can generate mock objects and run their tests.

Steps to write unit mock test cases:
1)Create testrunner.py file near your settings.py file:
    from django.test.runner import DiscoverRunner
class NoDbTestRunner(DiscoverRunner):
      “”” A test runner to test without database creation “””
    def setup_databases(self, **kwargs):
          “”” Override the database creation defined in parent class “””
    pass
   def teardown_databases(self, old_config, **kwargs):
   “”” Override the database teardown defined in parent class “””
   pass

2)Create no_db_settings.py file near testrunner file:
from boilerplate.settings import *

TEST_RUNNER = boilerplate.testrunner.NoDbTestRunner’

3)Write test cases in tests.py file:
from django.core.urlresolvers import reverse
from rest_framework.test import APITestCase

from model_mommy import mommy
from unittest.mock import patch

from .models import User

class UserAPIViewTestCase(APITestCase):

   def setUp(self):

       self.user1 = mommy.prepare(User)
       self.user2 = mommy.prepare(User)
       self.user3 = mommy.prepare(User)
       self.user = [self.user1, self.user2, self.user3]        self.URLS = {
           ‘list_users’: reverse(“users:list-users”)
           ‘login’: reverse(“users:login”),
           #”users” is the namespace defined for the app
       }

   @patch(‘users.models.User’)
   def test_list_users(self, mock_user):        

       mock_user.all.return_value = self.user
       response = self.client.get(self.URLS[‘list_users’])
       self.assertEqual(200, response.status_code)

4)Run test cases using:
python manange.py test –settings=boilerplate.no_db_settings

Logging
Logging is a means of tracking events that happen when some software runs. Logging is important for software developing, debugging and running. If we have used logging in our software then its very helpful in case of software failure to identify the cause. In boilerplate we have used requests and database logging, also a unique request id for each requests. So that we can know if there is something wrong in the API or Query executed.

Log formats used in boilerplate:
request log format– request_id:time:http_method:API_Path:data:URIPATHPARAM
db log format– request_id:query:execution_time_of_query:time

An easy configuration to write all the logs in a local file:

	LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'request_log_file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/path/to/django/request.log',
        },
        'db_log_file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/path/to/django/database.log',
        },
    },
    'loggers': {  
        'django': { # this is used for django logs such as API requests and responses.
            'handlers': ['request_log_file'],
            'level': 'DEBUG',
            'propagate': True,
       },
        'django.db.backends': { # this is used for database logs such as SQL query executed.
            'handlers': ['db_log_file'],
            'level': 'DEBUG',
        },
    },
}

Kibana
Kibana is an open source analytics and visualization platform designed to work with Elasticsearch. We can create great dashboards using predefined and custom Graphs for analysing metrics like request response time, number of requests per second etc parameters that are helpful in determining the performance of the APIs. Its simple, browser-based interface enables you to quickly create and share dynamic dashboards that display changes to Elasticsearch queries in real time.

Filebeat: Filebeat is used for transporting the generated logs by application to Kibana. Where they can be analysed better.
We can also search through these logs to find the particular requests. To forward the logs to kibana filebeat uses a pipeline, that pipeline should be in same format as the logs generated.

Example Usage :-
if you have log format type “request_id:time:http_method:api_url:response_message” , then pipeline should be like this:

pipeline.json
	{
    "description" : "my sample pipeline to show grok",
    "processors": [
    {
        "grok": {
            "field": "message",
            "patterns": ["%{NUMBER:request_id} %{DATESTAMP:date} %{WORD:httpmethod} %{URIPATHPARAM:request_url} %{WORD:message}"]
        }      
        }
    ]
}

To create this pipeline execute “curl -XPUT
https://your_host/_ingest/pipeline/test-pipeline‘ -d@pipeline.json -H ‘Content-Type: application/json’” command from terminal, and to get the created pipeline open browser and search for “https://your_host/_ingest/pipeline“.

Metricbeat: Metricbeat is used to monitor the services, cpu, memory, disc, processes etc usage and forwards the logs of them to kibana using elasticsearch. Metricbeat helps you monitor your servers by collecting metrics from the system and services running on the server.

Heartbeat: Heartbeat is used to monitor the health and status of the system and services running on the server, so that you can know when your service is up and running or down.

Celery:
Celery is an asynchronous task queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or multiple worker. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Steps to use celery:
1) Install celery using:
pip install celery

2) Create celery.py and __init__.py near your settings.py file.
Celery.py
import os
from celery import Celery
from django.conf import settings

# set the default Django settings module for the ‘celery’ program.
os.environ.setdefault(‘DJANGO_SETTINGS_MODULE’, ‘your_poject_name.settings’)
app = Celery(‘your_poject_name’)

app.config_from_object(‘django.conf:settings’)
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

__init__.py
from .celery import app as celery_app

3) Create a tasks.py file inside your app.
from __future__ import absolute_import, unicode_literals
from celery import shared_task

@shared_task
def add(x, y):
   return x + y

4) Use this task wherever you want as:
x = 4
y = 5
add.delay(x, y)

Swagger
Swagger is a framework for describing your API using a common language that everyone can understand.

There are two main ways to take advantage of Swagger:
1) Top-down approach, or design-first: This means you’re using Swagger to design your API before you’ve written any actual code.
2) Bottom-up approach, or code-first: This means you’ve already written the code for your API, and you’ll be using Swagger to document your API.

We have used django-rest-swagger(built-in support) in boilerplate to document our APIs.
The built-in API documentation includes:

  • Documentation of API endpoints.   
  • Automatically generated code samples for each API available.
  • Support for API interaction.

Usage of django-rest-swagger:
1) Install django rest swagger using:
pip install django-rest-swagger

2) Include django rest swagger in INSTALLED_APPS:

INSTALLED_APPS = [
    ...
    'rest_framework_swagger',
    ...
]

3)Create swagger.py near your settings.py file:
swagger.py
from rest_framework.response import Response
from rest_framework.decorators import api_view, permission_classes, renderer_classes
from rest_framework.permissions import AllowAny
from rest_framework_swagger.renderers import OpenAPIRenderer, SwaggerUIRenderer
from rest_framework.schemas import SchemaGenerator

@api_view()
@permission_classes((AllowAny, ))
@renderer_classes([OpenAPIRenderer, SwaggerUIRenderer])
def schema_view(request):
   “””
   Swagger test API
   “””
   generator = SchemaGenerator(title=’Rest Swagger’)
   schema = generator.get_schema(request=request)

   return Response(schema)

4) Use schema_view defined above in urls.py file:
url(r’^swagger/$’, schema_view, name=”schema_view”),

I hope that this blog might have helped you to get an insight on the mechanics of the boilerplate. Please feel free to fork this on GitHub and submit changes if you feel that they must be added.

Systango is an expert when it comes Python and Django Development. We have endless number of projects developed and successfully delivered for customers all across the globe. We have a dedicated team which is continuously involved in researching new Python and Django milestones and implement them in our projects. Contact Systango today for Python or Django web/mobile app development.

Mohit Rajput

September 21, 2018

Leave a Reply

Your email address will not be published. Required fields are marked *