Bradley Kirton's Blog

Published on June 1, 2021

Packaging Django projects

In this post I will demonstrate how to package a Django project as a Python package.

This post draws inspiration from the post Single-file Python/Django Deployments by Lincoln Loop which discusses alternatives to containerisation for Django projects.

I have made use of this technique for some of my own side projects (Including this blog) as well as in combination with Docker for some others. This post will focus on how to convert a new Django project into an installable Python package.

Creating a new project

I prefer poetry for managing my Python development environments but any of the existing Python tools (Pipenv, setuptools, flit etc) will work just fine as well.

$ poetry new example --name webapp # Created package webapp in example

When you run the command above poetry will generate a new package for you under the example folder named webapp. The package and folder names can be altered to your preference.

Our new package contains the following directory structure. Note that the test suite sits outside of the source code (More on this and why this is a good thing later).

$ tree example/
example/
├── pyproject.toml
├── README.rst
├── tests
│   ├── __init__.py
│   └── test_webapp.py
└── webapp
    └── __init__.py

2 directories, 5 files

Change directory into your new package and add Django as a dependency.

$ poetry add django
Creating virtualenv webapp in /home/bradleyk/Projects/example/.venv
Using version ^3.2.3 for Django

Updating dependencies
Resolving dependencies... (4.1s)

Writing lock file

Package operations: 12 installs, 0 updates, 0 removals

  • Installing pyparsing (2.4.7)
  • Installing asgiref (3.3.4)
  • Installing attrs (21.2.0)
  • Installing more-itertools (8.8.0)
  • Installing packaging (20.9)
  • Installing pluggy (0.13.1)
  • Installing py (1.10.0)
  • Installing pytz (2021.1)
  • Installing sqlparse (0.4.1)
  • Installing wcwidth (0.2.5)
  • Installing django (3.2.3)
  • Installing pytest (5.4.3)

If we inspect the pyproject.toml file you will notice that we rely on 2 top level packages, Django and pytest. Pytest is provided as the default test framework by poetry. If you have never heard of it I highly recommend you investigate using it to test your Django application alongside the pytest-cov and pytest-django plugins.

[tool.poetry]
name = "webapp"
version = "0.1.0"
description = ""
authors = ["Bradley Stuart Kirton <bradleykirton@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.9"
Django = "^3.2.3"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Bootstrapping Django

Next up we can use the django-admin CLI to generate our Django project and a Django app. The app is not strictly necessary at this stage but I have included it because we need to make a small change to the apps.py file for the Django apps module loading to work.

$ poetry run django-admin startproject config webapp/    # Create a new project
$ mkdir webapp/core # Create a directory for our `core` app
$ poetry run django-admin startapp core webapp/core # Create the core app

The above command creates a fresh Django project inside our webapp package. Our directory structure should now look as follows.

$ tree example/
├── poetry.lock
├── pyproject.toml
├── README.rst
├── tests
│   ├── __init__.py
│   └── test_webapp.py
└── webapp
    ├── config
    │   ├── asgi.py
    │   ├── __init__.py
    │   ├── settings.py
    │   ├── urls.py
    │   └── wsgi.py
    ├── core
    │   ├── admin.py
    │   ├── apps.py
    │   ├── __init__.py
    │   ├── migrations
    │   │   └── __init__.py
    │   ├── models.py
    │   ├── tests.py
    │   └── views.py
    ├── __init__.py
    └── manage.py

6 directories, 21 files

Making it into a package

Everything up to this point is quite standard issue (We have run a few commands that's all).

In order to make this regular looking Django project into an installable package we need to apply the following:

Update some module paths in the following files
- wsgi.py
- manage.py
- settings.py
Update the name attribute of the AppConfig instances defined in your apps.py modules
Update the app paths in the INSTALLED_APPS list in our settings to match the name attribute of the AppConfig
Include package data including templates into the build process

wsgi.py

The default wsgi.py file sets a value for DJANGO_SETTINGS_MODULE in the environ. Django uses this environment variable to locate your project settings when run as a wsgi application.

Update the dot path from "config.settings" to "webapp.config.settings".

webapp/config/wsgi.py

"""
WSGI config for config project.

It exposes the WSGI callable as a module-level variable named ``application``.

For more information on this file, see
https://docs.djangoproject.com/en/3.2/howto/deployment/wsgi/
"""

import os

from django.core.wsgi import get_wsgi_application

# We need to inform Django of the full namespace to the settings module
# Change "config.settings" to "webapp.config.settings"
# os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webapp.config.settings")

application = get_wsgi_application()

manage.py

We need to make an identical adjustment to the manage.py file. This is required so that Django can locate your project settings when you run management commands.

Update the dot path from "config.settings" to "webapp.config.settings".

webapp/manage.py

#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys


def main():
    """Run administrative tasks."""
    # We need to inform Django of the full namespace to the settings module
    # Change "config.settings" to "webapp.config.settings"
    # os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webapp.config.settings")
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == "__main__":
    main()

settings.py

In the settings.py file update the following settings to include the package namespace.

webapp/config/settings.py

# ROOT_URLCONF = "config.urls"
ROOT_URLCONF = "webapp.config.urls"

# WSGI_APPLICATION = "config.wsgi.application"
WSGI_APPLICATION = "webapp.config.wsgi.application"

apps.py

Each Django app contains an apps.py file which exposes an AppConfig instance. Update the name attribute of the your AppConfig instances to include the package name.

webapp/core/apps.py

from django.apps import AppConfig


class CoreConfig(AppConfig):
    default_auto_field = "django.db.models.BigAutoField"
    # name = "core"
    name = "webapp.core"

INSTALLED_APPS

Finally when adding apps to the INSTALLED_APPS list in the settings.py file be sure to include the package namespace.

webapp/config/settings.py

INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",

    # Project apps
    "webapp.core",
]

Including non Python files in the build

By default when you build a Python package only Python code will be included in the distribution. Poetry has some neat features to include and exclude application data within the build process.

Update the include attribute of your pyproject.toml to include your project templates and other files you require to be shipped in your distribution.

To keep your build artefact lean we can also make use of the exclude attribute of the pyproject.toml file. The glob patterns below ensure that no static files are included into the distribution. This makes sense as we can run manage.py collectstatic outside of the build process to bundle the static assets for deployment.

pyproject.toml

include = [
  "webapp/templates/*.svg",
  "webapp/templates/*.html",
  "webapp/**/templates/*.html",
]
exclude = [
  "webapp/db.sqlite3", # Include this if you are using sqlite
  "webapp/static/*",
  "webapp/**/static/*",
]

Bonus (Making manage.py into a CLI)

I like to expose the manage.py utility as a CLI tool when using this packaged approach. This can be achieved by adding the tool.poetry.scripts attribute to the pyproject.toml file.

A script is defined using the following syntax.

cli_tool_name = "dotted.path.to:function"

For this example project I have created a CLI named webapp as follows.

pyproject.toml

[tool.poetry.scripts]
webapp = "webapp.manage:main"

Once you added your script you will need to reinstall your application. Once you have run the install command the manage.py utility will be available as a CLI.

$ poetry install
$ poetry run webapp --help

Your pyproject.toml file should now look something like this.

[tool.poetry]
name = "webapp"
version = "0.1.0"
description = ""
authors = ["Bradley Stuart Kirton <bradleykirton@gmail.com>"]
include = [
  "webapp/templates/*.svg",
  "webapp/templates/*.html",
  "webapp/**/templates/*.html",
]
exclude = [
  "webapp/db.sqlite3", # Include this if you are using sqlite
  "webapp/static/*",
  "webapp/**/static/*",
]

[tool.poetry.scripts]
webapp = "webapp.manage:main"

[tool.poetry.dependencies]
python = "^3.9"
Django = "^3.2.3"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

A note on imports

At this stage your application can be built into a Python package. I think it is important to note that the way you import will now be different to the way you would import in a regular Django project. Since our project is now just a regular Python package we need to import like we would from a regular python package.

I will illustrate this with an example.

Create an index view in the core application.

webapp/core/views.py

"""Core application views."""

from django.http import HttpRequest, HttpResponse


def index_view(request: HttpRequest) -> HttpResponse:
    """Hello world example view."""

    return HttpResponse("Hello world")

Now in your project urls.py add the view to the urlpatterns list. Notice that we include the full namespace when importing the view. That is the only difference now that your project is a package.

"""URL config for project."""

from django.contrib import admin
from django.urls import path

# A regular django project would import from core.views
from webapp.core.views import index_view

urlpatterns = [
    path("admin/", admin.site.urls),
    path("", index_view, name="index"),
]

A note on testing

As mentioned above the test suite lives outside of the package source code. All tests can be written this way so long as we ensure the imports include the package namespace. One benefit of this approach is our built distribution excludes our development dependencies as well as our test source code.

Building your package

Our built project will consist of two artefacts, namely:

The package
The static files

We can achieve the above with two commands but before we can collect static files we will need to add the STATIC_ROOT attribute to the settings file.

Update the the settings.py file to place the static files within our root directory.

webapp/config/settings.py

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.2/howto/static-files/

STATIC_URL = "/static/"
STATIC_ROOT = BASE_DIR.parent / "staticfiles"

Now that the static root has been set we can build the application artefacts.

$ poetry run webapp/manage.py collectstatic --noinput
0 static files copied to '/home/bradleyk/Projects/example/staticfiles', 128 unmodified.

$ poetry build --format sdist
Building webapp (0.1.0)
  - Building sdist
  - Built webapp-0.1.0.tar.gz

The above commands generated a directory of static files and an source distribution which can be shipped off to a server. The source distribution can be pip installed while the static assets can be served however you decide.