Published on June 1, 2021Go home
Packaging Django projects
In this post I will demonstrate how to package a Django project as a Python package.
This post draws inspiration from the post Single-file Python/Django Deployments by Lincoln Loop which discusses alternatives to containerization for Django projects.
I have made use of this technique for some of my own side projects (Including this blog) as well as in combination with Docker for some others. This post will focus on how to convert a new Django project into an installable Python package.
Creating a new project
I prefer poetry for managing my Python development environments but any of the existing Python tools (Pipenv, setuptools, flit etc) will work just fine as well.
$ poetry new example --name webapp # Created package webapp in example
When you run the command above poetry will generate a new package for you under the
example folder named
webapp. The package and folder names can be altered to your preference.
Our new package contains the following directory structure. Note that the test suite sits outside of the source code (More on this and why this is a good thing later).
$ tree example/ example/ ├── pyproject.toml ├── README.rst ├── tests │ ├── __init__.py │ └── test_webapp.py └── webapp └── __init__.py 2 directories, 5 files
Change directory into your new package and add Django as a dependency.
$ poetry add django Creating virtualenv webapp in /home/bradleyk/Projects/example/.venv Using version ^3.2.3 for Django Updating dependencies Resolving dependencies... (4.1s) Writing lock file Package operations: 12 installs, 0 updates, 0 removals • Installing pyparsing (2.4.7) • Installing asgiref (3.3.4) • Installing attrs (21.2.0) • Installing more-itertools (8.8.0) • Installing packaging (20.9) • Installing pluggy (0.13.1) • Installing py (1.10.0) • Installing pytz (2021.1) • Installing sqlparse (0.4.1) • Installing wcwidth (0.2.5) • Installing django (3.2.3) • Installing pytest (5.4.3)
If we inspect the
pyproject.toml file you will notice that we rely on 2 top level packages, Django and pytest. Pytest is provided as the default test framework by poetry. If you have never heard of it I highly recommend you investigate using it to test your Django application alongside the
[tool.poetry] name = "webapp" version = "0.1.0" description = "" authors = ["Bradley Stuart Kirton <email@example.com>"] [tool.poetry.dependencies] python = "^3.9" Django = "^3.2.3" [tool.poetry.dev-dependencies] pytest = "^5.2" [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api"
Next up we can use the
django-admin CLI to generate our Django project and a Django app. The app is not strictly necessary at this stage but I have included it because we need to make a small change to the
apps.py file for the Django apps module loading to work.
$ poetry run django-admin startproject config webapp/ # Create a new project $ mkdir webapp/core # Create a directory for our `core` app $ poetry run django-admin startapp core webapp/core # Create the core app
The above command creates a fresh Django project inside our
webapp package. Our directory structure should now look as follows.
$ tree example/ ├── poetry.lock ├── pyproject.toml ├── README.rst ├── tests │ ├── __init__.py │ └── test_webapp.py └── webapp ├── config │ ├── asgi.py │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py ├── core │ ├── admin.py │ ├── apps.py │ ├── __init__.py │ ├── migrations │ │ └── __init__.py │ ├── models.py │ ├── tests.py │ └── views.py ├── __init__.py └── manage.py 6 directories, 21 files
Making it into a package
Everything up to this point is quite standard issue (We have run a few commands that's all).
In order to make this regular looking Django project into an installable package we need to apply the following:
- Update some module paths in the following files
- Update the
nameattribute of the AppConfig instances defined in your
- Update the app paths in the INSTALLED_APPS list in our settings to match the
nameattribute of the AppConfig
- Include package data including templates into the build process
wsgi.py file sets a value for
DJANGO_SETTINGS_MODULE in the environ. Django uses this environment variable to locate your project settings when run as a wsgi application.
Update the dot path from "config.settings" to "webapp.config.settings".
""" WSGI config for config project. It exposes the WSGI callable as a module-level variable named ``application``. For more information on this file, see https://docs.djangoproject.com/en/3.2/howto/deployment/wsgi/ """ import os from django.core.wsgi import get_wsgi_application # We need to inform Django of the full namespace to the settings module # Change "config.settings" to "webapp.config.settings" # os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings") os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webapp.config.settings") application = get_wsgi_application()
We need to make an identical adjustment to the
manage.py file. This is required so that Django can locate your project settings when you run management commands.
Update the dot path from "config.settings" to "webapp.config.settings".
#!/usr/bin/env python """Django's command-line utility for administrative tasks.""" import os import sys def main(): """Run administrative tasks.""" # We need to inform Django of the full namespace to the settings module # Change "config.settings" to "webapp.config.settings" # os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings") os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webapp.config.settings") try: from django.core.management import execute_from_command_line except ImportError as exc: raise ImportError( "Couldn't import Django. Are you sure it's installed and " "available on your PYTHONPATH environment variable? Did you " "forget to activate a virtual environment?" ) from exc execute_from_command_line(sys.argv) if __name__ == "__main__": main()
settings.py file update the following settings to include the package namespace.
# ROOT_URLCONF = "config.urls" ROOT_URLCONF = "webapp.config.urls" # WSGI_APPLICATION = "config.wsgi.application" WSGI_APPLICATION = "webapp.config.wsgi.application"
Each Django app contains an
apps.py file which exposes an AppConfig instance. Update the
name attribute of the your AppConfig instances to include the package name.
from django.apps import AppConfig class CoreConfig(AppConfig): default_auto_field = "django.db.models.BigAutoField" # name = "core" name = "webapp.core"
Finally when adding apps to the
INSTALLED_APPS list in the
settings.py file be sure to include the package namespace.
INSTALLED_APPS = [ "django.contrib.admin", "django.contrib.auth", "django.contrib.contenttypes", "django.contrib.sessions", "django.contrib.messages", "django.contrib.staticfiles", # Project apps "webapp.core", ]
Including non Python files in the build
By default when you build a Python package only Python code will be included in the distribution. Poetry has some neat features to include and exclude application data within the build process.
include attribute of your
pyproject.toml to include your project templates and other files you require to be shipped in your distribution.
To keep your build artifact lean we can also make use of the
exclude attribute of the
pyproject.toml file. The glob patterns below ensure that no static files are included into the distribution. This makes sense as we can run
manage.py collectstatic outside of the build process to bundle the static assets for deployment.
include = [ "webapp/templates/*.svg", "webapp/templates/*.html", "webapp/**/templates/*.html", ] exclude = [ "webapp/db.sqlite3", # Include this if you are using sqlite "webapp/static/*", "webapp/**/static/*", ]
Bonus (Making manage.py into a CLI)
I like to expose the
manage.py utility as a CLI tool when using this packaged approach. This can be achieved by adding the
tool.poetry.scripts attribute to the
A script is defined using the following syntax.
cli_tool_name = "dotted.path.to:function"
For this example project I have created a CLI named
webapp as follows.
[tool.poetry.scripts] webapp = "webapp.manage:main"
Once you added your script you will need to reinstall your application. Once you have run the install command the
manage.py utility will be available as a CLI.
$ poetry install $ poetry run webapp --help
pyproject.toml file should now look something like this.
[tool.poetry] name = "webapp" version = "0.1.0" description = "" authors = ["Bradley Stuart Kirton <firstname.lastname@example.org>"] include = [ "webapp/templates/*.svg", "webapp/templates/*.html", "webapp/**/templates/*.html", ] exclude = [ "webapp/db.sqlite3", # Include this if you are using sqlite "webapp/static/*", "webapp/**/static/*", ] [tool.poetry.scripts] webapp = "webapp.manage:main" [tool.poetry.dependencies] python = "^3.9" Django = "^3.2.3" [tool.poetry.dev-dependencies] pytest = "^5.2" [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api"
A note on imports
At this stage your application can be built into a Python package. I think it is important to note that the way you import will now be different to the way you would import in a regular Django project. Since our project is now just a regular Python package we need to import like we would from a regular python package.
I will illustrate this with an example.
Create an index view in the
"""Core application views.""" from django.http import HttpRequest, HttpResponse def index_view(request: HttpRequest) -> HttpResponse: """Hello world example view.""" return HttpResponse("Hello world")
Now in your project
urls.py add the view to the
urlpatterns list. Notice that we include the full namespace when importing the view. That is the only difference now that your project is a package.
"""URL config for project.""" from django.contrib import admin from django.urls import path # A regular django project would import from core.views from webapp.core.views import index_view urlpatterns = [ path("admin/", admin.site.urls), path("", index_view, name="index"), ]
A note on testing
As mentioned above the test suite lives outside of the package source code. All tests can be written this way so long as we ensure the imports include the package namespace. One benefit of this approach is our built distribution excludes our development dependencies as well as our test source code.
Building your package
Our built project will consist of two artifacts, namely:
- The package
- The static files
We can achieve the above with two commands but before we can collect static files we will need to add the
STATIC_ROOT attribute to the settings file.
Update the the
settings.py file to place the static files within our root directory.
Now that the static root has been set we can build the application artifacts.
$ poetry run webapp/manage.py collectstatic --noinput 0 static files copied to '/home/bradleyk/Projects/example/staticfiles', 128 unmodified. $ poetry build --format sdist Building webapp (0.1.0) - Building sdist - Built webapp-0.1.0.tar.gz
The above commands generated a directory of static files and an source distribution which can be shipped off to a server. The source distribution can be pip installed while the static assets can be served however you decide.