Acknowledgements

The most gratifying aspect of working on Django is the community. We’ve been especially lucky that Django has attracted such a smart, motivated, and friendly bunch. A segment of that community followed us over to the online “beta” release of this book. Their reviews and comments were indispensable; this book wouldn’t have been possible without all that wonderful peer review. Almost a thousand people left comments that helped us improve the clarity, quality, and flow of the final book; we’d like to thank each and every one of them.

We’re especially grateful to those who took the time to review the book in depth and left dozens (sometimes hundreds) of comments apiece: Marty Alchin, Max Battcher, Oliver Beat- tie, Rod Begbie, Paul Bissex, Matt Boersma, Robbin Bonthond, Peter Bowyer, Nesta Campbell, Jon Colverson, Jeff Croft, Chris Dary, Alex Dong, Matt Drew, Robert Dzikowski, Nick Efford, Ludvig Ericson, Eric Floehr, Brad Fults, David Grant, Simon Greenhill, Robert Haveman, Kent Johnson, Andrew Kember, Marek Kubica, Eduard Kucera, Anand Kumria, Scott Lamb, Fredrik Lundh, Vadim Macagon, Markus Majer, Orestis Markou, R. Mason, Yasushi Masuda, Kevin Menard, Carlo Miron, James Mulholland, R.D. Nielsen, Michael O’Keefe, Lawrence Oluyede, Andreas Pfrengle, Frankie Robertson, Mike Robinson, Armin Ronacher, Daniel Roseman, Johan Samyn, Ross Shannon, Carolina F. Silva, Paul Smith, Björn Stabell, Bob Stepno, Graeme Stevenson, Justin Stockton, Kevin Teague, Daniel Tietze, Brooks Travis, Peter Tripp, Matthias Urlichs, Peter van Kampen, Alexandre Vassalotti, Jay Wang, Brian Will, and Joshua Works.

Many thanks to our technical editor, Jeremy Dunck. Without Jeremy this book would be littered with errors, inaccuracies, and broken code. We feel very lucky that someone as talented as Jeremy found the time to help us out.

Specials thanks go to Simon Willison for writing the chapter on form processing. We really appreciate the help, and we’re thrilled that Simon’s excellent writing can be part of this book.

We’re grateful for all the hard work the folks at Apress put into this book. They’ve been amazingly supportive and patient; this book wouldn’t have come together without a lot of work on their part. We’re especially happy that Apress supported and even encouraged the free release of this book online; it’s wonderful seeing a publisher so embracing the spirit of open source.

Finally, of course, thanks to our friends, families, and coworkers who’ve graciously tolerated our mental absence while we finished this work.

About the Authors

Adrian Holovaty is a Web developer with a background in journalism. He’s known in journalism circles as one of the pioneers of “journalism via computer programming”, and in technical circles as “the guy who invented Django.”

He was lead developer at World Online for 2.5 years, during which time Django was developed and implemented on World Online’s sites. He’s the founder of EveryBlock, a “news feed for your block”.

Adrian lives in Chicago, USA.

Jacob Kaplan-Moss is a partner at Revolution Systems which provides support services around Django and related open source technologies. Jacob previously worked at World Online, where Django was invented, where he was the lead developer of Ellington, a commercial Web publishing platform for media companies.

Jacob lives in Lawrence, Kansas, USA.

About the Technical Reviewer

Jeremy Dunck was rescued from corporate IT drudgery by Free Software and, in part, Django. Many of Jeremy’s interests center around access to information.

Jeremy was the lead developer of Pegasus News, one of the first uses of Django outside World Online, and has since joined Votizen, a startup intent on reducing the influence of money in politics.

He serves as DSF Secretary, organizes and helps organize sprints, cares about the health and equity of the Django community. He has gone an embarrassingly long time without a working blog.

Jeremy lives in Mountain View, CA, USA.

What Is a Web Framework?

Django is a prominent member of a new generation of Web frameworks – but what does that term mean, precisely?

To answer that question, let’s consider the design of a Web application written in Python without a framework. Throughout this book, we’ll take this approach of showing you basic ways of getting work done without shortcuts, in the hope that you’ll recognize why shortcuts are so helpful. (It’s also valuable to know how to get things done without shortcuts because shortcuts aren’t always available. And most importantly, knowing why things work the way they do makes you a better Web developer.)

One of the simplest, most direct ways to build a Python Web app from scratch is to use the Common Gateway Interface (CGI) standard, which was a popular technique circa 1998. Here’s a high-level explanation of how it works: just create a Python script that outputs HTML, then save the script to a Web server with a ”.cgi” extension and visit the page in your Web browser. That’s it.

Here’s an example Python CGI script that displays the ten most recently published books from a database. Don’t worry about syntax details; just get a feel for the basic things it’s doing:

#!/usr/bin/env python

import MySQLdb

print "Content-Type: text/html\n"
print "<html><head><title>Books</title></head>"
print "<body>"
print "<h1>Books</h1>"
print "<ul>"

connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db')
cursor = connection.cursor()
cursor.execute("SELECT name FROM books ORDER BY pub_date DESC LIMIT 10")

for row in cursor.fetchall():
    print "<li>%s</li>" % row[0]

print "</ul>"
print "</body></html>"

connection.close()

First, to fulfill the requirements of CGI, this code prints a “Content-Type” line, followed by a blank line. It prints some introductory HTML, connects to a database and runs a query to retrieve the names of the latest ten books. Looping over those books, it generates an HTML list of the titles. Finally, it prints the closing HTML and closes the database connection.

With a one-off page like this one, the write-it-from-scratch approach isn’t necessarily bad. For one thing, this code is simple to comprehend – even a novice developer can read these 16 lines of Python and understand everything it does, from start to finish. There’s nothing else to learn, no other code to read. It’s also simple to deploy: just save this code in a file that ends with ”.cgi”, upload that file to a Web server, and visit that page with a browser.

But despite its simplicity, this approach has a number of problems and annoyances. Ask yourself these questions:

• What happens when multiple parts of your application need to connect to the database? Surely that database-connecting code shouldn’t need to be duplicated in each individual CGI script. The pragmatic thing to do would be to refactor it into a shared function.
• Should a developer really have to worry about printing the “Content-Type” line and remembering to close the database connection? This sort of boilerplate reduces programmer productivity and introduces opportunities for mistakes. These setup- and teardown-related tasks would best be handled by some common infrastructure.
• What happens when this code is reused in multiple environments, each with a separate database and password? At this point, some environment-specific configuration becomes essential.
• What happens when a Web designer who has no experience coding Python wishes to redesign the page? One wrong character could crash the entire application. Ideally, the logic of the page – the retrieval of book titles from the database – would be separate from the HTML display of the page, so that a designer could edit the latter without affecting the former.

These problems are precisely what a Web framework intends to solve. A Web framework provides a programming infrastructure for your applications, so that you can focus on writing clean, maintainable code without having to reinvent the wheel. In a nutshell, that’s what Django does.

The MVC Design Pattern

Let’s dive in with a quick example that demonstrates the difference between the previous approach and a Web framework’s approach. Here’s how you might write the previous CGI code using Django. The first thing to note is that that we split it over four Python files (models.py, views.py, urls.py) and an HTML template (latest_books.html):

# models.py (the database tables)

from django.db import models

class Book(models.Model):
    name = models.CharField(max_length=50)
    pub_date = models.DateField()


# views.py (the business logic)

from django.shortcuts import render
from models import Book

def latest_books(request):
    book_list = Book.objects.order_by('-pub_date')[:10]
    return render(request, 'latest_books.html', {'book_list': book_list})


# urls.py (the URL configuration)

from django.conf.urls.defaults import *
import views

urlpatterns = patterns('',
    (r'^latest/$', views.latest_books),
)


# latest_books.html (the template)

<html><head><title>Books</title></head>
<body>
<h1>Books</h1>
<ul>
{% for book in book_list %}
<li>{{ book.name }}</li>
{% endfor %}
</ul>
</body></html>

Again, don’t worry about the particulars of syntax; just get a feel for the overall design. The main thing to note here is the separation of concerns:

• The models.py file contains a description of the database table, represented by a Python class. This class is called a model. Using it, you can create, retrieve, update and delete records in your database using simple Python code rather than writing repetitive SQL statements.
• The views.py file contains the business logic for the page. The latest_books() function is called a view.
• The urls.py file specifies which view is called for a given URL pattern. In this case, the URL /latest/ will be handled by the latest_books() function. In other words, if your domain is example.com, any visit to the URL http://example.com/latest/ will call the latest_books() function.
• The latest_books.html file is an HTML template that describes the design of the page. It uses a template language with basic logic statements – e.g., {% for book in book_list %}.

Taken together, these pieces loosely follow a pattern called Model-View-Controller (MVC). Simply put, MVC is way of developing software so that the code for defining and accessing data (the model) is separate from request-routing logic (the controller), which in turn is separate from the user interface (the view). (We’ll discuss MVC in more depth in Chapter 5.)

A key advantage of such an approach is that components are loosely coupled. Each distinct piece of a Django-powered Web application has a single key purpose and can be changed independently without affecting the other pieces. For example, a developer can change the URL for a given part of the application without affecting the underlying implementation. A designer can change a page’s HTML without having to touch the Python code that renders it. A database administrator can rename a database table and specify the change in a single place, rather than having to search and replace through a dozen files.

In this book, each component of MVC gets its own chapter. Chapter 3 covers views, Chapter 4 covers templates, and Chapter 5 covers models.

Django’s History

Before we dive into more code, we should take a moment to explain Django’s history. We noted above that we’ll be showing you how to do things without shortcuts so that you more fully understand the shortcuts. Similarly, it’s useful to understand why Django was created, because knowledge of the history will put into context why Django works the way it does.

If you’ve been building Web applications for a while, you’re probably familiar with the problems in the CGI example we presented earlier. The classic Web developer’s path goes something like this:

1. Write a Web application from scratch.
2. Write another Web application from scratch.
3. Realize the application from step 1 shares much in common with the application from step 2.
4. Refactor the code so that application 1 shares code with application 2.
5. Repeat steps 2-4 several times.
6. Realize you’ve invented a framework.

This is precisely how Django itself was created!

Django grew organically from real-world applications written by a Web development team in Lawrence, Kansas, USA. It was born in the fall of 2003, when the Web programmers at the Lawrence Journal-World newspaper, Adrian Holovaty and Simon Willison, began using Python to build applications.

The World Online team, responsible for the production and maintenance of several local news sites, thrived in a development environment dictated by journalism deadlines. For the sites – including LJWorld.com, Lawrence.com and KUsports.com – journalists (and management) demanded that features be added and entire applications be built on an intensely fast schedule, often with only days’ or hours’ notice. Thus, Simon and Adrian developed a time-saving Web development framework out of necessity – it was the only way they could build maintainable applications under the extreme deadlines.

In summer 2005, after having developed this framework to a point where it was efficiently powering most of World Online’s sites, the team, which now included Jacob Kaplan-Moss, decided to release the framework as open source software. They released it in July 2005 and named it Django, after the jazz guitarist Django Reinhardt.

Now, several years later, Django is a well-established open source project with tens of thousands of users and contributors spread across the planet. Two of the original World Online developers (the “Benevolent Dictators for Life,” Adrian and Jacob) still provide central guidance for the framework’s growth, but it’s much more of a collaborative team effort.

This history is relevant because it helps explain two key things. The first is Django’s “sweet spot.” Because Django was born in a news environment, it offers several features (such as its admin site, covered in Chapter 6) that are particularly well suited for “content” sites – sites like Amazon.com, craigslist.org, and washingtonpost.com that offer dynamic, database-driven information. Don’t let that turn you off, though – although Django is particularly good for developing those sorts of sites, that doesn’t preclude it from being an effective tool for building any sort of dynamic Web site. (There’s a difference between being particularly effective at something and being ineffective at other things.)

The second matter to note is how Django’s origins have shaped the culture of its open source community. Because Django was extracted from real-world code, rather than being an academic exercise or commercial product, it is acutely focused on solving Web development problems that Django’s developers themselves have faced – and continue to face. As a result, Django itself is actively improved on an almost daily basis. The framework’s maintainers have a vested interest in making sure Django saves developers time, produces applications that are easy to maintain and performs well under load. If nothing else, the developers are motivated by their own selfish desires to save themselves time and enjoy their jobs. (To put it bluntly, they eat their own dog food.)

How to Read This Book

In writing this book, we tried to strike a balance between readability and reference, with a bias toward readability. Our goal with this book, as stated earlier, is to make you a Django expert, and we believe the best way to teach is through prose and plenty of examples, rather than providing an exhaustive but bland catalog of Django features. (As the saying goes, you can’t expect to teach somebody how to speak a language merely by teaching them the alphabet.)

With that in mind, we recommend that you read Chapters 1 through 12 in order. They form the foundation of how to use Django; once you’ve read them, you’ll be able to build and deploy Django-powered Web sites. Specifically, Chapters 1 through 7 are the “core curriculum,” Chapters 8 through 11 cover more advanced Django usage, and Chapter 12 covers deployment. The remaining chapters, 13 through 20, focus on specific Django features and can be read in any order.

The appendixes are for reference. They, along with the free documentation at http://www.djangoproject.com/, are probably what you’ll flip back to occasionally to recall syntax or find quick synopses of what certain parts of Django do.

Installing Python

Django itself is written purely in Python, so the first step in installing the framework is to make sure you have Python installed.

Python 2.7.3rc2 (default, Apr 22 2012, 22:30:17)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Installing Django

At any given time, two distinct versions of Django are available to you: the latest official release and the bleeding-edge development version. The version you decide to install depends on your priorities. Do you want a stable and tested version of Django, or do you want a version containing the latest features, perhaps so you can contribute to Django itself, at the expense of stability?

We’d recommend sticking with an official release, but it’s important to know that the development version exists, because you’ll find it mentioned in the documentation and by members of the community.

python setup.py install

Testing the Django installation

For some post-installation positive feedback, take a moment to test whether the installation worked. In a command shell, change into another directory (e.g., not the directory that contains the django directory) and start the Python interactive interpreter by typing python. If the installation was successful, you should be able to import the module django:

>>> import django
>>> django.VERSION
(1, 4, 2, 'final', 0)

>>> print """This is a
... string that spans
... three lines."""
This is a
string that spans
three lines.
>>> def my_function(value):
...     print value
>>> my_function('hello')
hello

Setting Up a Database

At this point, you could very well begin writing a Web application with Django, because Django’s only hard-and-fast prerequisite is a working Python installation. However, odds are you’ll be developing a database-driven Web site, in which case you’ll need to configure a database server.

If you just want to start playing with Django, skip ahead to the “Starting a Project” section – but keep in mind that all the examples in this book assume you have a working database set up.

Django supports four database engines:

• PostgreSQL (http://www.postgresql.org/)
• SQLite 3 (http://www.sqlite.org/)
• MySQL (http://www.mysql.com/)
• Oracle (http://www.oracle.com/)

For the most part, all the engines here work equally well with the core Django framework. (A notable exception is Django’s optional GIS support, which is much more powerful with PostgreSQL than with other databases.) If you’re not tied to any legacy system and have the freedom to choose a database backend, we recommend PostgreSQL, which achieves a fine balance between cost, features, speed and stability.

Setting up the database is a two-step process:

• First, you’ll need to install and configure the database server itself. This process is beyond the scope of this book, but each of the four database backends has rich documentation on its Web site. (If you’re on a shared hosting provider, odds are that they’ve set this up for you already.)
• Second, you’ll need to install the Python library for your particular database backend. This is a third-party bit of code that allows Python to interface with the database. We outline the specific, per-database requirements in the following sections.

If you’re just playing around with Django and don’t want to install a database server, consider using SQLite. SQLite is unique in the list of supported databases in that it doesn’t require either of the above steps. It merely reads and writes its data to a single file on your filesystem, and Python versions 2.5 and higher include built-in support for it.

On Windows, obtaining database driver binaries can be frustrating. If you’re eager to jump in, we recommend using Python 2.7 and its built-in support for SQLite.

Starting a Project

Once you’ve installed Python, Django and (optionally) your database server/library, you can take the first step in developing a Django application by creating a project.

A project is a collection of settings for an instance of Django, including database configuration, Django-specific options and application-specific settings.

If this is your first time using Django, you’ll have to take care of some initial setup. Create a new directory to start working in, perhaps something like /home/username/djcode/.

Change into the directory you created, and run the command django-admin.py startproject mysite. This will create a mysite directory in your current directory.

If you see a “permission denied” message when running django-admin.py startproject, you’ll need to change the file’s permissions. To do this, navigate to the directory where django-admin.py is installed (e.g., cd /usr/local/bin) and run the command chmod +x django-admin.py.

The startproject command creates a directory containing five files:

mysite/
    manage.py
    mysite/
        __init__.py
        settings.py
        urls.py
        wsgi.py

These files are as follows:

• mysite/: The outer mysite/ directory is just a container for your project. Its name doesn’t matter to Django; you can rename it to anything you like.
• manage.py: A command-line utility that lets you interact with this Django project in various ways. Type python manage.py help to get a feel for what it can do. You should never have to edit this file; it’s created in this directory purely for convenience.
• mysite/mysite/: The inner mysite/ directory is the actual Python package for your project. Its name is the Python package name you’ll need to use to import anything inside it (e.g. import mysite.settings).
• __init__.py: A file required for Python to treat the mysite directory as a package (i.e., a group of Python modules). It’s an empty file, and generally you won’t add anything to it.
• settings.py: Settings/configuration for this Django project. Take a look at it to get an idea of the types of settings available, along with their default values.
• urls.py: The URLs for this Django project. Think of this as the “table of contents” of your Django-powered site.
• wsgi.py: An entry-point for WSGI-compatible webservers to serve your project. See How to deploy with WSGI (https://docs.djangoproject.com/en/1.4/howto/deployment/wsgi/) for more details.

Despite their small size, these files already constitute a working Django application.

python manage.py runserver

Validating models...
0 errors found.

Django version 1.4.2, using settings 'mysite.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

python manage.py runserver 8080

python manage.py runserver 0.0.0.0:8000

What’s Next?

Now that you have everything installed and the development server running, you’re ready to :doc: learn the basics Chapter 3, of serving Web pages with Django.

Your First Django-Powered Page: Hello World

As our first goal, let’s create a Web page that outputs that famous example message: “Hello world.”

If you were publishing a simple “Hello world” Web page without a Web framework, you’d simply type “Hello world” into a text file, call it hello.html, and upload it to a directory on a Web server somewhere. Notice, in that process, you’ve specified two key pieces of information about that Web page: its contents (the string "Hello world") and its URL ( http://www.example.com/hello.html, or maybe http://www.example.com/files/hello.html if you put it in a subdirectory).

With Django, you specify those same two things, but in a different way. The contents of the page are produced by a view function, and the URL is specified in a URLconf. First, let’s write our “Hello world” view function.

from django.http import HttpResponse

def hello(request):
    return HttpResponse("Hello world")

from django.conf.urls import patterns, include, url

# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()

urlpatterns = patterns('',
    # Examples:
    # url(r'^$', 'mysite.views.home', name='home'),
    # url(r'^mysite/', include('mysite.foo.urls')),

    # Uncomment the admin/doc line below to enable admin documentation:
    # url(r'^admin/doc/', include('django.contrib.admindocs.urls')),

    # Uncomment the next line to enable the admin:
    # url(r'^admin/', include(admin.site.urls)),
)

from django.conf.urls.defaults import patterns, include, url

urlpatterns = patterns('',
)

from django.conf.urls.defaults import patterns, include, url
from mysite.views import hello

urlpatterns = patterns('',
    url(r'^hello/$', hello),
)

>>> import sys
>>> print sys.path

A Quick Note About 404 Errors

At this point, our URLconf defines only a single URLpattern: the one that handles requests to the URL /hello/. What happens when you request a different URL?

To find out, try running the Django development server and visiting a page such as http://127.0.0.1:8000/goodbye/ or http://127.0.0.1:8000/hello/subdirectory/, or even http://127.0.0.1:8000/ (the site “root”). You should see a “Page not found” message (see Figure 3-1). Django displays this message because you requested a URL that’s not defined in your URLconf.

Figure 3-1. Django’s 404 page

The utility of this page goes beyond the basic 404 error message. It also tells you precisely which URLconf Django used and every pattern in that URLconf. From that information, you should be able to tell why the requested URL threw a 404.

Naturally, this is sensitive information intended only for you, the Web developer. If this were a production site deployed live on the Internet, you wouldn’t want to expose that information to the public. For that reason, this “Page not found” page is only displayed if your Django project is in debug mode. We’ll explain how to deactivate debug mode later. For now, just know that every Django project is in debug mode when you first create it, and if the project is not in debug mode, Django outputs a different 404 response.

from mysite.views import hello, my_homepage_view

urlpatterns = patterns('',
    url(r'^$', my_homepage_view),
    # ...
)

How Django Processes a Request

Before continuing to our second view function, let’s pause to learn a little more about how Django works. Specifically, when you view your “Hello world” message by visiting http://127.0.0.1:8000/hello/ in your Web browser, what does Django do behind the scenes?

It all starts with the settings file. When you run python manage.py runserver, the script looks for a file called settings.py in the inner mysite directory. This file contains all sorts of configuration for this particular Django project, all in uppercase: TEMPLATE_DIRS, DATABASES, etc. The most important setting is called ROOT_URLCONF. ROOT_URLCONF tells Django which Python module should be used as the URLconf for this Web site.

Remember when django-admin.py startproject created the files settings.py and urls.py? The autogenerated settings.py contains a ROOT_URLCONF setting that points to the autogenerated urls.py. Open the settings.py file and see for yourself; it should look like this:

ROOT_URLCONF = 'mysite.urls'

This corresponds to the file mysite/urls.py.

When a request comes in for a particular URL – say, a request for /hello/ – Django loads the URLconf pointed to by the ROOT_URLCONF setting. Then it checks each of the URLpatterns in that URLconf, in order, comparing the requested URL with the patterns one at a time, until it finds one that matches. When it finds one that matches, it calls the view function associated with that pattern, passing it an HttpRequest object as the first parameter. (We’ll cover the specifics of HttpRequest later.)

As we saw in our first view example, a view function must return an HttpResponse. Once it does this, Django does the rest, converting the Python object to a proper Web response with the appropriate HTTP headers and body (i.e., the content of the Web page).

In summary:

1. A request comes in to /hello/.
2. Django determines the root URLconf by looking at the ROOT_URLCONF setting.
3. Django looks at all of the URLpatterns in the URLconf for the first one that matches /hello/.
4. If it finds a match, it calls the associated view function.
5. The view function returns an HttpResponse.
6. Django converts the HttpResponse to the proper HTTP response, which results in a Web page.

You now know the basics of how to make Django-powered pages. It’s quite simple, really – just write view functions and map them to URLs via URLconfs.

Your Second View: Dynamic Content

Our “Hello world” view was instructive in demonstrating the basics of how Django works, but it wasn’t an example of a dynamic Web page, because the content of the page are always the same. Every time you view /hello/, you’ll see the same thing; it might as well be a static HTML file.

For our second view, let’s create something more dynamic – a Web page that displays the current date and time. This is a nice, simple next step, because it doesn’t involve a database or any user input – just the output of your server’s internal clock. It’s only marginally more exciting than “Hello world,” but it’ll demonstrate a few new concepts.

This view needs to do two things: calculate the current date and time, and return an HttpResponse containing that value. If you have experience with Python, you know that Python includes a datetime module for calculating dates. Here’s how to use it:

>>> import datetime
>>> now = datetime.datetime.now()
>>> now
datetime.datetime(2008, 12, 13, 14, 9, 39, 2731)
>>> print now
2008-12-13 14:09:39.002731

That’s simple enough, and it has nothing to do with Django. It’s just Python code. (We want to emphasize that you should be aware of what code is “just Python” vs. code that is Django-specific. As you learn Django, we want you to be able to apply your knowledge to other Python projects that don’t necessarily use Django.)

To make a Django view that displays the current date and time, then, we just need to hook this datetime.datetime.now() statement into a view and return an HttpResponse. Here’s how that looks:

from django.http import HttpResponse
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    html = "<html><body>It is now %s.</body></html>" % now
    return HttpResponse(html)

As with our hello view function, this should live in views.py. Note that we’ve hidden the hello function from this example for brevity, but for the sake of completeness, here’s what the entire views.py looks like:

from django.http import HttpResponse
import datetime

def hello(request):
    return HttpResponse("Hello world")

def current_datetime(request):
    now = datetime.datetime.now()
    html = "<html><body>It is now %s.</body></html>" % now
    return HttpResponse(html)

(From now on, we won’t display previous code in code examples, except when necessary. You should be able to tell from context which parts of an example are new vs. old.)

Let’s step through the changes we’ve made to views.py to accommodate the current_datetime view.

• We’ve added an import datetime to the top of the module, so we can calculate dates.

• The new current_datetime function calculates the current date and time, as a datetime.datetime object, and stores that as the local variable now.

• The second line of code within the view constructs an HTML response using Python’s “format-string” capability. The %s within the string is a placeholder, and the percent sign after the string means “Replace the %s in the preceding string with the value of the variable now.” The now variable is technically a datetime.datetime object, not a string, but the %s format character converts it to its string representation, which is something like "2008-12-13 14:09:39.002731". This will result in an HTML string such as "<html><body>It is now 2008-12-13 14:09:39.002731.</body></html>".

  (Yes, our HTML is invalid, but we’re trying to keep the example simple and short.)

• Finally, the view returns an HttpResponse object that contains the generated response – just as we did in hello.

After adding that to views.py, add the URLpattern to urls.py to tell Django which URL should handle this view. Something like /time/ would make sense:

from django.conf.urls.defaults import patterns, include, url
from mysite.views import hello, current_datetime

urlpatterns = patterns('',
    url(r'^hello/$', hello),
    url(r'^time/$', current_datetime),
)

We’ve made two changes here. First, we imported the current_datetime function at the top. Second, and more importantly, we added a URLpattern mapping the URL /time/ to that new view. Getting the hang of this?

With the view written and URLconf updated, fire up the runserver and visit http://127.0.0.1:8000/time/ in your browser. You should see the current date and time.

URLconfs and Loose Coupling

Now’s a good time to highlight a key philosophy behind URLconfs and behind Django in general: the principle of loose coupling. Simply put, loose coupling is a software-development approach that values the importance of making pieces interchangeable. If two pieces of code are loosely coupled, then changes made to one of the pieces will have little or no effect on the other.

Django’s URLconfs are a good example of this principle in practice. In a Django web application, the URL definitions and the view functions they call are loosely coupled; that is, the decision of what the URL should be for a given function, and the implementation of the function itself, reside in two separate places. This lets you switch out one piece without affecting the other.

For example, consider our current_datetime view. If we wanted to change the URL for the application – say, to move it from /time/ to /current-time/ – we could make a quick change to the URLconf, without having to worry about the view itself. Similarly, if we wanted to change the view function – altering its logic somehow – we could do that without affecting the URL to which the function is bound.

Furthermore, if we wanted to expose the current-date functionality at several URLs, we could easily take care of that by editing the URLconf, without having to touch the view code. In this example, our current_datetime is available at two URLs. It’s a contrived example, but this technique can come in handy:

urlpatterns = patterns('',
    url(r'^hello/$', hello),
    url(r'^time/$', current_datetime),
    url(r'^another-time-page/$', current_datetime),
)

URLconfs and views are loose coupling in action. We’ll continue to point out examples of this important philosophy throughout this book.

Your Third View: Dynamic URLs

In our current_datetime view, the contents of the page – the current date/time – were dynamic, but the URL (/time/) was static. In most dynamic Web applications, though, a URL contains parameters that influence the output of the page. For example, an online bookstore might give each book its own URL, like /books/243/ and /books/81196/.

Let’s create a third view that displays the current date and time offset by a certain number of hours. The goal is to craft a site in such a way that the page /time/plus/1/ displays the date/time one hour into the future, the page /time/plus/2/ displays the date/time two hours into the future, the page /time/plus/3/ displays the date/time three hours into the future, and so on.

A novice might think to code a separate view function for each hour offset, which might result in a URLconf like this:

urlpatterns = patterns('',
    url(r'^time/$', current_datetime),
    url(r'^time/plus/1/$', one_hour_ahead),
    url(r'^time/plus/2/$', two_hours_ahead),
    url(r'^time/plus/3/$', three_hours_ahead),
    url(r'^time/plus/4/$', four_hours_ahead),
)

Clearly, this line of thought is flawed. Not only would this result in redundant view functions, but also the application is fundamentally limited to supporting only the predefined hour ranges – one, two, three or four hours. If we decided to create a page that displayed the time five hours into the future, we’d have to create a separate view and URLconf line for that, furthering the duplication. We need to do some abstraction here.

How, then do we design our application to handle arbitrary hour offsets? The key is to use wildcard URLpatterns. As we mentioned previously, a URLpattern is a regular expression; hence, we can use the regular expression pattern \d+ to match one or more digits:

urlpatterns = patterns('',
    # ...
    url(r'^time/plus/\d+/$', hours_ahead),
    # ...
)

(We’re using the # ... to imply there might be other URLpatterns that we trimmed from this example.)

This new URLpattern will match any URL such as /time/plus/2/, /time/plus/25/, or even /time/plus/100000000000/. Come to think of it, let’s limit it so that the maximum allowed offset is 99 hours. That means we want to allow either one- or two-digit numbers – and in regular expression syntax, that translates into \d{1,2}:

url(r'^time/plus/\d{1,2}/$', hours_ahead),

Now that we’ve designated a wildcard for the URL, we need a way of passing that wildcard data to the view function, so that we can use a single view function for any arbitrary hour offset. We do this by placing parentheses around the data in the URLpattern that we want to save. In the case of our example, we want to save whatever number was entered in the URL, so let’s put parentheses around the \d{1,2}, like this:

url(r'^time/plus/(\d{1,2})/$', hours_ahead),

If you’re familiar with regular expressions, you’ll be right at home here; we’re using parentheses to capture data from the matched text.

The final URLconf, including our previous two views, looks like this:

from django.conf.urls.defaults import *
from mysite.views import hello, current_datetime, hours_ahead

urlpatterns = patterns('',
    url(r'^hello/$', hello),
    url(r'^time/$', current_datetime),
    url(r'^time/plus/(\d{1,2})/$', hours_ahead),
)

With that taken care of, let’s write the hours_ahead view.

hours_ahead is very similar to the current_datetime view we wrote earlier, with a key difference: it takes an extra argument, the number of hours of offset. Here’s the view code:

from django.http import Http404, HttpResponse
import datetime

def hours_ahead(request, offset):
    try:
        offset = int(offset)
    except ValueError:
        raise Http404()
    dt = datetime.datetime.now() + datetime.timedelta(hours=offset)
    html = "<html><body>In %s hour(s), it will be %s.</body></html>" % (offset, dt)
    return HttpResponse(html)

Let’s step through this code one line at a time:

• The view function, hours_ahead, takes two parameters: request and offset.

  • request is an HttpRequest object, just as in hello and current_datetime. We’ll say it again: each view always takes an HttpRequest object as its first parameter.

  • offset is the string captured by the parentheses in the URLpattern. For example, if the requested URL were /time/plus/3/, then offset would be the string '3'. If the requested URL were /time/plus/21/, then offset would be the string '21'. Note that captured values will always be strings, not integers, even if the string is composed of only digits, such as '21'.

    (Technically, captured values will always be Unicode objects, not plain Python bytestrings, but don’t worry about this distinction at the moment.)

    We decided to call the variable offset, but you can call it whatever you’d like, as long as it’s a valid Python identifier. The variable name doesn’t matter; all that matters is that it’s the second argument to the function, after request. (It’s also possible to use keyword, rather than positional, arguments in an URLconf. We cover that in Chapter 8.)

• The first thing we do within the function is call int() on offset. This converts the string value to an integer.

  Note that Python will raise a ValueError exception if you call int() on a value that cannot be converted to an integer, such as the string 'foo'. In this example, if we encounter the ValueError, we raise the exception django.http.Http404, which, as you can imagine, results in a 404 “Page not found” error.

  Astute readers will wonder: how could we ever reach the ValueError case, anyway, given that the regular expression in our URLpattern – (\d{1,2}) – captures only digits, and therefore offset will only ever be a string composed of digits? The answer is, we won’t, because the URLpattern provides a modest but useful level of input validation, but we still check for the ValueError in case this view function ever gets called in some other way. It’s good practice to implement view functions such that they don’t make any assumptions about their parameters. Loose coupling, remember?

• In the next line of the function, we calculate the current date/time and add the appropriate number of hours. We’ve already seen datetime.datetime.now() from the current_datetime view; the new concept here is that you can perform date/time arithmetic by creating a datetime.timedelta object and adding to a datetime.datetime object. Our result is stored in the variable dt.

  This line also shows why we called int() on offset – the datetime.timedelta function requires the hours parameter to be an integer.

• Next, we construct the HTML output of this view function, just as we did in current_datetime. A small difference in this line from the previous line is that it uses Python’s format-string capability with two values, not just one. Hence, there are two %s symbols in the string and a tuple of values to insert: (offset, dt).

• Finally, we return an HttpResponse of the HTML. By now, this is old hat.

With that view function and URLconf written, start the Django development server (if it’s not already running), and visit http://127.0.0.1:8000/time/plus/3/ to verify it works. Then try http://127.0.0.1:8000/time/plus/5/. Then http://127.0.0.1:8000/time/plus/24/. Finally, visit http://127.0.0.1:8000/time/plus/100/ to verify that the pattern in your URLconf only accepts one- or two-digit numbers; Django should display a “Page not found” error in this case, just as we saw in the section “A Quick Note About 404 Errors” earlier. The URL http://127.0.0.1:8000/time/plus/ (with no hour designation) should also throw a 404.

Django’s Pretty Error Pages

Take a moment to admire the fine Web application we’ve made so far . . . now let’s break it! Let’s deliberately introduce a Python error into our views.py file by commenting out the offset = int(offset) lines in the hours_ahead view:

def hours_ahead(request, offset):
    # try:
    #     offset = int(offset)
    # except ValueError:
    #     raise Http404()
    dt = datetime.datetime.now() + datetime.timedelta(hours=offset)
    html = "<html><body>In %s hour(s), it will be %s.</body></html>" % (offset, dt)
    return HttpResponse(html)

Load up the development server and navigate to /time/plus/3/. You’ll see an error page with a significant amount of information, including a TypeError message displayed at the very top: "unsupported type for timedelta hours component: unicode".

What happened? Well, the datetime.timedelta function expects the hours parameter to be an integer, and we commented out the bit of code that converted offset to an integer. That caused datetime.timedelta to raise the TypeError. It’s the typical kind of small bug that every programmer runs into at some point.

The point of this example was to demonstrate Django’s error pages. Take some time to explore the error page and get to know the various bits of information it gives you.

Here are some things to notice:

• At the top of the page, you get the key information about the exception: the type of exception, any parameters to the exception (the "unsupported type" message in this case), the file in which the exception was raised, and the offending line number.

• Under the key exception information, the page displays the full Python traceback for this exception. This is similar to the standard traceback you get in Python’s command-line interpreter, except it’s more interactive. For each level (“frame”) in the stack, Django displays the name of the file, the function/method name, the line number, and the source code of that line.

  Click the line of source code (in dark gray), and you’ll see several lines from before and after the erroneous line, to give you context.

  Click “Local vars” under any frame in the stack to view a table of all local variables and their values, in that frame, at the exact point in the code at which the exception was raised. This debugging information can be a great help.

• Note the “Switch to copy-and-paste view” text under the “Traceback” header. Click those words, and the traceback will switch to a alternate version that can be easily copied and pasted. Use this when you want to share your exception traceback with others to get technical support – such as the kind folks in the Django IRC chat room or on the Django users mailing list.

  Underneath, the “Share this traceback on a public Web site” button will do this work for you in just one click. Click it to post the traceback to http://www.dpaste.com/, where you’ll get a distinct URL that you can share with other people.

• Next, the “Request information” section includes a wealth of information about the incoming Web request that spawned the error: GET and POST information, cookie values, and meta information, such as CGI headers. Appendix G has a complete reference of all the information a request object contains.

  Below the “Request information” section, the “Settings” section lists all of the settings for this particular Django installation. (We’ve already mentioned ROOT_URLCONF, and we’ll show you various Django settings throughout the book. All the available settings are covered in detail in Appendix D.)

The Django error page is capable of displaying more information in certain special cases, such as the case of template syntax errors. We’ll get to those later, when we discuss the Django template system. For now, uncomment the offset = int(offset) lines to get the view function working properly again.

Are you the type of programmer who likes to debug with the help of carefully placed print statements? You can use the Django error page to do so – just without the print statements. At any point in your view, temporarily insert an assert False to trigger the error page. Then, you can view the local variables and state of the program. Here’s an example, using the hours_ahead view:

def hours_ahead(request, offset):
    try:
        offset = int(offset)
    except ValueError:
        raise Http404()
    dt = datetime.datetime.now() + datetime.timedelta(hours=offset)
    assert False
    html = "<html><body>In %s hour(s), it will be %s.</body></html>" % (offset, dt)
    return HttpResponse(html)

Finally, it’s obvious that much of this information is sensitive – it exposes the innards of your Python code and Django configuration – and it would be foolish to show this information on the public Internet. A malicious person could use it to attempt to reverse-engineer your Web application and do nasty things. For that reason, the Django error page is only displayed when your Django project is in debug mode. We’ll explain how to deactivate debug mode in Chapter 12. For now, just know that every Django project is in debug mode automatically when you start it. (Sound familiar? The “Page not found” errors, described earlier in this chapter, work the same way.)

What’s next?

So far, we’ve been writing our view functions with HTML hard-coded directly in the Python code. We’ve done that to keep things simple while we demonstrated core concepts, but in the real world, this is nearly always a bad idea.

Django ships with a simple yet powerful template engine that allows you to separate the design of the page from the underlying code. We’ll dive into Django’s template engine in the next chapter Chapter 4.

Template System Basics

A Django template is a string of text that is intended to separate the presentation of a document from its data. A template defines placeholders and various bits of basic logic (template tags) that regulate how the document should be displayed. Usually, templates are used for producing HTML, but Django templates are equally capable of generating any text-based format.

Let’s start with a simple example template. This Django template describes an HTML page that thanks a person for placing an order with a company. Think of it as a form letter:

<html>
<head><title>Ordering notice</title></head>

<body>

<h1>Ordering notice</h1>

<p>Dear {{ person_name }},</p>

<p>Thanks for placing an order from {{ company }}. It's scheduled to
ship on {{ ship_date|date:"F j, Y" }}.</p>

<p>Here are the items you've ordered:</p>

<ul>
{% for item in item_list %}
    <li>{{ item }}</li>
{% endfor %}
</ul>

{% if ordered_warranty %}
    <p>Your warranty information will be included in the packaging.</p>
{% else %}
    <p>You didn't order a warranty, so you're on your own when
    the products inevitably stop working.</p>
{% endif %}

<p>Sincerely,<br />{{ company }}</p>

</body>
</html>

This template is basic HTML with some variables and template tags thrown in. Let’s step through it:

• Any text surrounded by a pair of braces (e.g., {{ person_name }}) is a variable. This means “insert the value of the variable with the given name.” (How do we specify the values of the variables? We’ll get to that in a moment.)

• Any text that’s surrounded by curly braces and percent signs (e.g., {% if ordered_warranty %}) is a template tag. The definition of a tag is quite broad: a tag just tells the template system to “do something.”

  This example template contains a for tag ({% for item in item_list %}) and an if tag ({% if ordered_warranty %}).

  A for tag works very much like a for statement in Python, letting you loop over each item in a sequence. An if tag, as you may expect, acts as a logical “if” statement. In this particular case, the tag checks whether the value of the ordered_warranty variable evaluates to True. If it does, the template system will display everything between the {% if ordered_warranty %} and {% else %}. If not, the template system will display everything between {% else %} and {% endif %}. Note that the {% else %} is optional.

• Finally, the second paragraph of this template contains an example of a filter, which is the most convenient way to alter the formatting of a variable. In this example, {{ ship_date|date:"F j, Y" }}, we’re passing the ship_date variable to the date filter, giving the date filter the argument "F j, Y". The date filter formats dates in a given format, as specified by that argument. Filters are attached using a pipe character (|), as a reference to Unix pipes.

Each Django template has access to several built-in tags and filters, many of which are discussed in the sections that follow. Appendix E contains the full list of tags and filters, and it’s a good idea to familiarize yourself with that list so you know what’s possible. It’s also possible to create your own filters and tags; we’ll cover that in Chapter 9.

Using the Template System

Let’s dive into Django’s template system so you can see how it works – but we’re not yet going to integrate it with the views that we created in the previous chapter. Our goal here is to show you how the system works independently of the rest of Django. (Put another way: usually you’ll be using the template system within a Django view, but we want to make it clear that the template system is just a Python library that you can use anywhere, not just in Django views.)

Here is the most basic way you can use Django’s template system in Python code:

1. Create a Template object by providing the raw template code as a string.
2. Call the render() method of the Template object with a given set of variables (the context). This returns a fully rendered template as a string, with all of the variables and template tags evaluated according to the context.

In code, here’s what that looks like:

>>> from django import template
>>> t = template.Template('My name is {{ name }}.')
>>> c = template.Context({'name': 'Adrian'})
>>> print t.render(c)
My name is Adrian.
>>> c = template.Context({'name': 'Fred'})
>>> print t.render(c)
My name is Fred.

The following sections describe each step in much more detail.

>>> from django.template import Template
>>> t = Template('My name is {{ name }}.')
>>> print t

<django.template.Template object at 0xb7d5f24c>

>>> from django.template import Template
>>> t = Template('{% notatag %}')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  ...
django.template.TemplateSyntaxError: Invalid block tag: 'notatag'

>>> from django.template import Context, Template
>>> t = Template('My name is {{ name }}.')
>>> c = Context({'name': 'Stephane'})
>>> t.render(c)
u'My name is Stephane.'

>>> from django.template import Template, Context
>>> raw_template = """<p>Dear {{ person_name }},</p>
...
... <p>Thanks for placing an order from {{ company }}. It's scheduled to
... ship on {{ ship_date|date:"F j, Y" }}.</p>
...
... {% if ordered_warranty %}
... <p>Your warranty information will be included in the packaging.</p>
... {% else %}
... <p>You didn't order a warranty, so you're on your own when
... the products inevitably stop working.</p>
... {% endif %}
...
... <p>Sincerely,<br />{{ company }}</p>"""
>>> t = Template(raw_template)
>>> import datetime
>>> c = Context({'person_name': 'John Smith',
...     'company': 'Outdoor Equipment',
...     'ship_date': datetime.date(2009, 4, 2),
...     'ordered_warranty': False})
>>> t.render(c)
u"<p>Dear John Smith,</p>\n\n<p>Thanks for placing an order from Outdoor
Equipment. It's scheduled to\nship on April 2, 2009.</p>\n\n\n<p>You
didn't order a warranty, so you're on your own when\nthe products
inevitably stop working.</p>\n\n\n<p>Sincerely,<br />Outdoor Equipment
</p>"

>>> from django.template import Template, Context
>>> t = Template('Hello, {{ name }}')
>>> print t.render(Context({'name': 'John'}))
Hello, John
>>> print t.render(Context({'name': 'Julie'}))
Hello, Julie
>>> print t.render(Context({'name': 'Pat'}))
Hello, Pat

# Bad
for name in ('John', 'Julie', 'Pat'):
    t = Template('Hello, {{ name }}')
    print t.render(Context({'name': name}))

# Good
t = Template('Hello, {{ name }}')
for name in ('John', 'Julie', 'Pat'):
    print t.render(Context({'name': name}))

>>> from django.template import Template, Context
>>> person = {'name': 'Sally', 'age': '43'}
>>> t = Template('{{ person.name }} is {{ person.age }} years old.')
>>> c = Context({'person': person})
>>> t.render(c)
u'Sally is 43 years old.'

>>> from django.template import Template, Context
>>> import datetime
>>> d = datetime.date(1993, 5, 2)
>>> d.year
1993
>>> d.month
5
>>> d.day
2
>>> t = Template('The month is {{ date.month }} and the year is {{ date.year }}.')
>>> c = Context({'date': d})
>>> t.render(c)
u'The month is 5 and the year is 1993.'

>>> from django.template import Template, Context
>>> class Person(object):
...     def __init__(self, first_name, last_name):
...         self.first_name, self.last_name = first_name, last_name
>>> t = Template('Hello, {{ person.first_name }} {{ person.last_name }}.')
>>> c = Context({'person': Person('John', 'Smith')})
>>> t.render(c)
u'Hello, John Smith.'

>>> from django.template import Template, Context
>>> t = Template('{{ var }} -- {{ var.upper }} -- {{ var.isdigit }}')
>>> t.render(Context({'var': 'hello'}))
u'hello -- HELLO -- False'
>>> t.render(Context({'var': '123'}))
u'123 -- 123 -- True'

>>> from django.template import Template, Context
>>> t = Template('Item 2 is {{ items.2 }}.')
>>> c = Context({'items': ['apples', 'bananas', 'carrots']})
>>> t.render(c)
u'Item 2 is carrots.'

>>> from django.template import Template, Context
>>> person = {'name': 'Sally', 'age': '43'}
>>> t = Template('{{ person.name.upper }} is {{ person.age }} years old.')
>>> c = Context({'person': person})
>>> t.render(c)
u'SALLY is 43 years old.'

>>> from django.template import Template, Context
>>> t = Template('Your name is {{ name }}.')
>>> t.render(Context())
u'Your name is .'
>>> t.render(Context({'var': 'hello'}))
u'Your name is .'
>>> t.render(Context({'NAME': 'hello'}))
u'Your name is .'
>>> t.render(Context({'Name': 'hello'}))
u'Your name is .'

>>> from django.template import Context
>>> c = Context({"foo": "bar"})
>>> c['foo']
'bar'
>>> del c['foo']
>>> c['foo']
Traceback (most recent call last):
  ...
KeyError: 'foo'
>>> c['newvariable'] = 'hello'
>>> c['newvariable']
'hello'

Basic Template Tags and Filters

As we’ve mentioned already, the template system ships with built-in tags and filters. The sections that follow provide a rundown of the most common tags and filters.

{% if today_is_weekend %}
    <p>Welcome to the weekend!</p>
{% endif %}

{% if today_is_weekend %}
    <p>Welcome to the weekend!</p>
{% else %}
    <p>Get back to work.</p>
{% endif %}

{% if athlete_list and coach_list %}
    Both athletes and coaches are available.
{% endif %}

{% if not athlete_list %}
    There are no athletes.
{% endif %}

{% if athlete_list or coach_list %}
    There are some athletes or some coaches.
{% endif %}

{% if not athlete_list or coach_list %}
    There are no athletes or there are some coaches.
{% endif %}

{% if athlete_list and not coach_list %}
    There are some athletes and absolutely no coaches.
{% endif %}

{% if athlete_list and coach_list or cheerleader_list %}

{% if athlete_list %}
    {% if coach_list or cheerleader_list %}
        We have athletes, and either coaches or cheerleaders!
    {% endif %}
{% endif %}

{% if athlete_list or coach_list or parent_list or teacher_list %}

{% if athlete_list %}
    <p>Here are the athletes: {{ athlete_list }}.</p>
{% else %}
    <p>No athletes are available.</p>
    {% if coach_list %}
        <p>Here are the coaches: {{ coach_list }}.</p>
    {% endif %}
{% endif %}

<ul>
{% for athlete in athlete_list %}
    <li>{{ athlete.name }}</li>
{% endfor %}
</ul>

{% for athlete in athlete_list reversed %}
...
{% endfor %}

{% for athlete in athlete_list %}
    <h1>{{ athlete.name }}</h1>
    <ul>
    {% for sport in athlete.sports_played %}
        <li>{{ sport }}</li>
    {% endfor %}
    </ul>
{% endfor %}

{% if athlete_list %}
    {% for athlete in athlete_list %}
        <p>{{ athlete.name }}</p>
    {% endfor %}
{% else %}
    <p>There are no athletes. Only computer programmers.</p>
{% endif %}

{% for athlete in athlete_list %}
    <p>{{ athlete.name }}</p>
{% empty %}
    <p>There are no athletes. Only computer programmers.</p>
{% endfor %}

{% ifequal user currentuser %}
    <h1>Welcome!</h1>
{% endifequal %}

{% ifequal section 'sitenews' %}
    <h1>Site News</h1>
{% endifequal %}

{% ifequal section "community" %}
    <h1>Community</h1>
{% endifequal %}

{% ifequal section 'sitenews' %}
    <h1>Site News</h1>
{% else %}
    <h1>No News Here</h1>
{% endifequal %}

{% ifequal variable 1 %}
{% ifequal variable 1.23 %}
{% ifequal variable 'foo' %}
{% ifequal variable "foo" %}

{% ifequal variable True %}
{% ifequal variable [1, 2, 3] %}
{% ifequal variable {'key': 'value'} %}

{# This is a comment #}

This is a {# this is not
a comment #}
test.

{% comment %}
This is a
multi-line comment.
{% endcomment %}

{{ name|lower }}

{{ my_list|first|upper }}

{{ bio|truncatewords:"30" }}

Philosophies and Limitations

Now that you’ve gotten a feel for the Django template language, we should point out some of its intentional limitations, along with some philosophies behind why it works the way it works.

More than any other component of Web applications, template syntax is highly subjective, and programmers’ opinions vary wildly. The fact that Python alone has dozens, if not hundreds, of open source template-language implementations supports this point. Each was likely created because its developer deemed all existing template languages inadequate. (In fact, it is said to be a rite of passage for a Python developer to write his or her own template language! If you haven’t done this yet, consider it. It’s a fun exercise.)

With that in mind, you might be interested to know that Django doesn’t require that you use its template language. Because Django is intended to be a full-stack Web framework that provides all the pieces necessary for Web developers to be productive, many times it’s more convenient to use Django’s template system than other Python template libraries, but it’s not a strict requirement in any sense. As you’ll see in the upcoming section “Using Templates in Views”, it’s very easy to use another template language with Django.

Still, it’s clear we have a strong preference for the way Django’s template language works. The template system has roots in how Web development is done at World Online and the combined experience of Django’s creators. Here are a few of those philosophies:

• Business logic should be separated from presentation logic. Django’s developers see a template system as a tool that controls presentation and presentation-related logic – and that’s it. The template system shouldn’t support functionality that goes beyond this basic goal.

  For that reason, it’s impossible to call Python code directly within Django templates. All “programming” is fundamentally limited to the scope of what template tags can do. It is possible to write custom template tags that do arbitrary things, but the out-of-the-box Django template tags intentionally do not allow for arbitrary Python code execution.

• Syntax should be decoupled from HTML/XML. Although Django’s template system is used primarily to produce HTML, it’s intended to be just as usable for non-HTML formats, such as plain text. Some other template languages are XML based, placing all template logic within XML tags or attributes, but Django deliberately avoids this limitation. Requiring valid XML to write templates introduces a world of human mistakes and hard-to-understand error messages, and using an XML engine to parse templates incurs an unacceptable level of overhead in template processing.

• Designers are assumed to be comfortable with HTML code. The template system isn’t designed so that templates necessarily are displayed nicely in WYSIWYG editors such as Dreamweaver. That is too severe a limitation and wouldn’t allow the syntax to be as friendly as it is. Django expects template authors to be comfortable editing HTML directly.

• Designers are assumed not to be Python programmers. The template system authors recognize that Web page templates are most often written by designers, not programmers, and therefore should not assume Python knowledge.

  However, the system also intends to accommodate small teams in which the templates are created by Python programmers. It offers a way to extend the system’s syntax by writing raw Python code. (More on this in Chapter 9.)

• The goal is not to invent a programming language. The goal is to offer just enough programming-esque functionality, such as branching and looping, that is essential for making presentation-related decisions.

Using Templates in Views

You’ve learned the basics of using the template system; now let’s use this knowledge to create a view. Recall the current_datetime view in mysite.views, which we started in the previous chapter. Here’s what it looks like:

from django.http import HttpResponse
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    html = "<html><body>It is now %s.</body></html>" % now
    return HttpResponse(html)

Let’s change this view to use Django’s template system. At first, you might think to do something like this:

from django.template import Template, Context
from django.http import HttpResponse
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    t = Template("<html><body>It is now {{ current_date }}.</body></html>")
    html = t.render(Context({'current_date': now}))
    return HttpResponse(html)

Sure, that uses the template system, but it doesn’t solve the problems we pointed out in the introduction of this chapter. Namely, the template is still embedded in the Python code, so true separation of data and presentation isn’t achieved. Let’s fix that by putting the template in a separate file, which this view will load.

You might first consider saving your template somewhere on your filesystem and using Python’s built-in file-opening functionality to read the contents of the template. Here’s what that might look like, assuming the template was saved as the file /home/djangouser/templates/mytemplate.html:

from django.template import Template, Context
from django.http import HttpResponse
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    # Simple way of using templates from the filesystem.
    # This is BAD because it doesn't account for missing files!
    fp = open('/home/djangouser/templates/mytemplate.html')
    t = Template(fp.read())
    fp.close()
    html = t.render(Context({'current_date': now}))
    return HttpResponse(html)

This approach, however, is inelegant for these reasons:

• It doesn’t handle the case of a missing file. If the file mytemplate.html doesn’t exist or isn’t readable, the open() call will raise an IOError exception.
• It hard-codes your template location. If you were to use this technique for every view function, you’d be duplicating the template locations. Not to mention it involves a lot of typing!
• It includes a lot of boring boilerplate code. You’ve got better things to do than to write calls to open(), fp.read(), and fp.close() each time you load a template.

To solve these issues, we’ll use template loading and template directories.

Template Loading

Django provides a convenient and powerful API for loading templates from the filesystem, with the goal of removing redundancy both in your template-loading calls and in your templates themselves.

In order to use this template-loading API, first you’ll need to tell the framework where you store your templates. The place to do this is in your settings file – the settings.py file that we mentioned last chapter, when we introduced the ROOT_URLCONF setting.

If you’re following along, open your settings.py and find the TEMPLATE_DIRS setting. By default, it’s an empty tuple, likely containing some auto-generated comments:

TEMPLATE_DIRS = (
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
)

This setting tells Django’s template-loading mechanism where to look for templates. Pick a directory where you’d like to store your templates and add it to TEMPLATE_DIRS, like so:

TEMPLATE_DIRS = (
    '/home/django/mysite/templates',
)

There are a few things to note:

• You can specify any directory you want, as long as the directory and templates within that directory are readable by the user account under which your Web server runs. If you can’t think of an appropriate place to put your templates, we recommend creating a templates directory within your project (i.e., within the mysite directory you created in Chapter 2).

• If your TEMPLATE_DIRS contains only one directory, don’t forget the comma at the end of the directory string!

  Bad:

  # Missing comma!
  TEMPLATE_DIRS = (
      '/home/django/mysite/templates'
  )
  

  Good:

  # Comma correctly in place.
  TEMPLATE_DIRS = (
      '/home/django/mysite/templates',
  )
  

  The reason for this is that Python requires commas within single-element tuples to disambiguate the tuple from a parenthetical expression. This is a common newbie gotcha.

• If you’re on Windows, include your drive letter and use Unix-style forward slashes rather than backslashes, as follows:

  TEMPLATE_DIRS = (
      'C:/www/django/templates',
  )
  

• It’s simplest to use absolute paths (i.e., directory paths that start at the root of the filesystem). If you want to be a bit more flexible and decoupled, though, you can take advantage of the fact that Django settings files are just Python code by constructing the contents of TEMPLATE_DIRS dynamically. For example:

  import os.path
  
  TEMPLATE_DIRS = (
      os.path.join(os.path.dirname(__file__), 'templates').replace('\\','/'),
  )
  

  This example uses the “magic” Python variable __file__, which is automatically set to the file name of the Python module in which the code lives. It gets the name of the directory that contains settings.py (os.path.dirname), then joins that with templates in a cross-platform way (os.path.join), then ensures that everything uses forward slashes instead of backslashes (in case of Windows).

  While we’re on the topic of dynamic Python code in settings files, we should point out that it’s very important to avoid Python errors in your settings file. If you introduce a syntax error, or a runtime error, your Django-powered site will likely crash.

With TEMPLATE_DIRS set, the next step is to change the view code to use Django’s template-loading functionality rather than hard-coding the template paths. Returning to our current_datetime view, let’s change it like so:

from django.template.loader import get_template
from django.template import Context
from django.http import HttpResponse
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    t = get_template('current_datetime.html')
    html = t.render(Context({'current_date': now}))
    return HttpResponse(html)

In this example, we’re using the function django.template.loader.get_template() rather than loading the template from the filesystem manually. The get_template() function takes a template name as its argument, figures out where the template lives on the filesystem, opens that file, and returns a compiled Template object.

Our template in this example is current_datetime.html, but there’s nothing special about that .html extension. You can give your templates whatever extension makes sense for your application, or you can leave off extensions entirely.

To determine the location of the template on your filesystem, get_template() combines your template directories from TEMPLATE_DIRS with the template name that you pass to get_template(). For example, if your TEMPLATE_DIRS is set to '/home/django/mysite/templates', the above get_template() call would look for the template /home/django/mysite/templates/current_datetime.html.

If get_template() cannot find the template with the given name, it raises a TemplateDoesNotExist exception. To see what that looks like, fire up the Django development server again by running python manage.py runserver within your Django project’s directory. Then, point your browser at the page that activates the current_datetime view (e.g., http://127.0.0.1:8000/time/). Assuming your DEBUG setting is set to True and you haven’t yet created a current_datetime.html template, you should see a Django error page highlighting the TemplateDoesNotExist error.

Figure 4-1: The error page shown when a template cannot be found.

This error page is similar to the one we explained in Chapter 3, with one additional piece of debugging information: a “Template-loader postmortem” section. This section tells you which templates Django tried to load, along with the reason each attempt failed (e.g., “File does not exist”). This information is invaluable when you’re trying to debug template-loading errors.

Moving along, create the current_datetime.html file within your template directory using the following template code:

<html><body>It is now {{ current_date }}.</body></html>

Refresh the page in your Web browser, and you should see the fully rendered page.

from django.shortcuts import render
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    return render(request, 'current_datetime.html', {'current_date': now})

t = get_template('dateapp/current_datetime.html')

return render(request, 'dateapp/current_datetime.html', {'current_date': now})

{% include 'nav.html' %}
{% include "nav.html" %}

{% include 'includes/nav.html' %}

{% include template_name %}

# mypage.html

<html>
<body>
{% include "includes/nav.html" %}
<h1>{{ title }}</h1>
</body>
</html>

# includes/nav.html

<div id="nav">
    You are in: {{ current_section }}
</div>

Template Inheritance

Our template examples so far have been tiny HTML snippets, but in the real world, you’ll be using Django’s template system to create entire HTML pages. This leads to a common Web development problem: across a Web site, how does one reduce the duplication and redundancy of common page areas, such as sitewide navigation?

A classic way of solving this problem is to use server-side includes, directives you can embed within your HTML pages to “include” one Web page inside another. Indeed, Django supports that approach, with the {% include %} template tag just described. But the preferred way of solving this problem with Django is to use a more elegant strategy called template inheritance.

In essence, template inheritance lets you build a base “skeleton” template that contains all the common parts of your site and defines “blocks” that child templates can override.

Let’s see an example of this by creating a more complete template for our current_datetime view, by editing the current_datetime.html file:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html lang="en">
<head>
    <title>The current time</title>
</head>
<body>
    <h1>My helpful timestamp site</h1>
    <p>It is now {{ current_date }}.</p>

    <hr>
    <p>Thanks for visiting my site.</p>
</body>
</html>

That looks just fine, but what happens when we want to create a template for another view – say, the hours_ahead view from Chapter 3? If we want again to make a nice, valid, full HTML template, we’d create something like:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html lang="en">
<head>
    <title>Future time</title>
</head>
<body>
    <h1>My helpful timestamp site</h1>
    <p>In {{ hour_offset }} hour(s), it will be {{ next_time }}.</p>

    <hr>
    <p>Thanks for visiting my site.</p>
</body>
</html>

Clearly, we’ve just duplicated a lot of HTML. Imagine if we had a more typical site, including a navigation bar, a few style sheets, perhaps some JavaScript – we’d end up putting all sorts of redundant HTML into each template.

The server-side include solution to this problem is to factor out the common bits in both templates and save them in separate template snippets, which are then included in each template. Perhaps you’d store the top bit of the template in a file called header.html:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html lang="en">
<head>

And perhaps you’d store the bottom bit in a file called footer.html:

    <hr>
    <p>Thanks for visiting my site.</p>
</body>
</html>

With an include-based strategy, headers and footers are easy. It’s the middle ground that’s messy. In this example, both pages feature a title – <h1>My helpful timestamp site</h1> – but that title can’t fit into header.html because the <title> on both pages is different. If we included the <h1> in the header, we’d have to include the <title>, which wouldn’t allow us to customize it per page. See where this is going?

Django’s template inheritance system solves these problems. You can think of it as an “inside-out” version of server-side includes. Instead of defining the snippets that are common, you define the snippets that are different.

The first step is to define a base template – a skeleton of your page that child templates will later fill in. Here’s a base template for our ongoing example:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html lang="en">
<head>
    <title>{% block title %}{% endblock %}</title>
</head>
<body>
    <h1>My helpful timestamp site</h1>
    {% block content %}{% endblock %}
    {% block footer %}
    <hr>
    <p>Thanks for visiting my site.</p>
    {% endblock %}
</body>
</html>

This template, which we’ll call base.html, defines a simple HTML skeleton document that we’ll use for all the pages on the site. It’s the job of child templates to override, or add to, or leave alone the contents of the blocks. (If you’re following along, save this file to your template directory as base.html.)

We’re using a template tag here that you haven’t seen before: the {% block %} tag. All the {% block %} tags do is tell the template engine that a child template may override those portions of the template.

Now that we have this base template, we can modify our existing current_datetime.html template to use it:

{% extends "base.html" %}

{% block title %}The current time{% endblock %}

{% block content %}
<p>It is now {{ current_date }}.</p>
{% endblock %}

While we’re at it, let’s create a template for the hours_ahead view from Chapter 3. (If you’re following along with code, we’ll leave it up to you to change hours_ahead to use the template system instead of hard-coded HTML.) Here’s what that could look like:

{% extends "base.html" %}

{% block title %}Future time{% endblock %}

{% block content %}
<p>In {{ hour_offset }} hour(s), it will be {{ next_time }}.</p>
{% endblock %}

Isn’t this beautiful? Each template contains only the code that’s unique to that template. No redundancy needed. If you need to make a site-wide design change, just make the change to base.html, and all of the other templates will immediately reflect the change.

Here’s how it works. When you load the template current_datetime.html, the template engine sees the {% extends %} tag, noting that this template is a child template. The engine immediately loads the parent template – in this case, base.html.

At that point, the template engine notices the three {% block %} tags in base.html and replaces those blocks with the contents of the child template. So, the title we’ve defined in {% block title %} will be used, as will the {% block content %}.

Note that since the child template doesn’t define the footer block, the template system uses the value from the parent template instead. Content within a {% block %} tag in a parent template is always used as a fallback.

Inheritance doesn’t affect the template context. In other words, any template in the inheritance tree will have access to every one of your template variables from the context.

You can use as many levels of inheritance as needed. One common way of using inheritance is the following three-level approach:

1. Create a base.html template that holds the main look and feel of your site. This is the stuff that rarely, if ever, changes.
2. Create a base_SECTION.html template for each “section” of your site (e.g., base_photos.html and base_forum.html). These templates extend base.html and include section-specific styles/design.
3. Create individual templates for each type of page, such as a forum page or a photo gallery. These templates extend the appropriate section template.

This approach maximizes code reuse and makes it easy to add items to shared areas, such as section-wide navigation.

Here are some guidelines for working with template inheritance:

• If you use {% extends %} in a template, it must be the first template tag in that template. Otherwise, template inheritance won’t work.
• Generally, the more {% block %} tags in your base templates, the better. Remember, child templates don’t have to define all parent blocks, so you can fill in reasonable defaults in a number of blocks, and then define only the ones you need in the child templates. It’s better to have more hooks than fewer hooks.
• If you find yourself duplicating code in a number of templates, it probably means you should move that code to a {% block %} in a parent template.
• If you need to get the content of the block from the parent template, use {{ block.super }}, which is a “magic” variable providing the rendered text of the parent template. This is useful if you want to add to the contents of a parent block instead of completely overriding it.
• You may not define multiple {% block %} tags with the same name in the same template. This limitation exists because a block tag works in “both” directions. That is, a block tag doesn’t just provide a hole to fill, it also defines the content that fills the hole in the parent. If there were two similarly named {% block %} tags in a template, that template’s parent wouldn’t know which one of the blocks’ content to use.
• The template name you pass to {% extends %} is loaded using the same method that get_template() uses. That is, the template name is appended to your TEMPLATE_DIRS setting.
• In most cases, the argument to {% extends %} will be a string, but it can also be a variable, if you don’t know the name of the parent template until runtime. This lets you do some cool, dynamic stuff.

What’s next?

You now have the basics of Django’s template system under your belt. What’s next?

Many modern Web sites are database-driven: the content of the Web site is stored in a relational database. This allows a clean separation of data and logic (in the same way views and templates allow the separation of logic and display.)

The next chapter Chapter 5 covers the tools Django gives you to interact with a database.

The “Dumb” Way to Do Database Queries in Views

Just as Chapter 3 detailed a “dumb” way to produce output within a view (by hard-coding the text directly within the view), there’s a “dumb” way to retrieve data from a database in a view. It’s simple: just use any existing Python library to execute an SQL query and do something with the results.

In this example view, we use the MySQLdb library (available via http://www.djangoproject.com/r/python-mysql/) to connect to a MySQL database, retrieve some records, and feed them to a template for display as a Web page:

from django.shortcuts import render
import MySQLdb

def book_list(request):
    db = MySQLdb.connect(user='me', db='mydb', passwd='secret', host='localhost')
    cursor = db.cursor()
    cursor.execute('SELECT name FROM books ORDER BY name')
    names = [row[0] for row in cursor.fetchall()]
    db.close()
    return render(request, 'book_list.html', {'names': names})

This approach works, but some problems should jump out at you immediately:

• We’re hard-coding the database connection parameters. Ideally, these parameters would be stored in the Django configuration.
• We’re having to write a fair bit of boilerplate code: creating a connection, creating a cursor, executing a statement, and closing the connection. Ideally, all we’d have to do is specify which results we wanted.
• It ties us to MySQL. If, down the road, we switch from MySQL to PostgreSQL, we’ll have to use a different database adapter (e.g., psycopg rather than MySQLdb), alter the connection parameters, and – depending on the nature of the SQL statement – possibly rewrite the SQL. Ideally, the database server we’re using would be abstracted, so that a database server change could be made in a single place. (This feature is particularly relevant if you’re building an open-source Django application that you want to be used by as many people as possible.)

As you might expect, Django’s database layer aims to solve these problems. Here’s a sneak preview of how the previous view can be rewritten using Django’s database API:

from django.shortcuts import render
from mysite.books.models import Book

def book_list(request):
    books = Book.objects.order_by('name')
    return render(request, 'book_list.html', {'books': books})

We’ll explain this code a little later in the chapter. For now, just get a feel for how it looks.

The MTV (or MVC) Development Pattern

Before we delve into any more code, let’s take a moment to consider the overall design of a database-driven Django Web application.

As we mentioned in previous chapters, Django is designed to encourage loose coupling and strict separation between pieces of an application. If you follow this philosophy, it’s easy to make changes to one particular piece of the application without affecting the other pieces. In view functions, for instance, we discussed the importance of separating the business logic from the presentation logic by using a template system. With the database layer, we’re applying that same philosophy to data access logic.

Those three pieces together – data access logic, business logic, and presentation logic – comprise a concept that’s sometimes called the Model-View-Controller (MVC) pattern of software architecture. In this pattern, “Model” refers to the data access layer, “View” refers to the part of the system that selects what to display and how to display it, and “Controller” refers to the part of the system that decides which view to use, depending on user input, accessing the model as needed.

Django follows this MVC pattern closely enough that it can be called an MVC framework. Here’s roughly how the M, V, and C break down in Django:

• M, the data-access portion, is handled by Django’s database layer, which is described in this chapter.
• V, the portion that selects which data to display and how to display it, is handled by views and templates.
• C, the portion that delegates to a view depending on user input, is handled by the framework itself by following your URLconf and calling the appropriate Python function for the given URL.

Because the “C” is handled by the framework itself and most of the excitement in Django happens in models, templates and views, Django has been referred to as an MTV framework. In the MTV development pattern,

• M stands for “Model,” the data access layer. This layer contains anything and everything about the data: how to access it, how to validate it, which behaviors it has, and the relationships between the data.
• T stands for “Template,” the presentation layer. This layer contains presentation-related decisions: how something should be displayed on a Web page or other type of document.
• V stands for “View,” the business logic layer. This layer contains the logic that access the model and defers to the appropriate template(s). You can think of it as the bridge between models and templates.

If you’re familiar with other MVC Web-development frameworks, such as Ruby on Rails, you may consider Django views to be the “controllers” and Django templates to be the “views.” This is an unfortunate confusion brought about by differing interpretations of MVC. In Django’s interpretation of MVC, the “view” describes the data that gets presented to the user; it’s not necessarily just how the data looks, but which data is presented. In contrast, Ruby on Rails and similar frameworks suggest that the controller’s job includes deciding which data gets presented to the user, whereas the view is strictly how the data looks, not which data is presented.

Neither interpretation is more “correct” than the other. The important thing is to understand the underlying concepts.

Configuring the Database

With all of that philosophy in mind, let’s start exploring Django’s database layer. First, we need to take care of some initial configuration; we need to tell Django which database server to use and how to connect to it.

We’ll assume you’ve set up a database server, activated it, and created a database within it (e.g., using a CREATE DATABASE statement). If you’re using SQLite, no such setup is required, because SQLite uses standalone files on the filesystem to store its data.

As with TEMPLATE_DIRS in the previous chapter, database configuration lives in the Django settings file, called settings.py by default. Edit that file and look for the database settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': '',                      # Or path to database file if using sqlite3.
        'USER': '',                      # Not used with sqlite3.
        'PASSWORD': '',                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
    }
}

Here’s a rundown of each setting.

• ENGINE tells Django which database engine to use. If you’re using a database with Django, ENGINE must be set to one of the strings shown in Table 5-1.

  Table 5-1. Database Engine Settings

  Setting	Database	Required Adapter
  django.db.backends.postgresql_psycopg2	PostgreSQL	psycopg version 2.x, http://www.djangoproject.com/r/python-pgsql/.
  django.db.backends.mysql	MySQL	MySQLdb, http://www.djangoproject.com/r/python-mysql/.
  django.db.backends.sqlite3	SQLite	No adapter needed.
  django.db.backends.oracle	Oracle	cx_Oracle, http://www.djangoproject.com/r/python-oracle/.

  Note that for whichever database back-end you use, you’ll need to download and install the appropriate database adapter. Each one is available for free on the Web; just follow the links in the “Required Adapter” column in Table 5-1. If you’re on Linux, your distribution’s package-management system might offer convenient packages. (Look for packages called python-postgresql or python-psycopg, for example.)

  Example:

  'ENGINE': 'django.db.backends.postgresql_psycopg2',
  

• NAME tells Django the name of your database. For example:

  'NAME': 'mydb',
  

  If you’re using SQLite, specify the full filesystem path to the database file on your filesystem. For example:

  'NAME': '/home/django/mydata.db',
  

  As for where you put that SQLite database, we’re using the /home/django directory in this example, but you should pick a directory that works best for you.

• USER tells Django which username to use when connecting to your database. For example: If you’re using SQLite, leave this blank.

• PASSWORD tells Django which password to use when connecting to your database. If you’re using SQLite or have an empty password, leave this blank.

• HOST tells Django which host to use when connecting to your database. If your database is on the same computer as your Django installation (i.e., localhost), leave this blank. If you’re using SQLite, leave this blank.

  MySQL is a special case here. If this value starts with a forward slash ('/') and you’re using MySQL, MySQL will connect via a Unix socket to the specified socket, for example:

  'HOST': '/var/run/mysql',
  

• PORT tells Django which port to use when connecting to your database. If you’re using SQLite, leave this blank. Otherwise, if you leave this blank, the underlying database adapter will use whichever port is default for your given database server. In most cases, the default port is fine, so you can leave this blank.

Once you’ve entered those settings and saved settings.py, it’s a good idea to test your configuration. To do this, run python manage.py shell as in the last chapter, from within the mysite project directory. (As we pointed out last chapter manage.py shell is a way to run the Python interpreter with the correct Django settings activated. This is necessary in our case, because Django needs to know which settings file to use in order to get your database connection information.)

In the shell, type these commands to test your database configuration:

>>> from django.db import connection
>>> cursor = connection.cursor()

If nothing happens, then your database is configured properly. Otherwise, check the error message for clues about what’s wrong. Table 5-2 shows some common errors.

Your First App

Now that you’ve verified the connection is working, it’s time to create a Django app – a bundle of Django code, including models and views, that lives together in a single Python package and represents a full Django application.

It’s worth explaining the terminology here, because this tends to trip up beginners. We’d already created a project, in Chapter 2, so what’s the difference between a project and an app? The difference is that of configuration vs. code:

• A project is an instance of a certain set of Django apps, plus the configuration for those apps.

  Technically, the only requirement of a project is that it supplies a settings file, which defines the database connection information, the list of installed apps, the TEMPLATE_DIRS, and so forth.

• An app is a portable set of Django functionality, usually including models and views, that lives together in a single Python package.

  For example, Django comes with a number of apps, such as a commenting system and an automatic admin interface. A key thing to note about these apps is that they’re portable and reusable across multiple projects.

There are very few hard-and-fast rules about how you fit your Django code into this scheme. If you’re building a simple Web site, you may use only a single app. If you’re building a complex Web site with several unrelated pieces such as an e-commerce system and a message board, you’ll probably want to split those into separate apps so that you’ll be able to reuse them individually in the future.

Indeed, you don’t necessarily need to create apps at all, as evidenced by the example view functions we’ve created so far in this book. In those cases, we simply created a file called views.py, filled it with view functions, and pointed our URLconf at those functions. No “apps” were needed.

However, there’s one requirement regarding the app convention: if you’re using Django’s database layer (models), you must create a Django app. Models must live within apps. Thus, in order to start writing our models, we’ll need to create a new app.

Within the mysite project directory, type this command to create a books app:

python manage.py startapp books

This command does not produce any output, but it does create a books directory within the mysite directory. Let’s look at the contents of that directory:

books/
    __init__.py
    models.py
    tests.py
    views.py

These files will contain the models and views for this app.

Have a look at models.py and views.py in your favorite text editor. Both files are empty, except for comments and an import in models.py. This is the blank slate for your Django app.

Defining Models in Python

As we discussed earlier in this chapter, the “M” in “MTV” stands for “Model.” A Django model is a description of the data in your database, represented as Python code. It’s your data layout – the equivalent of your SQL CREATE TABLE statements – except it’s in Python instead of SQL, and it includes more than just database column definitions. Django uses a model to execute SQL code behind the scenes and return convenient Python data structures representing the rows in your database tables. Django also uses models to represent higher-level concepts that SQL can’t necessarily handle.

If you’re familiar with databases, your immediate thought might be, “Isn’t it redundant to define data models in Python instead of in SQL?” Django works the way it does for several reasons:

• Introspection requires overhead and is imperfect. In order to provide convenient data-access APIs, Django needs to know the database layout somehow, and there are two ways of accomplishing this. The first way would be to explicitly describe the data in Python, and the second way would be to introspect the database at runtime to determine the data models.

  This second way seems cleaner, because the metadata about your tables lives in only one place, but it introduces a few problems. First, introspecting a database at runtime obviously requires overhead. If the framework had to introspect the database each time it processed a request, or even only when the Web server was initialized, this would incur an unacceptable level of overhead. (While some believe that level of overhead is acceptable, Django’s developers aim to trim as much framework overhead as possible.) Second, some databases, notably older versions of MySQL, do not store sufficient metadata for accurate and complete introspection.

• Writing Python is fun, and keeping everything in Python limits the number of times your brain has to do a “context switch.” It helps productivity if you keep yourself in a single programming environment/mentality for as long as possible. Having to write SQL, then Python, and then SQL again is disruptive.

• Having data models stored as code rather than in your database makes it easier to keep your models under version control. This way, you can easily keep track of changes to your data layouts.

• SQL allows for only a certain level of metadata about a data layout. Most database systems, for example, do not provide a specialized data type for representing email addresses or URLs. Django models do. The advantage of higher-level data types is higher productivity and more reusable code.

• SQL is inconsistent across database platforms. If you’re distributing a Web application, for example, it’s much more pragmatic to distribute a Python module that describes your data layout than separate sets of CREATE TABLE statements for MySQL, PostgreSQL, and SQLite.

A drawback of this approach, however, is that it’s possible for the Python code to get out of sync with what’s actually in the database. If you make changes to a Django model, you’ll need to make the same changes inside your database to keep your database consistent with the model. We’ll discuss some strategies for handling this problem later in this chapter.

Finally, we should note that Django includes a utility that can generate models by introspecting an existing database. This is useful for quickly getting up and running with legacy data. We’ll cover this in Chapter 18.

Your First Model

As an ongoing example in this chapter and the next chapter, we’ll focus on a basic book/author/publisher data layout. We use this as our example because the conceptual relationships between books, authors, and publishers are well known, and this is a common data layout used in introductory SQL textbooks. You’re also reading a book that was written by authors and produced by a publisher!

We’ll suppose the following concepts, fields, and relationships:

• An author has a first name, a last name and an email address.
• A publisher has a name, a street address, a city, a state/province, a country, and a Web site.
• A book has a title and a publication date. It also has one or more authors (a many-to-many relationship with authors) and a single publisher (a one-to-many relationship – aka foreign key – to publishers).

The first step in using this database layout with Django is to express it as Python code. In the models.py file that was created by the startapp command, enter the following:

from django.db import models

class Publisher(models.Model):
    name = models.CharField(max_length=30)
    address = models.CharField(max_length=50)
    city = models.CharField(max_length=60)
    state_province = models.CharField(max_length=30)
    country = models.CharField(max_length=50)
    website = models.URLField()

class Author(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=40)
    email = models.EmailField()

class Book(models.Model):
    title = models.CharField(max_length=100)
    authors = models.ManyToManyField(Author)
    publisher = models.ForeignKey(Publisher)
    publication_date = models.DateField()

Let’s quickly examine this code to cover the basics. The first thing to notice is that each model is represented by a Python class that is a subclass of django.db.models.Model. The parent class, Model, contains all the machinery necessary to make these objects capable of interacting with a database – and that leaves our models responsible solely for defining their fields, in a nice and compact syntax. Believe it or not, this is all the code we need to write to have basic data access with Django.

Each model generally corresponds to a single database table, and each attribute on a model generally corresponds to a column in that database table. The attribute name corresponds to the column’s name, and the type of field (e.g., CharField) corresponds to the database column type (e.g., varchar). For example, the Publisher model is equivalent to the following table (assuming PostgreSQL CREATE TABLE syntax):

CREATE TABLE "books_publisher" (
    "id" serial NOT NULL PRIMARY KEY,
    "name" varchar(30) NOT NULL,
    "address" varchar(50) NOT NULL,
    "city" varchar(60) NOT NULL,
    "state_province" varchar(30) NOT NULL,
    "country" varchar(50) NOT NULL,
    "website" varchar(200) NOT NULL
);

Indeed, Django can generate that CREATE TABLE statement automatically, as we’ll show you in a moment.

The exception to the one-class-per-database-table rule is the case of many-to-many relationships. In our example models, Book has a ManyToManyField called authors. This designates that a book has one or many authors, but the Book database table doesn’t get an authors column. Rather, Django creates an additional table – a many-to-many “join table” – that handles the mapping of books to authors.

For a full list of field types and model syntax options, see Appendix B.

Finally, note we haven’t explicitly defined a primary key in any of these models. Unless you instruct it otherwise, Django automatically gives every model an auto-incrementing integer primary key field called id. Each Django model is required to have a single-column primary key.

Installing the Model

We’ve written the code; now let’s create the tables in our database. In order to do that, the first step is to activate these models in our Django project. We do that by adding the books app to the list of “installed apps” in the settings file.

Edit the settings.py file again, and look for the INSTALLED_APPS setting. INSTALLED_APPS tells Django which apps are activated for a given project. By default, it looks something like this:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
)

Temporarily comment out all six of those strings by putting a hash character (#) in front of them. (They’re included by default as a common-case convenience, but we’ll activate and discuss them in subsequent chapters.) While you’re at it, comment out the default MIDDLEWARE_CLASSES setting, too; the default values in MIDDLEWARE_CLASSES depend on some of the apps we just commented out. Then, add 'books' to the INSTALLED_APPS list, so the setting ends up looking like this:

MIDDLEWARE_CLASSES = (
    # 'django.middleware.common.CommonMiddleware',
    # 'django.contrib.sessions.middleware.SessionMiddleware',
    # 'django.middleware.csrf.CsrfViewMiddleware',
    # 'django.contrib.auth.middleware.AuthenticationMiddleware',
    # 'django.contrib.messages.middleware.MessageMiddleware',
)

INSTALLED_APPS = (
    # 'django.contrib.auth',
    # 'django.contrib.contenttypes',
    # 'django.contrib.sessions',
    # 'django.contrib.sites',
    'books',
)

(As we mentioned last chapter when setting TEMPLATE_DIRS, you’ll need to be sure to include the trailing comma in INSTALLED_APPS, because it’s a single-element tuple. By the way, this book’s authors prefer to put a comma after every element of a tuple, regardless of whether the tuple has only a single element. This avoids the issue of forgetting commas, and there’s no penalty for using that extra comma.)

'mysite.books' refers to the books app we’re working on. Each app in INSTALLED_APPS is represented by its full Python path – that is, the path of packages, separated by dots, leading to the app package.

Now that the Django app has been activated in the settings file, we can create the database tables in our database. First, let’s validate the models by running this command:

python manage.py validate

The validate command checks whether your models’ syntax and logic are correct. If all is well, you’ll see the message 0 errors found. If you don’t, make sure you typed in the model code correctly. The error output should give you helpful information about what was wrong with the code.

Any time you think you have problems with your models, run python manage.py validate. It tends to catch all the common model problems.

If your models are valid, run the following command for Django to generate CREATE TABLE statements for your models in the books app (with colorful syntax highlighting available, if you’re using Unix):

python manage.py sqlall books

In this command, books is the name of the app. It’s what you specified when you ran the command manage.py startapp. When you run the command, you should see something like this:

BEGIN;
CREATE TABLE "books_publisher" (
    "id" serial NOT NULL PRIMARY KEY,
    "name" varchar(30) NOT NULL,
    "address" varchar(50) NOT NULL,
    "city" varchar(60) NOT NULL,
    "state_province" varchar(30) NOT NULL,
    "country" varchar(50) NOT NULL,
    "website" varchar(200) NOT NULL
)
;
CREATE TABLE "books_author" (
    "id" serial NOT NULL PRIMARY KEY,
    "first_name" varchar(30) NOT NULL,
    "last_name" varchar(40) NOT NULL,
    "email" varchar(75) NOT NULL
)
;
CREATE TABLE "books_book" (
    "id" serial NOT NULL PRIMARY KEY,
    "title" varchar(100) NOT NULL,
    "publisher_id" integer NOT NULL REFERENCES "books_publisher" ("id") DEFERRABLE INITIALLY DEFERRED,
    "publication_date" date NOT NULL
)
;
CREATE TABLE "books_book_authors" (
    "id" serial NOT NULL PRIMARY KEY,
    "book_id" integer NOT NULL REFERENCES "books_book" ("id") DEFERRABLE INITIALLY DEFERRED,
    "author_id" integer NOT NULL REFERENCES "books_author" ("id") DEFERRABLE INITIALLY DEFERRED,
    UNIQUE ("book_id", "author_id")
)
;
CREATE INDEX "books_book_publisher_id" ON "books_book" ("publisher_id");
COMMIT;

Note the following:

• Table names are automatically generated by combining the name of the app (books) and the lowercase name of the model (publisher, book, and author). You can override this behavior, as detailed in Appendix B.
• As we mentioned earlier, Django adds a primary key for each table automatically – the id fields. You can override this, too.
• By convention, Django appends "_id" to the foreign key field name. As you might have guessed, you can override this behavior, too.
• The foreign key relationship is made explicit by a REFERENCES statement.
• These CREATE TABLE statements are tailored to the database you’re using, so database-specific field types such as auto_increment (MySQL), serial (PostgreSQL), or integer primary key (SQLite) are handled for you automatically. The same goes for quoting of column names (e.g., using double quotes or single quotes). This example output is in PostgreSQL syntax.

The sqlall command doesn’t actually create the tables or otherwise touch your database – it just prints output to the screen so you can see what SQL Django would execute if you asked it. If you wanted to, you could copy and paste this SQL into your database client, or use Unix pipes to pass it directly (e.g., python manage.py sqlall books | psql mydb). However, Django provides an easier way of committing the SQL to the database: the syncdb command:

python manage.py syncdb

Run that command, and you’ll see something like this:

Creating table books_publisher
Creating table books_author
Creating table books_book
Installing index for books.Book model

The syncdb command is a simple “sync” of your models to your database. It looks at all of the models in each app in your INSTALLED_APPS setting, checks the database to see whether the appropriate tables exist yet, and creates the tables if they don’t yet exist. Note that syncdb does not sync changes in models or deletions of models; if you make a change to a model or delete a model, and you want to update the database, syncdb will not handle that. (More on this in the “Making Changes to a Database Schema” section toward the end of this chapter.)

If you run python manage.py syncdb again, nothing happens, because you haven’t added any models to the books app or added any apps to INSTALLED_APPS. Ergo, it’s always safe to run python manage.py syncdb – it won’t clobber things.

If you’re interested, take a moment to dive into your database server’s command-line client and see the database tables Django created. You can manually run the command-line client (e.g., psql for PostgreSQL) or you can run the command python manage.py dbshell, which will figure out which command-line client to run, depending on your DATABASE_SERVER setting. The latter is almost always more convenient.

Basic Data Access

Once you’ve created a model, Django automatically provides a high-level Python API for working with those models. Try it out by running python manage.py shell and typing the following:

>>> from books.models import Publisher
>>> p1 = Publisher(name='Apress', address='2855 Telegraph Avenue',
...     city='Berkeley', state_province='CA', country='U.S.A.',
...     website='http://www.apress.com/')
>>> p1.save()
>>> p2 = Publisher(name="O'Reilly", address='10 Fawcett St.',
...     city='Cambridge', state_province='MA', country='U.S.A.',
...     website='http://www.oreilly.com/')
>>> p2.save()
>>> publisher_list = Publisher.objects.all()
>>> publisher_list
[<Publisher: Publisher object>, <Publisher: Publisher object>]

These few lines of code accomplish quite a bit. Here are the highlights:

• First, we import our Publisher model class. This lets us interact with the database table that contains publishers.
• We create a Publisher object by instantiating it with values for each field – name, address, etc.
• To save the object to the database, call its save() method. Behind the scenes, Django executes an SQL INSERT statement here.
• To retrieve publishers from the database, use the attribute Publisher.objects, which you can think of as a set of all publishers. Fetch a list of all Publisher objects in the database with the statement Publisher.objects.all(). Behind the scenes, Django executes an SQL SELECT statement here.

One thing is worth mentioning, in case it wasn’t clear from this example. When you’re creating objects using the Django model API, Django doesn’t save the objects to the database until you call the save() method:

p1 = Publisher(...)
# At this point, p1 is not saved to the database yet!
p1.save()
# Now it is.

If you want to create an object and save it to the database in a single step, use the objects.create() method. This example is equivalent to the example above:

>>> p1 = Publisher.objects.create(name='Apress',
...     address='2855 Telegraph Avenue',
...     city='Berkeley', state_province='CA', country='U.S.A.',
...     website='http://www.apress.com/')
>>> p2 = Publisher.objects.create(name="O'Reilly",
...     address='10 Fawcett St.', city='Cambridge',
...     state_province='MA', country='U.S.A.',
...     website='http://www.oreilly.com/')
>>> publisher_list = Publisher.objects.all()
>>> publisher_list

Naturally, you can do quite a lot with the Django database API – but first, let’s take care of a small annoyance.

Adding Model String Representations

When we printed out the list of publishers, all we got was this unhelpful display that makes it difficult to tell the Publisher objects apart:

[<Publisher: Publisher object>, <Publisher: Publisher object>]

We can fix this easily by adding a method called __unicode__() to our Publisher class. A __unicode__() method tells Python how to display the “unicode” representation of an object. You can see this in action by adding a __unicode__() method to the three models:

from django.db import models

class Publisher(models.Model):
    name = models.CharField(max_length=30)
    address = models.CharField(max_length=50)
    city = models.CharField(max_length=60)
    state_province = models.CharField(max_length=30)
    country = models.CharField(max_length=50)
    website = models.URLField()

    def __unicode__(self):
        return self.name

class Author(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=40)
    email = models.EmailField()

    def __unicode__(self):
        return u'%s %s' % (self.first_name, self.last_name)

class Book(models.Model):
    title = models.CharField(max_length=100)
    authors = models.ManyToManyField(Author)
    publisher = models.ForeignKey(Publisher)
    publication_date = models.DateField()

    def __unicode__(self):
        return self.title

As you can see, a __unicode__() method can do whatever it needs to do in order to return a representation of an object. Here, the __unicode__() methods for Publisher and Book simply return the object’s name and title, respectively, but the __unicode__() for Author is slightly more complex – it pieces together the first_name and last_name fields, separated by a space.

The only requirement for __unicode__() is that it return a Unicode object. If __unicode__() doesn’t return a Unicode object – if it returns, say, an integer – then Python will raise a TypeError with a message like "coercing to Unicode: need string or buffer, int found".

For the __unicode__() changes to take effect, exit out of the Python shell and enter it again with python manage.py shell. (This is the simplest way to make code changes take effect.) Now the list of Publisher objects is much easier to understand:

>>> from books.models import Publisher
>>> publisher_list = Publisher.objects.all()
>>> publisher_list
[<Publisher: Apress>, <Publisher: O'Reilly>]

Make sure any model you define has a __unicode__() method – not only for your own convenience when using the interactive interpreter, but also because Django uses the output of __unicode__() in several places when it needs to display objects.

Finally, note that __unicode__() is a good example of adding behavior to models. A Django model describes more than the database table layout for an object; it also describes any functionality that object knows how to do. __unicode__() is one example of such functionality – a model knows how to display itself.

Inserting and Updating Data

You’ve already seen this done: to insert a row into your database, first create an instance of your model using keyword arguments, like so:

>>> p = Publisher(name='Apress',
...         address='2855 Telegraph Ave.',
...         city='Berkeley',
...         state_province='CA',
...         country='U.S.A.',
...         website='http://www.apress.com/')

As we noted above, this act of instantiating a model class does not touch the database. The record isn’t saved into the database until you call save(), like this:

>>> p.save()

In SQL, this can roughly be translated into the following:

INSERT INTO books_publisher
    (name, address, city, state_province, country, website)
VALUES
    ('Apress', '2855 Telegraph Ave.', 'Berkeley', 'CA',
     'U.S.A.', 'http://www.apress.com/');

Because the Publisher model uses an autoincrementing primary key id, the initial call to save() does one more thing: it calculates the primary key value for the record and sets it to the id attribute on the instance:

>>> p.id
52    # this will differ based on your own data

Subsequent calls to save() will save the record in place, without creating a new record (i.e., performing an SQL UPDATE statement instead of an INSERT):

>>> p.name = 'Apress Publishing'
>>> p.save()

The preceding save() statement will result in roughly the following SQL:

UPDATE books_publisher SET
    name = 'Apress Publishing',
    address = '2855 Telegraph Ave.',
    city = 'Berkeley',
    state_province = 'CA',
    country = 'U.S.A.',
    website = 'http://www.apress.com'
WHERE id = 52;

Yes, note that all of the fields will be updated, not just the ones that have been changed. Depending on your application, this may cause a race condition. See “Updating Multiple Objects in One Statement” below to find out how to execute this (slightly different) query:

UPDATE books_publisher SET
    name = 'Apress Publishing'
WHERE id=52;

Selecting Objects

Knowing how to create and update database records is essential, but chances are that the Web applications you’ll build will be doing more querying of existing objects than creating new ones. We’ve already seen a way to retrieve every record for a given model:

>>> Publisher.objects.all()
[<Publisher: Apress>, <Publisher: O'Reilly>]

This roughly translates to this SQL:

SELECT id, name, address, city, state_province, country, website
FROM books_publisher;

Let’s take a close look at each part of this Publisher.objects.all() line:

• First, we have the model we defined, Publisher. No surprise here: when you want to look up data, you use the model for that data.

• Next, we have the objects attribute. This is called a manager. Managers are discussed in detail in Chapter 10. For now, all you need to know is that managers take care of all “table-level” operations on data including, most important, data lookup.

  All models automatically get a objects manager; you’ll use it any time you want to look up model instances.

• Finally, we have all(). This is a method on the objects manager that returns all the rows in the database. Though this object looks like a list, it’s actually a QuerySet – an object that represents a specific set of rows from the database. Appendix C deals with QuerySets in detail. For the rest of this chapter, we’ll just treat them like the lists they emulate.

Any database lookup is going to follow this general pattern – we’ll call methods on the manager attached to the model we want to query against.

>>> Publisher.objects.filter(name='Apress')
[<Publisher: Apress>]

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
WHERE name = 'Apress';

>>> Publisher.objects.filter(country="U.S.A.", state_province="CA")
[<Publisher: Apress>]

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
WHERE country = 'U.S.A.'
AND state_province = 'CA';

>>> Publisher.objects.filter(name__contains="press")
[<Publisher: Apress>]

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
WHERE name LIKE '%press%';

>>> Publisher.objects.get(name="Apress")
<Publisher: Apress>

>>> Publisher.objects.get(country="U.S.A.")
Traceback (most recent call last):
    ...
MultipleObjectsReturned: get() returned more than one Publisher --
    it returned 2! Lookup parameters were {'country': 'U.S.A.'}

>>> Publisher.objects.get(name="Penguin")
Traceback (most recent call last):
    ...
DoesNotExist: Publisher matching query does not exist.

try:
    p = Publisher.objects.get(name='Apress')
except Publisher.DoesNotExist:
    print "Apress isn't in the database yet."
else:
    print "Apress is in the database."

>>> Publisher.objects.order_by("name")
[<Publisher: Apress>, <Publisher: O'Reilly>]

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
ORDER BY name;

>>> Publisher.objects.order_by("address")
[<Publisher: O'Reilly>, <Publisher: Apress>]

>>> Publisher.objects.order_by("state_province")
[<Publisher: Apress>, <Publisher: O'Reilly>]

>>> Publisher.objects.order_by("state_province", "address")
 [<Publisher: Apress>, <Publisher: O'Reilly>]

>>> Publisher.objects.order_by("-name")
[<Publisher: O'Reilly>, <Publisher: Apress>]

class Publisher(models.Model):
    name = models.CharField(max_length=30)
    address = models.CharField(max_length=50)
    city = models.CharField(max_length=60)
    state_province = models.CharField(max_length=30)
    country = models.CharField(max_length=50)
    website = models.URLField()

    def __unicode__(self):
        return self.name

    class Meta:
        ordering = ['name']

>>> Publisher.objects.filter(country="U.S.A.").order_by("-name")
[<Publisher: O'Reilly>, <Publisher: Apress>]

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
WHERE country = 'U.S.A'
ORDER BY name DESC;

>>> Publisher.objects.order_by('name')[0]
<Publisher: Apress>

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
ORDER BY name
LIMIT 1;

>>> Publisher.objects.order_by('name')[0:2]

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
ORDER BY name
OFFSET 0 LIMIT 2;

>>> Publisher.objects.order_by('name')[-1]
Traceback (most recent call last):
  ...
AssertionError: Negative indexing is not supported.

>>> Publisher.objects.order_by('-name')[0]

>>> p = Publisher.objects.get(name='Apress')
>>> p.name = 'Apress Publishing'
>>> p.save()

SELECT id, name, address, city, state_province, country, website
FROM books_publisher
WHERE name = 'Apress';

UPDATE books_publisher SET
    name = 'Apress Publishing',
    address = '2855 Telegraph Ave.',
    city = 'Berkeley',
    state_province = 'CA',
    country = 'U.S.A.',
    website = 'http://www.apress.com'
WHERE id = 52;

>>> Publisher.objects.filter(id=52).update(name='Apress Publishing')

UPDATE books_publisher
SET name = 'Apress Publishing'
WHERE id = 52;

>>> Publisher.objects.all().update(country='USA')
2

Deleting Objects

To delete an object from your database, simply call the object’s delete() method:

>>> p = Publisher.objects.get(name="O'Reilly")
>>> p.delete()
>>> Publisher.objects.all()
[<Publisher: Apress Publishing>]

You can also delete objects in bulk by calling delete() on the result of any QuerySet. This is similar to the update() method we showed in the last section:

>>> Publisher.objects.filter(country='USA').delete()
>>> Publisher.objects.all().delete()
>>> Publisher.objects.all()
[]

Be careful deleting your data! As a precaution against deleting all of the data in a particular table, Django requires you to explicitly use all() if you want to delete everything in your table. For example, this won’t work:

>>> Publisher.objects.delete()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'Manager' object has no attribute 'delete'

But it’ll work if you add the all() method:

>>> Publisher.objects.all().delete()

If you’re just deleting a subset of your data, you don’t need to include all(). To repeat a previous example:

>>> Publisher.objects.filter(country='USA').delete()

What’s Next?

Having read this chapter, you have enough knowledge of Django models to be able to write basic database applications. Chapter 10 will provide some information on more advanced usage of Django’s database layer.

Once you’ve defined your models, the next step is to populate your database with data. You might have legacy data, in which case Chapter 18 will give you advice about integrating with legacy databases. You might rely on site users to supply your data, in which case Chapter 7 will teach you how to process user-submitted form data.

But in some cases, you or your team might need to enter data manually, in which case it would be helpful to have a Web-based interface for entering and managing data. The next chapter Chapter 6 covers Django’s admin interface, which exists precisely for that reason.

The django.contrib packages

Django’s automatic admin is part of a larger suite of Django functionality called django.contrib – the part of the Django codebase that contains various useful add-ons to the core framework. You can think of django.contrib as Django’s equivalent of the Python standard library – optional, de facto implementations of common patterns. They’re bundled with Django so that you don’t have to reinvent the wheel in your own applications.

The admin site is the first part of django.contrib that we’re covering in this book; technically, it’s called django.contrib.admin. Other available features in django.contrib include a user authentication system (django.contrib.auth), support for anonymous sessions (django.contrib.sessions) and even a system for user comments (django.contrib.comments). You’ll get to know the various django.contrib features as you become a Django expert, and we’ll spend some more time discussing them in Chapter 16. For now, just know that Django ships with many nice add-ons, and django.contrib is generally where they live.

Activating the Admin Interface

The Django admin site is entirely optional, because only certain types of sites need this functionality. That means you’ll need to take a few steps to activate it in your project.

First, make a few changes to your settings file:

1. Add 'django.contrib.admin' to the INSTALLED_APPS setting. (The order of INSTALLED_APPS doesn’t matter, but we like to keep things alphabetical so it’s easy for a human to read.)
2. Make sure INSTALLED_APPS contains 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.messages' and 'django.contrib.sessions'. The Django admin site requires these three packages. (If you’re following along with our ongoing mysite project, note that we commented out these four INSTALLED_APPS entries in Chapter 5. Uncomment them now.)
3. Make sure MIDDLEWARE_CLASSES contains 'django.middleware.common.CommonMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware' and 'django.contrib.auth.middleware.AuthenticationMiddleware'. (Again, if you’re following along, note that we commented them out in Chapter 5, so uncomment them.)

Second, run python manage.py syncdb. This step will install the extra database tables that the admin interface uses. The first time you run syncdb with 'django.contrib.auth' in INSTALLED_APPS, you’ll be asked about creating a superuser. If you don’t do this, you’ll need to run python manage.py createsuperuser separately to create an admin user account; otherwise, you won’t be able to log in to the admin site. (Potential gotcha: the python manage.py createsuperuser command is only available if 'django.contrib.auth' is in your INSTALLED_APPS.)

Third, add the admin site to your URLconf (in urls.py, remember). By default, the urls.py generated by django-admin.py startproject contains commented-out code for the Django admin, and all you have to do is uncomment it. For the record, here are the bits you need to make sure are in there:

# Include these import statements...
from django.contrib import admin
admin.autodiscover()

# And include this URLpattern...
urlpatterns = patterns('',
    # ...
    (r'^admin/', include(admin.site.urls)),
    # ...
)

With that bit of configuration out of the way, now you can see the Django admin site in action. Just run the development server (python manage.py runserver, as in previous chapters) and visit http://127.0.0.1:8000/admin/ in your Web browser.

Using the Admin Site

The admin site is designed to be used by nontechnical users, and as such it should be pretty self-explanatory. Nevertheless, we’ll give you a quick walkthrough of the basic features.

The first thing you’ll see is a login screen, as shown in Figure 6-1.

Figure 6-1. Django’s login screen

Log in with the username and password you set up when you added your superuser. If you’re unable to log in, make sure you’ve actually created a superuser – try running python manage.py createsuperuser.

Once you’re logged in, the first thing you’ll see will be the admin home page. This page lists all the available types of data that can be edited on the admin site. At this point, because we haven’t activated any of our own models yet, the list is sparse: it includes only Groups and Users, which are the two default admin-editable models.

Figure 6-2. The Django admin home page

Each type of data in the Django admin site has a change list and an edit form. Change lists show you all the available objects in the database, and edit forms let you add, change or delete particular records in your database.

Click the “Change” link in the “Users” row to load the change list page for users.

Figure 6-3. The user change list page

This page displays all users in the database; you can think of it as a prettied-up Web version of a SELECT * FROM auth_user; SQL query. If you’re following along with our ongoing example, you’ll only see one user here, assuming you’ve added only one, but once you have more users, you’ll probably find the filtering, sorting and searching options useful. Filtering options are at right, sorting is available by clicking a column header, and the search box at the top lets you search by username.

Click the username of the user you created, and you’ll see the edit form for that user.

Figure 6-4. The user edit form

This page lets you change the attributes of the user, like the first/last names and various permissions. (Note that to change a user’s password, you should click “change password form” under the password field rather than editing the hashed code.) Another thing to note here is that fields of different types get different widgets – for example, date/time fields have calendar controls, boolean fields have checkboxes, character fields have simple text input fields.

You can delete a record by clicking the delete button at the bottom left of its edit form. That’ll take you to a confirmation page, which, in some cases, will display any dependent objects that will be deleted, too. (For example, if you delete a publisher, any book with that publisher will be deleted, too!)

You can add a record by clicking “Add” in the appropriate column of the admin home page. This will give you an empty version of the edit page, ready for you to fill out.

You’ll also notice that the admin interface also handles input validation for you. Try leaving a required field blank or putting an invalid date into a date field, and you’ll see those errors when you try to save, as shown in Figure 6-5.

Figure 6-5. An edit form displaying errors

When you edit an existing object, you’ll notice a History link in the upper-right corner of the window. Every change made through the admin interface is logged, and you can examine this log by clicking the History link (see Figure 6-6).

Figure 6-6. An object history page

Adding Your Models to the Admin Site

There’s one crucial part we haven’t done yet. Let’s add our own models to the admin site, so we can add, change and delete objects in our custom database tables using this nice interface. We’ll continue the books example from Chapter 5, where we defined three models: Publisher, Author and Book.

Within the books directory (mysite/books), create a file called admin.py, and type in the following lines of code:

from django.contrib import admin
from mysite.books.models import Publisher, Author, Book

admin.site.register(Publisher)
admin.site.register(Author)
admin.site.register(Book)

This code tells the Django admin site to offer an interface for each of these models.

Once you’ve done this, go to your admin home page in your Web browser (http://127.0.0.1:8000/admin/), and you should see a “Books” section with links for Authors, Books and Publishers. (You might have to stop and start the runserver for the changes to take effect.)

You now have a fully functional admin interface for each of those three models. That was easy!

Take some time to add and change records, to populate your database with some data. If you followed Chapter 5’s examples of creating Publisher objects (and you didn’t delete them), you’ll already see those records on the publisher change list page.

One feature worth mentioning here is the admin site’s handling of foreign keys and many-to-many relationships, both of which appear in the Book model. As a reminder, here’s what the Book model looks like:

class Book(models.Model):
    title = models.CharField(max_length=100)
    authors = models.ManyToManyField(Author)
    publisher = models.ForeignKey(Publisher)
    publication_date = models.DateField()

    def __unicode__(self):
        return self.title

On the Django admin site’s “Add book” page (http://127.0.0.1:8000/admin/books/book/add/), the publisher (a ForeignKey) is represented by a select box, and the authors field (a ManyToManyField) is represented by a multiple-select box. Both fields sit next to a green plus sign icon that lets you add related records of that type. For example, if you click the green plus sign next to the “Publisher” field, you’ll get a pop-up window that lets you add a publisher. After you successfully create the publisher in the pop-up, the “Add book” form will be updated with the newly created publisher. Slick.

How the Admin Site Works

Behind the scenes, how does the admin site work? It’s pretty straightforward.

When Django loads your URLconf from urls.py at server startup, it executes the admin.autodiscover() statement that we added as part of activating the admin. This function iterates over your INSTALLED_APPS setting and looks for a file called admin.py in each installed app. If an admin.py exists in a given app, it executes the code in that file.

In the admin.py in our books app, each call to admin.site.register() simply registers the given model with the admin. The admin site will only display an edit/change interface for models that have been explicitly registered.

The app django.contrib.auth includes its own admin.py, which is why Users and Groups showed up automatically in the admin. Other django.contrib apps, such as django.contrib.redirects, also add themselves to the admin, as do many third-party Django applications you might download from the Web.

Beyond that, the Django admin site is just a Django application, with its own models, templates, views and URLpatterns. You add it to your application by hooking it into your URLconf, just as you hook in your own views. You can inspect its templates, views and URLpatterns by poking around in django/contrib/admin in your copy of the Django codebase – but don’t be tempted to change anything directly in there, as there are plenty of hooks for you to customize the way the admin site works. (If you do decide to poke around the Django admin application, keep in mind it does some rather complicated things in reading metadata about models, so it would probably take a good amount of time to read and understand the code.)

Making Fields Optional

After you play around with the admin site for a while, you’ll probably notice a limitation – the edit forms require every field to be filled out, whereas in many cases you’d want certain fields to be optional. Let’s say, for example, that we want our Author model’s email field to be optional – that is, a blank string should be allowed. In the real world, you might not have an e-mail address on file for every author.

To specify that the email field is optional, edit the Author model (which, as you’ll recall from Chapter 5, lives in mysite/books/models.py). Simply add blank=True to the email field, like so:

class Author(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=40)
    email = models.EmailField(blank=True)

This tells Django that a blank value is indeed allowed for authors’ e-mail addresses. By default, all fields have blank=False, which means blank values are not allowed.

There’s something interesting happening here. Until now, with the exception of the __unicode__() method, our models have served as definitions of our database tables – Pythonic expressions of SQL CREATE TABLE statements, essentially. In adding blank=True, we have begun expanding our model beyond a simple definition of what the database table looks like. Now, our model class is starting to become a richer collection of knowledge about what Author objects are and what they can do. Not only is the email field represented by a VARCHAR column in the database; it’s also an optional field in contexts such as the Django admin site.

Once you’ve added that blank=True, reload the “Add author” edit form (http://127.0.0.1:8000/admin/books/author/add/), and you’ll notice the field’s label – “Email” – is no longer bolded. This signifies it’s not a required field. You can now add authors without needing to provide e-mail addresses; you won’t get the loud red “This field is required” message anymore, if the field is submitted empty.

CREATE TABLE "books_author" (
    "id" serial NOT NULL PRIMARY KEY,
    "first_name" varchar(30) NOT NULL,
    "last_name" varchar(40) NOT NULL,
    "email" varchar(75) NOT NULL
)
;

class Book(models.Model):
    title = models.CharField(max_length=100)
    authors = models.ManyToManyField(Author)
    publisher = models.ForeignKey(Publisher)
    publication_date = models.DateField(blank=True, null=True)

ALTER TABLE books_book ALTER COLUMN publication_date DROP NOT NULL;

Customizing Field Labels

On the admin site’s edit forms, each field’s label is generated from its model field name. The algorithm is simple: Django just replaces underscores with spaces and capitalizes the first character, so, for example, the Book model’s publication_date field has the label “Publication date.”

However, field names don’t always lend themselves to nice admin field labels, so in some cases you might want to customize a label. You can do this by specifying verbose_name in the appropriate model field.

For example, here’s how we can change the label of the Author.email field to “e-mail,” with a hyphen:

class Author(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=40)
    email = models.EmailField(blank=True, verbose_name='e-mail')

Make that change and reload the server, and you should see the field’s new label on the author edit form.

Note that you shouldn’t capitalize the first letter of a verbose_name unless it should always be capitalized (e.g., "USA state"). Django will automatically capitalize it when it needs to, and it will use the exact verbose_name value in other places that don’t require capitalization.

Finally, note that you can pass the verbose_name as a positional argument, for a slightly more compact syntax. This example is equivalent to the previous one:

class Author(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=40)
    email = models.EmailField('e-mail', blank=True)

This won’t work with ManyToManyField or ForeignKey fields, though, because they require the first argument to be a model class. In those cases, specifying verbose_name explicitly is the way to go.

Custom ModelAdmin classes

The changes we’ve made so far – blank=True, null=True and verbose_name – are really model-level changes, not admin-level changes. That is, these changes are fundamentally a part of the model and just so happen to be used by the admin site; there’s nothing admin-specific about them.

Beyond these, the Django admin site offers a wealth of options that let you customize how the admin site works for a particular model. Such options live in ModelAdmin classes, which are classes that contain configuration for a specific model in a specific admin site instance.

Customizing change lists

Let’s dive into admin customization by specifying the fields that are displayed on the change list for our Author model. By default, the change list displays the result of __unicode__() for each object. In Chapter 5, we defined the __unicode__() method for Author objects to display the first name and last name together:

class Author(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=40)
    email = models.EmailField(blank=True, verbose_name='e-mail')

    def __unicode__(self):
        return u'%s %s' % (self.first_name, self.last_name)

As a result, the change list for Author objects displays each other’s first name and last name together, as you can see in Figure 6-7.

Figure 6-7. The author change list page

We can improve on this default behavior by adding a few other fields to the change list display. It’d be handy, for example, to see each author’s e-mail address in this list, and it’d be nice to be able to sort by first and last name.

To make this happen, we’ll define a ModelAdmin class for the Author model. This class is the key to customizing the admin, and one of the most basic things it lets you do is specify the list of fields to display on change list pages. Edit admin.py to make these changes:

from django.contrib import admin
from mysite.books.models import Publisher, Author, Book

class AuthorAdmin(admin.ModelAdmin):
    list_display = ('first_name', 'last_name', 'email')

admin.site.register(Publisher)
admin.site.register(Author, AuthorAdmin)
admin.site.register(Book)

Here’s what we’ve done:

• We created the class AuthorAdmin. This class, which subclasses django.contrib.admin.ModelAdmin, holds custom configuration for a specific admin model. We’ve only specified one customization – list_display, which is set to a tuple of field names to display on the change list page. These field names must exist in the model, of course.

• We altered the admin.site.register() call to add AuthorAdmin after Author. You can read this as: “Register the Author model with the AuthorAdmin options.”

  The admin.site.register() function takes a ModelAdmin subclass as an optional second argument. If you don’t specify a second argument (as is the case for Publisher and Book), Django will use the default admin options for that model.

With that tweak made, reload the author change list page, and you’ll see it’s now displaying three columns – the first name, last name and e-mail address. In addition, each of those columns is sortable by clicking on the column header. (See Figure 6-8.)

Figure 6-8. The author change list page after list_display

Next, let’s add a simple search bar. Add search_fields to the AuthorAdmin, like so:

class AuthorAdmin(admin.ModelAdmin):
    list_display = ('first_name', 'last_name', 'email')
    search_fields = ('first_name', 'last_name')

Reload the page in your browser, and you should see a search bar at the top. (See Figure 6-9.) We’ve just told the admin change list page to include a search bar that searches against the first_name and last_name fields. As a user might expect, this is case-insensitive and searches both fields, so searching for the string "bar" would find both an author with the first name Barney and an author with the last name Hobarson.

Figure 6-9. The author change list page after search_fields

Next, let’s add some date filters to our Book model’s change list page:

from django.contrib import admin
from mysite.books.models import Publisher, Author, Book

class AuthorAdmin(admin.ModelAdmin):
    list_display = ('first_name', 'last_name', 'email')
    search_fields = ('first_name', 'last_name')

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'publisher', 'publication_date')
    list_filter = ('publication_date',)

admin.site.register(Publisher)
admin.site.register(Author, AuthorAdmin)
admin.site.register(Book, BookAdmin)

Here, because we’re dealing with a different set of options, we created a separate ModelAdmin class – BookAdmin. First, we defined a list_display just to make the change list look a bit nicer. Then, we used list_filter, which is set to a tuple of fields to use to create filters along the right side of the change list page. For date fields, Django provides shortcuts to filter the list to “Today,” “Past 7 days,” “This month” and “This year” – shortcuts that Django’s developers have found hit the common cases for filtering by date. Figure 6-10 shows what that looks like.

Figure 6-10. The book change list page after list_filter

list_filter also works on fields of other types, not just DateField. (Try it with BooleanField and ForeignKey fields, for example.) The filters show up as long as there are at least 2 values to choose from.

Another way to offer date filters is to use the date_hierarchy admin option, like this:

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'publisher', 'publication_date')
    list_filter = ('publication_date',)
    date_hierarchy = 'publication_date'

With this in place, the change list page gets a date drill-down navigation bar at the top of the list, as shown in Figure 6-11. It starts with a list of available years, then drills down into months and individual days.

Figure 6-11. The book change list page after date_hierarchy

Note that date_hierarchy takes a string, not a tuple, because only one date field can be used to make the hierarchy.

Finally, let’s change the default ordering so that books on the change list page are always ordered descending by their publication date. By default, the change list orders objects according to their model’s ordering within class Meta (which we covered in Chapter 5) – but you haven’t specified this ordering value, then the ordering is undefined.

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'publisher', 'publication_date')
    list_filter = ('publication_date',)
    date_hierarchy = 'publication_date'
    ordering = ('-publication_date',)

This admin ordering option works exactly as the ordering in models’ class Meta, except that it only uses the first field name in the list. Just pass a list or tuple of field names, and add a minus sign to a field to use descending sort order.

Reload the book change list to see this in action. Note that the “Publication date” header now includes a small arrow that indicates which way the records are sorted. (See Figure 6-12.)

Figure 6-12. The book change list page after ordering

We’ve covered the main change list options here. Using these options, you can make a very powerful, production-ready data-editing interface with only a few lines of code.

Customizing edit forms

Just as the change list can be customized, edit forms can be customized in many ways.

First, let’s customize the way fields are ordered. By default, the order of fields in an edit form corresponds to the order they’re defined in the model. We can change that using the fields option in our ModelAdmin subclass:

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'publisher', 'publication_date')
    list_filter = ('publication_date',)
    date_hierarchy = 'publication_date'
    ordering = ('-publication_date',)
    fields = ('title', 'authors', 'publisher', 'publication_date')

After this change, the edit form for books will use the given ordering for fields. It’s slightly more natural to have the authors after the book title. Of course, the field order should depend on your data-entry workflow. Every form is different.

Another useful thing the fields option lets you do is to exclude certain fields from being edited entirely. Just leave out the field(s) you want to exclude. You might use this if your admin users are only trusted to edit a certain segment of your data, or if part of your fields are changed by some outside, automated process. For example, in our book database, we could hide the publication_date field from being editable:

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'publisher', 'publication_date')
    list_filter = ('publication_date',)
    date_hierarchy = 'publication_date'
    ordering = ('-publication_date',)
    fields = ('title', 'authors', 'publisher')

As a result, the edit form for books doesn’t offer a way to specify the publication date. This could be useful, say, if you’re an editor who prefers that his authors not push back publication dates. (This is purely a hypothetical example, of course.)

When a user uses this incomplete form to add a new book, Django will simply set the publication_date to None – so make sure that field has null=True.

Another commonly used edit-form customization has to do with many-to-many fields. As we’ve seen on the edit form for books, the admin site represents each ManyToManyField as a multiple-select boxes, which is the most logical HTML input widget to use – but multiple-select boxes can be difficult to use. If you want to select multiple items, you have to hold down the control key, or command on a Mac, to do so. The admin site helpfully inserts a bit of text that explains this, but, still, it gets unwieldy when your field contains hundreds of options.

The admin site’s solution is filter_horizontal. Let’s add that to BookAdmin and see what it does.

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'publisher', 'publication_date')
    list_filter = ('publication_date',)
    date_hierarchy = 'publication_date'
    ordering = ('-publication_date',)
    filter_horizontal = ('authors',)

(If you’re following along, note that we’ve also removed the fields option to restore all the fields in the edit form.)

Reload the edit form for books, and you’ll see that the “Authors” section now uses a fancy JavaScript filter interface that lets you search through the options dynamically and move specific authors from “Available authors” to the “Chosen authors” box, and vice versa.

Figure 6-13. The book edit form after adding filter_horizontal

We’d highly recommend using filter_horizontal for any ManyToManyField that has more than 10 items. It’s far easier to use than a simple multiple-select widget. Also, note you can use filter_horizontal for multiple fields – just specify each name in the tuple.

ModelAdmin classes also support a filter_vertical option. This works exactly as filter_horizontal, but the resulting JavaScript interface stacks the two boxes vertically instead of horizontally. It’s a matter of personal taste.

filter_horizontal and filter_vertical only work on ManyToManyField fields, not ForeignKey fields. By default, the admin site uses simple <select> boxes for ForeignKey fields, but, as for ManyToManyField, sometimes you don’t want to incur the overhead of having to select all the related objects to display in the drop-down. For example, if our book database grows to include thousands of publishers, the “Add book” form could take a while to load, because it would have to load every publisher for display in the <select> box.

The way to fix this is to use an option called raw_id_fields. Set this to a tuple of ForeignKey field names, and those fields will be displayed in the admin with a simple text input box (<input type="text">) instead of a <select>. See Figure 6-14.

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'publisher', 'publication_date')
    list_filter = ('publication_date',)
    date_hierarchy = 'publication_date'
    ordering = ('-publication_date',)
    filter_horizontal = ('authors',)
    raw_id_fields = ('publisher',)

Figure 6-14. The book edit form after adding raw_id_fields

What do you enter in this input box? The database ID of the publisher. Given that humans don’t normally memorize database IDs, there’s also a magnifying-glass icon that you can click to pull up a pop-up window, from which you can select the publisher to add.

Users, Groups, and Permissions

Because you’re logged in as a superuser, you have access to create, edit, and delete any object. Naturally, different environments require different permission systems – not everybody can or should be a superuser. Django’s admin site uses a permissions system that you can use to give specific users access only to the portions of the interface that they need.

These user accounts are meant to be generic enough to be used outside of the admin interface, but we’ll just treat them as admin user accounts for now. In Chapter 14, we’ll cover how to integrate user accounts with the rest of your site (i.e., not just the admin site).

You can edit users and permissions through the admin interface just like any other object. We saw this earlier in this chapter, when we played around with the User and Group sections of the admin. User objects have the standard username, password, e-mail and real name fields you might expect, along with a set of fields that define what the user is allowed to do in the admin interface. First, there’s a set of three boolean flags:

• The “active” flag controls whether the user is active at all. If this flag is off and the user tries to log in, he won’t be allowed in, even with a valid password.
• The “staff” flag controls whether the user is allowed to log in to the admin interface (i.e., whether that user is considered a “staff member” in your organization). Since this same user system can be used to control access to public (i.e., non-admin) sites (see Chapter 14), this flag differentiates between public users and administrators.
• The “superuser” flag gives the user full access to add, create and delete any item in the admin interface. If a user has this flag set, then all regular permissions (or lack thereof) are ignored for that user.

“Normal” admin users – that is, active, non-superuser staff members – are granted admin access through assigned permissions. Each object editable through the admin interface (e.g., books, authors, publishers) has three permissions: a create permission, an edit permission and a delete permission. Assigning permissions to a user grants the user access to do what is described by those permissions.

When you create a user, that user has no permissions, and it’s up to you to give the user specific permissions. For example, you can give a user permission to add and change publishers, but not permission to delete them. Note that these permissions are defined per-model, not per-object – so they let you say “John can make changes to any book,” but they don’t let you say “John can make changes to any book published by Apress.” The latter functionality, per-object permissions, is a bit more complicated and is outside the scope of this book but is covered in the Django documentation.

You can also assign users to groups. A group is simply a set of permissions to apply to all members of that group. Groups are useful for granting identical permissions to a subset of users.

When and Why to Use the Admin Interface – And When Not to

After having worked through this chapter, you should have a good idea of how to use Django’s admin site. But we want to make a point of covering when and why you might want to use it – and when not to use it.

Django’s admin site especially shines when nontechnical users need to be able to enter data; that’s the purpose behind the feature, after all. At the newspaper where Django was first developed, development of a typical online feature – say, a special report on water quality in the municipal supply – would go something like this:

• The reporter responsible for the project meets with one of the developers and describes the available data.
• The developer designs Django models to fit this data and then opens up the admin site to the reporter.
• The reporter inspects the admin site to point out any missing or extraneous fields – better now than later. The developer changes the models iteratively.
• When the models are agreed upon, the reporter begins entering data using the admin site. At the same time, the programmer can focus on developing the publicly accessible views/templates (the fun part!).

In other words, the raison d’être of Django’s admin interface is facilitating the simultaneous work of content producers and programmers.

However, beyond these obvious data entry tasks, the admin site is useful in a few other cases:

• Inspecting data models: Once you’ve defined a few models, it can be quite useful to call them up in the admin interface and enter some dummy data. In some cases, this might reveal data-modeling mistakes or other problems with your models.
• Managing acquired data: For applications that rely on data coming from external sources (e.g., users or Web crawlers), the admin site gives you an easy way to inspect or edit this data. You can think of it as a less powerful, but more convenient, version of your database’s command-line utility.
• Quick and dirty data-management apps: You can use the admin site to build yourself a very lightweight data management app – say, to keep track of expenses. If you’re just building something for your own needs, not for public consumption, the admin site can take you a long way. In this sense, you can think of it as a beefed up, relational version of a spreadsheet.

One final point we want to make clear is: the admin site is not an end-all-be-all. Over the years, we’ve seen it hacked and chopped up to serve a variety of functions it wasn’t intended to serve. It’s not intended to be a public interface to data, nor is it intended to allow for sophisticated sorting and searching of your data. As we said early in this chapter, it’s for trusted site administrators. Keeping this sweet spot in mind is the key to effective admin-site usage.

What’s Next?

So far we’ve created a few models and configured a top-notch interface for editing data. In the next chapter Chapter 7, we’ll move on to the real “meat and potatoes” of Web development: form creation and processing.

Getting Data From the Request Object

We introduced HttpRequest objects in Chapter 3 when we first covered view functions, but we didn’t have much to say about them at the time. Recall that each view function takes an HttpRequest object as its first parameter, as in our hello() view:

from django.http import HttpResponse

def hello(request):
    return HttpResponse("Hello world")

HttpRequest objects, such as the variable request here, have a number of interesting attributes and methods that you should familiarize yourself with, so that you know what’s possible. You can use these attributes to get information about the current request (i.e., the user/Web browser that’s loading the current page on your Django-powered site), at the time the view function is executed.

# BAD!
def current_url_view_bad(request):
    return HttpResponse("Welcome to the page at /current/")

# GOOD
def current_url_view_good(request):
    return HttpResponse("Welcome to the page at %s" % request.path)

# BAD!
def ua_display_bad(request):
    ua = request.META['HTTP_USER_AGENT']  # Might raise KeyError!
    return HttpResponse("Your browser is %s" % ua)

# GOOD (VERSION 1)
def ua_display_good1(request):
    try:
        ua = request.META['HTTP_USER_AGENT']
    except KeyError:
        ua = 'unknown'
    return HttpResponse("Your browser is %s" % ua)

# GOOD (VERSION 2)
def ua_display_good2(request):
    ua = request.META.get('HTTP_USER_AGENT', 'unknown')
    return HttpResponse("Your browser is %s" % ua)

def display_meta(request):
    values = request.META.items()
    values.sort()
    html = []
    for k, v in values:
        html.append('<tr><td>%s</td><td>%s</td></tr>' % (k, v))
    return HttpResponse('<table>%s</table>' % '\n'.join(html))