Testing is important, I've been a big advocate of writing tests for a while, however when you've got tests you need to run them. This is a big problem in open source, Django works on something like six versions of Python (2.4, 2.5, 2.6, 2.7, Jython, and PyPy), 4 databases (SQLite, PostgreSQL, MySQL, Oracle, plus the GIS backends, and external backends), and I don't even know how many operating systems (at least the various Linuxes, OS X, and Windows). If I tried to run the tests in all those configurations for every commit I'd go crazy. Reusable applications have it even worse, ideally they should be tested under all those configurations, with each version of Django they support. For a Django application that wants to work on Django 1.1, 1.2, and all of those interpreters, databases, and operating systems you've got over 100 configurations. Crazy. John Resig faced a similar problem with jQuery (5 major browsers, multiple versions, mobile and desktop, different OSs), and the result was Test Swarm (note that at this time it doesn't appear to be up), an automated way for people to volunteer their machines to run tests. We need something like that for Python.
It'd be nice if it were as simple as users pointing their browser at a URL, but that's not practical with Python: the environments we want to test in are more complex than what can be detected, we need to know what external services are available (databases, for example). My suggestion is users should maintain a config file (.ini perhaps) somewhere on their system, it would say what versions of Python are available, and what external services are available (and how they can be accessed, e.g. DB passwords). Then the user downloads a bootstrap script and runs it. This script sees what services the user has available on their machine and queries a central server to see what tests need to be run, given the configuration they have available. The script downloads a test, creates a virtualenv, and does whatever setup it needs to do (e.g. writing a Django settings.py file given the available DB configuration), and runs the tests. Finally it sends the test results back to the central server.
It's very much like a standard buildbot system, except any user can download the script and start running tests. There are a number of problems to be solved, how do you verify that a project's tests aren't malicious (only allow trusted tests to start), how do you verify that the test results are valid, how do you actually write the configuration for a test suite? However, if solved I think this could be an invaluable resource for the Python community. Have a reusable app you want tested? Sign it up for PonySwarm, add a post-commit hook, and users will automatically run the tests for it.
You can find the rest here. There are view comments.
Lately I've had the opportunity to do some test-driven development with Django, which a) is awesome, I love testing, and b) means I've been working up a box full of testing utilities, and I figured I'd share them.
If you've done testing of views with Django you probably have some tests that look like:
def test_my_view(self):
response = self.client.get(reverse("my_url", kwargs={"pk": 1}))
response = self.client.post(reverse("my_url", kwargs={"pk": 1}), {
"key": "value",
})
This was a tad too verbose for my tastes so I wrote:
def get(self, url_name, *args, **kwargs):
return self.client.get(reverse(url_name, args=args, kwargs=kwargs))
def post(self, url_name, *args, **kwargs):
data = kwargs.pop("data", None)
return self.client.post(reverse(url_name, args=args, kwargs=kwargs), data)
Which are used:
def test_my_view(self):
response = self.get("my_url", pk=1)
response = self.post("my_url", pk=1, data={
"key": "value",
})
Much nicer.
The next big issue I had was logging in and out of multiple users was too verbose. I often want to switch between users, either to check different permissions or to test some inter-user workflow. That was solved with a simple context manager:
class login(object):
def __init__(self, testcase, user, password):
self.testcase = testcase
success = testcase.client.login(username=user, password=password)
self.testcase.assertTrue(
success,
"login with username=%r, password=%r failed" % (user, password)
)
def __enter__(self):
pass
def __exit__(self, *args):
self.testcase.client.logout()
def login(self, user, password):
return login(self, user, password)
This is used:
def test_my_view(self):
with self.login("username", "password"):
response = self.get("my_url", pk=1)
Again, a lot better.
Not quite a testing utility, but my app django-fixture-generator has made testing a lot easier for me. Fixtures are useful in getting data to work wit, but maintaining them is often a pain, you've got random scripts to generate them, or you just checkin some JSON to your repository with no way to regenerate it sanely (say if you add a new field to your model). django-fixture-generator gives you a clean way to manage the code for generating fixtures.
In general I've found context managers are a pretty awesome tool for writing clean, readable, succinct tests. I'm sure I'll have more utilities as I write more tests, hopefully someone finds these useful.
Following yesterday's post another hotly requested topic was testing in Django. Today I wanted to give a simple overview on how to get started writing tests for your Django applications. Since Django 1.1, Django has automatically provided a tests.py file when you create a new application, that's where we'll start.
For me the first thing I want to test with my applications is, "Do the views work?". This makes sense, the views are what the user sees, they need to at least be in a working state (200 OK response) before anything else can happen (business logic). So the most basic thing you can do to start testing is something like this:
from django.tests import TestCase
class MyTests(TestCase):
def test_views(self):
response = self.client.get("/my/url/")
self.assertEqual(response.status_code, 200)
By just making sure you run this code before you commit something you've already eliminated a bunch of errors, syntax errors in your URLs or views, typos, forgotten imports, etc. The next thing I like to test is making sure that all the branches of my code are covered, the most common place my views have branches is in views that handle forms, one branch for GET and one for POST. So I'll write a test like this:
from django.tests import TestCase
class MyTests(TestCase):
def test_forms(self):
response = self.client.get("/my/form/")
self.assertEqual(response.status_code, 200)
response = self.client.post("/my/form/", {"data": "value"})
self.assertEqual(response.status_code, 302) # Redirect on form success
response = self.client.post("/my/form/", {})
self.assertEqual(response.status_code, 200) # we get our page back with an error
Now I've tested both the GET and POST conditions on this view, as well the form is valid and form is invalid cases. With this strategy you can have a good base set of tests for any application with not a lot of work. The next step is setting up tests for your business logic. These are a little more complicated, you need to make sure models are created and edited in the right cases, emails are sent in the right places, etc. Django's testing documentation is a great place to read more on writing tests for your applications.
You can find the rest here. There are view comments.
This question came up in IRC yesterday, so I figured I'd run through it today. Django has a very extensive test suite that tests all of the components of Django itself. If you've ever written a patch for Django you probably know that tests are a requirement, for both new features and bug fixes. I'm going to try to run down how to setup your own testing envrioment.
First you need to have Django installed somewhere, for now I'll assume you have it in ~/django_src/. Somewhere on your python path, go ahead are use django-admin.py to create a new project. I've named this project django_test. Next, inside of that project create a folder named settings, and move settings.py into that folder and renmae it __init__.py. The reason we're going to have a settings directory is so that we can have subsettings for individual scenarios. Put some default settings for the various field in there now, for example my default settings provides the necessary options for SQLite. Now if there are any other subsettings you wish to setup create a file for them in the settings directory, and at the top of this file put from django_test.settings import *, followed by whatever settings you wish to overide. For example I have a mysql.py that overides my default SQLite database settings with MySQL values.
Now that we have our test settings, go to the directory where you have Django installed. To run the tests do ./tests/runtests.py --settings=django_test.settings. You can also use the DJANGO_SETTINGS_MODULE enviromental variable in place of passing in the settings like this. You can also provide a verbosity argument, the default is 0, I prefer to run with verbosity=1 because this keeps you up to date on the progress of the tests, without too much extraneus output.
Usually when you are working on a patch you don't need to run the entire test suite, your patch only affects a few tests. Therefore, you can provide a list of tests you'd like to run, so for example you could do ./tests/runtests.py --settings=django_test.settings.mysql -v 1 model_forms model_formsets. This will run the model_forms and model_formsets tests, with your mysql settings, at verbosity level 1.
And that's all it takes to run the Django test suite.
You can find the rest here. There are view comments.