Getting the most out of tox

tox is a recent Python testing tool, by Holger Krekel. It's stated purpose is to make testing Python projects against multiple versions of Python (or different interpreters, like PyPy and Jython) much easier. However, it can be used for so much more. Yesterday I set it up for django-taggit, and it's an absolute dream, it automates testing against four different versions of Python, two different versions of Django, and it automates building the docs and checking for any warnings from Sphinx. I'll try to give a run through on what exactly you need to do to set this up with your project.

First create a tox.ini at the root of your project (i.e. in the same directory as your setup.py). Next create a [tox] section, and list out the enviroments you'd like to be tested (i.e. which Pythons):

[tox]
envlist =
    py25, py26 , py27, pypy

The enviroments we've listed out are a few of the ones included with tox, they point at specific versions of Python and use the default testing setup. Now add a [testenv] section which will tell tox how to actually run your tests:

[testenv]
commands =
    python setup.py test
deps =
    django==1.2.3

commands is the list of commands tox will run, and deps specifies any dependencies that are needed to run the tests (tox creates a virtualenv for each enviroment and doesn't include system wide site-packages, so you need to make sure you list everything needed by default here). If you want to use this same python setup.py test formulation you'll need to be using setuptools or distribute for your setup.py and provide the test_suite argument, Eric Holscher provides a good run down for how to do this for Django projects.

Now you should be able to just type tox into your command line and it will try to run your tests in each of the enviroments you specified. Hopefully they're all passing (future test runs will go faster, for the first run it has to install all the dependencies). The next thing you may want to do is get it setup to build your documentation. To do this create a [testenv:docs] section:

[testenv:docs]
changedir = docs
deps =
    sphinx
commands =
    sphinx-build -W -b html -d {envtmpdir}/doctrees . {envtmpdir}/html

This tells tox a few things. First changedir tells it that to run these commands it should cd into the docs/ directory (if you're docs live elsewhere, change as appropriate). Next it has sphinx as a dependency. Finally the commands invoke sphinx-build, -W makes warnings into errors (so you get an approrpiate failure status code), -b html uses the HTML builder, and the rest of the parameters tell Sphinx where the docs live and to put the output in the temporary directory that tox creates for each env.

Now all you need to do is add docs to the envlist, and a tox run will build your documentation.

The last thing you might want to do is set it up to test against multiple versions of a package (such as Django 1.1, Django 1.2, and trunk). To do this create another section whose name includes both the Python version and dependency version, e.g. [testenv:py25-trunk]. In it place:

[testenv:py25-trunk]
basepython = python2.5
deps =
    http://www.djangoproject.com/download/1.3-alpha-1/tarball/

This "inherits" from the default testenv, so it still has its commands, but we specify the basepython indicating this testenv is for python 2.5, and a different set of dependencies, here we're using the Django 1.3 alpha. You'll need to do a bit of copy-paste and create one of these for each version of Python you're testing against, and make sure to add each of these to the envlist.

At this point you should have a lean, mean, testing setup. With one command you can test your package with different dependencies, different pythons, and build your documentation. The tox documentation features tons of examples so you should use it as a reference.

django-taggit 0.9 Released

It's been a long time coming, since taggit 0.8 was released in June, however 0.9 is finally here, and it brings a ton of cool bug fixes, improvements, and cleanups. If you don't already know, django-taggit is an application for django to make tagging simple and awesome. The biggest changes in this release are:

• The addition of a bunch of locales.
• Support for custom tag models.
• Moving taggit.contrib.suggest into an external package.
• Changed the filter syntax from filter(tags__in=["foo"]) to filter(tags__name__in=["foo"]). This change is backwards incompatible, but was necessary to support lookups on all fields.

You can checkout the docs for complete details. The goals for the 1.0 release are going to be adding some useful widgets for use in the admin and forms, hopefully adding a class based generic view to replace the current one, and adding a migration command to move data from django-tagging into the django-taggit models.

You can get the latest release on PyPi. Enjoy.

A Tour of the django-taggit Internals

In a previous post I talked about a cool new customization API that django-taggit has. Now I'm going to dive into the internals.

The public API is almost exclusively exposed via a class named TaggableManager, you attach one of these to your model and it has some cool tagging APIs. This class basically masquerades as ManyToManyField, this is how it gets cool things like filtering and forms automatically. If you look at its definition you'll see it has a bunch of attributes that it never actually uses, basically all of these act to emulate the Field interface. This class is also the entry point for the new customization API, exposed via the through parameter. This basically acts as an analogue to the through parameter on actual ManyToManyFields (documented here). The final crucial method is __get__, which turns TaggableManager into a descriptor.

This descriptor exposes an _TaggableManager class, which holds some of the internal logic. This class exposes all of the "managery" type methods, add(), set(), remove(), and clear(). This class is pretty simple, basically it just proxies bewteen the methods called and it's through model. This class is, unlike TaggableManager, actually a subclass of models.Manager, it just defines get_query_set() to return a QuerySet of all the tags for that model, or instance, and then filtering, ordering, and more falls out naturally.

Beyond that there's not too much going on. The code is fairly simple, and it's not particularly long. I've found this to be a pretty good pattern for extensibility, and it really resolves the need to have dozens of parameters, or GenericForeignKeys popping out every which way.