Debian package helpers for OpenERP buildouts

Reminders : why Buildout is so great

Buildout is a powerful tool to deploy custom installations, and insulate them from system-wide python libraries and from each other. It further offers a way to achieve a good repeatability of the installation, with its version pinning system.

We have witnessed too often applications that went down because of a system upgrade impacting one of the needed libraries for some unrelated reasons, and conversely way too many systems hosting business code that could not be upgraded at all, because of lack of control of the impacts.

All of this makes buildout especially interesting for fully integrated OpenERP-based applications, be it on developper machines or production environments. That's why we didn't hesitate to write a dedicated buildout recipe, more than a year ago : our past experiences teached us that it was worth it. Today, this recipe has matured a lot, and is used on a regular basis, even in unattended ways such as within our buildbot setups.

So… if you want to setup quickly 10 different OpenERP servers, on different versions, with custom code to sit side by side on your system,  buildout is definitely the way to go. Check the recipe's README for examples of buildout configurations.

Need of system dependencies

Now flexibility and insulation come at a cost : buildout will have to install all required python packages in the correct versions, and some of these actually go through compilation against a system library.

For example, the Python ldap package is essentially a wrapper around libldap. To expose libldap's API in the Python world, there's a bit of compilation to be done. Similarly, lxml is both a wrapper around libxml2 and written in Cython.

As far as insulation is concerned, this is not a big deal in our opinion. The major source of trouble we have seen comes from the fact that software written in a dynamical language such as Python tend to evolve much faster, go through API changes etc than traditional libraries. In other words, changes in Python code are more frequent and dangerous than changes in the underlying system libraries.

Still, this means that the user has to install the compilation tools, and all needed libraries, with their development files before he can unleash the buildout. This can be really cumbersome, for newcomers as well as experienced users (I'm personnally very tired of repeating the same lists of dependencies over and over again).

So, the solution is to provide system-level packages to provide all these dependencies. We started this internally a few months ago to hasten our setups, for Debian-based GNU/Linux distributions only. Now it's time to clean-up a bit and publish.

Remark: starting with OpenERP 6.1, you should also be able to do a regular "python setup.py install" on the openerp distribution with them, provided you circumvent the flaws in setup.py by installing pychart and babel first.

How to use our Debian package repository

Registration of the repository

Put the following line in your /etc/apt/sources.list, or, better, in a separate file in /etc/apt/sources.list.d (call it something.list) :

deb http://apt.anybox.fr/openerp common main

If you don't want your system to protest for unsigned packages, install our GPG key:

sudo apt-key adv --keyserver hkp://subkeys.pgp.net --recv-keys 0xE38CEB07

The key servers can sometimes be overloaded. In that case, wait a few minutes and try again. Alternatively, you may want to download the key directly from here and install it directly:

wget http://anybox.fr/docs/clef-de-signature-de-paquets/at_download/file/anybox_packagers.asc
sudo apt-key add anybox_packagers.asc

Then don't forget to refresh the list of available packages:

sudo aptitude update

Basic usage

To get all needed dependencies, just do :

sudo aptitude install openerp-server-system-build-deps

We prefer to use aptitude because of its auto-cleaning capabilities (see later).

Now you can launch some builds, this could look like this:

cd my_buildout
python bootstrap.py
bin/buildout

Cleanups

Finally, in a production environment, one typically doesn't want compilers to stay installed, and one tends to appreciate to spare space on the root filesystem. You can remove the build package, and install the run package instead

sudo aptitude install openerp-server-system-run-deps
sudo aptitude remove openerp-server-system-build-deps

This way, aptitude will remove all those packages that came with the build-deps package and have not been explicitely installed by the user (so-called automatically installed packages) or as a dependency of the run-deps package.

Conclusion

We are in early tests for this version of the packages.

For now, this is mostly tested on Debian 6.0 (Squeeze), but we expect at least the build-deps package to work with little variation on most Debian based systems, and actually witnessed that on Ubuntu 12.04 and Debian Wheezy (current testing). That's why the release is called "common". The run-deps package is expected to need adaptations for specific distributions.

Don't hesitate to report working and non working systems, so that we can make the needed adjustments for them.