Text indexing for Pootle

Pootle provides searching functionality, which is a great way to do searches over all files in a project. If there are many strings to search through, then performance can be slow, but installing a text indexing library will speed things up, allowing searching even in very large projects.

Supported indexing engines

The following indexing engines are supported:

  • Lucene: This should be the fastest, but is not packaged for many Linux distributions, and is a bit harder to build.
  • Xapian (v1.0 or higher): Note that you need at least version 1.0.13 to run under Apache with mod_wsgi or mod_python.

Pootle’s usage of the indexing engine

The indexing database helps to speed up search queries performed from the Pootle interface.

Installation

If you want to use an indexing engine to speed up text searches, then you need to install one of the supported indexing engines and its Python binding.

The indexing engine will be used, as soon as the required Python bindings are available.

See below for details.

Lucene

Xapian

The Xapian tools packaged is required for the xapian-check command which is used to determines whether the Xapian version is compatible with Pootle.

Note

If you are deploying using a virtualenv and want to make use of Xapian then you will need to, either:

  1. Install your virtualenv with access to the system packages using the –system-site-packages option:

    virtualenv --system-site-packages ENV

  2. Compile the Python bindings

  3. Symbolically link the Xapian bindings into your virtualenv, as follows on an Ubuntu system:

    $ mkdir ${ENV}/lib/python2.6/dist-packages
$ cd ${ENV}/lib/python2.6/dist-packages
$ ln -s /usr/lib/python2.6/dist-packages/xapian.py
$ ln -s /usr/lib/python2.6/dist-packages/_xapian.so

Debugging

If you want to check which indexing engines are currently detected on your system you can execute the self-tests of the indexing engine interface of Pootle:

python translate/search/indexing/test_indexers.py

This will display the installed engines and check if they work as expected.

Note

Please file a bug report if you encounter any errors when running these tests.

The actual test for xapian is xapian-check --version.