Postfix OpenLDAP LMDB Howto

Introduction

Postfix uses databases of various kinds to store and look up information. Postfix databases are specified as "type:name". OpenLDAP LMDB (called "LMDB" from here on) implements the Postfix database type "lmdb". The name of a Postfix LMDB database is the name of the database file without the ".lmdb" suffix.

This document describes:

• Building Postfix with LMDB support.

• Configuring LMDB settings.

• Using LMDB maps with non-Postfix programs.

• Required minimum LMDB patchlevel.

• Credits.

Building Postfix with LMDB support

Postfix normally does not enable LMDB support. To build Postfix with LMDB support, use something like:

% make makefiles CCARGS="-DHAS_LMDB -I/usr/local/include" \
    AUXLIBS="-L/usr/local/lib -llmdb"
% make

Solaris may need this:

% make makefiles CCARGS="-DHAS_LMDB -I/usr/local/include" \
    AUXLIBS="-R/usr/local/lib -L/usr/local/lib -llmdb"
% make

The exact pathnames depend on how LMDB was installed.

When building Postfix fails with:

undefined reference to `pthread_mutexattr_destroy'
undefined reference to `pthread_mutexattr_init'
undefined reference to `pthread_mutex_lock'

Add the "-lpthread" library to the "make makefiles" command.

% make makefiles .... AUXLIBS="... -lpthread"

Configuring LMDB settings

Postfix provides one configuration parameter that controls LMDB database behavior.

• lmdb_map_size (default: 16777216). This setting specifies the initial LMDB database size limit in bytes. Each time a database becomes "full", its size limit is doubled. The maximum size is the largest signed integer value of "long".

Using LMDB maps with non-Postfix programs

Programs that use LMDB's built-in locking protocol will corrupt a Postfix LMDB database or will read garbage.

Postfix does not use LMDB's built-in locking protocol, because that would require world-writable lockfiles, and would violate Postfix security policy. Instead, Postfix uses external locks based on fcntl(2) to prevent writers from corrupting the database, and to prevent readers from receiving garbage.

See lmdb_table(5) for a detailed description of the locking protocol that all programs must use when they access a Postfix LMDB database.

Required minimum LMDB patchlevel

Currently, Postfix requires LMDB 0.9.11 or later. The required minimum LMDB patchlevel has evolved over time, as the result of Postfix deployment experience:

• LMDB 0.9.11 allows Postfix daemons to log an LMDB error message, instead of falling out of the sky without any notification.

• LMDB 0.9.10 closes an information leak where LMDB was writing up to 4-kbyte chunks of uninitialized heap memory to the database. This would persist information that was not meant to be persisted, or share information that was not meant to be shared.

• LMDB 0.9.9 allows Postfix to use external (fcntl()-based) locks, instead of having to use world-writable LMDB lock files, violating the Postfix security model in multiple ways.

• LMDB 0.9.8 allows Postfix to recover from a "database full" error without having to close the database. This version adds support to update the database size limit on-the-fly. This is necessary because Postfix database sizes vary with mail server load.

• LMDB 0.9.7 allows the postmap(1) and postalias(1) commands to use a bulk-mode transaction larger than the amount of physical memory. This is necessary because LMDB supports databases larger than physical memory.

Credits

• Howard Chu contributed the initial Postfix dict_lmdb driver.

• Wietse Venema wrote an abstraction layer (slmdb) that behaves more like Berkeley DB, NDBM, etc. This layer automatically retries an LMDB request when a database needs to be resized, or after a database was resized by a different process.

• Howard and Wietse went through many iterations with changes to both LMDB and Postfix, with input from Viktor Dukhovni.