<html> <!--Warning, preformatted content! --> <head> <title> Postfix Rate Controls</title> </head> <body> <h1><a href="big-picture.html"><img src="small-picture.gif" width="115" height="45"></a> Postfix Rate Controls</h1> <hr> <a href="index.html">Up one level</a> | <a href="basic.html">Basic Configuration</a> | <a href="uce.html">UCE Controls</a> | Rate Controls | <a href="resource.html">Resource Controls</a> | <a href="rewrite.html">Address Manipulation</a> <h2> Introduction</h2> Building a high-performance mail delivery system is one thing; building one that does not knock over other systems is a different story. Some mailers suffer from the <i>thundering herd</i> syndrome: they literally flood other systems with mail. Postfix tries to be a fast mailer and a good neighbor at the same time. <p> On the inbound side, the Postfix <a href="smtpd.8.html">SMTP server</a> has defenses in place against malicious or confused clients. They won't protect against an all-out denial of service attack on your infrastructure, but then nothing will except pulling the plug. <p> Unless indicated otherwise, all parameters described here are in the <b>main.cf</b> file. If you change parameters of a running Postfix system, don't forget to issue a <b>postfix reload</b> command. <ul> <a href="#process">Process limits</a> <p> <a href="#destination">Destination concurrency</a> <p> <a href="#recipients">Recipient limits</a> <p> <a href="#postponing">Always postponing delivery</a> <p> <a href="#backoff">Backoff from dead hosts</a> <p> <a href="#slowdown">Slowing down bad clients</a> </ul> <a name="process"><h2> Process limits</h2> The <b>default_process_limit</b> parameter (default: 100) gives direct control over inbound and outbound delivery rates. This parameter controls the number of concurrent processes that implement a Postfix service (smtp client, smtp server, local delivery, etc.). On small systems, or on systems connected via dialup networks, a <b>default_process_limit</b> of 10 is probably more than adequate. Use a larger value if your machine is a major mail hub. <p> You can override this setting for specific Postfix daemons by editing the <b>master.cf</b> file. For example, if you do not wish to receive 100 SMTP messages at the same time, you could specify: <dl> <dd> <pre> # ========================================================================== # service type private unpriv chroot wakeup maxproc command + args # (yes) (yes) (yes) (never) (100) # ========================================================================== . . . smtp inet n - - - 5 smtpd . . . </pre> </dl> <a name="destination"><h2> Destination concurrency</h2> So, you have this huge mailhub with tons of disk and memory, and have configured Postfix to run up to 1000 SMTP client processes at the same time. Congratulations. But do you really want to make 1000 simultaneous connections to the same remote system? Probably not. <p> The Postfix queue manager comes to the rescue. This program implements the analog of the TCP <i> slow start</i> flow control strategy: when delivering to a site, send a small number of messages first, then increase the rate as long as all goes well; back off in the face of congestion. <p> The <b>initial_destination_concurrency</b> parameter (default: 2) controls how many messages are initially sent to the same destination before adapting delivery concurrency. Of course, this setting is effective only as long as it does not exceed the <b><a href="#process">process limit</a></b> and the destination concurrency limit for the specific mail transport channel. <p> The <b>default_destination_concurrency_limit</b> parameter (default: 20) controls how many messages may be sent to the same destination simultaneously. You can override this setting for specific delivery channels (<b>local, smtp, uucp</b> etc.). The <b>main.cf</b> file recommends the following: <dl> <dd> <b>local_destination_concurrency_limit = 2</b> <dd> <b>default_destination_concurrency_limit = 20</b> </dl> The <b>local_destination_concurrency_limit</b> parameter controls how many messages are delivered simultaneously to the same local recipient. The recommended limit is low because delivery to the same mailbox must happen sequentially, so massive parallelism is not useful. Another good reason to limit delivery concurrency to the same recipient: if the recipient has an expensive shell command in her <b>.forward </b> file, or if the recipient is a mailing list manager, you don't want to run too many instances at the same time. <p> A destination concurrency limit of 20 for SMTP delivery seems enough to noticeably load a system without bringing it to its knees. Be careful when changing this to a much larger number. <a name="recipients"><h2> Recipient limits</h2> The <b>default_destination_recipient_limit</b> parameter (default: 50) controls how many recipients a Postfix delivery agent (<b>smtp</b>, <b>uucp</b>, etc.) will send with each copy of an email message. If an email message has more than <b>$default_destination_recipient_limit</b> recipients at the same destination, the list of recipients will be broken up into smaller lists, and multiple copies of the message will be sent. <p> You can override this setting for specific Postfix delivery agents (<b>smtp, uucp,</b> etc.). For example: <dl> <dd> <b>uucp_destination_recipient_limit = 100</b> </dl> would limit the number of recipients per <b>UUCP</b> delivery to 100. <p> You must be careful when increasing the recipient limit; some SMTP servers abort the connection when they run out of memory or when a hard recipient limit is reached, so the mail won't get through. <p> The <b>smtpd_recipient_limit</b> parameter (default: 1000) controls how many recipients the SMTP server will take per delivery. That's more than any reasonable SMTP client would send. The limit exists just to protect the local mail system against a malicious or confused client. <a name="postponing"><h2> Always postponing delivery</h2> The <b>defer_transports</b> parameter allows you to specify what mail should always be deferred until Postfix is explicitly asked to deliver. <p> A small site that is on-line only part of the time, and that wants to defer all deliveries until the command <b>sendmail -q</b> is executed (e.g., from a PPP dialout script) would use: <p> <dl> <dd><b>defer_transports = smtp</b> </dl> <p> An ISP can use the <b>defer_transports</b> feature for customers that are off-line most of the time. The customer can trigger delivery by issuing an <b>ETRN</b> command at the SMTP port. The following examples show how to configure such a customer: <p> <dl> <dt>/etc/postfix/main.cf: <p> <dd><b>defer_transports = hold</b> <p> You can specify any number of transports here. The example gives just one. <p> <dt>/etc/postfix/<a href="transport.5.html">transport</a>: <p> <dd><b>customer.com hold:[gateway.customer.com]</b> <dd><b>.customer.com hold:[gateway.customer.com]</b> <p> The [] are necessary to avoid MX lookups, which might point to your local machine. The second entry is necessary only if you want to relay mail for customer subdomains. <p> <dt>/etc/postfix/master.cf: <p> <dd><b>hold unix - - n - - smtp</b> <p> This is just the <b>master.cf</b> entry for regular SMTP, with the first field changed to <b>hold</b>. </dl> <a name="backoff"><h2> Backoff from unreachable hosts</h2> When a Postfix delivery agent (<b>smtp, local, uucp,</b> etc.) is unable to deliver a message it may blame the message itself or the receiving party. <ul> <li> If the delivery agent blames the message, the queue manager gives the queue file a time stamp into the future, so it won't be looked at for a while. By default, the amount of time to cool down is the amount of time that has passed since the message arrived. This results in so-called <i>exponential backoff</i> behavior. <p> <li> If the delivery agent blames the receiving party (for example a local recipient user, or a remote host), the queue manager not only advances the queue file time stamp, but also puts the receiving party on a "dead" list so that it will be skipped for some amount of time. </ul> <p> As you would expect, this whole process is governed by a bunch of little parameters. <dl> <dt> <b>queue_run_delay</b> (default: 1000 seconds) <dd> How often the queue manager scans the queue for deferred mail. <p> <dt> <b>maximal_queue_lifetime</b> (default: 5 days) <dd> How long a message stays in the queue before it is sent back as undeliverable. Specify 0 for mail that should be returned immediately after the first unsuccessful delivery attempt. <p> <dt> <b>minimal_backoff_time</b> (default: 1000 seconds) <dd> The minimal amount of time a message won't be looked at, and the minimal amount of time to stay away from a "dead" destination. <p> <dt> <b>maximal_backoff_time</b> (default: 4000 seconds) <dd> The maximal amount of time a message won't be looked at after a delivery failure. <p> <dt> <b>qmgr_message_recipient_limit</b> (default: 20000) <dd> The size of many in-memory queue manager data structures. Among others, this parameter limits the size of the short-term, in-memory "dead" list. Destinations that don't fit the list are not added. </dl> <a name="slowdown"><h2> Slowing down bad clients</h2> First of all, no defense will protect against an all-out denial of service attack. I just don't want to raise impossible expectations. But there are a few simple things one can do in order to deal with confused or malicious client programs. <p> Some defenses are part of a more general strategy: for example, how long a line of text may be before it is broken up into pieces, and how much text may be carried in a multi-line message header. See the <a href="resource.html">resource controls</a> documentation for details. <p> The Postfix <a href="smtpd.8.html">SMTP server</a> increments a per-session error counter whenever a client request is unrecognized or unimplemented, or whenever a client request violates <a href="uce.html">UCE restrictions</a> or other reasons. The error counter is reset when a message is transferred successfully. <p> As the per-session error count increases, the SMTP server changes behavior. The idea is to limit the damage by slowing down the client. The behavior is controlled by the following parameters: <p> <dl> <a name="smtpd_error_sleep_time"> <dt> <b>smtpd_error_sleep_time</b> (default: 1 second) <dd> When the per-session error count is small, the SMTP server pauses only when reporting a problem to a client. The purpose is to prevent naive clients from going into a fast <i>connect-error-disconnect</i> loop. <p> <a name="smtpd_soft_error_limit"> <dt> <b>smtpd_soft_error_limit</b> (default: 10) <dd> When the per-session error count exceeds this value, the SMTP server sleeps <b>error_count</b> seconds before responding to a client request. <p> <a name="smtpd_hard_error_limit"> <dt> <b>smtpd_hard_error_limit</b> (default: 20) <dd> When the per-session error count exceeds this value, the SMTP server disconnects. </dl> <p> Unfortunately, the Postfix SMTP server does not yet know how to limit the number of connections <i> from the same client</i>, other than by limiting the total number of SMTP server processes (see <a href="#process">process limit</a>). Things could be worse: some mailers don't even implement an SMTP server process limit. That's of course no excuse. I'm still looking for a good solution. <hr> <a href="index.html">Up one level</a> | <a href="basic.html">Basic Configuration</a> | <a href="uce.html">UCE Controls</a> | Rate Controls | <a href="resource.html">Resource Controls</a> | <a href="rewrite.html">Address Manipulation</a> </body> </html>