SMTPUTF8_README.html [plain text]
<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Postfix SMTPUTF8 support</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">
Postfix SMTPUTF8 support
</h1>
<hr>
<h2> Overview </h2>
<p> This document describes Postfix support for Email Address
Internationalization (EAI) as defined in RFC 6531 (SMTPUTF8 extension),
RFC 6532 (Internationalized email headers) and RFC 6533 (Internationalized
delivery status notifications). Introduced with Postfix version
3.0, this fully supports UTF-8 email addresses and UTF-8 message
header values. </p>
<p> Topics covered in this document: </p>
<ul>
<li><a href="#building">Building with/without SMTPUTF8 support</a>
<li><a href="#enabling">Enabling Postfix SMTPUTF8 support</a>
<li><a href="#using">Using Postfix SMTPUTF8 support</a>
<li><a href="#detecting">SMTPUTF8 autodetection</a>
<li><a href="#limitations">Limitations of the current implementation</a>
<li><a href="#compatibility">Compatibility with pre-SMTPUTF8 environments</a>
<li><a href="#idna2003">Compatibility with IDNA2003</a>
<li><a href="#credits">Credits</a>
</ul>
<h2> <a name="building">Building Postfix with/without SMTPUTF8 support</a> </h2>
<p> Postfix will build with SMTPUTF8 support if the ICU version
≥ 46 library and header files are installed on the system. The
package name varies with the OS distribution. The table shows package
names for a number of platforms at the time this text was written.
</p>
<blockquote>
<table border="1">
<tr> <th> OS Distribution </th> <th> Package </th> </tr>
<tr> <td> FreeBSD, NetBSD, etc. </td> <td> icu </td> </tr>
<tr> <td> Centos, Fedora, RHEL </td> <td> libicu-devel </td> </tr>
<tr> <td> Debian, Ubuntu </td> <td> libicu-dev </td> </tr>
</table>
</blockquote>
<p> To force Postfix to build without SMTPUTF8, specify: </p>
<blockquote>
<pre>
$ <b>make makefiles CCARGS="-DNO_EAI ..."</b>
</pre>
</blockquote>
<p> See the INSTALL document for more "make makefiles" options. </p>
<h2> <a name="enabling">Enabling Postfix SMTPUTF8 support</a> </h2>
<p> There is more to SMTPUTF8 than just Postfix itself. The rest
of your email infrastructure also needs to be able to handle UTF-8
email addresses and message header values. This includes SMTPUTF8
protocol support in SMTP-based content filters (Amavisd), LMTP
servers (Dovecot), and down-stream SMTP servers. </p>
<p> Postfix SMTPUTF8 support is enabled by default, but it may be
disabled as part of a backwards-compatibility safety net (see the
COMPATIBILITY_README file). </p>
<p> SMTPUTF8 support is enabled by setting the smtputf8_enable
parameter in main.cf:</p>
<blockquote>
<pre>
# <b>postconf "smtputf8_enable = yes"</b>
# <b>postfix reload</b>
</pre>
</blockquote>
<p> (With Postfix ≤ 3.1, you may also need to specify "<b>option_group
= client</b>" in Postfix MySQL client files, to enable UTF8 support
in MySQL queries. This setting is the default as of Postfix 3.2.) </p>
<p> With SMTPUTF8 support enabled, Postfix changes behavior with
respect to earlier Postfix releases: </p>
<ul>
<li> <p> UTF-8 is permitted in the myorigin parameter value. However,
the myhostname and mydomain parameters must currently specify
ASCII-only domain names. This limitation may be removed later. </p>
<li> <p> UTF-8 is the only form of non-ASCII text that Postfix
supports in access tables, address rewriting tables, and other
tables that are indexed with an email address, hostname, or domain
name. </p>
<li> <p> The header_checks-like and body_checks-like features are
not UTF-8 enabled, and therefore they do not enforce UTF-8 syntax
rules on inputs and outputs. The reason is that non-ASCII text may
be sent in encodings other than UTF-8, and that real email sometimes
contains malformed headers. Instead of skipping non-UTF-8 content,
Postfix should be able to filter it. You may try to enable UTF-8
processing by starting a PCRE pattern with the sequence (*UTF8),
but this is will result in "message not accepted, try again later"
errors when the PCRE pattern matcher encounters non-UTF-8 input.
Other features that are not UTF-8 enabled are smtpd_command_filter,
smtp_reply_filter, the *_delivery_status_filter features, and the
*_dns_reply_filter features (the latter because DNS is by definition
an ASCII protocol). </p>
<li> <p> The Postfix SMTP server announces SMTPUTF8 support in the
EHLO response. </p>
<pre>
220 server.example.com ESMTP Postfix
<b>EHLO client.example.com</b>
250-server.example.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 SMTPUTF8
</pre>
<li> <p> The Postfix SMTP server accepts the SMTPUTF8 request in
MAIL FROM and VRFY commands. </p>
<pre>
<b>MAIL FROM:<address> SMTPUTF8 ...</b>
<b>VRFY address SMTPUTF8</b>
</pre>
<li> <p> The Postfix SMTP client may issue the SMTPUTF8 request in
MAIL FROM commands. </p>
<li> <p> The Postfix SMTP server accepts UTF-8 in email address
domains, but only after the remote SMTP client issues the
SMTPUTF8 request in MAIL FROM or VRFY commands. </p>
</ul>
<p> Postfix already permitted UTF-8 in message header values
and in address localparts. This does not change. </p>
<h2> <a name="using">Using Postfix SMTPUTF8 support</a> </h2>
<p> After Postfix SMTPUTF8 support is turned on, Postfix behavior
will depend on 1) whether a remote SMTP client requests SMTPUTF8
support, 2) the presence of UTF-8 content in the message envelope
and headers, and 3) whether a down-stream SMTP (or LMTP) server
announces SMTPUTF8 support. </p>
<ul>
<li> <p> When the Postfix SMTP server receives a message WITHOUT
the SMTPUTF8 request, Postfix handles the message as it has always
done (at least that is the default, see autodetection below).
Specifically, the Postfix SMTP server does not accept UTF-8 in the
envelope sender domain name or envelope recipient domain name, and
the Postfix SMTP client does not issue the SMTPUTF8 request when
delivering that message to an SMTP or LMTP server that announces
SMTPUTF8 support (again, that is the default). Postfix will accept
UTF-8 in message header values and in the localpart of envelope
sender and recipient addresses, because it has always done that.
</p>
<li> <p> When the Postfix SMTP server receives a message WITH the
SMTPUTF8 request, Postfix will issue the SMTPUTF8 request when
delivering that message to an SMTP or LMTP server that announces
SMTPUTF8 support. This is not configurable. </p>
<li> <p> When a message is received with the SMTPUTF8 request,
Postfix will deliver the message to a non-SMTPUTF8 SMTP or LMTP
server ONLY if: </p>
<ul>
<li> <p> No message header value contains UTF-8. </p>
<li> <p> The envelope sender address contains no UTF-8, </p>
<li> <p> No envelope recipient address for that specific
SMTP/LMTP delivery transaction contains UTF-8. </p>
<blockquote> <p> NOTE: Recipients in other email delivery
transactions for that same message may still contain UTF-8.
</p> </blockquote>
</ul>
<p> Otherwise, Postfix will return the recipient(s) for that
email delivery transaction as undeliverable. The delivery status
notification message will be an SMTPUTF8 message. It will therefore
be subject to the same restrictions as email that is received
with the SMTPUTF8 request. </p>
<li> <p> When the Postfix SMTP server receives a message with the
SMTPUTF8 request, that request also applies after the message is
forwarded via a virtual or local alias, or $HOME/.forward file.
</p>
</ul>
<h2> <a name="detecting">SMTPUTF8 autodetection</a> </h2>
<p> This section applies only to systems that have SMTPUTF8 support
turned on (smtputf8_enable = yes). </p>
<p> For compatibility with pre-SMTPUTF8 environments, Postfix does
not automatically set the "SMTPUTF8 requested" flag on messages
from non-SMTPUTF8 clients that contain an UTF-8 header value or
UTF-8 address localpart. This would make such messages undeliverable
to non-SMTPUTF8 servers, and could be a barrier to SMTPUTF8 adoption.
</p>
<p> By default, Postfix sets the "SMTPUTF8 requested" flag only on
address verification probes and on Postfix sendmail submissions
that contain UTF-8 in the sender address, UTF-8 in a recipient
address, or UTF-8 in a message header value. </p>
<blockquote>
<pre>
/etc/postfix/main.cf:
smtputf8_autodetect_classes = sendmail, verify
</pre>
</blockquote>
<p> However, if you have a non-ASCII myorigin or mydomain setting,
or if you have a configuration that introduces UTF-8 addresses with
virtual aliases, canonical mappings, or BCC mappings, then you may
have to apply SMTPUTF8 autodetection to all email: </p>
<blockquote>
<pre>
/etc/postfix/main.cf:
smtputf8_autodetect_classes = all
</pre>
</blockquote>
<p> This will, of course, also flag email that was received without
SMTPUTF8 request, but that contains UTF-8 in a sender address
localpart, receiver address localpart, or message header value.
Such email was not standards-compliant, but Postfix would have
delivered it if SMTPUTF8 support was disabled. </p>
<h2> <a name="limitations">Limitations of the current implementation</a>
</h2>
<p> The Postfix implementation is a work in progress; limitations
are steadily being removed. The text below describes the situation
at one point in time. </p>
<h3> No automatic conversions between ASCII and UTF-8 domain names. </h3>
<p> Some background: According to RFC 6530 and related documents,
an internationalized domain name can appear in two forms: the UTF-8
form, and the ASCII (xn--mumble) form. An internationalized address
localpart must be encoded in UTF-8; the RFCs do not define an ASCII
alternative form. </p>
<p> Postfix currently does not convert internationalized domain
names from UTF-8 into ASCII (or from ASCII into UTF-8) before using
domain names in SMTP commands and responses, before looking up
domain names in lists such as mydestination, relay_domains or in
lookup tables such as access tables, etc., before using domain names
in a policy daemon or Milter request, or before logging events.
</p>
<p> Postfix does, however, casefold domain names and email addresses
before matching them against a Postfix configuration parameter or
lookup table. </p>
<p> In order to use Postfix SMTPUTF8 support: </p>
<ul>
<li> <p> The Postfix parameters myhostname and mydomain must be in
ASCII form. One is a substring of the other, and the myhostname
value is used in SMTP commands and responses that require ASCII.
The parameter myorigin (added to local addresses without domain)
supports UTF-8. </p>
<li> <p> You need to configure both the ASCII and UTF-8 forms of
an Internationalized domain name in Postfix parameters such as
mydestination and relay_domains, as well as lookup table search
keys. </p>
<li> <p> Milters, content filters, policy servers and logfile
analysis tools need to be able to handle both the ASCII and UTF-8
forms of Internationalized domain names. </p>
</ul>
<h2> <a name="compatibility">Compatibility with pre-SMTPUTF8
environments</a> </h2>
<h3> Mailing lists with UTF-8 and non-UTF-8 subscribers </h3>
<p> With Postfix, there is no need to split mailing lists into UTF-8 and
non-UTF-8 members. Postfix will try to deliver the non-UTF8 subscribers
over "traditional" non-SMTPUTF8 sessions, as long as the message
has an ASCII envelope sender address and all-ASCII header values.
The mailing list manager may have to apply RFC 2047 encoding to
satisfy that last condition. </p>
<h3> Pre-existing non-ASCII email flows </h3>
<p> With "smtputf8_enable = no", Postfix handles email with non-ASCII
in address localparts (and in headers) as before. The vast majority
of email software is perfectly capable of handling such email, even
if pre-SMTPUTF8 standards do not support such practice. </p>
<p> However, when you specify "smtputf8_enable = yes", Postfix
requires that non-ASCII address information is encoded in UTF-8 and
will reject other encodings such as ISO-8859. It is not practical
for Postfix to support multiple encodings at the same time. There
is no problem with RFC 2047 encodings such as "=?ISO-8859-1?Q?text?=",
because those use only characters from the ASCII characterset. </p>
<h2> <a name="idna2003">Compatibility with IDNA2003</a> </h2>
<p> Postfix ≥ 3.2 by default disables the 'transitional'
compatibility between IDNA2003 and IDNA2008, when converting UTF-8
domain names to/from the ASCII form that is used in DNS lookups.
This makes Postfix behavior consistent with current versions of the
Firefox and Chrome web browsers. Specify "enable_idna2003_compatibility
= yes" to get the historical behavior. </p>
This affects the conversion of domain names that contain for example
the German sz (ß) and the Greek zeta (ς). See
http://unicode.org/cldr/utility/idna.jsp for more examples. </p>
<h2> <a name="credits">Credits</a> </h2>
<ul>
<li> <p> May 15, 2014: Arnt Gulbrandsen posted his patch for Unicode
email support. This work was sponsored by CNNIC. </p>
<li> <p> July 15, 2014: Wietse integrated Arnt Gulbrandsen's code
and released Postfix with SMTPUTF8 support. </p>
<li> <p> January 2015: Wietse added UTF-8 support for casefolding
in Postfix lookup tables and caseless string comparison in Postfix
list-based features. </p>
</ul>
</body>
</html>