libidn.texi [plain text]

\input texinfo   @c -*- mode: texinfo; coding: us-ascii; -*-
@c This file is part of GNU Libidn.
@c See below for copyright and license.

@setfilename libidn.info
@include version.texi
@settitle GNU Libidn
@finalout

@syncodeindex pg cp

@copying
This manual is last updated @value{UPDATED} for version
@value{VERSION} of GNU Libidn.

Copyright @copyright{} 2002, 2003, 2004, 2005 Simon Josefsson.

@quotation
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with the
Invariant Sections being ``Commercial Support'', no Front-Cover Texts,
and no Back-Cover Texts.  A copy of the license is included in the
section entitled ``GNU Free Documentation License''.
@end quotation
@end copying

@dircategory GNU Libraries
@direntry
* libidn: (libidn).	Internationalized string processing library.
@end direntry

@dircategory GNU utilities
@direntry
* idn: (libidn)Invoking idn.		Command line interface to GNU Libidn.
@end direntry

@dircategory Emacs
@direntry
* IDN Library: (libidn)Emacs API.	Emacs API for IDN functions.
@end direntry

@titlepage
@title GNU Libidn
@subtitle Internationalized string processing for the GNU system
@subtitle for version @value{VERSION}, @value{UPDATED}
@author Simon Josefsson
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents

@ifnottex
@node Top
@top GNU Libidn

@insertcopying
@end ifnottex

@menu
* Introduction::		How to use this manual.
* Preparation::			What you should do before using the library.
* Utility Functions::		Unicode transformation utility functions.
* Stringprep Functions::	Stringprep functions.
* Punycode Functions::		Punycode functions.
* IDNA Functions::		IDNA functions.
* TLD Functions::		TLD functions.
* PR29 Functions::		Detect strings non-idempotent under NFKC.
* Examples::                    Demonstrate how to use the library.
* Invoking idn::		Command line interface to the library.
* Emacs API::                   Emacs Lisp API for Libidn.
* Java API::                    Notes on the Java port of Libidn.
* C# API::                      Notes on the C# port of Libidn.
* Acknowledgements::            Whom to blame.
* Milestones::                  Rough outline of development history.

Indices

* Concept Index::
* Function and Variable Index::

Appendices

* PR29 discussion::             Implementation aspects of the PR29 flaw.
* Library Copying::		How you can copy and share GNU Libidn.
* Copying This Manual::		How you can copy and share this manual.

@end menu

@node Introduction
@chapter Introduction

GNU Libidn is an implementation of the Stringprep, Punycode and IDNA
specifications defined by the IETF Internationalized Domain Names
(IDN) working group, used for internationalized domain names.  The
package is available under the GNU Lesser General Public License.

The library contains a generic Stringprep implementation that does
Unicode 3.2 NFKC normalization, mapping and prohibitation of
characters, and bidirectional character handling.  Profiles for
Nameprep, iSCSI, SASL and XMPP are included.  Punycode and ASCII
Compatible Encoding (ACE) via IDNA are supported.  A mechanism to
define Top-Level Domain (TLD) specific validation tables, and to
compare strings against those tables, is included.  Default tables for
some TLDs are also included.

The Stringprep API consists of two main functions, one for converting
data from the system's native representation into UTF-8, and one
function to perform the Stringprep processing.  Adding a new
Stringprep profile for your application within the API is
straightforward.  The Punycode API consists of one encoding function
and one decoding function.  The IDNA API consists of the ToASCII and
ToUnicode functions, as well as an high-level interface for converting
entire domain names to and from the ACE encoded form.  The TLD API
consists of one set of functions to extract the TLD name from a domain
string, one set of functions to locate the proper TLD table to use
based on the TLD name, and core functions to validate a string against
a TLD table, and some utility wrappers to perform all the steps in one
call.

The library is used by, e.g., GNU SASL and Shishi to process user
names and passwords.  Libidn can be built into GNU Libc to enable a
new system-wide getaddrinfo flag for IDN processing.

Libidn is developed for the GNU/Linux system, but runs on over 20 Unix
platforms (including Solaris, IRIX, AIX, and Tru64) and Windows.
Libidn is written in C and (parts of) the API is accessible from C,
C++, Emacs Lisp, Python and Java.  A native Java and C# port is also
provided.

@menu
* Getting Started::
* Features::
* Library Overview::
* Supported Platforms::
* Getting help::
* Commercial Support::
* Downloading and Installing::
* Bug Reports::
* Contributing::
@end menu

@node Getting Started
@section Getting Started

This manual documents the library programming interface.  All
functions and data types provided by the library are explained.
Included are also examples, and documentation for the command line
tool @file{idn} that provide a quick interface to the library.  The
Emacs Lisp bindings for the library is also discussed.

The reader is assumed to possess basic familiarity with
internationalization concepts and network programming in C or C++.

This manual can be used in several ways.  If read from the beginning
to the end, it gives a good introduction into the library and how it
can be used in an application.  Forward references are included where
necessary.  Later on, the manual can be used as a reference manual to
get just the information needed about any particular interface of the
library.  Experienced programmers might want to start looking at the
examples at the end of the manual (@pxref{Examples}), and then only
read up those parts of the interface which are unclear.

@node Features
@section Features

This library might have a couple of advantages over other libraries
doing a similar job.

@table @asis
@item It's Free Software
Anybody can use, modify, and redistribute it under the terms of the
GNU Lesser General Public License.

@item It's thread-safe
No global state is kept in the library.  All functions are reentrant.

@item It's portable
The code is intended to be written in pure ANSI C89.  It has been
tested on many Unix like operating systems, and Windows.

@item It's modularized
The library is composed of several modules, and the only interaction
between modules is through each modules' public API.  If you only need
one piece of functionality, it is possible to take the files you need
and incorporate them into your own project.

@item It's not bloated
The design of the library is based on the smallest API necessary to
implement the basic functionality.  It has been carefully extended
with a small number of high-level wrappers to make it comfortable to
use the library.  However, it does not implement additional
functionality just for the sake of completeness.

@item It's documented
Sadly, not all software comes with documentation these days.  This one
does.

@end table

@node Library Overview
@section Library Overview

The following illustration show the components that make up Libidn,
and how your application relates to the library.  In the illustration,
various components are shown as boxes.  You see the generic StringPrep
component, the various StringPrep profiles including Nameprep, the
Punycode component, the IDNA component, and the TLD component.  The
arrows indicate aggregation, e.g., IDNA uses Punycode and Nameprep,
and in turn Nameprep uses the generic StringPrep interface.  The
interfaces to all components are available for applications, no
component within the library is hidden from the application.

@image{components}

@node Supported Platforms
@section Supported Platforms

Libidn has at some point in time been tested on the following
platforms.

@enumerate

@item Debian GNU/Linux 3.0 (Woody)
@cindex Debian

GCC 2.95.4 and GNU Make. This is the main development platform.
@code{alphaev67-unknown-linux-gnu}, @code{alphaev6-unknown-linux-gnu},
@code{arm-unknown-linux-gnu}, @code{armv4l-unknown-linux-gnu},
@code{hppa-unknown-linux-gnu}, @code{hppa64-unknown-linux-gnu},
@code{i686-pc-linux-gnu}, @code{ia64-unknown-linux-gnu},
@code{m68k-unknown-linux-gnu}, @code{mips-unknown-linux-gnu},
@code{mipsel-unknown-linux-gnu}, @code{powerpc-unknown-linux-gnu},
@code{s390-ibm-linux-gnu}, @code{sparc-unknown-linux-gnu},
@code{sparc64-unknown-linux-gnu}.

@item Debian GNU/Linux 2.1
@cindex Debian

GCC 2.95.1 and GNU Make. @code{armv4l-unknown-linux-gnu}.

@item Tru64 UNIX
@cindex Tru64

Tru64 UNIX C compiler and Tru64 Make. @code{alphaev67-dec-osf5.1},
@code{alphaev68-dec-osf5.1}.

@item SuSE Linux 7.1
@cindex SuSE

GCC 2.96 and GNU Make. @code{alphaev6-unknown-linux-gnu},
@code{alphaev67-unknown-linux-gnu}.

@item SuSE Linux 7.2a
@cindex SuSE Linux

GCC 3.0 and GNU Make. @code{ia64-unknown-linux-gnu}.

@item SuSE Linux
@cindex SuSE Linux

GCC 3.2.2 and GNU Make.  @code{x86_64-unknown-linux-gnu} (AMD64
Opteron ``Melody'').

@item SuSE Enterprise Server 9 on IBM OpenPower 720 
@cindex SuSE Linux
@cindex OpenPower 720

GCC 3.3.3 and GNU Make.  @code{powerpc64-unknown-linux-gnu}.

@item RedHat Linux 7.2
@cindex RedHat

GCC 2.96 and GNU Make. @code{alphaev6-unknown-linux-gnu},
@code{alphaev67-unknown-linux-gnu}, @code{ia64-unknown-linux-gnu}.

@item RedHat Linux 8.0
@cindex RedHat

GCC 3.2 and GNU Make. @code{i686-pc-linux-gnu}.

@item RedHat Advanced Server 2.1
@cindex RedHat Advanced Server

GCC 2.96 and GNU Make. @code{i686-pc-linux-gnu}.

@item Slackware Linux 8.0.01
@cindex RedHat

GCC 2.95.3 and GNU Make. @code{i686-pc-linux-gnu}.

@item Mandrake Linux 9.0
@cindex Mandrake

GCC 3.2 and GNU Make. @code{i686-pc-linux-gnu}.

@item IRIX 6.5
@cindex IRIX

MIPS C compiler, IRIX Make. @code{mips-sgi-irix6.5}.

@item AIX 4.3.2
@cindex AIX

IBM C for AIX compiler, AIX Make.  @code{rs6000-ibm-aix4.3.2.0}.

@item Microsoft Windows 2000 (Cygwin)
@cindex Windows

GCC 3.2, GNU make. @code{i686-pc-cygwin}.

@item HP-UX 11
@cindex HP-UX

HP-UX C compiler and HP Make. @code{ia64-hp-hpux11.22},
@code{hppa2.0w-hp-hpux11.11}.

@item SUN Solaris 2.7
@cindex Solaris

GCC 3.0.4 and GNU Make. @code{sparc-sun-solaris2.7}.

@item SUN Solaris 2.8
@cindex Solaris

Sun WorkShop Compiler C 6.0 and SUN Make. @code{sparc-sun-solaris2.8}.

@item SUN Solaris 2.9
@cindex Solaris

Sun Forte Developer 7 C compiler and GNU
Make. @code{sparc-sun-solaris2.9}.

@item NetBSD 1.6
@cindex NetBSD

GCC 2.95.3 and GNU Make. @code{alpha-unknown-netbsd1.6},
@code{i386-unknown-netbsdelf1.6}.

@item OpenBSD 3.1 and 3.2
@cindex OpenBSD

GCC 2.95.3 and GNU Make. @code{alpha-unknown-openbsd3.1},
@code{i386-unknown-openbsd3.1}.

@item FreeBSD 4.7 and 4.8
@cindex FreeBSD

GCC 2.95.4 and GNU Make. @code{alpha-unknown-freebsd4.7},
@code{alpha-unknown-freebsd4.8}, @code{i386-unknown-freebsd4.7},
@code{i386-unknown-freebsd4.8}.

@item MacOS X 10.2 Server Edition
@cindex MacOS X

GCC 3.1 and GNU Make. @code{powerpc-apple-darwin6.5}.

@item MacOS X 10.4 ``Tiger'' with Xcode 2.0
@cindex MacOS X

GCC 4.0 and GNU Make. @code{powerpc-apple-darwin8.0}.

@item Cross compiled to uClinux/uClibc on Motorola Coldfire
@cindex Motorola Coldfire
@cindex uClinux
@cindex uClibc

GCC 3.4 and GNU Make @code{m68k-uclinux-elf}.

@item Cross compiled to ARM using Glibc
@cindex ARM

GCC 2.95 and GNU Make @code{arm-linux}.

@end enumerate

If you use Libidn on, or port Libidn to, a new platform please report
it to the author.

@node Getting help
@section Getting help

A mailing list where users of Libidn may help each other exists, and
you can reach it by sending e-mail to @email{help-libidn@@gnu.org}.
Archives of the mailing list discussions, and an interface to manage
subscriptions, is available through the World Wide Web at
@url{http://lists.gnu.org/mailman/listinfo/help-libidn}.

@node Commercial Support
@section Commercial Support

Commercial support is available for users of GNU Libidn.  The kind of
support that can be purchased may include:

@itemize

@item Implement new features.
Such as country code specific profiling to support a restricted subset
of Unicode.

@item Port Libidn to new platforms.
This could include porting Libidn to an embedded platforms that may
need memory or size optimization.

@item Integrating IDN support in your existing project.

@item System design of components related to IDN.

@end itemize

If you are interested, please write to:

@verbatim
Simon Josefsson Datakonsult
Hagagatan 24
113 47 Stockholm
Sweden

E-mail: simon@josefsson.org
@end verbatim

If your company provide support related to GNU Libidn and would like
to be mentioned here, contact the author (@pxref{Bug Reports}).

@node Downloading and Installing
@section Downloading and Installing
@cindex Installation
@cindex Download

The package can be downloaded from several places, including:

@url{http://josefsson.org/libidn/releases/}

The latest version is stored in a file, e.g.,
@samp{gsasl-@value{VERSION}.tar.gz} where the @samp{@value{VERSION}}
value is the highest version number in the directory.

The package is then extracted, configured and built like many other
packages that use Autoconf.  For detailed information on configuring
and building it, refer to the @file{INSTALL} file that is part of the
distribution archive.

Here is an example terminal session that download, configure, build
and install the package.  You will need a few basic tools, such as
@samp{sh}, @samp{make} and @samp{cc}.

@example
$ wget -q http://josefsson.org/libidn/releases/libidn-@value{VERSION}.tar.gz
$ tar xfz libidn-@value{VERSION}.tar.gz
$ cd libidn-@value{VERSION}/
$ ./configure
...
$ make
...
$ make install
...
@end example

After that Libidn should be properly installed and ready for use.

A few @code{configure} options may be relevant, summarized in the
table.

@table @code

@item --enable-java
Build the Java port into a *.JAR file.  @xref{Java API}, for more
information.

@item --disable-tld
Disable the TLD module.  This would typically only be useful if you
are building on a memory restricted platforms.  @xref{TLD Functions},
for more information.

@item --enable-csharp[=IMPL]
Build the C3 port into a *.DLL file.  @xref{C# API}, for more
information.  Here, @code{IMPL} is @code{pnet} or @code{mono},
indicating whether the PNET @command{cscc} compiler or the Mono
@command{mcs} compiler should be used, respectively.

@end table

For the complete list, refer to the output from @code{configure
--help}.

@node Bug Reports
@section Bug Reports
@cindex Reporting Bugs

If you think you have found a bug in Libidn, please investigate it and
report it.

@itemize @bullet

@item Please make sure that the bug is really in Libidn, and
preferably also check that it hasn't already been fixed in the latest
version.

@item You have to send us a test case that makes it possible for us to
reproduce the bug.

@item You also have to explain what is wrong; if you get a crash, or
if the results printed are not good and in that case, in what way.
Make sure that the bug report includes all information you would need
to fix this kind of bug for someone else.

@end itemize

Please make an effort to produce a self-contained report, with
something definite that can be tested or debugged.  Vague queries or
piecemeal messages are difficult to act on and don't help the
development effort.

If your bug report is good, we will do our best to help you to get a
corrected version of the software; if the bug report is poor, we won't
do anything about it (apart from asking you to send better bug
reports).

If you think something in this manual is unclear, or downright
incorrect, or if the language needs to be improved, please also send a
note.

Send your bug report to:

@center @samp{bug-libidn@@gnu.org}

@node Contributing
@section Contributing
@cindex Contributing
@cindex Hacking

If you want to submit a patch for inclusion -- from solve a typo you
discovered, up to adding support for a new feature -- you should
submit it as a bug report (@pxref{Bug Reports}).  There are some
things that you can do to increase the chances for it to be included
in the official package.

Unless your patch is very small (say, under 10 lines) we require that
you assign the copyright of your work to the Free Software Foundation.
This is to protect the freedom of the project.  If you have not
already signed papers, we will send you the necessary information when
you submit your contribution.

For contributions that doesn't consist of actual programming code, the
only guidelines are common sense.  Use it.

For code contributions, a number of style guides will help you:

@itemize @bullet

@item Coding Style.
Follow the GNU Standards document (@pxref{top, GNU Coding Standards,,
standards}).

If you normally code using another coding standard, there is no
problem, but you should use @samp{indent} to reformat the code
(@pxref{top, GNU Indent,, indent}) before submitting your work.

@item Use the unified diff format @samp{diff -u}.

@item Return errors.
No reason whatsoever should abort the execution of the library.  Even
memory allocation errors, e.g. when malloc return NULL, should work
although result in an error code.

@item Design with thread safety in mind.
Don't use global variables and the like.

@item Avoid using the C math library.
It causes problems for embedded implementations, and in most
situations it is very easy to avoid using it.

@item Document your functions.
Use comments before each function headers, that, if properly
formatted, are extracted into GTK-DOC web pages.  Don't forget to
update the Texinfo manual as well.

@item Supply a ChangeLog and NEWS entries, where appropriate.

@end itemize

@c **********************************************************
@c *******************  Preparation  ************************
@c **********************************************************
@node Preparation
@chapter Preparation

To use `Libidn', you have to perform some changes to your sources and
the build system.  The necessary changes are small and explained in
the following sections.  At the end of this chapter, it is described
how the library is initialized, and how the requirements of the
library are verified.

A faster way to find out how to adapt your application for use with
`Libidn' may be to look at the examples at the end of this manual
(@pxref{Examples}).

@menu
* Header::
* Initialization::
* Version Check::
* Building the source::
* Autoconf tests::
@end menu

@node Header
@section Header

The library contains a few independent parts, and each part export the
interfaces (data types and functions) in a header file.  You must
include the appropriate header files in all programs using the
library, either directly or through some other header file, like this:

@example
#include <stringprep.h>
@end example

The header files and the functions they define are categorized as
follows:

@table @asis
@item stringprep.h

The low-level stringprep API entry point.  For IDN applications, this
is usually invoked via IDNA. Some applications, specifically non-IDN
ones, may want to prepare strings directly though, and should include
this header file.

The name space of the stringprep part of Libidn is @code{stringprep*}
for function names, @code{Stringprep*} for data types and
@code{STRINGPREP_*} for other symbols.  In addition,
@code{_stringprep*} is reserved for internal use and should never be
used by applications.

@item punycode.h

The entry point to Punycode encoding and decoding functions.  Normally
punycode is used via the idna.h interface, but some application may
want to perform raw punycode operations.

The name space of the punycode part of Libidn is @code{punycode_*} for
function names, @code{Punycode*} for data types and @code{PUNYCODE_*}
for other symbols.  In addition, @code{_punycode*} is reserved for
internal use and should never be used by applications.
@item idna.h

The entry point to the IDNA functions.  This is the normal entry point
for applications that need IDN functionality.

The name space of the IDNA part of Libidn is @code{idna_*} for
function names, @code{Idna*} for data types and @code{IDNA_*} for
other symbols.  In addition, @code{_idna*} is reserved for internal
use and should never be used by applications.

@item tld.h

The entry point to the TLD functions.  Normal applications are not
expected to need this functionality, but it is present for
applications that are used by TLDs to validate customer input.

The name space of the TLD part of Libidn is @code{tld_*} for function
names, @code{Tld_*} for data types and @code{TLD_*} for other symbols.
In addition, @code{_tld*} is reserved for internal use and should
never be used by applications.

@item pr29.h

The entry point to the PR29 functions.  These functions are used to
detect ``problem sequences'' (@pxref{PR29 Functions}), mostly for use
in security critical applications.

The name space of the PR29 part of Libidn is @code{pr29_*} for
function names, @code{Pr29_*} for data types and @code{PR29_*} for
other symbols.  In addition, @code{_pr29*} is reserved for internal
use and should never be used by applications.

@end table

@node Initialization
@section Initialization

Libidn is stateless and does not need any initialization.

@node Version Check
@section Version Check

It is often desirable to check that the version of `Libidn' used is
indeed one which fits all requirements.  Even with binary
compatibility new features may have been introduced but due to problem
with the dynamic linker an old version is actually used.  So you may
want to check that the version is okay right after program startup.

@include texi/stringprep_check_version.texi

The normal way to use the function is to put something similar to the
following first in your @code{main}:

@example
  if (!stringprep_check_version (STRINGPREP_VERSION))
    @{
      printf ("stringprep_check_version() failed:\n"
              "Header file incompatible with shared library.\n");
      exit(1);
    @}
@end example

@node Building the source
@section Building the source
@cindex Compiling your application

If you want to compile a source file including e.g. the `idna.h' header
file, you must make sure that the compiler can find it in the
directory hierarchy.  This is accomplished by adding the path to the
directory in which the header file is located to the compilers include
file search path (via the @option{-I} option).

However, the path to the include file is determined at the time the
source is configured.  To solve this problem, `Libidn' uses the
external package @command{pkg-config} that knows the path to the
include file and other configuration options.  The options that need
to be added to the compiler invocation at compile time are output by
the @option{--cflags} option to @command{pkg-config libidn}.  The
following example shows how it can be used at the command line:

@example
gcc -c foo.c `pkg-config libidn --cflags`
@end example

Adding the output of @samp{pkg-config libidn --cflags} to the
compilers command line will ensure that the compiler can find e.g. the
idna.h header file.

A similar problem occurs when linking the program with the library.
Again, the compiler has to find the library files.  For this to work,
the path to the library files has to be added to the library search
path (via the @option{-L} option).  For this, the option
@option{--libs} to @command{pkg-config libidn} can be used.  For
convenience, this option also outputs all other options that are
required to link the program with the `libidn' libarary.  The example
shows how to link @file{foo.o} with the `libidn' library to a program
@command{foo}.

@example
gcc -o foo foo.o `pkg-config libidn --libs`
@end example

Of course you can also combine both examples to a single command by
specifying both options to @command{pkg-config}:

@example
gcc -o foo foo.c `pkg-config libidn --cflags --libs`
@end example

@node Autoconf tests
@section Autoconf tests
@cindex Autoconf tests
@cindex Configure tests

If your project uses Autoconf (@pxref{top, GNU Autoconf,, autoconf})
to check for installed libraries, you might find the following snippet
illustrative.  It add a new @file{configure} parameter
@code{--with-libidn}, and check for @file{idna.h} and @samp{-lidn}
(possibly below the directory specified as the optional argument to
@code{--with-libidn}), and define the @acronym{CPP} symbol
@code{LIBIDN} if the library is found.  The default behaviour is to
search for the library and enable the functionality (that is, define
the symbol) when the library is found, but if you wish to make the
default behaviour of your package be that Libidn is not used (even if
it is installed on the system), change @samp{libidn=yes} to
@samp{libidn=no} on the third line.

@example
AC_ARG_WITH(libidn, AC_HELP_STRING([--with-libidn=[DIR]],
                                [Support IDN (needs GNU Libidn)]),
  libidn=$withval, libidn=yes)
if test "$libidn" != "no"; then
  if test "$libidn" != "yes"; then
    LDFLAGS="$@{LDFLAGS@} -L$libidn/lib"
    CPPFLAGS="$@{CPPFLAGS@} -I$libidn/include"
  fi
  AC_CHECK_HEADER(idna.h,
    AC_CHECK_LIB(idn, stringprep_check_version,
      [libidn=yes LIBS="$@{LIBS@} -lidn"], libidn=no),
    libidn=no)
fi
if test "$libidn" != "no" ; then
  AC_DEFINE(LIBIDN, 1, [Define to 1 if you want IDN support.])
else
  AC_MSG_WARN([Libidn not found])
fi
AC_MSG_CHECKING([if Libidn should be used])
AC_MSG_RESULT($libidn)
@end example

If you require that your users have installed @code{pkg-config} (which
I cannot recommend generally), the above can be done more easily as
follows.

@example
AC_ARG_WITH(libidn, AC_HELP_STRING([--with-libidn=[DIR]],
                                [Support IDN (needs GNU Libidn)]),
  libidn=$withval, libidn=yes)
if test "$libidn" != "no" ; then
  PKG_CHECK_MODULES(LIBIDN, libidn >= 0.0.0, [libidn=yes], [libidn=no])
  if test "$libidn" != "yes" ; then
    libidn=no
    AC_MSG_WARN([Libidn not found])
  else
    libidn=yes
    AC_DEFINE(LIBIDN, 1, [Define to 1 if you want Libidn.])
  fi
fi
AC_MSG_CHECKING([if Libidn should be used])
AC_MSG_RESULT($libidn)
@end example

@c **********************************************************
@c ********************  Utility Functions ******************
@c **********************************************************
@node Utility Functions
@chapter Utility Functions
@cindex Utility Functions

The rest of this library makes extensive use of Unicode characters.
In order to interface this library with the outside world, your
application may need to make various Unicode transformations.

@section Header file @code{stringprep.h}

To use the functions explained in this chapter, you need to include
the file @file{stringprep.h} using:

@example
#include <stringprep.h>
@end example

@section Unicode Encoding Transformation

@include texi/stringprep_unichar_to_utf8.texi
@include texi/stringprep_utf8_to_unichar.texi
@include texi/stringprep_ucs4_to_utf8.texi
@include texi/stringprep_utf8_to_ucs4.texi

@section Unicode Normalization

@include texi/stringprep_ucs4_nfkc_normalize.texi
@include texi/stringprep_utf8_nfkc_normalize.texi

@section Character Set Conversion

@include texi/stringprep_locale_charset.texi
@include texi/stringprep_convert.texi
@include texi/stringprep_locale_to_utf8.texi
@include texi/stringprep_utf8_to_locale.texi

@c **********************************************************
@c ******************  Stringprep Functions *****************
@c **********************************************************
@node Stringprep Functions
@chapter Stringprep Functions
@cindex Stringprep Functions

Stringprep describes a framework for preparing Unicode text strings in
order to increase the likelihood that string input and string
comparison work in ways that make sense for typical users throughout
the world. The stringprep protocol is useful for protocol identifier
values, company and personal names, internationalized domain names,
and other text strings.

@section Header file @code{stringprep.h}

To use the functions explained in this chapter, you need to include
the file @file{stringprep.h} using:

@example
#include <stringprep.h>
@end example

@section Defining A Stringprep Profile

Further types and structures are defined for applications that want to
specify their own stringprep profile.  As these are fairly obscure,
and by necessity tied to the implementation, we do not document them
here.  Look into the @file{stringprep.h} header file, and the
@file{profiles.c} source code for the details.

@section Control Flags

@deftypevr {Stringprep flags} {Stringprep_profile_flags} {STRINGPREP_NO_NFKC}
Disable the NFKC normalization, as well as selecting the non-NFKC case
folding tables.  Usually the profile specifies BIDI and NFKC settings,
and applications should not override it unless in special situations.
@end deftypevr

@deftypevr {Stringprep flags} {Stringprep_profile_flags} {STRINGPREP_NO_BIDI}
Disable the BIDI step.  Usually the profile specifies BIDI and NFKC
settings, and applications should not override it unless in special
situations.
@end deftypevr

@deftypevr {Stringprep flags} {Stringprep_profile_flags} {STRINGPREP_NO_UNASSIGNED}
Make the library return with an error if string contains unassigned
characters according to profile.
@end deftypevr

@section Core Functions

@include texi/stringprep_4i.texi
@include texi/stringprep_4zi.texi
@include texi/stringprep.texi
@include texi/stringprep_profile.texi

@section Error Handling

@include texi/stringprep_strerror.texi

@section Stringprep Profile Macros

@deftypefun {int} stringprep_nameprep_no_unassigned (char * @var{in}, int @var{maxlen})