<?xml version="1.0"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml" lang="en"><head><title>Twisted Documentation: Using Deferreds</title><link href="../howto/stylesheet.css" type="text/css" rel="stylesheet" /></head><body bgcolor="white"><h1 class="title">Using Deferreds</h1><div class="toc"><ol><li><a href="#auto0">Introduction</a></li><li><a href="#auto1">The Problem that Deferreds Solve</a></li><li><a href="#auto2">The Context</a></li><ul><li><a href="#auto3">Dealing with Blocking Code</a></li><li><a href="#auto4">Don't Call Us, We'll Call You</a></li></ul><li><a href="#auto5">Deferreds</a></li><ul><li><a href="#auto6">Visual Explanation</a></li><li><a href="#auto7">More about callbacks</a></li><li><a href="#auto8">More about errbacks</a></li><li><a href="#auto9">Unhandled Errors</a></li></ul><li><a href="#auto10">Class Overview</a></li><ul><li><a href="#auto11">Basic Callback Functions</a></li><li><a href="#auto12">Chaining Deferreds</a></li><li><a href="#auto13">Automatic Error Conditions</a></li><li><a href="#auto14">A Brief Interlude: Technical Details</a></li><li><a href="#auto15">Advanced Processing Chain Control</a></li></ul><li><a href="#auto16">Handling either synchronous or asynchronous results</a></li><ul><li><a href="#auto17">Handling possible Deferreds in the library code</a></li><li><a href="#auto18">Returning a Deferred from synchronous functions</a></li></ul><li><a href="#auto19">DeferredList</a></li><ul><li><a href="#auto20">Other behaviours</a></li></ul></ol></div><div class="content"><span></span><h2>Introduction<a name="auto0"></a></h2><p> Twisted is a framework that allows programmers to develop <a href="async.html">asynchronous</a> networked programs. <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.Deferred.html" title="twisted.internet.defer.Deferred">twisted.internet.defer.Deferred</a></code> objects are one of the key concepts that you should understand in order to develop asynchronous code that uses the Twisted framework: they are a signal to the calling function that a result is pending. </p><p> This HOWTO first describes the problem that Deferreds solve: that of managing tasks that are waiting for data without blocking. It then illustrates the difference between blocking Python code, and non-blocking code which returns a Deferred; describes Deferreds in more details; describes the details of the class interfaces; and finally describes DeferredList. </p><h2>The Problem that Deferreds Solve<a name="auto1"></a></h2><p> Deferreds are designed to enable Twisted programs to wait for data without hanging until that data arrives. </p><p> Many computing tasks take some time to complete, and there are two reasons why a task might take some time: </p><ol><li>it is computationally intensive (for example factorising large numbers) and requires a certain amount of CPU time to calculate the answer; or</li><li>it is not computationally intensive but has to wait for data to be available to produce a result.</li></ol><p> It is the second class of problem — non-computationally intensive tasks that involve an appreciable delay — that Deferreds are designed to help solve. Functions that wait on hard drive access, database access, and network access all fall into this class, although the time delay varies. </p><p> The basic idea behind Deferreds, and other solutions to this problem, is to keep the CPU as active as possible. If one task is waiting on data, rather than have the CPU (and the program!) idle waiting for that data (a process normally called "blocking"), the program performs other operations in the meantime, and waits for some signal that data is ready to be processed before returning to that process. </p><p> In Twisted, a function signals to the calling function that it is waiting by returning a Deferred. When the data is available, the program activates the callbacks on that Deferred to process the data. </p><h2>The Context<a name="auto2"></a></h2><h3>Dealing with Blocking Code<a name="auto3"></a></h3><p>When coding I/O based programs - networking code, databases, file access - there are many APIs that are blocking, and many methods where the common idiom is to block until a result is gotten.</p><pre class="python"> <span class="py-src-keyword">class</span> <span class="py-src-identifier">Getter</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">def</span> <span class="py-src-identifier">getData</span><span class="py-src-op">(</span><span class="py-src-parameter">self</span><span class="py-src-op">,</span> <span class="py-src-parameter">x</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span> <span class="py-src-comment"># imagine I/O blocking code here </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-string">"blocking"</span><span class="py-src-newline"> </span> <span class="py-src-keyword">import</span> <span class="py-src-variable">time</span><span class="py-src-newline"> </span> <span class="py-src-variable">time</span><span class="py-src-op">.</span><span class="py-src-variable">sleep</span><span class="py-src-op">(</span><span class="py-src-number">4</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-keyword">return</span> <span class="py-src-variable">x</span> <span class="py-src-op">*</span> <span class="py-src-number">3</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-variable">g</span> <span class="py-src-op">=</span> <span class="py-src-variable">Getter</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-keyword">print</span> <span class="py-src-variable">g</span><span class="py-src-op">.</span><span class="py-src-variable">getData</span><span class="py-src-op">(</span><span class="py-src-number">3</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><h3>Don't Call Us, We'll Call You<a name="auto4"></a></h3><p>Twisted cannot support blocking calls in most of its code, since it is single threaded, and event based. The solution for this issue is to refactor the code, so that instead of blocking until data is available, we return immediately, and use a callback to notify the requester once the data eventually arrives.</p><pre class="python"> <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span><span class="py-src-op">.</span><span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">reactor</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-keyword">class</span> <span class="py-src-identifier">Getter</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">def</span> <span class="py-src-identifier">getData</span><span class="py-src-op">(</span><span class="py-src-parameter">self</span><span class="py-src-op">,</span> <span class="py-src-parameter">x</span><span class="py-src-op">,</span> <span class="py-src-parameter">callback</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span> <span class="py-src-comment"># this won't block </span><span class="py-src-indent"> </span><span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">callLater</span><span class="py-src-op">(</span><span class="py-src-number">2</span><span class="py-src-op">,</span> <span class="py-src-variable">callback</span><span class="py-src-op">,</span> <span class="py-src-variable">x</span> <span class="py-src-op">*</span> <span class="py-src-number">3</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">printData</span><span class="py-src-op">(</span><span class="py-src-parameter">d</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-variable">d</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-variable">g</span> <span class="py-src-op">=</span> <span class="py-src-variable">Getter</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">g</span><span class="py-src-op">.</span><span class="py-src-variable">getData</span><span class="py-src-op">(</span><span class="py-src-number">3</span><span class="py-src-op">,</span> <span class="py-src-variable">printData</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-comment"># startup the event loop, exiting after 4 seconds </span><span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">callLater</span><span class="py-src-op">(</span><span class="py-src-number">4</span><span class="py-src-op">,</span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">stop</span><span class="py-src-op">)</span><span class="py-src-op">;</span><span class="py-src-newline"> </span><span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">run</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><p>There are several things missing in this simple example. There is no way to know if the data never comes back; no mechanism for handling errors. The example does not handle multiple callback functions, nor does it give a method to merge arguments before and after execution. Further, there is no way to distinguish between different calls to gotData from different producer objects. <code>Deferred</code> solves these problems, by creating a single, unified way to handle callbacks and errors from deferred execution.</p><h2>Deferreds<a name="auto5"></a></h2><p>A <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.Deferred.html" title="twisted.internet.defer.Deferred">twisted.internet.defer.Deferred</a></code> is a promise that a function will at some point have a result. We can attach callback functions to a Deferred, and once it gets a result these callbacks will be called. In addition Deferreds allow the developer to register a callback for an error, with the default behavior of logging the error. The deferred mechanism standardizes the application programmer's interface with all sorts of blocking or delayed operations.</p><pre class="python"> <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span><span class="py-src-op">.</span><span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">reactor</span><span class="py-src-op">,</span> <span class="py-src-variable">defer</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-keyword">class</span> <span class="py-src-identifier">Getter</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">def</span> <span class="py-src-identifier">getData</span><span class="py-src-op">(</span><span class="py-src-parameter">self</span><span class="py-src-op">,</span> <span class="py-src-parameter">x</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span> <span class="py-src-comment"># this won't block </span><span class="py-src-indent"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">callLater</span><span class="py-src-op">(</span><span class="py-src-number">2</span><span class="py-src-op">,</span> <span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">,</span> <span class="py-src-variable">x</span> <span class="py-src-op">*</span> <span class="py-src-number">3</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-keyword">return</span> <span class="py-src-variable">d</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">printData</span><span class="py-src-op">(</span><span class="py-src-parameter">d</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-variable">d</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-variable">g</span> <span class="py-src-op">=</span> <span class="py-src-variable">Getter</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">g</span><span class="py-src-op">.</span><span class="py-src-variable">getData</span><span class="py-src-op">(</span><span class="py-src-number">3</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printData</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">callLater</span><span class="py-src-op">(</span><span class="py-src-number">4</span><span class="py-src-op">,</span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">stop</span><span class="py-src-op">)</span><span class="py-src-op">;</span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">run</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><p><em>Deferreds do not make the code magically not block.</em> Once you have rewritten your code to not block, Deferreds give you a nice way to build an interface to that code.</p><p>As we said, multiple callbacks can be added to a Deferred. The first callback in the Deferred's callback chain will be called with the result, the second with the result of the first callback, and so on. Why do we need this? Well, consider a Deferred returned by twisted.enterprise.adbapi - the result of a SQL query. A web widget might add a callback that converts this result into HTML, and pass the Deferred onwards, where the callback will be used by twisted to return the result to the HTTP client. The callback chain will be bypassed in case of errors or exceptions.</p><pre class="python"> <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span><span class="py-src-op">.</span><span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">reactor</span><span class="py-src-op">,</span> <span class="py-src-variable">defer</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-keyword">class</span> <span class="py-src-identifier">Getter</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">def</span> <span class="py-src-identifier">gotResults</span><span class="py-src-op">(</span><span class="py-src-parameter">self</span><span class="py-src-op">,</span> <span class="py-src-parameter">x</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-string">"""The Deferred mechanism provides a mechanism to signal error conditions. In this case, odd numbers are bad. """</span> <span class="py-src-newline"> </span> <span class="py-src-keyword">if</span> <span class="py-src-variable">x</span> <span class="py-src-op">%</span> <span class="py-src-number">2</span> <span class="py-src-op">==</span> <span class="py-src-number">0</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-variable">self</span><span class="py-src-op">.</span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">(</span><span class="py-src-variable">x</span><span class="py-src-op">*</span><span class="py-src-number">3</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-dedent"></span><span class="py-src-keyword">else</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-variable">self</span><span class="py-src-op">.</span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">errback</span><span class="py-src-op">(</span><span class="py-src-variable">ValueError</span><span class="py-src-op">(</span><span class="py-src-string">"You used an odd number!"</span><span class="py-src-op">)</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span> <span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">_toHTML</span><span class="py-src-op">(</span><span class="py-src-parameter">self</span><span class="py-src-op">,</span> <span class="py-src-parameter">r</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">return</span> <span class="py-src-string">"Result: %s"</span> <span class="py-src-op">%</span> <span class="py-src-variable">r</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span> <span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">getData</span><span class="py-src-op">(</span><span class="py-src-parameter">self</span><span class="py-src-op">,</span> <span class="py-src-parameter">x</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-string">"""The Deferred mechanism allows for chained callbacks. In this example, the output of gotResults is first passed through _toHTML on its way to printData. """</span> <span class="py-src-newline"> </span> <span class="py-src-variable">self</span><span class="py-src-op">.</span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">callLater</span><span class="py-src-op">(</span><span class="py-src-number">2</span><span class="py-src-op">,</span> <span class="py-src-variable">self</span><span class="py-src-op">.</span><span class="py-src-variable">gotResults</span><span class="py-src-op">,</span> <span class="py-src-variable">x</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-variable">self</span><span class="py-src-op">.</span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">self</span><span class="py-src-op">.</span><span class="py-src-variable">_toHTML</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-keyword">return</span> <span class="py-src-variable">self</span><span class="py-src-op">.</span><span class="py-src-variable">d</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">printData</span><span class="py-src-op">(</span><span class="py-src-parameter">d</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-variable">d</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">printError</span><span class="py-src-op">(</span><span class="py-src-parameter">failure</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">import</span> <span class="py-src-variable">sys</span><span class="py-src-newline"> </span> <span class="py-src-variable">sys</span><span class="py-src-op">.</span><span class="py-src-variable">stderr</span><span class="py-src-op">.</span><span class="py-src-variable">write</span><span class="py-src-op">(</span><span class="py-src-variable">str</span><span class="py-src-op">(</span><span class="py-src-variable">failure</span><span class="py-src-op">)</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-comment"># this will print an error message </span><span class="py-src-dedent"></span><span class="py-src-variable">g</span> <span class="py-src-op">=</span> <span class="py-src-variable">Getter</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">g</span><span class="py-src-op">.</span><span class="py-src-variable">getData</span><span class="py-src-op">(</span><span class="py-src-number">3</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printData</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addErrback</span><span class="py-src-op">(</span><span class="py-src-variable">printError</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-comment"># this will print "Result: 12" </span><span class="py-src-variable">g</span> <span class="py-src-op">=</span> <span class="py-src-variable">Getter</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">g</span><span class="py-src-op">.</span><span class="py-src-variable">getData</span><span class="py-src-op">(</span><span class="py-src-number">4</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printData</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addErrback</span><span class="py-src-op">(</span><span class="py-src-variable">printError</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">callLater</span><span class="py-src-op">(</span><span class="py-src-number">4</span><span class="py-src-op">,</span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">stop</span><span class="py-src-op">)</span><span class="py-src-op">;</span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">run</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><h3>Visual Explanation<a name="auto6"></a></h3><div hlint="off" align="center"><img src="../img/deferred-attach.png" /></div><ol><li>Requesting method (data sink) requests data, gets Deferred object.</li><li>Requesting method attaches callbacks to Deferred object.</li></ol><img src="../img/deferred-process.png" hlint="off" align="left" /><ol><li>When the result is ready, give it to the Deferred object. <code>.callback(result)</code> if the operation succeeded, <code>.errback(failure)</code> if it failed. Note that <code>failure</code> is typically an instance of a <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.python.failure.Failure.html" title="twisted.python.failure.Failure">twisted.python.failure.Failure</a></code> instance.</li><li>Deferred object triggers previously-added (call/err)back with the <code>result</code> or <code>failure</code>. Execution then follows the following rules, going down the chain of callbacks to be processed. <ul><li>Result of the callback is always passed as the first argument to the next callback, creating a chain of processors.</li><li>If a callback raises an exception, switch to errback.</li><li>An unhandled failure gets passed down the line of errbacks, this creating an asynchronous analog to a series to a series of <code>except:</code> statements.</li><li>If an errback doesn't raise an exception or return a <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.python.failure.Failure.html" title="twisted.python.failure.Failure">twisted.python.failure.Failure</a></code> instance, switch to callback.</li></ul></li></ol><br hlint="off" clear="all" /><h3>More about callbacks<a name="auto7"></a></h3><p>You add multiple callbacks to a Deferred:</p><pre class="python"> <span class="py-src-variable">g</span> <span class="py-src-op">=</span> <span class="py-src-variable">Getter</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">g</span><span class="py-src-op">.</span><span class="py-src-variable">getResult</span><span class="py-src-op">(</span><span class="py-src-number">3</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">processResult</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printResult</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><p>Each callback feeds its return value into the next callback (callbacks will be called in the order you add them). Thus in the previous example, <code class="python">processResult</code>'s return value will be passed to <code class="python">printResult</code>, instead of the value initially passed into the callback. This gives you a flexible way to chain results together, possibly modifying values along the way (for example, you may wish to pre-process database query results).</p><h3>More about errbacks<a name="auto8"></a></h3><p>Deferred's error handling is modeled after Python's exception handling. In the case that no errors occur, all the callbacks run, one after the other, as described above.</p><p>If the errback is called instead of the callback (e.g. because a DB query raised an error), then a <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.python.failure.Failure.html" title="twisted.python.failure.Failure">twisted.python.failure.Failure</a></code> is passed into the first errback (you can add multiple errbacks, just like with callbacks). You can think of your errbacks as being like <code class="python">except</code> blocks of ordinary Python code.</p><p>Unless you explicitly <code class="python">raise</code> an error in except block, the <code class="python">Exception</code> is caught and stops propagating, and normal execution continues. The same thing happens with errbacks: unless you explicitly <code class="python">return</code> a <code class="python">Failure</code> or (re-)raise an exception, the error stops propagating, and normal callbacks continue executing from that point (using the value returned from the errback). If the errback does returns a <code class="python">Failure</code> or raise an exception, then that is passed to the next errback, and so on.</p><p><em>Note:</em> If an errback doesn't return anything, then it effectively returns <code class="python">None</code>, meaning that callbacks will continue to be executed after this errback. This may not be what you expect to happen, so be careful. Make sure your errbacks return a <code class="python">Failure</code> (probably the one that was passed to it), or a meaningful return value for the next callback.</p><p>Also, <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.python.failure.Failure.html" title="twisted.python.failure.Failure">twisted.python.failure.Failure</a></code> instances have a useful method called trap, allowing you to effectively do the equivalent of:</p><pre class="python"> <span class="py-src-keyword">try</span><span class="py-src-op">:</span><span class="py-src-newline"> </span> <span class="py-src-comment"># code that may throw an exception </span><span class="py-src-indent"> </span><span class="py-src-variable">cookSpamAndEggs</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-keyword">except</span> <span class="py-src-op">(</span><span class="py-src-variable">SpamException</span><span class="py-src-op">,</span> <span class="py-src-variable">EggException</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span> <span class="py-src-comment"># Handle SpamExceptions and EggExceptions </span><span class="py-src-indent"> </span><span class="py-src-op">.</span><span class="py-src-op">.</span><span class="py-src-op">.</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-endmarker"></span></pre><p>You do this by:</p><pre class="python"> <span class="py-src-keyword">def</span> <span class="py-src-identifier">errorHandler</span><span class="py-src-op">(</span><span class="py-src-parameter">failure</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-variable">failure</span><span class="py-src-op">.</span><span class="py-src-variable">trap</span><span class="py-src-op">(</span><span class="py-src-variable">SpamException</span><span class="py-src-op">,</span> <span class="py-src-variable">EggException</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-comment"># Handle SpamExceptions and EggExceptions </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">cookSpamAndEggs</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addErrback</span><span class="py-src-op">(</span><span class="py-src-variable">errorHandler</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><p>If none of arguments passed to <code class="python">failure.trap</code> match the error encapsulated in that <code class="python">Failure</code>, then it re-raises the error.</p><p>There's another potential <q>gotcha</q> here. There's a method <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.Deferred.addCallbacks.html" title="twisted.internet.defer.Deferred.addCallbacks">twisted.internet.defer.Deferred.addCallbacks</a></code> which is similar to, but not exactly the same as, <code class="python">addCallback</code> followed by <code class="python">addErrback</code>. In particular, consider these two cases:</p><pre class="python"> <span class="py-src-comment"># Case 1 </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">getDeferredFromSomewhere</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">callback1</span><span class="py-src-op">)</span> <span class="py-src-comment"># A</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addErrback</span><span class="py-src-op">(</span><span class="py-src-variable">errback1</span><span class="py-src-op">)</span> <span class="py-src-comment"># B</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">callback2</span><span class="py-src-op">)</span> <span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addErrback</span><span class="py-src-op">(</span><span class="py-src-variable">errback2</span><span class="py-src-op">)</span> <span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-comment"># Case 2 </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">getDeferredFromSomewhere</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallbacks</span><span class="py-src-op">(</span><span class="py-src-variable">callback1</span><span class="py-src-op">,</span> <span class="py-src-variable">errback1</span><span class="py-src-op">)</span> <span class="py-src-comment"># C</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallbacks</span><span class="py-src-op">(</span><span class="py-src-variable">callback2</span><span class="py-src-op">,</span> <span class="py-src-variable">errback2</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><p>If an error occurs in <code class="python">callback1</code>, then for Case 1 <code class="python">errback1</code> will be called with the failure. For Case 2, <code class="python">errback2</code> will be called. Be careful with your callbacks and errbacks.</p><p>What this means in a practical sense is in Case 1, "A" will handle a success condition from <code>getDeferredFromSomewhere</code>, and "B" will handle any errors that occur <em>from either the upstream source, or that occur in 'A'</em>. In Case 2, "C"'s errback1 <em>will only handle an error condition raised by <code>getDeferredFromSomewhere</code></em>, it will not do any handling of errors raised in callback1.</p><h3>Unhandled Errors<a name="auto9"></a></h3><p>If a Deferred is garbage-collected with an unhandled error (i.e. it would call the next errback if there was one), then Twisted will write the error's traceback to the log file. This means that you can typically get away with not adding errbacks and still get errors logged. Be careful though; if you keep a reference to the Deferred around, preventing it from being garbage-collected, then you may never see the error (and your callbacks will mysteriously seem to have never been called). If unsure, you should explicitly add an errback after your callbacks, even if all you do is:</p><pre class="python"> <span class="py-src-comment"># Make sure errors get logged </span><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span><span class="py-src-op">.</span><span class="py-src-variable">python</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">log</span><span class="py-src-newline"> </span><span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addErrback</span><span class="py-src-op">(</span><span class="py-src-variable">log</span><span class="py-src-op">.</span><span class="py-src-variable">err</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><h2>Class Overview<a name="auto10"></a></h2><p>This is the overview API reference for Deferred. It is not meant to be a substitute for the docstrings in the Deferred class, but can provide guidelines for its use.</p><h3>Basic Callback Functions<a name="auto11"></a></h3><ul><li><code class="py-prototype">addCallbacks(self, callback[, errback, callbackArgs, errbackArgs, errbackKeywords, asDefaults])</code><p>This is the method you will use to interact with Deferred. It adds a pair of callbacks <q>parallel</q> to each other (see diagram above) in the list of callbacks made when the Deferred is called back to. The signature of a method added using addCallbacks should be <code>myMethod(result, *methodArgs, **methodKeywords)</code>. If your method is passed in the callback slot, for example, all arguments in the tuple <code>callbackArgs</code> will be passed as <code>*methodArgs</code> to your method.</p><p>There are various convenience methods that are derivative of addCallbacks. I will not cover them in detail here, but it is important to know about them in order to create concise code.</p><ul><li><code class="py-prototype">addCallback(callback, *callbackArgs, **callbackKeywords)</code><p>Adds your callback at the next point in the processing chain, while adding an errback that will re-raise its first argument, not affecting further processing in the error case.</p><p>Note that, while addCallbacks (plural) requires the arguments to be passed in a tuple, addCallback (singular) takes all its remaining arguments as things to be passed to the callback function. The reason is obvious: addCallbacks (plural) cannot tell whether the arguments are meant for the callback or the errback, so they must be specifically marked by putting them into a tuple. addCallback (singular) knows that everything is destined to go to the callback, so it can use Python's <q>*</q> and <q>**</q> syntax to collect the remaining arguments.</p></li><li><code class="py-prototype">addErrback(errback, *errbackArgs, **errbackKeywords)</code><p>Adds your errback at the next point in the processing chain, while adding a callback that will return its first argument, not affecting further processing in the success case.</p></li><li><code class="py-prototype">addBoth(callbackOrErrback, *callbackOrErrbackArgs, **callbackOrErrbackKeywords)</code><p>This method adds the same callback into both sides of the processing chain at both points. Keep in mind that the type of the first argument is indeterminate if you use this method! Use it for <code>finally:</code> style blocks.</p></li></ul></li><li><code class="py-prototype">callback(result)</code><p>Run success callbacks with the given result. <em>This can only be run once.</em> Later calls to this or <code>errback</code> will raise <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.AlreadyCalledError.html" title="twisted.internet.defer.AlreadyCalledError">twisted.internet.defer.AlreadyCalledError</a></code>. If further callbacks or errbacks are added after this point, addCallbacks will run the callbacks immediately.</p></li><li><code class="py-prototype">errback(failure)</code><p>Run error callbacks with the given failure. <em>This can only be run once.</em> Later calls to this or <code>callback</code> will raise <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.AlreadyCalledError.html" title="twisted.internet.defer.AlreadyCalledError">twisted.internet.defer.AlreadyCalledError</a></code>. If further callbacks or errbacks are added after this point, addCallbacks will run the callbacks immediately.</p></li></ul><h3>Chaining Deferreds<a name="auto12"></a></h3><p>If you need one Deferred to wait on another, all you need to do is return a Deferred from a method added to addCallbacks. Specifically, if you return Deferred B from a method added to Deferred A using A.addCallbacks, Deferred A's processing chain will stop until Deferred B's .callback() method is called; at that point, the next callback in A will be passed the result of the last callback in Deferred B's processing chain at the time.</p><p>If this seems confusing, don't worry about it right now -- when you run into a situation where you need this behavior, you will probably recognize it immediately and realize why this happens. If you want to chain deferreds manually, there is also a convenience method to help you.</p><ul><li><code class="py-prototype">chainDeferred(otherDeferred)</code><p>Add <code>otherDeferred</code> to the end of this Deferred's processing chain. When self.callback is called, the result of my processing chain up to this point will be passed to <code>otherDeferred.callback</code>. Further additions to my callback chain do not affect <code>otherDeferred</code></p><p>This is the same as <code class="python">self.addCallbacks(otherDeferred.callback, otherDeferred.errback)</code></p></li></ul><h3>Automatic Error Conditions<a name="auto13"></a></h3><ul><li><code class="py-prototype">setTimeout(seconds[, timeoutFunc])</code><p>Set a timeout function to be triggered if this Deferred is not called within that time period. By default, this will raise a TimeoutError after <code>seconds</code>.</p></li></ul><h3>A Brief Interlude: Technical Details<a name="auto14"></a></h3><p>Deferreds greatly simplify the process of writing asynchronous code by providing a standard for registering callbacks, but there are some subtle and sometimes confusing rules that you need to follow if you are going to use them. This mostly applies to people who are writing new systems that use Deferreds internally, and not writers of applications that just add callbacks to Deferreds produced and processed by other systems. Nevertheless, it is good to know.</p><p>Deferreds are one-shot. A generalization of the Deferred API to generic event-sources is in progress -- watch this space for updates! -- but Deferred itself is only for events that occur once. You can only call <code>Deferred.callback</code> or <code>Deferred.errback</code> once. The processing chain continues each time you add new callbacks to an already-called-back-to Deferred.</p><p>The important consequence of this is that <strong>sometimes, addCallbacks will call its argument synchronously, and sometimes it will not</strong>. In situations where callbacks modify state, it is highly desirable for the chain of processing to halt until all callbacks are added. For this, it is possible to <code>pause</code> and <code>unpause</code> a Deferred's processing chain while you are adding lots of callbacks.</p><p>Be careful when you use these methods! If you <code>pause</code> a Deferred, it is <em>your</em> responsibility to make sure that you unpause it; code that calls <code>callback</code> or <code>errback</code> should <strong>never</strong> call <code>unpause</code>, as this would negate its usefulness!</p><h3>Advanced Processing Chain Control<a name="auto15"></a></h3><ul><li><code class="py-prototype">pause()</code><p>Cease calling any methods as they are added, and do not respond to <code>callback</code>, until <code>self.unpause()</code> is called.</p></li><li><code class="py-prototype">unpause()</code><p>If <code>callback</code> has been called on this Deferred already, call all the callbacks that have been added to this Deferred since <code>pause</code> was called.</p><p>Whether it was called or not, this will put this Deferred in a state where further calls to <code>addCallbacks</code> or <code>callback</code> will work as normal.</p></li></ul><h2>Handling either synchronous or asynchronous results<a name="auto16"></a></h2><p> In some applications, there are functions that might be either asynchronous or synchronous. For example, a user authentication function might be able to check in memory whether a user is authenticated, allowing the authentication function to return an immediate result, or it may need to wait on network data, in which case it should return a Deferred to be fired when that data arrives. However, a function that wants to check if a user is authenticated will then need to accept both immediate results <em> and</em> Deferreds. </p><p> In this example, the library function <code>authenticateUser</code> uses the application function <code>isValidUser</code> to authenticate a user: </p><pre class="python"> <span class="py-src-keyword">def</span> <span class="py-src-identifier">authenticateUser</span><span class="py-src-op">(</span><span class="py-src-parameter">isValidUser</span><span class="py-src-op">,</span> <span class="py-src-parameter">user</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">if</span> <span class="py-src-variable">isValidUser</span><span class="py-src-op">(</span><span class="py-src-variable">user</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-string">"User is authenticated"</span><span class="py-src-newline"> </span> <span class="py-src-dedent"></span><span class="py-src-keyword">else</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-string">"User is not authenticated"</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-endmarker"></span></pre><p> However, it assumes that <code>isValidUser</code> returns immediately, whereas <code>isValidUser</code> may actually authenticate the user asynchronously and return a Deferred. It is possible to adapt this trivial user authentication code to accept either a synchronous <code>isValidUser</code> or an asynchronous <code>isValidUser</code>, allowing the library to handle either type of function. It is, however, also possible to adapt synchronous functions to return Deferreds. This section describes both alternatives: handling functions that might be synchronous or asynchronous in the library function (<code>authenticateUser</code>) or in the application code. </p><h3>Handling possible Deferreds in the library code<a name="auto17"></a></h3><p> Here is an example of a synchronous user authentication function that might be passed to <code>authenticateUser</code>: </p><pre class="python"> <span class="py-src-keyword">def</span> <span class="py-src-identifier">synchronousIsValidUser</span><span class="py-src-op">(</span><span class="py-src-parameter">d</span><span class="py-src-op">,</span> <span class="py-src-parameter">user</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">return</span> <span class="py-src-variable">user</span> <span class="py-src-keyword">in</span> <span class="py-src-op">[</span><span class="py-src-string">"Alice"</span><span class="py-src-op">,</span> <span class="py-src-string">"Angus"</span><span class="py-src-op">,</span> <span class="py-src-string">"Agnes"</span><span class="py-src-op">]</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-endmarker"></span></pre><p> However, here's an <code>asynchronousIsValidUser</code> function that returns a Deferred: </p><pre class="python"> <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span><span class="py-src-op">.</span><span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">reactor</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-keyword">def</span> <span class="py-src-identifier">asynchronousIsValidUser</span><span class="py-src-op">(</span><span class="py-src-parameter">d</span><span class="py-src-op">,</span> <span class="py-src-parameter">user</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-variable">reactor</span><span class="py-src-op">.</span><span class="py-src-variable">callLater</span><span class="py-src-op">(</span><span class="py-src-number">2</span><span class="py-src-op">,</span> <span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">,</span> <span class="py-src-variable">user</span> <span class="py-src-keyword">in</span> <span class="py-src-op">[</span><span class="py-src-string">"Alice"</span><span class="py-src-op">,</span> <span class="py-src-string">"Angus"</span><span class="py-src-op">,</span> <span class="py-src-string">"Agnes"</span><span class="py-src-op">]</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-keyword">return</span> <span class="py-src-variable">d</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-endmarker"></span></pre><p> Our original implementation of <code>authenticateUser</code> expected <code>isValidUser</code> to be synchronous, but now we need to change it to handle both synchronous and asynchronous implementations of <code>isValidUser</code>. For this, we use <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.maybeDeferred.html" title="twisted.internet.defer.maybeDeferred">maybeDeferred</a></code> to call <code>isValidUser</code>, ensuring that the result of <code>isValidUser</code> is a Deferred, even if <code>isValidUser</code> is a synchronous function: </p><pre class="python"> <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span><span class="py-src-op">.</span><span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">defer</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-keyword">def</span> <span class="py-src-identifier">printResult</span><span class="py-src-op">(</span><span class="py-src-parameter">result</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">if</span> <span class="py-src-variable">result</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-string">"User is authenticated"</span><span class="py-src-newline"> </span> <span class="py-src-dedent"></span><span class="py-src-keyword">else</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-string">"User is not authenticated"</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">authenticateUser</span><span class="py-src-op">(</span><span class="py-src-parameter">isValidUser</span><span class="py-src-op">,</span> <span class="py-src-parameter">user</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">maybeDeferred</span><span class="py-src-op">(</span><span class="py-src-variable">isValidUser</span><span class="py-src-op">,</span> <span class="py-src-variable">user</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printResult</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-endmarker"></span></pre><p> Now <code>isValidUser</code> could be either <code>synchronousIsValidUser</code> or <code>asynchronousIsValidUser</code>. </p><h3>Returning a Deferred from synchronous functions<a name="auto18"></a></h3><p> An alternative is for <code>authenticateUser</code> to require that the implementation of <code>isValidUser</code> return a Deferred: </p><pre class="python"> <span class="py-src-keyword">def</span> <span class="py-src-identifier">printResult</span><span class="py-src-op">(</span><span class="py-src-parameter">result</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">if</span> <span class="py-src-variable">result</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-string">"User is authenticated"</span><span class="py-src-newline"> </span> <span class="py-src-dedent"></span><span class="py-src-keyword">else</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-string">"User is not authenticated"</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-dedent"></span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">authenticateUser</span><span class="py-src-op">(</span><span class="py-src-parameter">isValidUser</span><span class="py-src-op">,</span> <span class="py-src-parameter">user</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-variable">d</span> <span class="py-src-op">=</span> <span class="py-src-variable">isValidUser</span><span class="py-src-op">(</span><span class="py-src-variable">user</span><span class="py-src-op">)</span><span class="py-src-newline"> </span> <span class="py-src-variable">d</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printResult</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-endmarker"></span></pre><p> In this case, the author of <code>synchronousIsValidUser</code> would use <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.succeed.html" title="twisted.internet.defer.succeed">defer.succeed</a></code> to return a fired Deferred which will call the first callback with <code>result</code>, rather than returning the result itself: </p><pre class="python"> <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span><span class="py-src-op">.</span><span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">defer</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-keyword">def</span> <span class="py-src-identifier">immediateIsValidUser</span><span class="py-src-op">(</span><span class="py-src-parameter">d</span><span class="py-src-op">,</span> <span class="py-src-parameter">user</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-variable">result</span> <span class="py-src-op">=</span> <span class="py-src-variable">user</span> <span class="py-src-keyword">in</span> <span class="py-src-op">[</span><span class="py-src-string">"Alice"</span><span class="py-src-op">,</span> <span class="py-src-string">"Angus"</span><span class="py-src-op">,</span> <span class="py-src-string">"Agnes"</span><span class="py-src-op">]</span><span class="py-src-newline"> </span> <span class="py-src-keyword">return</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">succeed</span><span class="py-src-op">(</span><span class="py-src-variable">result</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-endmarker"></span></pre><h2>DeferredList<a name="auto19"></a></h2><p>Sometimes you want to be notified after several different events have all happened, rather than waiting for each one individually. For example, you may want to wait for all the connections in a list to close. <code class="API"><a href="http://twistedmatrix.com/documents/TwistedDocs/TwistedDocs-1.3.0/api/twisted.internet.defer.DeferredList.html" title="twisted.internet.defer.DeferredList">twisted.internet.defer.DeferredList</a></code> is the way to do this.</p><p>To create a DeferredList from multiple Deferreds, you simply pass a list of the Deferreds you want it to wait for:</p><pre class="python"> <span class="py-src-comment"># Creates a DeferredList </span><span class="py-src-variable">dl</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">DeferredList</span><span class="py-src-op">(</span><span class="py-src-op">[</span><span class="py-src-variable">deferred1</span><span class="py-src-op">,</span> <span class="py-src-variable">deferred2</span><span class="py-src-op">,</span> <span class="py-src-variable">deferred3</span><span class="py-src-op">]</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-endmarker"></span></pre><p>You can now treat the DeferredList like an ordinary Deferred; you can call <code>addCallbacks</code> and so on. The DeferredList will call its callback when all the deferreds have completed. The callback will be called with a list of the results of the Deferreds it contains, like so:</p><pre class="python"> <span class="py-src-keyword">def</span> <span class="py-src-identifier">printResult</span><span class="py-src-op">(</span><span class="py-src-parameter">result</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-variable">result</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-variable">deferred1</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred2</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred3</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">dl</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">DeferredList</span><span class="py-src-op">(</span><span class="py-src-op">[</span><span class="py-src-variable">deferred1</span><span class="py-src-op">,</span> <span class="py-src-variable">deferred2</span><span class="py-src-op">,</span> <span class="py-src-variable">deferred3</span><span class="py-src-op">]</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">dl</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printResult</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred1</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">(</span><span class="py-src-string">'one'</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred2</span><span class="py-src-op">.</span><span class="py-src-variable">errback</span><span class="py-src-op">(</span><span class="py-src-string">'bang!'</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred3</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">(</span><span class="py-src-string">'three'</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-comment"># At this point, dl will fire its callback, printing: </span><span class="py-src-comment"># [(1, 'one'), (0, 'bang!'), (1, 'three')] </span><span class="py-src-comment"># (note that defer.SUCCESS == 1, and defer.FAILURE == 0) </span><span class="py-src-endmarker"></span></pre><p>A standard DeferredList will never call errback.</p><div class="note"><strong>Note: </strong><p>If you want to apply callbacks to the individual Deferreds that go into the DeferredList, you should be careful about when those callbacks are added. The act of adding a Deferred to a DeferredList inserts a callback into that Deferred (when that callback is run, it checks to see if the DeferredList has been completed yet). The important thing to remember is that it is <em>this callback</em> which records the value that goes into the result list handed to the DeferredList's callback.</p><!-- TODO: add picture here: three columns of callback chains, with a value being snarfed out of the middle of each and handed off to the DeferredList --><p>Therefore, if you add a callback to the Deferred <em>after</em> adding the Deferred to the DeferredList, the value returned by that callback will not be given to the DeferredList's callback. To avoid confusion, we recommend not adding callbacks to a Deferred once it has been used in a DeferredList.</p></div><pre class="python"> <span class="py-src-keyword">def</span> <span class="py-src-identifier">printResult</span><span class="py-src-op">(</span><span class="py-src-parameter">result</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">print</span> <span class="py-src-variable">result</span><span class="py-src-newline"> </span><span class="py-src-dedent"></span><span class="py-src-keyword">def</span> <span class="py-src-identifier">addTen</span><span class="py-src-op">(</span><span class="py-src-parameter">result</span><span class="py-src-op">)</span><span class="py-src-op">:</span><span class="py-src-newline"> </span><span class="py-src-indent"> </span><span class="py-src-keyword">return</span> <span class="py-src-variable">result</span> <span class="py-src-op">+</span> <span class="py-src-string">" ten"</span><span class="py-src-newline"> </span><span class="py-src-nl"> </span><span class="py-src-comment"># Deferred gets callback before DeferredList is created </span><span class="py-src-dedent"></span><span class="py-src-variable">deferred1</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred2</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred1</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">addTen</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">dl</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">DeferredList</span><span class="py-src-op">(</span><span class="py-src-op">[</span><span class="py-src-variable">deferred1</span><span class="py-src-op">,</span> <span class="py-src-variable">deferred2</span><span class="py-src-op">]</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">dl</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printResult</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred1</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">(</span><span class="py-src-string">"one"</span><span class="py-src-op">)</span> <span class="py-src-comment"># fires addTen, checks DeferredList, stores "one ten"</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred2</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">(</span><span class="py-src-string">"two"</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-comment"># At this point, dl will fire its callback, printing: </span><span class="py-src-comment"># [(1, 'one ten'), (1, 'two')] </span><span class="py-src-nl"> </span><span class="py-src-comment"># Deferred gets callback after DeferredList is created </span><span class="py-src-variable">deferred1</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred2</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">Deferred</span><span class="py-src-op">(</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">dl</span> <span class="py-src-op">=</span> <span class="py-src-variable">defer</span><span class="py-src-op">.</span><span class="py-src-variable">DeferredList</span><span class="py-src-op">(</span><span class="py-src-op">[</span><span class="py-src-variable">deferred1</span><span class="py-src-op">,</span> <span class="py-src-variable">deferred2</span><span class="py-src-op">]</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred1</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">addTen</span><span class="py-src-op">)</span> <span class="py-src-comment"># will run *after* DeferredList gets its value</span><span class="py-src-newline"> </span><span class="py-src-variable">dl</span><span class="py-src-op">.</span><span class="py-src-variable">addCallback</span><span class="py-src-op">(</span><span class="py-src-variable">printResult</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred1</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">(</span><span class="py-src-string">"one"</span><span class="py-src-op">)</span> <span class="py-src-comment"># checks DeferredList, stores "one", fires addTen</span><span class="py-src-newline"> </span><span class="py-src-variable">deferred2</span><span class="py-src-op">.</span><span class="py-src-variable">callback</span><span class="py-src-op">(</span><span class="py-src-string">"two"</span><span class="py-src-op">)</span><span class="py-src-newline"> </span><span class="py-src-comment"># At this point, dl will fire its callback, printing: </span><span class="py-src-comment"># [(1, 'one), (1, 'two')] </span><span class="py-src-endmarker"></span></pre><h3>Other behaviours<a name="auto20"></a></h3><p>DeferredList accepts two keywords arguments that modify its behaviour: <code>fireOnOneCallback</code>, <code>fireOnOneErrback</code> and <code>consumeErrors</code>. If <code>fireOnOneCallback</code> is set, the DeferredList will immediately call its callback as soon as any of its Deferreds call their callback. Similarly, <code>fireOnOneErrback</code> will call errback as soon as any of the Deferreds call their errback. Note that DeferredList is still one-shot, like ordinary Deferreds, so after a callback or errback has been called the DeferredList will do nothing further (it will just silently ignore any other results from its Deferreds).</p><p>The <code>fireOnOneErrback</code> option is particularly useful when you want to wait for all the results if everything succeeds, but also want to know immediately if something fails.</p><p>The <code>consumeErrors</code> argument will stop the DeferredList from propagating any errors along the callback chains of any Deferreds it contains (usually creating a DeferredList has no effect on the results passed along the callbacks and errbacks of their Deferreds). Stopping errors at the DeferredList with this option will prevent <q>Unhandled error in Deferred</q> warnings from the Deferreds it contains without needing to add extra errbacks<a href="#footnote-1" title="Unless of course a later callback starts a fresh error &mdash; but as we've already noted, adding callbacks to a Deferred after its used in a DeferredList is confusing and usually avoided."><super>1</super></a>.</p><h2>Footnotes</h2><ol><li><a name="footnote-1"><span xmlns="http://www.w3.org/1999/xhtml" class="footnote">Unless of course a later callback starts a fresh error — but as we've already noted, adding callbacks to a Deferred after its used in a DeferredList is confusing and usually avoided.</span></a></li></ol></div><p><a href="../howto/index.html">Index</a></p><span class="version">Version: 1.3.0</span></body></html>