<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Using Perspective Broker</title>
<style src="stylesheet-unprocessed.css"></style>
</head>
<body>
<h1>Using Perspective Broker</h1>
<h2>Basic Example</h2>
<p>The first example to look at is a complete (although somewhat trivial)
application. It uses <code>PBServerFactory()</code> on the server side, and
<code>PBClientFactory()</code> on the client side.</p>
<a href="../examples/pbsimple.py" class="py-listing" skipLines="16"
>pbsimple.py</a>
<a href="../examples/pbsimpleclient.py" class="py-listing" skipLines="16"
>pbsimpleclient.py</a>
<p>First we look at the server. This defines an Echoer class (derived from
<code class="API" base="twisted.spread">pb.Root</code>), with a method called
<code>remote_echo()</code>.
<code class="API" base="twisted.spread">pb.Root</code> objects (because of
their inheritance of
<code class="API" base="twisted.spread">pb.Referenceable</code>, described
later) can define methods with names of the form <code>remote_*</code>; a
client which obtains a remote reference to that
<code class="API" base="twisted.spread">pb.Root</code> object will be able to
invoke those methods.</p>
<p>The <code class="API" base="twisted.spread">pb.Root</code>-ish object is
given to a
<code class="API" base="twisted.spread">pb.PBServerFactory</code><code>()</code>.
This is a <code class="API" base="twisted.internet.protocol">Factory</code>
object like any other: the
<code class="API" base="twisted.internet.protocol">Protocol</code> objects it
creates for new connections know how to speak the PB protocol. The object you
give to <code>pb.PBServerFactory()</code> becomes the <q>root object</q>, which
simply makes it available for the client to retrieve. The client may only
request references to the objects you want to provide it: this helps you
implement your security model. Because it is so common to export just a single
object (and because a <code>remote_*</code> method on that one can return a
reference to any other object you might want to give out), the simplest example
is one where the <code class="API" base="twisted.spread.pb">PBServerFactory</code>
is given the root object, and the client retrieves it.</p>
<p>The client side uses
<code class="API" base="twisted.spread">pb.PBClientFactory</code> to make a
connection to a given port. This is a two-step process involving opening
a TCP connection to a given host and port and requesting the root object
using <code>.getRootObject()</code>.</p>
<p>Because <code>.getRootObject()</code> has to wait until a network
connection has been made and exchange some data, it may take a while,
so it returns a Deferred, to which the gotObject() callback is
attached. (See the documentation on <a href="defer.xhtml">Deferring
Execution</a> for a complete explanation of <code class="API"
base="twisted.internet.defer">Deferred</code>s). If and when the
connection succeeds and a reference to the remote root object is
obtained, this callback is run. The first argument passed to the
callback is a remote reference to the distant root object. (you can
give other arguments to the callback too, see the other parameters for
<code>.addCallback()</code> and <code>.addCallbacks()</code>).</p>
<p>The callback does:</p>
<pre class="python">
object.callRemote("echo", "hello network")
</pre>
<p>which causes the server's <code>.remote_echo()</code> method to be invoked.
(running <code>.callRemote("boom")</code> would cause
<code>.remote_boom()</code> to be run, etc). Again because of the delay
involved, <code>callRemote()</code> returns a
<code class="API" base="twisted.internet.defer">Deferred</code>. Assuming the
remote method was run without causing an exception (including an attempt to
invoke an unknown method), the callback attached to that
<code class="API" base="twisted.internet.defer">Deferred</code> will be
invoked with any objects that were returned by the remote method call.</p>
<p>In this example, the server's <code>Echoer</code> object has a method
invoked, <em>exactly</em> as if some code on the server side had done:</p>
<pre class="python">
echoer_object.remote_echo("hello network")
</pre>
<p>and from the definition of <code>remote_echo()</code> we see that this just
returns the same string it was given: <q>hello network</q>.</p>
<p>From the client's point of view, the remote call gets another
<code class="API" base="twisted.internet.defer">Deferred</code> object
instead of that string. <code>callRemote()</code> <em>always</em> returns a
<code class="API" base="twisted.internet.defer">Deferred</code>. This is why PB
is described as a system for <q>translucent</q> remote method calls instead of
<q>transparent</q> ones: you cannot pretend that the remote object is really
local.
Trying to do so (as some other RPC mechanisms do, coughCORBAcough) breaks down
when faced with the asynchronous nature of the network. Using Deferreds turns
out to be a very clean way to deal with the whole thing.</p>
<p>The remote reference object (the one given to <code>getRootObject()</code>'s
success callback) is an instance the
<code class="API" base="twisted.spread.pb">RemoteReference</code> class. This
means you can use it to invoke methods on the remote object that it refers to.
Only instances of
<code class="API" base="twisted.spread.pb">RemoteReference</code> are eligible for
<code>.callRemote()</code>. The
<code class="API" base="twisted.spread.pb">RemoteReference</code> object is the
one that lives on the remote side (the client, in this case), not the local
side (where the actual object is defined).</p>
<p>In our example, the local object is that <code>Echoer()</code> instance,
which inherits from <code class="API" base="twisted.spread">pb.Root</code>,
which inherits from
<code class="API" base="twisted.spread">pb.Referenceable</code>. It is that
<code>Referenceable</code> class that makes the object eligible to be available
for remote method calls<span class="footnote">There are a few other classes
that can bestow this ability, but pb.Referenceable is the easiest to
understand; see 'flavors' below for details on the others.</span>. If you have
an object that is Referenceable, then any client that manages to get a
reference to it can invoke any <code>remote_*</code> methods they please.</p>
<div class="note">
<p>The <em>only</em> thing they can do is invoke those
methods. In particular, they cannot access attributes. From a security point
of view, you control what they can do by limiting what the
<code>remote_*</code> methods can do.</p>
<p>Also note: the other classes like
<code class="API" base="twisted.spread.pb">Referenceable</code> allow access to
other methods, in particular <code>perspective_*</code> and <code>view_*</code>
may be accessed. Don't write local-only methods with these names, because then
remote callers will be able to do more than you intended.</p>
<p>Also also note: the other classes like
<code class="API" base="twisted.spread">pb.Copyable</code> <em>do</em> allow
access to attributes, but you control which ones they can see.</p>
</div>
<p>You don't have to be a
<code class="API" base="twisted.spread">pb.Root</code> to be remotely callable,
but you do have to be
<code class="API" base="twisted.spread">pb.Referenceable</code>. (Objects that
inherit from <code class="API" base="twisted.spread">pb.Referenceable</code>
but not from <code class="API" base="twisted.spread">pb.Root</code> can be
remotely called, but only
<code class="API" base="twisted.spread">pb.Root</code>-ish objects can be given
to the <code class="API" base="twisted.spread.pb">PBServerFactory</code>.)</p>
<h2>Complete Example</h2>
<a href="listings/TwistedQuotes/pbquote.py"
class="py-listing">QuoteReader Root object</a>
<p>For examples of these, we're returning to the TwistedQuotes project
discussed in <a href="plugin.xhtml">Writing Plugins</a>. To use the
examples in this HOWTO, we need to make a TML file to refer to our new set
of examples:
</p>
<a href="listings/TwistedQuotes/plugins2.tml" class="py-listing">Twisted Quotes
Plug-in registration</a>
<p>
The root object for
TwistedQuotes is pretty small. The only thing it needs to keep track
of for itself is the quoter object.
</p>
<p>The QuoteReader publishes one method. By subclassing <code class="API"
base="twisted.spread.pb">Root</code>, we are declaring that all
methods with the <code class="python">remote_</code> prefix are
remotely accessible.</p>
<p>In order to get this Root published, so that we can actually connect to
it, we need to re-visit the TAP building plugin, so we can actually get an
Application that has a PBServerFactory listening on a port. (The default
port for PB is 8787.) </p>
<a href="listings/TwistedQuotes/quotetap2.py" class="py-listing">TAP Plugin
with PB Quotes support</a>
<p>In the TAP builder, all we need to do is create our QuoteReader
instance (making sure to pass it our quoter object), give it to a
PBServerFactory, and create a TCPServer so that it can listen on a TCP
port.</p>
<p>Accessing this through a client is fairly easy, as we use the
<code class="API" base="twisted.spread">pb.PBClientFactory.getRootObject</code>
method.</p>
<a href="listings/TwistedQuotes/pbquoteclient.py" class="py-listing">PB Quotes
Client Code</a>
<p><code class="API"
base="twisted.spread">pb.PBClientFactory.getRootObject</code> will
handle all the details of waiting for the creation of a connection.
It returns a <code class="API"
base="twisted.internet.defer">Deferred</code>, which will have its
callback called when the reactor connects to the remote server and
<code class="API" base="twisted.">pb.PBClientFactory</code> gets the
root, and have its <code class="python">errback</code> called when the
object-connection fails for any reason, whether it was host lookup
failure, connection refusal, or some server-side error.
</p>
<p>In this example, the <code class="python">connected</code> callback should
be made when the script is run. Looking at the code, it should be clear that
in the event of a connection success, the client will print out a quote and
exit. If you start up a server, you can see:</p>
<pre class="shell">
% <em>mktap qotd --pb 8787</em>
% <em>twistd -f qotd.tap</em>
% <em>python -c 'import TwistedQuotes.pbquoteclient'</em>
An apple a day keeps the doctor away.
</pre>
<p>The argument to this callback, <code class="python">root</code>, is
a <code class="API" base="twisted.spread.pb">RemoteReference</code>.
It represents a reference to the <code
class="python">QuoteReader</code> object.</p>
<p><code class="API" base="twisted.spread.pb">RemoteReference</code>
objects have one method which is their purpose for being: <code
class="python">callRemote</code>. This method allows you to call a
remote method on the object being referred to by the Reference. <code
class="API"
base="twisted.spread.pb">RemoteReference.callRemote</code>, like <code
class="API"
base="twisted.spread">pb.PBClientFactory.getRootObject</code>, returns
a <code class="API" base="twisted.internet.defer">Deferred</code>.
When a response to the method-call being sent arrives, the <code
class="API" base="twisted.internet.defer">Deferred</code>'s <code
class="python">callback</code> or <code class="python">errback</code>
will be made, depending on whether an error occurred in processing the
method call.</p>
<p>This introduction to PB does not showcase all of the features that it
provides, but hopefully it gives you a good idea of where to get started
setting up your own application. Here are some of the other building blocks
you can use.</p>
<h2>Passing more references</h2>
<p>Here is an example of using
<code class="API" base="twisted.spread">pb.Referenceable</code> in a second
class. The second <code class="API" base="twisted.spread">Referenceable</code>
object can have remote methods invoked too, just like the first. In this
example, the initial root object has a method that returns a reference to the
second object.</p>
<a href="listings/pb/pb1server.py" class="py-listing">pb1server.py</a>
<a href="listings/pb/pb1client.py" class="py-listing">pb1client.py</a>
<p>The root object has a method called <code>remote_getTwo</code>, which
returns the <code>Two()</code> instance. On the client end, the callback gets a
<code class="API" base="twisted.spread.pb">RemoteReference</code> to that
instance. The client can then invoke two's <code>.remote_three()</code>
method.</p>
<p>You can use this technique to provide access to arbitrary sets of objects.
Just remember that any object that might get passed <q>over the wire</q> must
inherit from <code class="API" base="twisted.spread.pb">Referenceable</code>
(or one of the other flavors). If you try to pass a non-Referenceable object
(say, by returning one from a <code>remote_*</code> method), you'll get an
<code class="API" base="twisted.spread.jelly">InsecureJelly</code>
exception<span class="footnote">This can be overridden, by subclassing one of
the Serializable flavors and defining custom serialization code for your
class. See <a href="pb-copyable.xhtml">Passing Complex Types</a> for
details.</span>.</p>
<h2>References can come back to you</h2>
<p>If your server gives a reference to a client, and then that client gives
the reference back to the server, the server will wind up with the same
object it gave out originally. The serialization layer watches for returning
reference identifiers and turns them into actual objects. You need to stay
aware of where the object lives: if it is on your side, you do actual method
calls. If it is on the other side, you do
<code>.callRemote()</code><span class="footnote">The binary nature of this
local vs. remote scheme works because you cannot give RemoteReferences to a
third party. If you could, then your object A could go to B, B could give it to
C, C might give it back to you, and you would be hard pressed to tell if the
object lived in C's memory space, in B's, or if it was really your own object,
tarnished and sullied after being handed down like a really ugly picture that
your great aunt owned and which nobody wants but which nobody can bear to throw
out. Ok, not really like that, but you get the idea.</span>.</p>
<a href="listings/pb/pb2server.py" class="py-listing">pb2server.py</a>
<a href="listings/pb/pb2client.py" class="py-listing">pb2client.py</a>
<p>The server gives a <code>Two()</code> instance to the client, who then
returns the reference back to the server. The server compares the <q>two</q>
given with the <q>two</q> received and shows that they are the same, and that
both are real objects instead of remote references.</p>
<p>A few other techniques are demonstrated in <code>pb2client.py</code>. One
is that the callbacks are are added with <code>.addCallback</code> instead
of <code>.addCallbacks</code>. As you can tell from the <a
href="defer.xhtml">Deferred</a> documentation, <code>.addCallback</code> is a
simplified form which only adds a success callback. The other is that to
keep track of state from one callback to the next (the remote reference to
the main One() object), we create a simple class, store the reference in an
instance thereof, and point the callbacks at a sequence of bound methods.
This is a convenient way to encapsulate a state machine. Each response kicks
off the next method, and any data that needs to be carried from one state to
the next can simply be saved as an attribute of the object.</p>
<p>Remember that the client can give you back any remote reference you've
given them. Don't base your zillion-dollar stock-trading clearinghouse
server on the idea that you trust the client to give you back the right
reference. The security model inherent in PB means that they can <em>only</em>
give you back a reference that you've given them for the current connection
(not one you've given to someone else instead, nor one you gave them last
time before the TCP session went down, nor one you haven't yet given to the
client), but just like with URLs and HTTP cookies, the particular reference
they give you is entirely under their control.</p>
<h2>References to client-side objects</h2>
<p>Anything that's Referenceable can get passed across the wire, <em>in either
direction</em>. The <q>client</q> can give a reference to the
<q>server</q>, and then the server can use .callRemote() to invoke methods on
the client end. This fuzzes the distinction between <q>client</q> and
<q>server</q>: the only real difference is who initiates the original TCP
connection; after that it's all symmetric.</p>
<a href="listings/pb/pb3server.py" class="py-listing">pb3server.py</a>
<a href="listings/pb/pb3client.py" class="py-listing">pb3client.py</a>
<p>In this example, the client gives a reference to its own object to the
server. The server then invokes a remote method on the client-side
object.</p>
<h2>Raising Remote Exceptions</h2>
<p>Everything so far has covered what happens when things go right. What
about when they go wrong? The Python Way is to raise an exception of some
sort. The Twisted Way is the same.</p>
<p>The only special thing you do is to define your
<code>Exception</code> subclass by deriving it from <code class="API"
base="twisted.spread">pb.Error</code>. When any remotely-invokable
method (like <code>remote_*</code> or <code>perspective_*</code>)
raises a <code>pb.Error</code>-derived exception, a serialized form of
that Exception object will be sent back over the wire<span
class="footnote">To be precise, the Failure will be sent if
<em>any</em> exception is raised, not just pb.Error-derived ones. But
the server will print ugly error messages if you raise ones that
aren't derived from pb.Error.</span>. The other side (which did
<code>callRemote</code>) will have the <q><code>errback</code></q>
callback run with a <code class="API"
base="twisted.python.failure">Failure</code> object that contains a
copy of the exception object. This <code>Failure</code> object can be
queried to retrieve the error message and a stack traceback.</p>
<p><code class="API" base="twisted.python.failure">Failure</code> is a
special class, defined in <code>twisted/python/failure.py</code>, created to
make it easier to handle asynchronous exceptions. Just as exception handlers
can be nested, <code>errback</code> functions can be chained. If one errback
can't handle the particular type of failure, it can be <q>passed along</q> to a
errback handler further down the chain.</p>
<p>For simple purposes, think of the <code>Failure</code> as just a container
for remotely-thrown <code>Exception</code> objects. To extract the string that
was put into the exception, use its <code>.getErrorMessage()</code> method.
To get the type of the exception (as a string), look at its
<code>.type</code> attribute. The stack traceback is available too. The
intent is to let the errback function get just as much information about the
exception as Python's normal <code>try:</code> clauses do, even though the
exception occurred in somebody else's memory space at some unknown time in
the past.</p>
<a href="listings/pb/exc_server.py" class="py-listing">exc_server.py</a>
<a href="listings/pb/exc_client.py" class="py-listing">exc_client.py</a>
<pre class="shell">
% ./exc_client.py
got remote Exception
.__class__ = twisted.spread.pb.CopiedFailure
.getErrorMessage() = fall down go boom
.type = __main__.MyError
Main loop terminated.
</pre>
<p>Oh, and what happens if you raise some other kind of exception? Something
that <em>isn't</em> subclassed from <code>pb.Error</code>? Well, those are
called <q>unexpected exceptions</q>, which make Twisted think that something
has <em>really</em> gone wrong. These will raise an exception on the
<em>server</em> side. This won't break the connection (the exception is
trapped, just like most exceptions that occur in response to network
traffic), but it will print out an unsightly stack trace on the server's
stderr with a message that says <q>Peer Will Receive PB Traceback</q>, just
as if the exception had happened outside a remotely-invokable method. (This
message will go the current log target, if <code class="API"
base="twisted.python">log.startLogging</code> was used to redirect it). The
client will get the same <code>Failure</code> object in either case, but
subclassing your exception from <code>pb.Error</code> is the way to tell
Twisted that you expect this sort of exception, and that it is ok to just
let the client handle it instead of also asking the server to complain. Look
at <code>exc_client.py</code> and change it to invoke <code>broken2()</code>
instead of <code>broken()</code> to see the change in the server's
behavior.</p>
<p>If you don't add an <code>errback</code> function to the <code
class="API" base="twisted.internet.defer">Deferred</code>, then a remote
exception will still send a <code>Failure</code> object back over, but it
will get lodged in the <code>Deferred</code> with nowhere to go. When that
<code>Deferred</code> finally goes out of scope, the side that did
<code>callRemote</code> will emit a message about an <q>Unhandled error in
Deferred</q>, along with an ugly stack trace. It can't raise an exception at
that point (after all, the <code>callRemote</code> that triggered the
problem is long gone), but it will emit a traceback. So be a good programmer
and <em>always add <code>errback</code> handlers</em>, even if they are just
calls to <code class="API" base="twisted.python">log.err</code>.</p>
<h2>Try/Except blocks and <code class="API"
base="twisted.python.failure">Failure.trap</code> </h2>
<p>To implement the equivalent of the Python try/except blocks (which can
trap particular kinds of exceptions and pass others <q>up</q> to
higher-level <code>try/except</code> blocks), you can use the
<code>.trap()</code> method in conjunction with multiple
<code>errback</code> handlers on the <code>Deferred</code>. Re-raising an
exception in an <code>errback</code> handler serves to pass that new
exception to the next handler in the chain. The <code>trap</code> method is
given a list of exceptions to look for, and will re-raise anything that
isn't on the list. Instead of passing unhandled exceptions <q>up</q> to an
enclosing <code>try</code> block, this has the effect of passing the
exception <q>off</q> to later <code>errback</code> handlers on the same
<code>Deferred</code>. The <code>trap</code> calls are used in chained
errbacks to test for each kind of exception in sequence. </p>
<a href="listings/pb/trap_server.py" class="py-listing">trap_server.py</a>
<a href="listings/pb/trap_client.py" class="py-listing">trap_client.py</a>
<pre class="shell">
% ./trap_client.py
callOne: call with safe object
method successful, response: response
callTwo: call with dangerous object
InsecureJelly: you tried to send something unsafe to them
callThree: call that raises remote exception
remote raised a MyException
telling them to shut down
callFour: call on stale reference
stale reference: the client disconnected or crashed
%
</pre>
<p>In this example, <code>callTwo</code> tries to send an instance of a
locally-defined class through <code>callRemote</code>. The default security
model implemented by <code class="API" base="twisted.spread">pb.Jelly</code>
on the remote end will not allow unknown classes to be unserialized (i.e.
taken off the wire as a stream of bytes and turned back into an object: a
living, breathing instance of some class): one reason is that it does not
know which local class ought to be used to create an instance that
corresponds to the remote object<!-- eat breaking space
--><span class="footnote"><p>The naive approach of simply doing <code>import
SomeClass</code> to match a remote caller who claims to have an object of
type <q>SomeClass</q> could have nasty consequences for some modules that do
significant operations in their <code>__init__</code> methods (think
<code>telnetlib.Telnet(host='localhost', port='chargen')</code>, or even
more powerful classes that you have available in your server program).
Allowing a remote entity to create arbitrary classes in your namespace is
nearly equivalent to allowing them to run arbitrary code.</p>
<p>The <code class="API" base="twisted.spread">pb.InsecureJelly</code>
exception arises because the class being sent over the wire has not been
registered with the serialization layer (known as <code class="API"
base="twisted.spread">jelly</code>). The easiest way to make it possible to
copy entire class instances over the wire is to have them inherit from <code
class="API" base="twisted.spread">pb.Copyable</code>, and then to use
<code>setUnjellyableForClass(remoteClass, localClass)</code> on the
receiving side. See <a href="pb-copyable.xhtml">Passing Complex Types</a>
for an example.</p></span>.
The receiving end of the connection gets to decide what to accept and what
to reject. It indicates its disapproval by raising a <code class="API"
base="twisted.spread">pb.InsecureJelly</code> exception. Because it occurs
at the remote end, the exception is returned to the caller asynchronously,
so an <code>errback</code> handler for the associated <code>Deferred</code>
is run. That errback receives a <code>Failure</code> which wraps the
<code>InsecureJelly</code>.</p>
<p>Remember that <code>trap</code> re-raises exceptions that it wasn't asked
to look for. You can only check for one set of exceptions per errback
handler: all others must be checked in a subsequent handler.
<code>check_MyException</code> shows how multiple kinds of exceptions can be
checked in a single errback: give a list of exception types to
<code>trap</code>, and it will return the matching member. In this case, the
kinds of exceptions we are checking for (<code>MyException</code> and
<code>MyOtherException</code>) may be raised by the remote end: they inherit
from <code class="API" base="twisted.spread">pb.Error</code>.</p>
<p>The handler can return <code>None</code> to terminate processing of the
errback chain (to be precise, it switches to the callback that follows the
errback; if there is no callback then processing terminates). It is a good
idea to put an errback that will catch everything (no <code>trap</code>
tests, no possible chance of raising more exceptions, always returns
<code>None</code>) at the end of the chain. Just as with regular <code>try:
except:</code> handlers, you need to think carefully about ways in which
your errback handlers could themselves raise exceptions. The extra
importance in an asynchronous environment is that an exception that falls
off the end of the <code>Deferred</code> will not be signalled until that
<code>Deferred</code> goes out of scope, and at that point may only cause a
log message (which could even be thrown away if <code class="API"
base="twisted.python">log.startLogging</code> is not used to point it at
stdout or a log file). In contrast, a synchronous exception that is not
handled by any other <code>except:</code> block will very visibly terminate
the program immediately with a noisy stack trace.</p>
<p><code>callFour</code> shows another kind of exception that can occur
while using <code>callRemote</code>: <code class="API"
base="twisted.spread">pb.DeadReferenceError</code>. This one occurs when the
remote end has disconnected or crashed, leaving the local side with a stale
reference. This kind of exception happens to be reported right away (XXX: is
this guaranteed? probably not), so must be caught in a traditional
synchronous <code>try: except pb.DeadReferenceError</code> block. </p>
<p>Yet another kind that can occur is a <code class="API"
base="twisted.spread">pb.PBConnectionLost</code> exception. This occurs
(asynchronously) if the connection was lost while you were waiting for a
<code>callRemote</code> call to complete. When the line goes dead, all
pending requests are terminated with this exception. Note that you have no
way of knowing whether the request made it to the other end or not, nor how
far along in processing it they had managed before the connection was
lost. XXX: explain transaction semantics, find a decent reference.</p>
</body> </html>