Distributed Computing with Python by Pierfederici Francesco

Author:Pierfederici, Francesco [Pierfederici, Francesco]
Language: eng
Format: azw3, mobi, epub
Publisher: Packt Publishing
Published: 2016-04-12T04:00:00+00:00

The pip command will install Pyro version 4.x as well as Serpent, the serializer that Pyro uses behind the scenes to encode and decode Python objects.

Rewriting the previous currency exchange rate application using Pyro will be only a tiny bit more complex than with Python-RQ, and it will involve one extra piece of software: the Pyro nameserver. We will, however, not need any broker or result backend as all Pyro objects will communicate directly with each other.

The high-level description of how Pyro works is as follows. Every object that we choose to access remotely is wrapped by the framework in a socket server listening for connections. Once some piece of code calls a method on one such remote object, the method call, together with its parameters, is serialized and sent over the wire to the appropriate object/server. At this point, the remote object performs whatever actions have been requested and sends the results back over the same connection (again, transparently serialized) to the calling code.

Since each remote object can itself call any other remote object, this architecture can be quite decentralized. In addition, once established, communication between objects is peer to peer, which is radically different from the loosely coupled architecture of distributed task queues. Another interesting aspect of Pyro applications is that each remote object can function as both a master and a worker.

Let's rewrite the currency exchange rate application using Pyro so that we can see how the code performs under a different architecture. Starting from the currency example, create a new Python script (pyro/worker.py) with the following code:

import urllib.request import Pyro4 URL = 'http://finance.yahoo.com/d/quotes.csv?s={}=X&f=p' @Pyro4.expose(instance_mode="percall") class Worker(object): def get_rate(self, pair, url_tmplt=URL): with urllib.request.urlopen(url_tmplt.format(pair)) as res: body = res.read() return (pair, float(body.strip())) # Create a Pyro daemon which will run our code. daemon = Pyro4.Daemon() uri = daemon.register(Worker) Pyro4.locateNS().register('MyWorker', uri) # Sit in an infinite loop accepting connections print('Accepting connections') try: daemon.requestLoop() except KeyboardInterrupt: daemon.shutdown() print('All done')

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.