Giter Club home page Giter Club logo

dis-lab-3's Introduction

Distributed Lab 3: Message Broker in Go

Using the lab sheet

There are two ways to use the lab sheet, you can either:

Ratings

Each question is rated to help you balance your work:

  • ๐Ÿ”ดโšชโšชโšชโšช - Easy, strictly necessary.
  • ๐Ÿ”ด๐Ÿ”ดโšชโšชโšช - Medium, still necessary.
  • ๐Ÿ”ด๐Ÿ”ด๐Ÿ”ดโšชโšช - Hard, necessary if you're aiming for higher marks.
  • ๐Ÿ”ด๐Ÿ”ด๐Ÿ”ด๐Ÿ”ดโšช - Hard, useful for coursework extensions.
  • ๐Ÿ”ด๐Ÿ”ด๐Ÿ”ด๐Ÿ”ด๐Ÿ”ด - Hard, beyond what you need for any part of the coursework.

Introduction

We are going to use a variant of the publish-subscribe model which uses a message broker for distributing work between many contributors.

The typical publish-subscribe model sends messages to all subscribers signed up to a topic. This works for distributing events that all subscribers should see or act on. For the case where we want just one subscriber to see an event, we want something more like a worker pool that accepts new machines joining the pool remotely and uses 'take' semantics to avoid workers duplicating work.

Part 1: The Multiply Factory ๐Ÿ”ด๐Ÿ”ดโšชโšชโšช

Look at the skeleton code you've been given. There are two complete components:

  1. A broker that handles creating channels, publishing stubs.Pair events to those channels, and subscriptions to those channels. When a subscriber sends a stubs.Subscription to a running instance of the broker, it will get callbacks for work whenever it is available.

  2. A miner that generates stubs.Pair events and publishes them to a multiply channel on the broker.

You should be able to launch the broker with go run broker/broker.go and the miner with go run miner/miner.go -- nothing visible should happen except that the broker will print out a notification that the multiply channel has been created.

Your first task is to complete the Factory in factory/factory.go.

This factory should be a worker that:

  • sets up an RPC server that registers a Multiply procedure. This procedure should accept a stubs.Pair and respond with a stubs.JobReport. You may also want to print out the operation, so you can see what the instance is doing while it's running.
  • connects to the broker as an RPC client.
  • sends a stubs.Subscription request to the multiply channel, containing its own ip:port string and the correct string for the broker to use to call its Multiply procedure.

The required mechanism is illustrated in the sequence diagram below.

Part 1

You'll know if this is working correctly because once the factory is subscribed the broker will start sending it work and printing out the results of jobs.

You should be able to:

  1. Stop and restart the factory.

  2. run a second instance of the factory, and have it also process work. Note: you will need to tell the second factory to use a different port for its RPC server.

Part 2: The Multiply-Divide Pipeline

Part 2a: Division ๐Ÿ”ด๐Ÿ”ดโšชโšชโšช

Following the same steps, add a Divide procedure. It should also accept a stubs.Pair as the request and respond with a stubs.JobReport.

Test your new procedure using two miners:

Miner 1
go run miner/miner.go

Miner 2
go run miner/miner.go -topic divide

You'll know the code is working when the broker reports results for division operations. You should be able run multiple instances of your Factory (on different ports), and stop and start each of them.

Part 2b: Creating a Pipeline ๐Ÿ”ด๐Ÿ”ด๐Ÿ”ดโšชโšช

In this part you will link your Multiply and Divide procedures to create a pipeline. For every two Multiply results produced, the factory should ask the broker to Divide them by each other.

This can be achieved by adding a new goroutine in the factory. When a Multiply procedure is completed, the factory should send each result to this goroutine. Once the goroutine has received two values, it should Publish a new Pair to the broker under the divide topic. It should then continue to wait for further results from the Multiply procedure.

The required mechanism is illustrated in the sequence diagram below.

Part 2

Test your pipeline using a single "multiply" miner. Do not use a "divide" miner this time.

go run miner/miner.go

You'll know the code is working when the broker reports results for division operations (the divisions will be the result of the pipeline). Again, you should be able run multiple instances of your Factory (on different ports), and stop and start each of them.

Part 3: Triplets ๐Ÿ”ด๐Ÿ”ด๐Ÿ”ดโšชโšช

The entire pipeline currently operates on stubs.Pairs. Modify it so that it instead works with Triplets of three integers. As well as the factory, you will need to edit the miner and broker code to accomplish this, and make a decision about what Divide means for three integer arguments.

dis-lab-3's People

Contributors

seth9022 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.