Message Oriented Communication

11 minute read

Updated: February 09, 2020

Layered Protocols
Middleware Protocols
Types of Communications
Remote Procedure Call (RPC)
- RPC operation
- Interaction using Asynchronous RPC
The Message Passing Interface (MPI)
Summary

Goal

HOW the communication takes place in DS - Interprocess communication is at the heart of all ds
It is important to examine the ways that processes on different machines can exchange information
- But def of DS emphasizes on message passing is highlighted
Communication in ds is always based on low level message passing as offered by the underlying network
Some widely used models for communication
- Remote Procedure Call (RPC)
- Remote Method Invocation
- Message Oriented communication

Layered Protocols

Due to the absence of shared memory, all communication in DS is based on sending and receiving low level messages.
There are number of rules that communicating processes must adhere to - protocols and structuring these protocols in the form of layers.
Each layer provides an interface to the layer above it.
This model, OSI model, there are 7 layers that communicate w each other. In theory the application layer on the sending side will communicate with the receiving side, going through all the layers in btwn.
OSI – Open System Interconnection – we don’t use this model; we use simplified version with 5 layers
- In TCP/IP, scrapping out session & physical layers
Importance of !!Layer, interfaces and protocols

Layered Protocols II

Each layer will add a header to the front of the message and passes the resulting message. – including usage and source/destination addresses.
- Use of overhead – since it will use network resources. It’s a price to pay to add a header
- Some will add a trailer too – data link layer does.
End up with a message with 6 different headers. This is how the messages are gonna be converted into bits by sending it to the physical layer which is then transported to link.
When it hits the bottom, physical layer transmits the message by putting git to a medium.
When it gets to the receiving side, each header will strip off the header and examine it. → you obtain original
→v imp that the original msg is passed back!!!

Middleware Protocols

Sits btwn the application and OS (in fact a transport layer).we need some form of layer that will provide us with a single coherent view
It provides common services and communication protocols that can be used by many different applications
However, there will be issues:
- Marshalling of data – if we are sending the data through network, we have to marshall and unmarshall. U’ll need to build a msg and tell them what ur gonna do with this msg
  - Marshalling and unmarshalling itself is an overhead because the data is serialized into 1s and 0s, and when marshalling, it has to compose the msgs
- Naming protocols – in middleware, you are accessing resources all over the place. So you need some sort of protocol to name those processes
- Security – Paramount. A secure communication in DS is essential -need security protocols
- Scaling mechanisms - some sort of service with replication and caching – middleware should deal with this transparently!!very imp in terms of scaling
  - Since it replicates and caches, it takes up a lot of data

Middleware: Adapted layering Scheme

Very easy to take this scheme and map it in a network architecture- TCP IP
We communicate one to one, and the LHS will have to communicate with RHS- it has to go through all this layer
!!! imp role is that Network layer has to ensure that this communication takes plac e!!!

Types of Communications

If we take client server model and we take an email system. how does it acc work in terms of client server? → core is the middleware communication service.
- Client sends a request, server deals with it, server will reply
- What’s interesting is, in email system, email needs to be stored in the server before sending

First need to synchronize – server and client.
Client sends the msg to the mail delivery system
1. Once the msg gets through the server, the message is stored in a storage facility.
Server connects to the mail delivery system and does the rq - deliver the message
Notify the client that its done - another synchronization

Constant communication is always there

Types of Communications II

Asynchronous and synchronous
- Synchronous – if you have client and server, they both need to be running, and they communicate in a way that the other side is listening.
- Asynchronous – client can send a msg and can do sth else. The message is temporarily stored.
Transient and persistent
- Transient – is only stored as long as the sending and receiving application are both!! executing .
  - If the app on the other side is unavailable and when you can’t communicate, it discards the message → does not ensure the message sending
  - All transport layer typically provides this.
- Persistent – message that has been submitted for transmission is stored by the communication middleware as long as it takes to deliver the message
  - So not necessary of sending app to continue application
You can take transient and make it synchronous, or take persistent and make it asynchronous !

Types of Communications III

In this case, client/ server is usually transient synchronous communication where the server and client has to be active at the time of communication
- Client sends a rq, Server waits for the incoming rq
- Client issues rq and blocks until it receives reply
- drawback
  - They have to be active at the time! → They can’t do other work
Msg oriented middleware aims for higher level persistent asynchronous communication!! - best of both worlds
- Processes queued message
- Persistent means that the copy will always be stored!! And won’t be lost → ensures fault tolerance
- Sender does not have to wait to reply.
How you can find it in the real world – whether that is first or second

Remote Procedure Call (RPC)

In the real world, this communication is implemented as RPC
Is at the heart of DS communication - !!! very imp mechanism
Sending and receiving msgs – the process isn’t concealed.
To locally call a procedure that is effectively implemented on a remote machine
- Process on machine A calls a procedure on machine B, the calling process on A is suspended, and execution of the called. Procedure takes place on B.
Problem
- Since the calling and called procedures run on diff machine, they execute in diff address space.
- Parameters and results have to be passed → complicated in diff machines
- Either or both machines can crash and failure can happen

RPC operation

Client is gonna invoke a procedure in client stub.
1. With 2 parameter a and b – 2 diff data types
The client stub builds a message , and calls local OS
Client OS takes the msg and send it to the remote (Server) OS
Server OS place the msg to the server stub
Server stub unpacks parameters , calls server
Server understands what to do, do it and returns result to stub
Server stub builds a message and calls local OS
Server’s OS sends the msg to the client OS
Client OS gives msg to client stub
Client stub unpacks the msg, and return it to the client In a concept of architecture – we have object-based style marshalling. We put RPC in object based – we end up with RMI – Remote Method Invocation, they are really the same

RPC: Parameter Passing

Although its very nice, when it comes to implementation, there are number of problems to deal with
- Need to understand how marshalling and unmarshalling works
- Client and the server machines might have diff data representation – 2 diff architecture, data encoding could be diff. so how do we make them talk?
- Wrapping paramete r – do you start with last or first byte when marshalling and unmarshalling?
- Same encoding – server and client has to agree on same encoding
- Basic data values (float, int) can be represented differently
- Marshalling and unmarshalling complex data value (arrays, unions), how do you do it?
In order for RPC to work, client and server need to correctly understand the msgs, and hopefully, will be able to transform them into machine dependent representation

Data Marshalling Issues

Diff encoding for charterers
- IBM uses EBCDIC - Extended Binary Coded Decimal Interchange Code,
- Ascii – American standard for code information interchange
When it comes to implementing integers and floats, what kinda encoding?
- Little endian – start with least representative
  - 44,33,22,11
- Big endian - start with the most representative
  - Send 11, 22, 33, 44
- Usually big endian is used to transfer bytes
They need to agree on the format!!

Interaction using Asynchronous RPC

Different from the traditional RPC, in asynchronous RPC, client can continue without waiting for an answer from the server!!
Calls request → waits for acceptance, and continue doing other thing → when results are available, server sends a callback to client
But the problem is, reliability is not guaranteed and we dunno if its gonna be processed.
Client and server does not need to synchronize constantly, and that you can do sth else

Message Oriented Communication: Transient Messaging with sockets

So with sockets, you have server listening to it, client connects to server and communication takes place. Once the comm is done, they both close it
Complicated coz you gotta know the port and shit but it served us well

The Message Passing Interface (MPI) : Advanced Transient Messaging

We have 2 models -shared memory and private memory models → they communicate through msg passing
Supports transient communication.
Parallel computers have interconnection networks. But historically, the HPC were shipped with proprietary communication libraries
- Problematic coz the software might not be compatible with diff machines (btwn HPCs)
Sooo MPI was designed for parallel computations
- Assumes communication takes place within a known group of processes
  - Each group and process is assigned an identifier.
- Makes direct use of underlying network

The Message Passing Interface (MPI)

Large number 150 primitives available
- 6 ways of sending messages – MPI_bsend, send, ssend, sendrecv, etc
V imp for HPC community to easily write highly efficient apps

Message Oriented Persistent Communication: Message-Queuing Model

New way of thinking and designing MPI – queuing
Provides support for Asynchronous persistent – and middleware does this by implementing a queue, and neither sender nor receiver has to be active
Queues correspond to buffers – it goes to the queue, and you fetch the messages
- Put – append
- Get – get the msg from the queue
- Poll – check if the msg is in the queue
- Notify – handler – as a sever, if I’m interested in event that hasn’t arrived in the queue, if it arrives notify me!
a) Both sender and receiver is active
b) Sender is active but receiver is not active – msg can’t be delivered
- Append msg in the queue and get it once they are active

c) Send the message and went to do sth else, receiver is active – receiver can read msg that was sent, but not necessary that the sender is executing
d) Both sender and receiver are inactive

They are loosely coupled – no need for them to be synchronized as everything goes through a queue!!

General Architecture of a Message-Queuing System

If we take this queue-level addressing and link it to the network- level addressing, its so similar
We communicate through queue, and client need to know how the destination queue is addressed. So we need to know the name of the queue and the address of where it can be accessed . → queue manager
Each name is associated with contact address (host, pair)
U need lookup that makes the name-to-address be available for queue manger
- Lookup says, I’m a sender, I need to append this msg, give me the name and the address of that queue.
- I want to send a msg to this receiver, where is its queue → it then finds the address of the queue, and the msg will be routed to that queue
Is what DNS implements, in the sense that it passes the address when you give it a name and we have an address lookup database
Can have router of queue managers. The request will be routed to an adequate queue manager, which will then find an appropriate queue using lookup
Main mechanism is a queue where senders and receivers post messages

AMQP: Advanced Message Queuing Protocol

There’s been a standard protocol - AMQP that is extensively used in the real world to ensure the safe transportation of information btwn applications btwn organisations
- In areas such as finance, transportation, smart grid, online gaming
In number of OS, they do have support for AMQP implementation.
There are cloud based offerings, i.e. amazon, you may be able to use cloud services, and they will potentially give you opportunity to use AMQP cloud base offerings
It’s a standard, which can be implemented in different flavours
- Rabbit MQ
- ActiveMQ

Example: OpenStack Compute - Management Stack Control Built on Queues

Example of how AMQP protocol works
Openstack – virtual infrastructure manager extensively used in cloud computing
- Supports the interaction btwn virtual infrastructure manager such as Openstack
- Message queues are there to provide communication and manage data persistence in all the components in Openstack
- Openstack has a lot of components, and they have to interact → through a queue! Using rabbitMQ
  - Had a default queue of mySQL DB, and can attach 1 queue to 1 database, and potentially have up to 1000 machines in the cloud, interacting with each other through this queue
  - 1 queue up to 1000 machine
- The communication through component is through routing , it gets routed to the right queue, and every time component needs sth, it uses an API , and this API is some form of message that displays on the queue
- Then you have server listening to this queue, putting out messages from the queue, and potentially responding to request through the queue
→ example in the real world where the protocol is actively used

Summary

We have discussed the rules that communication processes must adhere to
Looked at a number of widely used models for communication
- RPC
- Sockets
- Message Oriented Middleware
Importance of message queuing systems