What is RabbitMQ and Why we should use it?
We often get this question in our mind when we look to any project and see RabbitMQ in its technical stack. So, this blog will give you a brief introduction to RabbitMQ.
RabbitMQ is a message-broker whose basic functionality is to accept and forward the message. It is an open-source enterprise messaging system modeled on the Advanced Message Queuing Protocol (AMQP) standard.
Let’s first discuss the scenario where RabbitMQ could be useful:
Suppose you own a restaurant. Now imagine the situation where users are placing the order simultaneously and back-end servers are not processing the orders as fast as they should, or some backend error has occurred or hardware is malfunctioned. Since all the request were directly handled by the server, these request will not be processed.
Now let’s see how RabbitMQ can resolve this:
We can place a service in between the two services, i.e Frontend, and Backend. That service is Rabbit. It will consume all messages from Front-end and will only release when the backend is ready to process it.
Message brokers do many things such as:
- Decouple message publisher and consumer: A message queue provides an asynchronous communications protocol. You have the option to send a message from one service to another without having to know if another service is able to handle it immediately or not. Messages can wait until the responsible service is ready. A service publishing a message does not need to know anything about the inner workings of the services that will process that message. This way of handling messages decouple the producer from the consumer.
- Store the messages
- Routing of messages
- Monitoring and management of messages
Let’s understand a few terms first:
Producer: A program that sends messages.
Consumer: A program that receives messages.
Channel: A channel is a virtual connection inside a connection. When you are publishing or consuming messages from a queue — it’s all done over a channel.
Connection: A connection is a TCP connection between your application and the RabbitMQ broker.
Routing Key: Routing keybinding with the queue are ruled that allow the exchange to put messages into the queue.
Exchange: The exchange receives messages from the producer and from the other side it pushes them to the queues. The exchange must know exactly what to do with the messages it receives. Should it be appended to a particular queue? Should it be appended to many queues? Or should it get discarded? The rules for determining that are defined by the exchange type.
AMQP 0–9–1 (Advanced Message Queuing Protocol) is a messaging protocol that enables conforming client applications to communicate with conforming messaging middleware brokers.
The transfer of messages between producer and consumer does not take place directly. Instead, we have an exchange and a queue in between them. Now, the question comes what is an exchange and what is a queue here for?
Basically, an exchange is a very simple thing. On one side, it receives messages from producers and the other side it pushes them to queues. The exchange is responsible for deciding what should happen to the message. Should it be appended to a particular queue? Should it be appended to many queues? Or should it get discarded? All these rules for message routing are defined by the exchange type.
A queue, on the other hand, is a buffer that stores messages. In the example below, we will be creating a basic RabbitMQ producer and consumer in python using Pika client. You can install it using the command
pip install pikaIn producer.py, we will be sending Hi!! and the same message will be received by consumer.py
Here is the code for producer.py
import pika
# Establish a connection with RabbitMQ server
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()# Create a greeting queue to which the message will be delivered. If we send a message to non-existing location, RabbitMQ will just drop the message.
channel.queue_declare(queue='greeting')# Publish message with default exchange, routing key and message body
channel.basic_publish(exchange='',
routing_key='hi',
body='Hi!')
print("message sent!!")# Flush network buffers and close the connection
connection.close()
In the above code, we have used a default exchange which is a pre-declared direct exchange with no name, usually referred by the empty string “”. In this case, your message is delivered to the queue with a name equal to the routing key of the message. Every queue is automatically bound to the default exchange with a routing key which is the same as the queue name.
There are other types of exchanges as well like topic, fanout, headers. Let us discuss the types of exchanges in brief:
Direct: It delivers messages to the queues based on the routing key.
Topic: It delivers messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange.
Fanout: It delivers messages to all of the queues, bounded with that exchange and the routing key is ignored.
Header: A headers exchange is designed for routing on multiple attributes that are expressed as message headers.
Now, its time for creating our consumer. Here is the code for consumer.py
import pika# Establish a connection with RabbitMQ server.
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()# Create a greeting queue. Creating a queue is idempotent.
channel.queue_declare(queue='greeting')# Subscribe to callback to print the response on the screen.
def callback(ch, method, properties, body):
print("Received %r" % body)# Tell RabbitMQ that the above callback should recieve messages from greeting queue
channel.basic_consume(callback,
queue='greeting',
no_ack=True)print('Waiting for messages....')
channel.start_consuming()
To see the code in action, run on consumer.py on one terminal using the command
python consumer.py# => Waiting for messages....
# => Received 'Hi!'
Also, run your producer.py on the other terminal.
python producer.py# => message sent!!
You can see, we were able to send out the first message using RabbitMQ.
I hope, you have liked my blog. If you have any doubt or any suggestions to make please drop a comment. Thanks!
