Basic of Message Queue| RabbitMQ | Python | Kafka

What is Message Queue ?

Queue

Benefits of Message Queue

  1. Improve Web application page reload time
  2. Asynchronous Messaging
  3. Break Larger task into smaller task internally
  4. Improve Response Timing
  5. Message queues are asynchronous and reliable etc..

Different Types of MQ

  1. RabbitMQ is the message queue currently being used by OpenPaaS, so no migration would be necessary. It offers a good and mature community. Some problems regarding clustering have been reported, including message loss and manual reconciliation upon partition. The bad point of this solution is that it doesn’t fit the advanced management features needed by James. We selected it for further investigation, and the quality of its documentation have been a decisive factor
  2. Kafka is a cutting edge streaming platform. It fulfill the requested features. As it exposes a distributed log, some features of a mail queue are easier to implement. Its community is strong and mature, and it is thought to clustering as a primary concern. Replay is a core concept. However, it’s architecture is complex and involve a ZooKeeper quorum. We selected it as well.
  3. RocketMQ is a promising, newly born Apache project. However, despite good performances, and an impressive feature set, the community is not very mature, mostly centred around the Alibaba company. The project is still under development. So we considered despite all of its advantages choosing it would not be a wise choice.
  4. Artemis is the HornetQ, donated to the Apache foundation, and adopted by the ActiveMQ project. A rock-solid message queue, but sadly clustering it is hard. Some old school technologies (including XML!) are involved. Thus we decided to not investigate further.
  5. NSO is decentralised messaging system. The messaging patterns we want are supported, but only a hack allows to gain *durability*. Clustering is a primary citizen concept, but at the cost of features. AMQP is not supported, replay neither. We decided it was not worth paying the cost of a migration to it.

Some Points to Need to remember

  • Priority: You might want to give a higher priority to your organisation email, compared to spam
  • Delays: Maybe you don’t want to send too much mails at once. Maybe you need to wait a bit before re-sending an email to a remote mail server in case of errors
  • Management: Mail server administrators expects to inspect the content of the mail queue, remove the elements they want, amongst other…….

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store