Hello,
I'm working at a university and currently, I'm researching Message Queue (MQ) Software, such as Amazon SQS, Anypoint MQ, IBM MQ and Apache Kafka.
I would like to hear about examples of using asynchronous messages in request/response messaging patterns. What SW would you recommend?
I have personally worked on projects which relied on MQ technologies since it helps in the decoupling of services. My work has mainly been in the capital market domain. I can give you a couple of examples of queue usage.
One was typically to have data for stock charts wherein data couldn't be missed and data had to be in order.
We had used MSMQ since we were on the Microsoft platform at that time. But any queuing mechanism like RabbitMQ would have worked fantastic.
Another example is pub/sub usage wherein some control messages have to flow between multiple services. I have used RabbitMQ for such situations.
Though there is a lot of good word around Kafka, I feel it's a bit complex and overkill in some situations. However, there are specific areas where only Kafka is suitable.
In conclusion, for simple queueing or pub/sub, I would recommend RabbitMQ.
Hi - this is a really good page to understand more about the differences between message queuing technologies and Kafka: https://developer.ibm.com/arti...
In short, if you need conversational messaging (request/response) or targeted reliable delivery (exactly-once delivery), and you don't need the data stored for historical purposes then message queuing is perfect.
If you want to expose data for insight and status, and you don't have a specific intended recipient for that data, then something like Kafka (or a software product that offers publish/subscribe capabilities) is a good fit.
In terms of products, IBM MQ is the market leader (both in terms of capabilities and the number of customers using the technology in production) for Message Queuing and it also provides support for fine-grained events through Pub/Sub capabilities. Many of the world's leading banks, healthcare organizations, manufacturing businesses, etc use IBM MQ either on-premise in their own private data centers or on the public cloud.
If you want to try it out, here is a really good set of developer materials to get you started and there is a badge at the end to demonstrate that you can create a production-ready messaging solution. It takes about 2 hours to complete: https://developer.ibm.com/lear...
I would recommend Apache Kafka as the preferred option.
It is an open-source project and has most of the user community members to contribute/enhance. It is relatively mature.