What is our primary use case?
We are a bank whose core banking system is not so advanced. It is still running on an AS/400 system. Credit Card system is are deployed on IBM mainframes. About 70 to 80 percent of the bank's core systems rely on IBM AS/400 and mainframes. The enterprise service bus is used in conjunction with MQ to break synchronous web service /TCP calls into asynchronous MQ calls and expose them a web services-based or API-based service for both internal and external customers.
As part of enterprise architecture principles, we have enforced all connectivity to be service/ interface based by using ESB, MFT or API. Minimize the point to point connectivity.
We are using dedicated IBM power/pure-app servers to run IBM Integration Bus, IBM MQ, and WebSphere Application Server. These are the three components being used for the bank's enterprise service bus.
How has it helped my organization?
People have started using the likes of Kafka, Spark and other new messaging technologies. But when you take the likes of banks into consideration, which mostly are running on mainframes and AS/400, implementing advanced technologies are not an easy job. Getting an MQ-certified guy is not that difficult in the ASEAN market. There are a lot of certified professionals. That's one of the reasons we still use MQ for most of our messaging. We are still looking at open-source deployments but we have not yet implemented anything like that because of the knowledge GAP & dependency on the existing products. We do not have a dedicated team to take on that task yet.
With IBM MQ still in the bank, and a dedicated team which has expertise in it, we really cut down the time-to-market, from a few months to a few weeks. The development framework is already there. If business comes up with requirements, technology team already know what needs to be done. And by building the in-house team, it gives us the facility where we don't need to ask the IBM guys or other vendors in the market to help us every time we have a new requirement.
Another way MQ has improved the way our organization functions is customer notification. Whenever payments are happening, such as incoming payments to the bank, we need to notify the customer. With MQ we can actually do that asynchronously. There are requirements for notifying customers on a real-time base & also for each and every payment sometimes, once a day. These are be enabled with the help of MQ.
In addition, there are fewer failures during the end-to-end payment process. MQ comes in very handy because we don't lose messages in transit (message persistence). It gives us the ability to store and forward messages when required. We heavily rely on MQ for these kinds of requirements.
Also, we have certain applications that want to receive the messages in both production and the disaster-recovery data center at the same time. Without MQ in the picture, it would have been very difficult for us to configure that. MQ Publish subscribe capability is very helpful in that scenario.
MQ has helped to reduce integration costs, mostly by acquiring the enterprise license of MQ. We can actually set up multiple MQ servers in the same environment and each MQ server is dedicated to a particular application. We also use MQ up to a level where the messages are coming from multiple host systems and they go back to a single channel. When written back, the response goes to the exact host that had sent the request (Message Affinity). Without a tool and without a messaging architecture that is as good as MQ's, it would take a lot of time in hard-coding to achieve that. Prior to the team being set up and having these frameworks in place, it took roughly two to three months to deliver any of these integrations. Now it takes three to four weeks. It has helped reduce the effort and man-hours by half.
What is most valuable?
We have been using the normal messaging along with channel authorization.
At certain times — although not in this bank, in my previous experience — we had used the message authorization on top of the queue. That meant that the moment a user tried to write a message it would be authorized before it was written.
In the ESB architecture, MQ is used to decouple synchronous consumer specific web services calls from the host system calls, by implementing state management. Request and response message matching rely on MQ Message ID & Correlation ID (MQMD header properties).
Message load balancing being implemented using MQ or achieving high throughput, while Message affinity ensures response messages are propagated to the very same host system which had generated the request message.
MQ publish subscribe feature being used to for notification where multiple instance of one or more applications can subscribe to the same topic at different time.
Message expiry features ensures the redundant or unwanted messages are expired automatically from the MQ based on the settings. This feature is being used at times to ensure no duplication of payments.
Message persistence feature ensures the message availability even after any planned/unplanned downtime.
MQMFT feature ensures files are delivered asynchronously and complete file transfers are automated.
MQ authorization can help to ensure high level of security in accessing the messages from a MQ server or sending messages via MQ channels between two or more systems.
What needs improvement?
Day-to-day, I don't really see anything much that we are lacking, but I have never really compared MQ with other products to see what it lacks.
I am well aware of the way that IBM sells the suite of products. But I would like to see it integrate with the newer ways of messaging, such as Kafka. They might say that you have IBM Integration Bus to do that stuff, but it would be great if MQ could, out-of-the-box, listen to public Kafka.
One of the other improvements that I would like to see from MQ is for it to be containerized. It may already have that functionality.
For how long have I used the solution?
I have worked with IBM MQ for more than 10 years. I started using it in 2006. Recently, in the last three to four years, I have not had any work in the development area. I have moved on to enterprise architecture, so I don't really get a chance to use the solution every day. I haven't used it in the last three years.
I do read up on the new features when I get a chance to read, but I don't exactly have hands-on now-a-days. I now guide the team when they have issues, on things like how to set it up, how to have particular architectures on MQ. I still consult on it and I'm still familiar with it.
What do I think about the stability of the solution?
It is stable.
We have only had issues with the IBM GPFS File System, but that is a different product. We had an issue when we wanted to have an Active-Active deployment architecture across our production and disaster recovery site. We wanted the same MQ server to fail over to the disaster recovery site and come up from the state in which it went down on the production site. To achieve that we need a synchronous file system that is able to replicate the same data on the other side.
What do I think about the scalability of the solution?
We haven't had any scalability issues with MQ. We are running on a scalable hardware platform with a goal of virtualizing deployment up to multiple cores, and it can add on more and more compute and RAM when required.
For at least the next five years we are sticking with the existing implementation, while we are looking to implement new features, such as containerization.
How are customer service and technical support?
Our in-house support members are all certified. Most of them have 5 to 10 years of hands-on experience. They don't fall back on IBM for day-to-day support/issues. But if there is a Severity-1, they do log a problem ticket for further expert advises from IBM.
We have only had one issue that I can recall from the last four or five years, to do with the file synchronization. To figure it out, it took a few days.
Overall, the support has been good. As IBM doesn't have a lot of resources in Malaysia, they rely on India or Singapore region for support sometimes.
How was the initial setup?
I am MQ version 6, & 7 -certified solutions architect & system administrator. So I find the setup to be very easy. I have been using MQ since very early of my career, and I was also with IBM for quite some time. So for me, it's very straightforward. The installers come with a nice, quick-setup guide, and most of the time, after the training, users can go set up their own MQ.
The amount of time it would take to bring it to production depends on the scope of the services. If I just have to install MQ, it is not more than an hour. But if I have to install MQ for specific servers, I would probably have to take account of the log size, its location, and what the volume is per day or per week, availability aspects, so it would take a bit longer.
Most of the time, when we implement MQ, we implement it along with other products. It depends on the use case. If you just want to query the server to get the information, I would recommend that to be asynchronous because inquiries are huge in volume. If the use case is payments, it could be defined as synchronous mode, and within the flow we could still break it into two or three parts.
From the design point of view, it is still pretty straightforward, depending on what licensing model you want to go for. If you want to use one MQ server with multiple clients it's doable, but if you have more critical systems running, then you should go for multiple MQ servers so that you have a dedicated server for each particular application. Those are the guiding principles that we use internally for projects to follow.
For deployment, we have written our own scripts. We are mostly relying on AIX/ Linux server. Our scripts are pretty much standard to do things like create/ change queue, channels and it's properties, shut down, restart the MQ services. All these are already scripted, so for our support team it takes them a few minutes to run through them, one after the other, and wait for scripts to be executed.
What about the implementation team?
We have a fully dedicated team in-house to support MQ. There are teams that can help. TCS is one of them and IBM itself is one of them. And there are a few local vendors in the market that are quite proficient in it.
We have a support/ maintenance team of four pax and we are running 200 to 300 services on MQ. The solution doesn't exactly provide a user interface to the business user. The solution is more of a technology layer to support applications and we have 15 to 20 applications that are connected via ESB & MQ.
What's my experience with pricing, setup cost, and licensing?
We have a multi-year OIO (open infrastructure offering) with IBM and if there is any additional licensing required it gets deducted from the OIO. We have been using IBM's other services as well.
Which other solutions did I evaluate?
We have been looking at some competitors; for example, TIBCO Messaging and MuleSoft from Salesforce. One difference I have seen is that TIBCO is already a containerized edition. I have yet to discuss with IBM MQ if it is available on container. Another thing TIBCO has is that its messaging framework comes with a package for Kafka support as well as plugins for MQ connectivity. It allows you to connect to MQ or to Kafka for messaging.
We are also going to look at the IBM API-led integration. We have been running IBM products for some years so we may want to compare & see how these gets compared with their counter-parts.
What other advice do I have?
Overall, MQ is good, capability-wise. You still need a messaging platform and MQ is quite a reliable messaging platform. I have not seen hiccups using MQ across multiple environments in the bank. I have been using it since 2006 and I have never experienced any issues with the product itself. The guidelines of the product, the way it is used, the way things are done, are pretty self-explanatory. There are multiple blogs/ online helps available and there is a lot of help available from experts around the world.
Have a look at the features. If they complement the requirements you have, go ahead with it. If you are very technical and want to understand more about the open-source tools and features, that may require a notable learning curve.
The product has been around for a long time. It's probably time to see what MQ is going to add to its features. We have not started using IBM Cloud Pak with Red Hat OpenShift yet. We are also looking at using containerization but probably it may take some time.
Which deployment model are you using for this solution?
On-premises
Disclosure: My company does not have a business relationship with this vendor other than being a customer.
Hi Sunil, great review. On the two things you wished IBM MQ had...
Containerised IBM MQ, this is something it's had for a number of years, from simple developer images on DockerHub https://hub.docker.com/r/ibmco... to fully supported and certified images and Operator support in IBM Cloud Pak for Integration and OpenShift https://www.ibm.com/support/kn...
Connectivity to Kafka, this is also something you're able to do, either using the open source connectors https://github.com/ibm-messagi... / https://github.com/ibm-messagi... or within Cloud Pak for Integration when connecting to IBM's Kafka offering, IBM Event Streams https://ibm.github.io/event-st...