Broadcom Service Virtualization Reviews

I use VSI, VSM, and SOA testing. I hardly use performance testing and automated GUI testing.

The tool is currently mostly focused on SOA-related and performance testing. Its GUI automation testing is not that powerful compared to other tools, such as HPE UFT, and Selenium, in terms of, particularly, object recognition, smart identification, results repository, frameworks driven, integration with test management repositories, and ETL testing.

Today, customers look for one consolidated vendor to manage licenses and best usage under TEM (Test Environment Management) services.

The following areas of the product have room for improvement: automated GUI testing, performance testing of various HTML5 applications and its recording capabilities. There are SAP, Siebel, Oracle Apps, Citrix, Remote Desktop and RTE applications that require a multi-faceted approach for sniffing the requests and preparing the scripts.

Additionally, it does not have a robust test management repository. This impacts connecting the various phases of testing life cycle.

We have stability issues when creating stubs for performance testing. It provides the internal native web server where the VSIs are hosted/deployed, which are not scalable at higher load. They should more flexible and able to use multiple application servers. Sometimes, the VSIs stop responding too.

There is a scalability issue regarding licenses. The Developer Edition should be free and only charge for the hosting licenses. There are charges even for transactions and hits also, which makes it difficult to track usage and optimization.

While basic questions are answered, it becomes difficult with tricky questions such as, is the request payload non-plain text or Java de-serialized objects, and about customizing them. When we approach support, they redirect us to professional services. We need more helpful support here.

We used SOAPUI Pro for when web services and JMS queues had to be mocked. We shifted to CA for other protocols. But customers are looking at the return on the investment they made. Convincing them becomes a challenge.

Initial setup was OK.

Licensing definitely needs more flexible models. Too many complicated parameters also makes it difficult for us to explain them to our customers. Pricing definitely is too high. Not sure if CA is aware of SOAPUI Mock Services – a free alternative. And, for Pro, it is only $120 per year. If a customer has only a Web Services context, CA should explain how to bring up that story.

We have evaluated several products more than once, starting with SOAPUI Pro, IBM RTVS (earlier Green Hat), and Parasoft Virtualize.

Service virtualization is the most valuable feature because we can create end-to-end services without actual hardware, so that saves a lot of cost and money.

We do application performance management or testing, where we have two options: Either we have to build an infrastructure end-to-end or we have to use the service virtualization capability that can mimic the actual endpoints, which makes it much easier for testing.

There is always room for improvement. Microservices are coming, and I don't know how microservices are going to be virtualized.

They still have a long way to go because service virtualization is a new area and there is a lot happening in this world. They have to keep building on their product and services for us to get it in the future. They have to adapt in more areas.

They already know what we are looking for. We asked them to support the MASL. They worked it out and then they provided us support. Other areas they can improve on are the scalability, failover capabilities, so that you don't have to depend upon a single instance of the server.

Stability’s pretty good so far. We have been using it for the last three years, so it's pretty good.

Stability’s one area that needs work; it's good, but the challenge is that scaling at a very high level is slightly problematic at this point in time. It meets our requirements so far, but I think they need to improve it.

Technical support are very proactive; we have very good support.

We did not previously use a different solution. We had a homegrown solution, and that was easy and cheaper, but the problem was maintaining that homegrown solution over a long period of time. It was quite an investment, so buying this product made sense.

When selecting a vendor, we look equally at the process and technology aspects, from end to end; what are the services they can provide.

My team did the initial setup, and it's pretty straightforward.

I don't want to disclose which vendors were on the short list, but there were a couple of vendors, and we found out that those products were not mature enough.

Web service virtualization solution: It’s cost effective over a period of time and provides flexibility in product validation in the SOA space.

Specifically, service virtualization has reduced infrastructure costs by close to 40%. It helps in early problem detection and early fix, leading to early product release.

It would be great to have improved logging for trace requests and responses at the virtualized service level. The existing feature needs improving.

I have used this solution for nearly four years.

I have noticed performance issues such as the virtual server requires a restart after using it for a certain amount of time, to make sure it responds better.

Technical support is 8/10.

I had only used SoapUI before. LISA has better built-in capabilities that reduce the need for scripting/coding. Also, modeling is appealing and easily understandable.

Initial setup for service testing was easy, but setup of virtualization server was bit complex and needed help from an expert.

We built it in house. It was bit complex in the beginning but we gradually understood the setup better and are comfortable with it now.

My personal feeling is that the licensing cost keeps smaller organisations away due to affordability, but bigger organisations can afford it easily. However, an important point is that over time, there is a good ROI, given that tool is integrated well for virtualization of provider systems.

My personal opinion is that the tool is really good as a service virtualization solution rather than a service testing tool.

We were using this product for service virtualization, and SoapUI/REST API testing for the US’s largest retail customer in the home improvement space.

For service virtualization, we had a lot of subsystems in our application; and we had lot of dependencies on the downstream applications. We extensively used service virtualization to reduce these dependencies, which could be due to readiness, or downstream applications having downtime. We also had few third-party subsystems whose technical support was not available during US night hours. This resulted in a complete loss of the offshore effort in India. This was hampering our release dates, which are very critical in the retail business. Missing a release has a directly proportional impact on revenue.

For web services testing (SoapUI/Rest API), we initially had our web services test cases automated using a custom built Groovy/Selenium/SoapUI framework. Though the framework was robust, it was difficult for us to maintain the automation scripts. When our customer bought CA Service Virtualization, we were able to move our 3000+ automation scripts to Service Virtualization scripts within 6 months while supporting new features and producing regular releases. This was commendable.

Training our folks on Service Virtualization was also pretty easy. We hired three Service Virtualization experts from the market, and were able to build a team of 14 Service Virtualization experts within three to four months. CA Service Virtualization is a very robust product and is highly configurable. This helped us to take more user stories, compared to the days when we were using our home grown framework.

To the best of my knowledge (and memory), listed below are the few challenges we were facing at that time. It could be possible that CA would have already addressed a few of these limitations and short falls.

• Simultaneous execution of automation scripts was not possible; or, at least, not readily available. We had to write some wrappers to handle this situation.
• Integration with Jenkins was not available. We had to write small utilities to achieve this.
• Integration with JIRA was not available to automatically log a defect if there were script failures.
• It consumes more memory and makes the system very slow, which was stopping simultaneous execution.

I have used this solution for more than two years.

As mentioned in the room for improvement section.

I don’t remember any issues as such, apart from the ones mentioned in the room for improvement section.

As I mentioned, we were using a home grown Groovy/Selenium/SoapUI framework. Once a customer bought the CA Service Virtualization license, we were forced to use this. However, we found the product to be more robust than our framework.

Initial setup was pretty straightforward in terms of installation and configuration. Later, as we encountered a few challenges, we built a couple of utilities.

It is pretty expensive. On the licensing part, we don’t have much insight because the customer was handling this part.

We had evaluated multiple frameworks because we were working in a multi-vendor scenario. Every vendor brought their frameworks and solutions to the table. I don’t know if the customer evaluated any other commercial products. A few of our team members had already worked on iTKO/Service Virtualization on other projects; and they provided good feedback on the product.

I would recommend this product for clients who are heavy on web services. This product is very easy to learn, implement, and configure.

The DevTest Workstation (JMS steps) and CA Service Virtualization are the most valuable features.

By using the DevTest tool, we have successfully automated the testing in the Dev and QM spaces. We have also carried out virtualization of a few systems and intend to apply it on a larger scale in future.

There is need to improve the DevTest Workstation with UI-based testing features.

I have used this solution for one year.

It was easy to deploy, as proper documentation and additional support was provided by CA.

The product has been stable and patches have been provided on a regular basis.

The product has scaled to our needs.

The customer service was excellent.

The technical support was excellent.

Previously, we were using Java-based, homegrown tools for our automation processes.

Initial setup was complex, as we were new to the tool and to reach the current scenario, a learning curve in terms of the tool was required.

A PoC was carried out with a CA Service Virtualization technical resource and he met our expectations.

Applying automation using the DevTest Workstation for automation of middleware validation, has decreased manual effort by 40%.

We looked at the HPE UFT software.

You should try to do a PoC and then go through different use cases in order to understand where exactly this tool is applicable in your enterprise.

• The description of the components that pop up when the mouse hovers over them

It helped in simulating the environment for testing for a very tedious project, which otherwise would have involved lot of manhours to test.

The REST test step could include other methods. The currently available ones are: PUT, POST, GET and DELETE. It would be great if the DELETE method could have the Content tab enabled so the request payload can be sent.

I have been using this product for 1.5 years.

We have not had stability issues.

We haven’t had much issues with scalability, but when I did once, the support team was very helpful in resolving the issue.

I rate technical support 4/5.

I have used LISA version 7.5, DevTest Solutions version 8.1 and am currently using DevTest 9.1. The tool always had some additional features included in each of its releases and it fixed the bugs of the previous release. That made migrating to the newer version more exciting.

Setup was straightforward. The instructions in the specification document were crisp and clear.

I did not evaluate other options.

It a great tool for simulating the service.

Virtualization is very important for me so as to mitigate my dependency on real back-end system and push the QA testing life cycle earlier.

Especially in QA/Test/Pre prod environment because of rapid changes and non availability, CA LISA is helpful in replacing the back-end with stub responses which can be helpful in pushing the QA life cycle much earlier in the overall development cycle.

It's currently very sensitive to app changes/responses & the setup to capture the responses is a bit cumbersome .

3 years

Only some security issues with SSL/RSA.

Setting up the proxy to capture responses for the model.

We didn't encounter any issues with stability.

Right now we're only using it for pre-production so we haven't yet seen how it scales if we use it for production volumes.

Pretty Good.

Good

Ease of use and reliable solution.

It's a bit complex as far as creating the model and then improvising it. Also maintaining the model is cumbersome.

We implemented via a vendor team - their support was good and they had a high level of technical expertise.

ROI is NOT that great since the product license is expensive.

Pricing and Licensing is expensive.

Over the past 3 years which I've been using it, I've found it to be a good solution. Only drawback is price, setup & maintenance.

The most valuable features are the web service harness and the service virtualization.

We have been able to enact a 'shift left' in our testing. In addition, I built an automated regression test pack for our teams. Thanks to CA Service Virtualization, we have new functionality (the ability to stub services we rely on) and we can test our own APIs in an efficient manner.

The GUI can be hard to navigate. The ITR keeps pulling the screen to the front. The speed can be improved as well.

We have been using this solution for close to two years.

We did not encounter any issues with stability.

We did not encounter any issues with scalability.

Technical support was provided from our work team. It was sufficient, but the response time was slow.

Initially, we used scripts, JCL, and CICS transactions. CA Service Virtualization has simplified and enhanced this process.

The setup was straightforward. The development of the harnesses was more challenging.

Before buying the license, make sure that you try the product and are happy that it can provide the service testing capabilities that you require. Have people in your team who can invest the time to gain skills in CA Service Virtualization.

Our team did evaluate other products, such as SoapUI and Topaz.

Make sure you understand the potential of the solution and that it will take time to make it work.

The ease with which we can create virtual services and view the flow in the model are valuable features of this product.

CA Service Virtualization has enabled us to give the client demos in a very effective manner even when the environment is not available. It has also enabled us to shift-left.

There should be an option to search a transaction that is VSI based through a transaction number. It would really be a great addition.

I have been using this tool for three and half years now.

Stability issues do arise if there is not enough system memory.

We have not encountered any scalability issues.

The technical support is good. I would give it a 8/10 rating.

The initial setup was quite straightforward because the steps to install and configure were clear.

Pricing is a bit on the higher side. Owing to its high cost, proper business justification needs to be provided to the client, if this tool is to be proposed.

We evaluated another solution, SoapUI NG Pro.

I would recommend others to use CA Service Virtualization for all their virtualization needs as it is easy to use and the support provided via the CA Communities is also great.

The most valuable feature of CA Service Virtualization is that overall, it is a system-based product for virtualizing the services. There are many tools that offer RESTful calls. However, with this product, we have many other options such as database service virtualization, mainframe virtualization and other wide range of virtualization options.

There are a lot of benefits of this solution but I will give you the top benefit from my experience. Since I am from the testing team, I have to test a particular service and the service needs to be developed either by a third party or I am dependent on my development team to complete the service development, only then I can start my testing process. However, with CA Service Virtualization, what I can do is I can start my testing from Scrum 1. I can do this by grabbing details from the developer in my team or the third-party service and ask for sample request responses. Using these sample request responses, I can create a virtual service and then no longer am I dependent on any third party or a developer for my testing process.

The only thing is that I create the service, deploy the virtual service and then use it for my testing; it's that simple.Thus it saves a lot of my time and if I am using any third-party services, then it saves me money as well.

This is what I have spoken to the CA people as well, I would like them to include mainframe testing within this tool.

Right now, we can do service virtualization but there is no mainframe testing feature in this tool such as batch processing, etc.; it is the feature that I need.

It is pretty stable. The service is available 24/7.

It is pretty scalable.

They have some cool technical support. What happens is that when we are using the features of CA Service Virtualization, then if we are facing any issues, they have technical support for the same. Based on the complexity of the issue, they can tell us what can be fixed or they will give us a workaround for the same.

I was not involved in the initial setup of this solution.

For me, the number one criteria while selecting a vendor are the features of the product. The next factor involved is how other clients are making use of it because we cannot afford to spend time to find out new features or bugs in the tool. So, these are really important factors and I would like to make sure it fits our needs.

Creating a virtual service is a matter of ten minutes. It depends on the complexity of the service, like the mainframe virtualization such as CICS or database virtualization, where there are some agents involved that take a day or two. It is simple. The only thing is that the developer of the service should have some technical idea involved as to how the service works and finding the right URL. That is important.