Broadcom Service Virtualization Reviews

It provides for automated testing of applications, including the following features:

• Wide technology support (SOAP/REST web services, message queues, web UI using Selenium, database queries);
• Support to build modular test cases using reusable blocks (called sub-processes in DevTest);
• Extensibility using Java;
• Integration with Jenkins continuous integration tool; and

• Detailed test execution reports.

We've already created and switched over 10,000 test cases from manual to automated execution. Our QA team can now focus more on complex manual test cases.

I would like to see a completely web-oriented automated test case design (currently need to use desktop application called DevTest Workstation). I'd also like to see IntelliSense-like functionality for custom scripting, as well as improvements to reports for test cases using datasets to quickly point to/locate results for specific data iteration.

I've used it for one year.

We experienced some minor issues, but they've been fixed in the current release.

We experienced some minor issues, but they've been fixed in the current release.

We experienced some minor issues, but they've been fixed in the current release.

Very good, as support person is usually trying to do even more than expected. There’s also a web site by CA called Communities where product users can help each other.

I’ve used HP QTP/UFT tool for other clients, specifically for automated GUI-based testing of desktop applications (.NET/WPF, Stingray, Java Swing). I can’t directly answer question about choosing CA DevTest as I was not in the tool decision group for current clients, but it’s very suitable for SOAP, REST, JMS, DB testing. CA DevTest would not be suitable for GUI-based testing of desktop applications which our current client does not need.

The initial set-up was not complicated thanks to the detailed documentation available.

We used an in-house team with occasional help from the vendor support team.

I believe this product will meet your expectations if you’re looking for automated testing solution for any of the supported technologies. It also offers other major functionalities such as CAI for transaction recording and automated generation of test cases and virtual services which we’re planning to use.

The two most valuable features for us are:

• Service Virtualization – It provides us with the ability to replace real downstream systems with virtual systems.
• Test Automation – It enables us to trigger events and automate test cases.

• It reduces cost when we can replace expensive downstream systems.
• It improves speed of testing when systems are not always available.

The biggest area of improvement is in the licensing. The licensing costs are very high.

I've used it for over four years.

There were no issues with the deployment.

There were no issues with the stability.

There have been no issues scaling it.

Technical support is excellent. I rate is a 9 out of 10.

We originally used Green Hat, but CA Service Virtualization provides greater flexibility.

The product is relatively simple to install and use. The complexities generally relate to system connectivity restrictions within the organization.

We developed using our existing contracting partners with support from the vendor.

Think “outside the box” when you think about how this product can be used. Don’t just think it can emulate systems because it can do much, much more.

• Codeless testing: There are no test scripts to write and maintain in DevTest. It helps to easily design test scripts without any programming knowledge.
• Complete test coverage for heterogeneous, distributed architectures: in a single test environment, DevTest invokes and verifies functionality at every layer of the application - from dynamic web UIs, to web services, ESB/JMS messaging layers, EJB servers, Java objects, databases, file systems, legacy objects and much more.
• Service Virtualization: Virtualization reduces the dependency of third-party services/system and allows us shift-left testing.

Our application requires the integration of many services across many APIs both internal and external. Due to this increased complexity, it is important that the testing of complex systems also starts early as possible. Leaving the testing until the end of product delivery increases project risk and the cost of fixing any defects found during later development stage. Using the power of CA Service Virtualization, it is now possible to perform integration testing on every application build. For application components still under development, service virtualization allows integration testing to occur even earlier in the development cycle simulating the missing parts of the system. Service virtualization also allows driving of error scenarios from both internal and external systems that may be hard or even impossible to produce, even with the real systems.

They still need to improve the reporting functionality for better customization. Currently the report functionality does not allow us to go beyond 5000 records where the display show all the records also more filters needs to provide to generate/customize test reports.

I've been using DevTest 8.4 and Application Insight. 8.4 for the last five years.

There were no issues with the deployment.

There have been no issues with the stability.

There have been no issues scaling it.

The customer service is excellent.

The technical support is excellent.

For API test previously I used Selenium. To create/maintain a test script in Selenium Java coding is required. With DevTest, Java programming knowledge is not mandatory. We can do most of the things using in built DevTest assertions and filters without any scripting.

Setup/installation is straightforward. The product installation guide helps lot.

We did it in-house. We have our own product expert team for implementation.

The CA community will help us with any difficulties with the implementation. You can find it https://communities.ca.com/places.

The ability for us to be able to record the live transactions, and create a virtual services out of that, as well as be able to actually use the request and response to create new virtual services for the back-ends that we don't have access to. We have a bunch of use cases there from that perspective.

With the radio's hardware costs allow the development team to what they call shift left, and start to develop their code against the other back-end services that we may not have available in time. These are pretty much the benefits there that we see.

I would like to see more integration support with Docker for release automation, a few things on service monitoring that's pretty much provided in the prior version of the product. I would like to see that a little bit more usable in the future.

It's really good. We use it to support performance testing as well. That can scale up to tens of thousands of users concurrently.

It's great. Basically, it's designed so that we have a load balancing virtual service environment, so we can scale it up as much as we need to.

That's pretty much the key thing I like about CA compared to other vendors I work with. Their support is great. We have an account manager that's really helpful. He helps us kind of navigate through a different escalation path as well after we open the ticket. We never felt like we were left hanging when we run into any issues with the product.

I think it's a little more technically involved to get it started, but once you get the hang of it you can pretty much do even internal knowledge transfer to any other team. We rely on CA-provided training to a really small group of our team, and then after that we first do our internal training and knowledge transfer to other teams to scale it up.

We quantify what kind of benefits that we have against the cost that we have to pay on the solution. It turns out to be a really good exercise there. It forces us to do the ROI on the technology product that we buy against the benefits that we reap from the product there.

I would rate it an 8/10. From a capabilities perspective it's there. Like I mentioned, the learning curve can be a little high. Usability in the prior version that we used, version 8.2, usability is not great. That's why I give it an eight. It requires a Thick Client, so we need to install that on the machine. It's not the most intuitive UI ever, so we'd like to see it improved a little bit from that perspective. I know that they're coming out with version 9, which is supposed to improve that quite a lot, so I'll be looking forward to that.

What we start off with is that we had what we called a enter of excellence that comprised of a few very technical individual to try out the platform, to install the software, and be the product expert within the organization. Then what we did is that we pretty much scaled the development of the virtual services to other teams and that have more of a knowledge of how the service is supposed to behave, so that the COE doesn't need to be in all of these type of sessions and what not. We actually used the start out small and then use that team to empower other teams to actually be able to develop their virtual services that way.

One of the key benefits is that it really reduces the need for an organization to have all these backend infrastructure systems which are very, very costly to actually procure and to maintain. Service Virtualization allows you to test without actually having physical backends there. It pretends that they're there.

By virtualizing application services, you can actually give them to whoever needs to use it. They make a request, get a response, and they use the application without even knowing that it's not connected to a real live system in the backend.

One of the things I like about it is it actually does what it claims to do. It virtualizes application services. A lot of the times companies come in with their glossies and all this kind of stuff, and they put it up on a nice presentation, and they tell you all about it. Then we try to do it, and it just doesn't do it.

This product just works really, really well.

The problem is getting the developers to do more than just hooking into QA backends. They don't do integrated testing so that the first time you bump into an integrative problem is when it hits our QA environment, which are integrated. They discover a defect later on down the cycle and it's more costly to fix. The developers are not finding their own bugs and fixing them.

This tool would allow them to do simulated integration tests without having all this backend infrastructure. They can run on it so we build the services for them, but they're not using it for that capability because it hasn't been presented it to them in that way.

So really the biggest area of improvement is just in terms of who CA should be focusing the tool's use. It would be best to focus on the developers.

It was purchased from another larger corporate project and just kind of sat there, paid for but on the shelf. What happened was the maintenance comes up once a year, and our VP says, "You know, I'm paying 250 grand for this thing. Who's used this?"

We've been running one server for a couple years now. This year we've decided to spread our wings a little, and we're going to procure four more. It can replicate one setup on all five servers, or replicate five different setups, one on each.

The support is there when we need it.

We've had this one for several years, but we just started using over the last two years or so. No other options were used.

I would suggest that you do a proof of concept. Get it in your shop and try it out. See what you like. Look at other tools and compare them. IBM has got I think a tool called Green Hat. Hit the big guys that sell software like that, and do proofs of concept to make your own decision.

Use the web portal that allows you to develop services without actually buying the product. You can go in there and test it.

The most valuable feature is that it's available 24/7. If I'm not able to procure all the infrastructure and I have to share it, I'm not afraid to share it. It doesn't depend on who else is using it because I know it's available.

The performance testing guys really depend heavily on us for their volume testing. The developers use Service Virtualization so that they can present a better product to QA. Sometimes by using it, we're able to implement something without even going to QA. We're bypassing QA.

For our current needs, it's doing what we need right now, although it should automatically generate tests to further make our development to implementation process more efficient.

I've used it for two or three years. My colleagues and I inherited it. It was purchased and sitting on the shelf somewhere until we started using it.

We had no issues deploying it.

It's highly stable.

It depend on how much you're willing to spend and how large your company is. We have 5 servers and it's scaled just fine for us. In a presentation I went to, American Airlines has 50 servers and it's scaled just fine for them.

As soon as you drop an email, someone responds.

It's easy to install as there's lots of documentation. Whether it's a small installation, a single server, or multiple servers, it's pretty easy. Again, American Airlines has it deployed on 60 servers. We have only 5.

We implemented it in-house ourselves.

It's got a standardized format for endpoints as well as containerized, virtualized endpoints that are used by everyone on our team. We can spin up these endpoints if necessary.

We're still working with it, but it's given us a common deliverable. The overall testing strategy allows us to have more common strategies amongst various products.

The user interface and usability of the tool is not the best. They could definitely be improved upon in the next release.

Also, we are hamstrung by different releases and are not using all of the features, but it is useful.

It’s not stable as of yet, as it needs a lot of infrastructure.

We haven't done so yet, but we're hoping it will be scalable.

We interact with an internal department that works between CA and ourselves. They have all of the info we need.

I think we were just using many different tools to achieve the same thing. We needed one tool to do it all, and this fit the bill.

The initial setup was straightforward.

You should make an overall testing plan, otherwise it’s difficult to implement quickly.

I’d compare it to open-source solutions that already exists. Consider your organization before diving deep. For us, we are building up our maturity, and this solution is built to a higher standard.

The features that I find most useful would be how easy it is to quickly get something up and running as a small test sample. The first test we had was simple transactions to it and emulating responses from there. We have Java and .NET APIs that we use to access the legacy system. Dev test works by hijacking the code and then you put an agent in it.

Also, it looks to see what classes you're calling. You can say, okay, if it hits this class, then I want to know. Then, the data that comes through there, you can analyze it and you can replay that data -- Java, .NET, or C#.

I wouldn't say that there's a tremendous amount that could use improvement. I think that with applications like this, it's easy to use it for the wrong reasons. Early on, I think we had ideas about leveraging it in places where it wasn't a good fit for us, where it wasn't going to have a large impact. We had to have a bit of guidance from CA as to where we should be focusing. After that, it wasn't as much of a problem. There are some interactions with customers, depending on what they use. Where maybe we were thinking about doing something that was more internal, but didn't make a tremendous amount of sense for us. There was a case where we tried to virtualize too much. There are limits to what can be done in that kind of testing. It's easy to just take the tool and go virtualize something that you really didn't need to virtualize.

It does require a higher level of support. This is not the kind of tool that you can just haul off just go make decisions on. You need some understanding as to what the right move is and I think CA's the only group that's going to be able to help you with that.

We haven't been using it extensively yet, as we're still in the POC phase, but the POC's that we've run through have been pretty easy to set up. We haven't had a truly deep dive. We've been fairly impressed with it just initially. What we're doing is we're trying to emulate connectivity from a legacy-backing system.

We're a little too early on to be able to comment on that yet.

There was no problem. We did our own setup just for our POC and it was no problem.

With not much other knowledge than maybe two days of training at CA, we were able to go in there and hook it up to our application that writes out to the legacy system, intercept the data that was going in, store that, and then virtualize the response. We did all of that within four hours.

It was decided at the enterprise level well before I got involved.

I guess to sum up, you will need to engage with CA to understand what the right virtualization is, to ensure that you're not wasting your time.

I would say that you need to have good buy-in at the enterprise level, you need to have good buy-in at your director-plus level.

We do testing, so obviously the virtualized service is going to be our big use. A lot of the times, services and/or components are not available, whether it's because the equipment is not available or is being used for something else. With Service Virtualization, we can emulate services to test against them instead of waiting on infrastructure.

And it does it pretty well. We do the recordings and get the response-request pairs. We also have development groups that are probably going to be using these services because they won't let us test earlier in the life cycle.

We're actually ahead of the development groups on this, and they're looking into docking it, but they don't know how use it and they don't know how to create their own services. So we'll probably create the virtualized services for them and we'll support it, and then the development groups will test against our stuff.

We're fairly new adopters as we've only been doing this for less than a year. Because we didn't know what we were doing in the beginning, there was probably two months of ramp-up when we had to learn, and now we're getting more and more comfortable and we're getting pretty good at it.

The features that I think we need weren't in 7.5.2 and the 8s. They're all going to be in version 9. These are some reporting and scalability features, as well an expanded virtual service in version 9.

The automated generation of tests could be improved. Right now, we have to generate them all ourselves. We want to be able to run that against the service and have it creates our tests.

It really isn't scalable, other than the licensing. We can pretty much test away and we can easily clone services if one group needs a slightly different variation.

Technical support has been great. They've been supporting us well and had people on site, which helped a lot with our architecture questions. We had a three-day class, but that's not the same as really them helping us out.

No other vendors were looked at.

It's better than I would've originally anticipated. Just understand there is a slight ramp-up, but once you get past that, the value is really there. I think, wow, we can do a lot of stuff, save a lot of time, and save a lot of money.

It's allowed us to be able to set up virtualization web services quickly and to test unhappy path scenarios. It’s easy to use in setting up virtualization and testing edge-case scenarios.

The time savings it's provided us are huge. We've been able to have a reduction in headcount as well because it requires less staff to manage it. We're simply able to operate more efficiently from a time and human resources perspective.

From a technical use perspective, the improvement in our organization comes from being able to test situations before they arise in production.

I would really like more relevant online documentation. There’s not really enough of it. They should also provide some case studies so that I can see what others have done and how their experiences might help me in my situations.

I'd have no issues with deployment.

I haven’t had any issues as it seems stable.

I haven’t had any issues scaling, so it seems it works well as we grow.

I've never had to use technical support.

I was not involved in the initial setup. I have also not been involved the setup of any new releases.

It loses points in documentation. I’m sure there’s a ton of stuff out there, but the out-of-the-box documentation is lacking.