Parasoft SOAtest Reviews

• Being able to load existing .EDN and .XML messages and translate them into segments. I can use them to automate the generation of new messages.
• Generating new messages, based on the existing .EDN and .XML messages, is a crucial part or the testing project that I’m currently in.

We don’t have to manually create .EDN and .XML messages. We can do this with just one click now. This drastically decreases the time that is required to create these messages.

Parasoft SOAtest has an internal refresh function where you can refresh the software to show the changes you’ve made in your projects. Unfortunately this function does not work properly, because it often does not show the changes after you’ve hit te refresh button a few times. I have to restart the software for it to see the changes I’ve made. I have been in contact with the Parasoft Support Division and they told me that it was a known issue and that it has something to do with Java.

I’ve been using SOAtest for about four months now.

I haven’t experienced any stability issues.

I haven’t experienced any scalability issues.

I would give technical support a rating of 9/10. When I encountered an issue, I could always ask for support from Parasoft and they replied with answers very quickly.

The project hasn’t used any other solution before this one.

The setup was very easy. Just a plain installation on Windows and the activation of the license key are required.

From what I understand, Parasoft SOAtest isn’t the cheapest option. But it has a lot to offer. We are only using it for generating test messages, but there is so much more that you can do with it. I suggest getting full support from Parasoft.

I joined my current project at a later stage, but I’m sure that other options were evaluated before choosing Parasoft SOAtest for this project. Unfortunately, I don’t know which other options were evaluated.

You need to check all the features that Parasoft SOAtest offers. If you just need a specific feature from the program, then there could be a cheaper alternative option that works just as well. (This is an assumption.) If you’re looking for a complete testing software/suite, Parasoft SOAtest is a serious option.

XML assertions, XML data bank, various tools available internallly

Achieved testing middleware services and automating them with the features in SOATest. API testing is made easy. Automation and regression test suite was achieved without the need to write much code.

Performance of the tool

5 years

No

Performance issues if there are too many sessions open on your system.

Good

Good

No

Easy

Knowledgeable

Web services and API test automation was made simple without writing much code.

Automated execution and chaining of test validators is the most valuable feature; it is easily extensible via the built-in custom scripting API.

It has allowed us to build a large library of repeatable automated tests that are able to be quickly executed on demand. This builds a solid foundation for future initiatives expanding into a more continuous integration and the agile methodology adoption.

Memory Management suffers when using solutions other than the data repository for managing the test cases.

The platform can become unstable as the memory usage increases. For example, we use Excel spreadsheets for managing test cases, so the size of these test cases must be tightly controlled.

Scaling tests beyond a certain point can trigger issues with Memory Management, but good design practices usually indicate a need to break up a test before it would reach that point.

Technical support from Parasoft has been excellent.

The setup was straightforward. It was very easy to set up a test scenario that consumed assets in our web service layer and to run validation against the results. The tool itself allows for some more complex chaining of components within a test scenario, so familiarizing yourself with the tool enough to get more creative in your test setups can be more complex.

If you plan on using a floating-license model, plan on a buffer of at least 10 user tokens between the maximum allocated by your license and the expected usage for the tool.

I looked at SoapUI NG Pro.

Keep your test scenarios small and focused rather than trying to cover an entire web service with one scenario.

Repeatable automated regression tests in a SOA architecture IT landscape, for several SOAP/REST webservices including Web Gui end-to-end testing, all scenario-based for 10+ applications. Also, scheduled for test runs at night, using Jenkins.

It provides us with fast(er) test results, more predictable results and automatic (regression) test runs outside the employee workstations. This tooling made us more agile and more ready for continuous integration. We use the night hours to run different regression tests over systems, for which we only need to check the results in the morning.

• Fast and easy automated tests for UI
• Interfaces (SOAP, REST)
• Additional database queries
• Easy validations on responses (soap, rest, json, sql, Gui)

Most valuable for us is the combination of all these items in one solution for us. Broad user story scenario's from A to Z, though a chain of systems, including test data prerequisites.
The next most valuable benefit over the past year is the possibility to use the tooling for continuous testing purposes, using the server-side run option 'as a web service' for (nightly) batch runs.

In general, its user-friendliness needs a little improvement since there are a few bugs that still remain. It needs to be (possibly) more lightweight and have more validation options. The bulk changes in the test cases are not always easy. But all in all, these are minor details compared to the benefits that we have.

Very seldom, there were some stability issues but they do appear only from time to time. Since the 9.10 release we haven't seen this any more. Stable!

We only encountered scalability issues with large test scenarios, combining interfaces with a lot of browser GUI tests. Work-around is cutting up the test cases into smaller scenarios.

The technical support is fairly okay. Technical tool-wise they help a lot if needed, but when it comes to complex test scenarios or weird issues, they do not always have a solution.

An exchange of examples is difficult, since we use tooling for internal applications. So, it is not accessible from the outside world, which makes combined debugging impossible.

Initially, we were using some lightweight open-source (free) tools, but none met with our requirements fully; so we switched solutions.

The setup was fairly easy. They provide good instructions. The workstation install is a piece of cake and some additional (simple) config is needed, for Git/SVN etc. The license server install can be tricky, but it turned out to be fairly easy compared to the earlier versions.

We did it our self, with a little help from the seller.

Use floating licenses, so it’s not attached to one workstation. Furthermore, the pricing is something for the staff to handle, not for us. ;)

We mainly looked into the SoapUI solution, along with some Selenium software and besides that, other stand-alone database tooling options. We also evaluated manual testing.

Start off simple. Build your test setup from there. Do think about setting up some specific test cases as building blocks (reference material) for other test scenarios to use, so that the maintenance in the future is more easily done.

Also be sure to read the Help function, might help you on your way when you're facing some difficulties.

We use a lot of tools from Parasoft toolset, but with SOAtest specifically, we use it for creating and running integration tests.

They have a feature where they can record traffic and create tests on the report traffic. That is one of the most valuable features that we've found. So we can run our UI automation tests and directly create services tests from that, which is really, really helpful.

The product is very slow to start up, and that is a bit of a problem, actually. It is not as fast and it's a bit clunky, especially if you use it for the whole day or for a long time, it doesn't save the data, for example. So, it's a bit clunky and slow.

The scalability is good.

The stability is good. Currently, in our organization, we have about twelve to fifteen people using the solution and as the organization grows, and the "number of people" requirement is growing, we have plans to increase our usage. We also plan to move to other areas and try to get other people to use it as well. We do plan to increase the usage in the next one or two years.

The technical support is very good. Any time we have any issues and we raise a question, we get fast answers and we get fairly good answers, because being a licensed tool, we don't get a lot of online applications, but if you raise a request, support is pretty good.

The initial setup was fairly straightforward, easy to install, and easy to get the license keys and use them. It's very easy.

They do have a confusing licensing structure. Their licensing structure is really complex and being able to find out the right product for you, for an organization specifically, is a bit harder, but once you find the thing, their scalability and stability is pretty good. 

It's not a simple licensing model. It's not a licensing model that just, "do this and you get this", it's a bit more complicated and quite complex. For a person who is not used to this at all, it's a bit complex.

We did evaluate the other options in the market, Tosca and SoapUI, which we evaluated. Ultimately, we went with this tool.

It is a good solution, and if you're looking to migrate from an existing toolset as well, it's a pretty good solution. It is quite vast, and they have a lot of other toolsets in there. It has a lot of other toolsets like virtualize which is really helpful. The toolset is really good, I'm really impressed with it.

In the base licensing, we would like to have CLI support included because CLI is the most basic stuff that we need. Currently, it's a separate licensing that we need for that. If I'm getting an application and I need to use it. I would like to use the command line interface as well. I would expect that to be part of the base licensing.

I would rate this solution at an eight and a half out of ten.

The facilities for testing API’s (services, REST API’s, messaging clients, etc..). Especially the features for parametrizing requests and checking responses are very powerful. But the product really shines in end-to-end testing, not only because of all the built-in tools supporting this, but also because of the great extensibility of the tool.

If you want something that’s not provided out of the box, then you can write it yourself and integrate it with SOAtest.

As an expert user, you can accomplish a lot with this tool.

Using this tool, we were able to set up a nightly, still growing, company-wide regression test -- dramatically decreasing time being spent on (manual) regression testing.

• Web-testing facilities. This was not very good when we were using version 9.2, so we built our own Selenium framework and integrated this with SOAtest (this being possible at all is a very positive point in my opinion ;-).
• The user-interface is not very lightweight or friendly, sometimes buggy and cumbersome. The product has a steep learning curve, but very well worth the effort.
• There is virtually no (online) user community, except for an outdated forum. When having issues you have to find everything out yourself (or call in help from Parasoft).
• Reporting facilities can be better.
• Last but not least: using the tool in current continuous integration/delivery practices is not very easy or straightforward.

Five years.

Yes, the product can be buggy or crash sometimes. But not to the extent that it hinders working with it.

We had some issues with stability and performance -- with a growing testbase and/or ‘big’ tests. But these also were minor issues.

8 out of 10.

Technical staff living nearby are always prepared to come by and help with issues. Communicating with developers abroad is also satisfactory.

Not really, QTP was used a little before but was not the right tool for our purposes.

It was complex, but that had more to do with our own complex infrastructure than with the product itself. But it took some time to get the setup of the product right.

It is an expensive product, so think carefully about whether it fits your purposes and is the right tool for you. Support pack is helpful. Due to high license costs, the tool is not easy to use by many people in your organization (like with open-source tools) and so to make test automation something for the whole team.

Yes, but before I was involved, so I don’t know more than that.

Take your time to get to know SOAtest very well, don’t get put off by the steep learning curve, it’s very well worth the effort.

For integration tests, it’s hard to set up component tests due to dependencies. This means you cannot test without using the other components in the loop. The combination of SOATest + Virtualize gives us the opportunity to do this much easier and also to test very early in the development cycle (#Fail Fast, #Fail Early).

We can now test as early as possible, so we are achieving better control over our components.

I would like to see more concrete examples, easy to download studies of very simple Java projects, or TIBCO stuff. Now, I’m sometimes stuck and only finding whitepapers or other reading material, without having concrete examples.

Sometimes I’m stuck, and in that case, it’s useful to have simple examples. I see some shipped examples that are good, but I focus more on the Test automation as part of Continuous Delivery.

For example, yes, there is maven integration and the plugins are explained on a high abstract level. I would to like see simple examples just to showcase and give people a jump start. I found it out by experimenting, I will make this public on my blog in a couple of weeks, just to help the community.

Normally, I’m looking first for some git-repo’s supported with wiki’s, this is what I sometimes miss with Parasoft.

Another topic: SOATest is an integration testing tools, so I was expecting more examples around Tibco BW. Since it commonly just in combination.

Bottom line, modernize the way of documentation.

We are using the solution for four months already. We are using Parasoft SOATest 9.9 and Virtualize 9.9.

Till now, it seems to be stable with no issues.

We didn't encounter any issues with scalability.

We have good support of our solution architect of Parasoft, but sometimes they don’t know all the answers and then it’s harder to find more knowledge in technical support from US teams, which slows things down.

I used to use more open-source tooling. SOATest speeds up our test implementation in the integration context.

With open source, you have to build a lot of plumbing code to get it to work. With SOATest, you can directly focus on the test effort.

BUT, within these open-source frameworks, I did more BDD-oriented testing than seems to be possible with SOATest. So I need to spend more time to find the correct direction within the combined SOATest/Virtualize. Compare this with frameworks like Cucumber or FitNesse and there seems to be more control in organizing the test code.

I think it’s worth the price as long you can set up the test implementation & maintenance.

We evaluated IBM RTW. I think it has a little bit more control, but it’s more expensive.

My advice to others would depend on where they come from and what they want.