OpenText Core Performance Engineering (LoadRunner Cloud) Reviews

The charts dashboard, especially the results screen, is the most valuable feature. I’ve also configured the tests, so I’ve also used the test definition page, which is very easy to use, self-explanatory, and simple. The real value comes from the results screen.

StormRunner Load helped us evaluate the performance of the product version we were about to launch. It showed us bottlenecks, where our product reaches its limits, and different characteristics that affect this. For example, we had metrics we thought were affecting our response time. Initially we thought there were two metrics, but we found another significant metric we should have taken into account. Without StormRunner Load we wouldn’t have known about this "missing" metric.

I think the report page could be improved, although we have used the reporting feature. It wasn’t accurate enough to use as a report that I could send my managers. I had to take additional screenshots to help fill in the reporting. I actually, ultimately, generated a report on my own.

I've been using it for two months.

No issues encountered.

No issues encountered.

No issues encountered.

There was no previous solution in place.

This solution immediately helped us to determine the priorities for what to develop next and where we would reach our limits. Because we had a clear prediction, we knew when it would be urgent to develop features, in various cases. Before StormRunner Load, we didn’t have a clear developer roadmap, and now its very clear where we should invest, our weakest links, and what we need to do to strengthen weak links.

If you’re using third-parties for networking, cloud based servers, load balancer, VPN providers etc., all of them should be informed about the expected load. We had a VPN provider that was expecting thirty users per hour and calibrated accordingly, and the minute we started our load test it shut down. This third-party was not ready. Once we informed them that we were increasing our load, they increased their ability to support the load test.

No, this was my first choice.

It is very cost effective to start with a low number of virtual users for a short amount of time in order to verify that your company system is running properly. Start with a small load test, run it until completion, verify that you see the logs you intended to see, and all people involved in the environment are ready to see what they expected.

• Load generator setup is not required so we can be running high concurrency tests in few minutes.
• Scenario setup for running tests was very easy.
• Distribution of the load from different locations was very easy.
• SiteScope and New Relic monitoring tools can be easily integrated with StormRunner Load.
• SLA configuration was available to monitor response times.
• Navigation breakdown report - to create a client navigation breakdown report.
• Real time graphs for monitoring results.
• Email reports.
• Benchmark- comparing previous test results with current tests.
• Anomalies detection.

Being able to setup and run 300,000 load tests using LoadRunner Performance Center takes around 1 to 2 hours when creating four scenarios of 75,000 each, including allocating/blocking load generators required for testing, initializing the fusers and running the test.

Running a 300,000 test was very easy using StormRunner. We just had to upload the script and create scenarios for 300,000 and run the test in 20 minutes.

Running back to back high concurrency test was very easy using StormRunner.

• Improved PDF reports and creating custom reports
• Real world- custom scenario setup
• Update existing scripts online without uploading them again
• Run time settings update in scenario

I've used it for five months.

None - there's no deployment required for running a load test.

During the initial days of high concurrency testing (300,000 and 600,000) we had issues with the StormRunner infrastructure e.g. graphs “disappeared” in the middle of a test, and HP had to give an online fix. Also, there were cases where tests appeared as “initializing” even though it actually started and HP again had to give an online fix. The StormRunner R&D team was very proactive in fixing the issues in very small turnaround time and supporting us running the high concurrency tests.

Scalability issues were seen during initial days of testing but it was resolved by StormRunner R&D team.

8/10.

8/10 - their team was very proactive in resolving the issues and provided hotfixes to support us running performance tests whenever required.

We were using Performance Center to run performance tests, so we had to switch to StormRunner to run high concurrency tests.

As StormRunner Load is a web-based application, it's simple setup, just registering users through an HP site.

In-house implementation and evaluation.

There were no other solutions evaluated.

StormRunner is one of the fastest-growing cloud based performance testing tools for running high concurrency tests very quickly.

I can use it just as much as I need. It is quick and intuitive.

We are able to ensure very quickly and frequently that our product is ready for prime time. Using StormRunner, Codefresh has managed to identify performance issues and when these issues are introduced.

When you start a test, it takes times to initiate.

We have been using the solution for 7 months.

No issues with deployment.

No issues with stability.

We never encountered issues with scalability.

Excellent customer service.

Excellent technical support.

Very intuitive.

In-house technical people.

StormRunner does the job in a very simple and cost effective way. HP has always lead this market.

Flexibility and speed to provision and start large (high concurrency) performance testing cycles.

This software solution has allowed us to run larger, high concurrency performance testing cycles in a very repetitive manner…a requirement for validating our application solutions and supporting infrastructure for customer facing web apps.

Out of the box reporting could be improved.

We have been using the solution for 9 months.

Some issues with UI but HP did provide high levels of support and the issue(s) were quickly fixed (within 3 days of initial product usage).

No issues with stability.

No issues with scalability.

Customer service was an 8/10.

Technical support 9/10.

Utilized a different HP toolset (ALM/PC) for performance testing. SRL provided a better solution from a pricing and scaling perspective.

Initial setup was straightforward…cloud based solution with a simple, flexible UI.

Vendor team…expertise (10).

Looked briefly at Neotys…actively doing a POC of CloudTest for comparison purposes.

You need to effectively plan anticipated usage of the toolset. This will enable you to purchase the appropriate amount of Vusers hours for your contractual period (in our case this was 12 months).

• The ability to spin upload really fast.
• The ability to create a script easily against an application.
• The ability to provision the load so it comes from multiple geographies across the world. 

The documentation could absolutely be improved and scripts. For example, during a performance test, the monitoring graph is not very clear as to what its representing. I wasn’t sure what the graphs represented so my guess is a lot of customers feel the same way.

One month - were doing a trial.

No issues.

No, the only thing I remember seeing as a “bug” when you hover your mouse over the graph, it will jumps around and doesn’t follow the mouse movement, other than that I don’t remember seeing anything from a stability perspective.

No, but haven’t really scaled it yet.

That's where the documentation could be really helpful, apparently I had an older version on my machine, which was creating conflict. If everything had gone as supposed to would have been simple but the documentation could have better and clearer.

Customers should understand the amount of time it takes to build a script - benefit of StormRunner as opposed to other cloud based Load testing tools is the ease of scripting. End users should understand the impact of the time it takes to build scripts. If StormRunner can save you 50% of your scripting, you need to understand how that translates into value. Expertise of StormRunner and LoadRunner includes these elements.

• The fact that you don't have to worry about the infrastructure setup.
• The fact that 1000's of virtual users can log on to the cloud without having to set up.
• Provides a solution for new or existing customers.
• Alternative to LoadRunner, for new or existing clients with agile.

At the moment, its a bit early for us to tell - does everything we need it to. HP is looking to expand to see what exactly it can be used for.

We did a trial since it was launched.

No issues with deployment.

No issues with stability.

We were only running a small set of numbers, but no issues.

It's not something we've really had to worry about - we were able to crack it open and begin.

The help files on the web are good and the video links/ guides are good.

We use LoadRunner extensively for Performance testing.

Setup was straightforward.

In-house implementation.

I can't calculate the ROI right now but I would assume a lot considering savings on infrastructure cost and license costs. It will still take some time to run a performance test but setup saves a lot of time.

We evaluate a lot of testing tools. NeoLoad, LoadRunner, we prefer HP.

If you're looking at an application that can be seen on the WAN and using agile approach, then its a tool that should be recommended and used.