OpenText Enterprise Performance Engineering (LoadRunner Enterprise) Reviews

High scalability. Web-based testing. The interface. If you're familiar with the days of using LoadRunner, when you had to have the 32-bit client, using a web-based client is fantastic. You can spin it up relatively quickly despite the fact that it's enterprise software. You can test a huge variety of applications, not just web-based systems, but SAP, Oracle, web services, pretty much anything out in the market place, but it's mobile-based testing.

In my current organization, I honestly don't know so much. But in my previous organization, when I was doing consulting, we helped huge amounts of customers prepare not to fail under scale. So whether you have a large amount of base driven things like Super Bowl, or a major sale, release of a new product like Samsung S8, iPhone 7, etc. Basically when you get a huge push.

Canned reports are always a challenge and a question with customers because customers want to see sexy reports. They want to be able to show something to the CIO. So I think the dashboards are one of the features I'd like to see most.

I think it's more of getting into the world where you've got tableau and dashboarding. I think that reporting needs to be a little bit more fancy, as people expect the sexier reporting. They don't expect just to have, "I ran a test. The test ran for this long." I think the consumer's expectations for what reporting looks like have changed a lot. You do an Excel report or a Word report versus, "No, it needs to be a very pretty dashboard."

The product itself, I think it's pretty good. I can't think of anything off the top of my head.

It's great. I don't have a problem with stability at all, as long as you have it scaled properly and you have sufficient hardware in place. If you're running it all on a VM, you're going to have a problem, but if you run it with the proper infrastructure, it's a very solid product.

The nature of Performance Center is scalable, so you have the application server and then, when you need to have more generators to generate more load, you spin those up pretty quickly. You can use cloud-based generators as well, so that's a huge plus.

It's been a long time since I needed to use tech support. Normally, as a consultant, I am the tech support, so I don't typically have to use tech support. But when I have, I normally am able to get quickly to either R&D-level or a level-two support because it's a real problem with the product, not necessarily just, "I can't figure this out."

I help customers with this process all the time. I'm usually advising them on what, why, when, what the feature benefits are.

Unfortunately, as is human nature, customers decide that they need Performance Center because they've had a disaster. Hopefully not a horrible disaster, but they've had some kind of case where they released a product and it didn't scale. They didn't plan for their own success. A classic example is HealthCare.gov. Politics aside, when you've got the entire American population ready to enroll for healthcare and it tanks, it's a very bad experience for everyone. And that's not an uncommon occurrence across the board.

So then they realize, "Oh, well, we better do performance testing," and then they realize they didn't plan for that in the project lifecycle, so now they need to come and talk to Micro Focus about standing that up, or to talk to a partner at Micro Focus about how to do that for them.

There was a reason, for the longest time, that it had one of the largest market shares of any type of solution in the world, and now that Micro Focus has Silk and the LoadRunner/Performance Center product, they've got that market cornered.

I have set up many, many instances of Performance Center. Recently, it's much more straightforward. A long time ago it was very complex. But it's pretty straightforward. You set up the application center, you set up your generators, you set up your controllers, database.

When selecting a vendor I would judge them on the criteria that I have myself: they've got to have experience, they've got to have done the testing on the solutions that they've worked on. I think seniority is good too, little gray hairs don't hurt anything.

Regarding advice to others, invest in training. Invest in mentoring. Invest in experienced people that have done the job before. Don't go into it thinking that you're going to open the box, get it out, and it's going to be perfect. It's a complicated tool for a reason. You don't want someone operating on you who says, "Well, I read a book on brain surgery." It's complicated for a reason.

Its integration with the Application Lifecycle Management platform makes it very flexible and easy to learn.

My organization teaches Performance Center, and we find over and over that its the top choice for an enterprise solution to performance testing. Because it integrates with Application Lifecycle Management, it is a natural choice for any organization that already uses HP software solutions. While the installation of any server-based application can be complicated, Performance Center is remarkably straightforward by comparison, and for some time HP software has been known for its ease of use and, consequently, appeal to a broad user base.

Performance Center was a stand-alone server application until its marriage with Application Lifecycle Management in 12.x, when both the architecture and the user interface underwent a major overhaul. As such, it has been rapidly evolving for the past several years as HP worked through the initial UI and performance challenge in terms of a response to improve its stability, then begin to focus on adding features that would facilitate communication of performance data.

Today, they continue to enhance reporting and analysis as the My Performance Center dashboard matures, but the web-based interface is still a little clunky in terms of response to user input, especially in the Workload area where a load test is built. Communication between load testing hosts and components is very complex and sometimes fragile, making manual intervention still too often necessary should an anomaly occur during testing.

I've used it since v9.x.

As I said before, Performance Center is certainly part of a complex environment, especially now that it is an extension of Application Lifecycle Management. Yet it constantly impresses me with its relative ease of installation, and the flexibility of its architecture.

Initially (11.x) its stability left a little to be desired, but I feel HP has worked diligently to fine tune the tool so that today, the most noticeable lag occurs during the startup of a load test. Once the test is running, the tool performs as expected.

As I mentioned, I teach Performance Center classes as an independent contractor, so I do not frequently have the need to contact HP support. In my classes, naturally I hear feedback on different support experiences, and I would say they are an unsurprising mixture of favorable and unfavorable opinions. I can only imagine how challenging it must be to troubleshoot such a complex installation when every enterprise surely has a unique set of variables.

Other than HP LoadRunner, I have not. They were the first tools I learned, and have continued to use them because they constantly meet or exceed my expectations.

Performance Center 12.x is compatible with a very specific set of operating system, database, hardware, and file system options. The installation process scans for those exact options, and will not proceed even if a slight variation is detected. By comparison, the previous 11.x could be configured to use Oracle 11g (the free, development version of the database), useful when testing a trial version of the software. Now, the supported options are a bit more rigid, but still the installation process is remarkably straightforward. I have a reasonable technical background, and after working through a couple specific challenges I have now installed and configured the application half a dozen times.

We did implement the software using our in-house team, but I would advise engaging HP consultants for larger installations. The complexity of our environment cannot be compared to the needs of a typical enterprise.

The HP sales team is much more qualified to discuss topics related to pricing and licensing. HP is very focused on building long-lasting relationships with its clients, and as such is willing to negotiate a combination that is favorable for both them and the client.

HP has very masterfully re-engineered this tool and integrated it with Application Lifecycle Management to create one of the most seamless project management suites I've used in years. When simultaneously integrated with Unified Functional Testing, which also supports service testing, I feel the richness of testing information that can be managed from within a single tool (ALM) is hard to beat--despite the price tag.

We use OpenText LoadRunner Enterprise for load testing. 

The tool is very easy to set up and get running. 

OpenText LoadRunner Enterprise needs to improve reporting. 

I have been using the solution for 13 years. 

I rate OpenText LoadRunner Enterprise's stability a nine out of ten. 

I rate the tool's scalability an eight out of ten. My company has 120 users. 

I rate the tool's deployment a nine out of ten. It can be completed in two to three days. 

I rate the product's pricing a three out of ten. 

I rate OpenText LoadRunner Enterprise a ten out of ten. 

As a tool, it's something that we can use in all aspects of performance testing, whether it's services, whether it's web, whether it's customized APIs, like Citrix. It's the tool for performance testing, and it's definitely the industry leader that I've been using for years. It's a very powerful tool.

Truthfully, I think performance is a fairly mature space now. There are not too many things popping up that we're saying, "You know, Performance Center doesn't do X, Y or Z." It's a pretty mature tool and we're happy with it.

In terms of having one tool it is helpful in terms of training. It has really great reporting features, so not only is it a good tool to do testing with, but when it comes to helping you produce good results that you can present up, it's an all around tool that has everything that you need within it. You don't have to go to other third-party tools for reporting or for different types of testing. It's a tool that is "one size fits all".

I think better or more integration with some of the monitoring tools that we're considering.

We're looking to bring in maybe AppDynamics. I personally don't know the integrations but having good integration tools is going to help us in the future.

I think some of the reporting features could be better. I haven't seen much change in terms of that aspect of it, the report analysis piece. It's been good enough but I haven't seen as much advancement in that space, the reporting analysis.

I find it very stable. I will say that with the changes to virtual machines and things like that, It is harder now to manage. That's one of the reasons we are looking at not only using Performance Center, but also StormRunner. That gets us away from needing to scale up; doing that within Performance Center can be difficult because you have to deal with the aspects of all that infrastructure. It's not the tool itself, but it's the underlying infrastructure that you have to manage. Something like StormRunner is promising because it gets you away from that a little bit. Somebody else is taking care of that aspect of it.

It's an enterprise type tool. It scales very well. Again, it's not the product that we have issue with scaling.

We have challenges with the number of VMs that we have. The issue with scaling up is we come up against our infrastructure team that wants to limit the number of VMs. They have to manage them. The whole VM-type design seems to be a lot more VMs, and when you need to scale up, it's even more VMs. I understand the cost savings with VMs but in terms of management, if you don't have that nailed down in terms of automation and all that monitoring, it's a challenge to scale up.

I think that's where our current Performance Center implementation is probably going to remain, but if we're going to scale up, we're really looking toward something with StormRunner, where we can scale up as we need and not have to worry about the whole managing of the infrastructure, cause that's a challenge.

I have used tech support. Not recently, not in the last 12 or 18 months, but yes, I've used tech support. They're responsive. I've had good support. They get to the point. There's not a lot of hand-holding, they expect you to know what you're doing. I have no problem with that. As long as I can get the answer, get what I need and get it done, I'm happy with that.

I wasn't involved in the initial setup, it was pre-existing. We use Performance Center, but as the HPE ALM piece is managed by a separate tools group, that's a challenge because we don't have control over the whole implementation.

I've assisted with the Performance Center upgrade piece a little bit, installing it, but in general, we have a whole separate group that does it.

When looking for a vendor to work with the number one thing is does a tool do what it needs to do? Second, of course, support. Stability and the ability to scale are pretty important but I think that's grouped under the tool itself. It has to be an enterprise ready scalable tool.

Regarding the vendor itself, support, being responsive, having a way to access the support that's not overly obtrusive. I don't mind doing emails or logging onto a website, just as long as it's not too convoluted. Sometimes I feel like you have to go through 20 steps to get somebody to call you back and every customer support or technical support has their process. As long as it's not overly going through hoops to be able to access that.

In terms of advice, you have to do the math. There are a lot of free tools or tools that you write yourself. You just have to make sure that, long term, are those things maintainable, supportable? Do you have the training? Do you have the support? You have to bake all that in before you make a decision. It's not to say those other tools aren't valid, and people do a lot with them but, for example, if the tool needs programming skills, do you have those skills? Do you have a team with those skills? And how much is it going to cost to keep, hire, or maintain your staff with those tools? So you have to do the math to make that kind of decision, what the right tool is.

I think the tool does what it really needs to do and I've never had an issue with their support. I think they're definitely the industry leading product for performance.

Performance Center is actually run by a performance engineering group so in every release you have a performance engineering phase that runs at the same time as UAT. That's pretty much used to ensure that we're going to hit our production stability, scalability, etc., when a product finally goes in.

That is used in a phase gate, it's not continually run. That's one of the things we're eventually going to get to, that you continually run it.

Our performance engineering team has feedback on Performance Center, so they don't utilize as many of the features as they should. They have it, part of it could be training, but they're not using it as much as they should. They use it, but they're not taking full advantage of all the features in it, meaning they are not combining it with UFT, etc.

They just run it to run the virtual users and then load and stress test, and that's pretty much what they're doing. They're not really taking advantage of the whole stack like we do. That's another group as well.

Stable in the fact that it does what it does, it does it well. Why they use other tools, that's where I would say there's some poly-functionality that has to be improved in the product. I'd have to specifically interrogate them on what they're not getting out of it. It's stable, it's up, and it runs, but if you want to look at is it as highly leveraged as it could be, it's not so much.

We're already at enterprise scale, so it's used across the enterprise. I would say that we're at that point.

Originally we were using LoadRunner, and then we had to upgrade to Performance Center because with LoadRunner we didn't maintain currency with the license. That's another group that manages that. They just kind of were using it as it is. Then when we upgraded to version 11, we had compatibility issues and we had to go to Performance Center. I think they just didn't get used to it. I don't know exactly what they are or are not using in that stack, but part of it is we were forced to upgrade.

I have an entire team, so I'm a director and I have an entire tools team that does that. I did get involved in the planning and the strategy of how we're going to do it. My team said that first installation is relatively easy. When we go to upgrade and migrate, that's where there's pain.

We have to use other products like Selenium and a lot of custom scripting, et cetera, but that's the nature of the business. That ends up happening everywhere.

Have a well-defined process, have a strong reporting structure, meaning in your process you want a lot of measurability. If you define your output, the reports and the questions you need to answer from what you're doing, which your process should be managing for you. In our company, we are very specific about what our executives and stakeholders want.

We have a very well-defined set of measurements that we have to take. We then put a process designed to ensure those measurements are always taken. That then allows you to deal with your outputs and your reporting structure, which then allows you to properly architect your tooling. The technology is very flexible. You have to decide as a client area how you really want to use it and that's going to start with what your business needs are the values that you're trying to get out of it.

That's the biggest advice that I have, it's not even on the technology. The technology will do great things for you if you have a plan and a structure and you know what you want it to do for you. Half the time they don't know, they want the tool to do it for them and it's the other way around. So that's what I advise people to do.

Think about it, have a vision, have a plan, tie that to outcomes, and measure those outcomes. If you're answering the right questions and asking the right questions, your technology will really enable you. You've got to look at it from that standpoint.

We still have a legacy LoadRunner, but we are moving towards this tool. For our web applications, we are moving towards Performance Center, because it is more GUI and user-friendly, and with all the latest technologies. It's pretty easy to learn how to use.

I don't know if it is available or it depends on the licenses, but reports from Performance Center need to be improved. If we can improve the reporting structure and we can view the report on a smartphone, that would be huge.

We always have to get connected if there is a report. It's doesn't need to be the full tool, but just like how HPE Discover has an app, if we can have an app just for reports, that would help.

I've been using it for about six to eight months.

We are still learning and evaluating it.

All of our applications are not yet on Performance Center.

It was even before I joined. We were using LoadRunner. LoadRunner is a leading load testing tool in the market. Whenever a customer or anyone looks for a tool, the first thing which comes to mind is HP. We have seen StormRunner and we are here at HPE Discover 2016 to check out tools. We will see how they scale, and probably will make a decision soon.

Nothing wrong with it, but I have to learn more about it to see if it's going to match our needs. Performance Center has everything which LoadRunner offers, plus additional things.

• Auto/manual correlation
• Test Analysis
• Test script customization
• Test scheduling

Actually we’re implementing and consulting with our customers in terms of performance testing using LoadRunner or Performance Center.

For example, we’re in the middle of a project for implementing a distributed Performance Center system, with six load generators for testing an Oracle ERP system that holds about 800,000 users, with an average annual increase of 40,000 users.

Our target is to implement Performance Center, integrate it with HP ALM, design the load tests for simulating 70K virtual users that are hitting the system with different scenarios, and finally we will present reports to ERP management staff about the response trend of the system, such as virtual users passed/failed, server throughput/sec, transactions taken time etc.

I think there is a lot of configuration and customization that can come in for optimizing the recorded load test in terms of CPU and memory used and, accordingly, the service/process that are running this v-user script at load generator. This would be very helpful especially when running a huge number of v-users simultaneously.

I've used it for two years.

Running virtual users especially with a huge number is a very challenging task. The load generators may over utilize its resources (CPU/ memory) due to improper default configurations, or improper use of LoadRunner and Performance Center. Many considerations should be taken into account when optimizing resources when doing this task. For example, running the recorded script as a service at the load generator is different from running it as a process.

Also, customizing the auto-generated script in virual user generation may cause problems when ramping up the number of users. For example, simulating a generated list of variables that would be used by every virtual user is important.

Lastly, taking care of variable sizing and limitation should be reviewed, and otherwise errors may appear at certain point of running the script.

Above moderate, but HP is trying to enhance its support level.

Above moderate, but HP is trying to enhance its support level.

The initial setup is straightforward when following the installation guide steps.

I did the implementation by myself as part of a project for one of our customers. My advice is to understand the business need that would guide to a proper scope and design guidelines. Performance Center can be implemented in different ways (standalone, distributed, with/without integration with HP-ALM or HP Diagnostics, number of load generators and their location etc.). The implementor should arrange for the best and quickest setup that will fulfill the goals, otherwise the implementation would end up as a trip of trial and error, and probably fail, or exceed the time plan.

Our customers think of the following benefits as ROI of using HP Performance Center:

• Reducing downtime in operations phase that may come due to load on system by fixing or customizing their application to afford the maximum number of concurrent users they expect
• Prior knowledge of their system limitation and its break point, may let them take precautions steps at application server or network level to prohibit overload traffic that may crash their system.

The license of Performance Center is not cheap and may be very expensive for some protocols, such as SAP and Oracle. Accordingly, optimizing the license to what is only needed – in terms of protocol and number of virtual users can save a lot of money. When the load testing is limited with a start and end date plan, I recommend using temporary licenses, or a pay-as-you-go model of license, it may be more expensive in the short run, but of course it would be more economical in the long run.

From outside HP no, but inside HP there are two solutions for doing the load testing; LoadRunner, and Performance Center. The concept is almost the same, but Performance Center excels in big projects, and working with different teams besides scheduling tests feature.

My advice is to identify well the scenarios, protocols and maximum number of virtual users needed for load testing. Also, simulating a real-world load testing scenario is very important if you need to get a near-real results. For example, simulating network speed to reflect the real case. Some scenarios may require using a paid e-service, which may cost a lot when simulations repeat with a huge number of virtual users. At this point, another product called HP Service Virtualization could be used to sniff the in/out going traffic, then simulating the e-service function later after learning its function. At this point we can then replace the real e-services communications with HP Service Virtualization, so a budget-wise trade-off may be held between using HP Service Virtualization and the paid e-services. I recommend consulting HP presales to get the most proper setup with least possible licensing.

Initially, I've been using it for small use cases, just to test scenarios of less than 1,000 users. I think generally it's been very good. My team has even deployed it for clients within banking. It's still a go-to tool; although, as far as SaaS goes, recently we have had more suggestions to go with Neosyde. 

I think the number one feature everybody likes is the capability to easily generate virtual users as well as the reporting. Recently, we are starting to look at things more from the diagnostic perspective as well as from the troubleshooting perspective. It gives us many more options for troubleshooting and presenting reports. The other reason why LoadRunner is quite popular for us is that it has a long track record. We know if we need to look for a solution we can still search and find a use case or a solution quite easily.

I like the new pricing model. It helps us to ramp up much better, especially when we were trying to use this for SaaS applications. They have a much more practical pricing model now. It allows us to break it down smaller and also build-up towards a price model that works for the client. I think that was a big bottleneck in the past — now it looks much better.

From a technical perspective, LoadRunner has always been good. You can trust that it can deliver. The big bottleneck in the past has always been the pricing model. Now, with the new approach, with the use of SaaS, we are currently in proposals to recommend LoadRunner as a solution for one of our government clients. We are doing an implementation there. 

I think LoadRunner is still getting into grips with me — maybe, I've not used it that much. It's not that popular on the cloud. Also, we have not tried this on mobile platforms with mobile virtual users.

I have used LoadRunner for quite some time — roughly 10 years.

Both the scalability and stability are strong points for LoadRunner. We have no complaints so far. Of course, there's always this concern around if we have sufficient use on the hardware to create the required scale for the number of users, but I think that's easy to workaround. This is what enterprise users do; we don't really have that much of a complaint there.

We work quite closely with the local team in Malaysia — they do their job.

Generally, the initial setup has not been much of a problem. If you have some level of intermediate knowledge on networks as well as some quick training on LoadRunner, you should be able to set it up within a week or two.

Proper training is important. If you have teams that want to use the product, you need to ensure that they go through the right training. Get your guys to sit through the LoadRunner training or get someone experienced to train them.

Make sure that your team trains before they go and apply the system because LoadRunner is not actually something that you do, plug-and-play. You do need a little bit of configuration, and it's not for beginners. It is meant for people with at least an intermediate understanding of networks, and an intermediate understanding of performance application — you need to have that. I would say it's always important to ensure that you work very closely with the development team. To get the best out of the tool, you need to have a solid collaboration. When you want to troubleshoot, you want to review or uncover the performance issues; you need to make sure that you work quite closely with the development teams as well.

On a scale from one to ten, I would give LoadRunner a rating of eight. We have not used it for global distributed testing, and we also don't know its full capability from a mobile perspective. That's an area that I cannot comment on yet, so I'm reserving my judgment on that. That's the reason why I am giving it an eight.

From my perspective, there's still a gap in terms of the area that LoadRunner is being marketed to. Its biggest strength, in my opinion, is the reporting. If they could keep the reporting, but give it a lighter engine to generate virtual users, that would be perfect.