OpenText Enterprise Performance Engineering (LoadRunner Enterprise) Reviews

The typical use case for OpenText Enterprise Performance Engineering (LoadRunner Enterprise) involves ensuring performance before go-live or ensuring performance after upgrades, or sometimes big data center migrations and big platform changes.

The best features of this solution are easy scripting and broad platform support.

The distributed testing feature of OpenText Enterprise Performance Engineering (LoadRunner Enterprise) is also a fantastic feature.

The analytics and reporting features can be improved, though they are good enough. If you have expertise, you can manage with what is included. However, it could be much better, especially with modern AI capabilities.

When considering areas for improvement in OpenText Enterprise Performance Engineering (LoadRunner Enterprise), there is a need for automated analysis and code-level support.

I have experience working with OpenText Enterprise Performance Engineering (LoadRunner Enterprise) since the early 2000s when it was Mercury.

Regarding stability, I would rate it an eight or nine. From time to time, we encounter bugs that are unexpected from a 20-plus years old solution. However, it remains quite stable.

OpenText Enterprise Performance Engineering (LoadRunner Enterprise) is a scalable solution.

I rate the scalability of OpenText Enterprise Performance Engineering (LoadRunner Enterprise) as ten when using a scale from one to ten, with one being low.

The customer service and technical support for OpenText Enterprise Performance Engineering (LoadRunner Enterprise) is reasonable, not impressive, but provides adequate assistance.

Positive

The initial setup of OpenText Enterprise Performance Engineering (LoadRunner Enterprise) is straightforward in terms of adaptability.

I have seen an ROI from this tool, as it provides enormous value. It ensures that you are not setting yourself up for failure. It functions as an insurance.

The price of OpenText Enterprise Performance Engineering (LoadRunner Enterprise), including pricing, licensing, and setup cost, is reasonable. It is neither cheap nor expensive.

For those looking into using OpenText Enterprise Performance Engineering (LoadRunner Enterprise), my advice is that what they see is what they get. It is a solid performance testing solution. If it works, it works. If it does not work, good luck. It is a technical solution for most platforms.

On a scale of one to ten, I rate OpenText Enterprise Performance Engineering (LoadRunner Enterprise) an eight.

I always consider the purposes and use cases from an enterprise version perspective as a user of the product.

The advantages of OpenText Enterprise Performance Engineering (LoadRunner Enterprise) are clear. The analytics and reporting features provide beneficial impacts on my organization. The scalable testing environment is very beneficial for my performance testing processes.

Scalability is a main important factor when identifying bottlenecks. For example, when I target about 100 users but scale up the system to 120 or 150 users, I can identify where the bottleneck starts to occur.

Regarding negative sides or areas for improvement, I do not see any disadvantages so far. OpenText Enterprise Performance Engineering (LoadRunner Enterprise) might have some drawbacks, but I did not go to that extent to find any. Regarding new features they could add, I am not getting anything at this time.

I have been using OpenText Enterprise Performance Engineering (LoadRunner Enterprise) for two years.

In terms of stability, I would give a rating of nine. The product is stable.

For technical support, I would give a rating of eight. Although technical support issues are not directly related to OpenText Enterprise Performance Engineering (LoadRunner Enterprise) but related to partners, I need to contact support through them since the partners are in between. The technical support experience is a bit limited due to this indirect relationship.

Positive

When discussing price, OpenText Enterprise Performance Engineering (LoadRunner Enterprise) is very expensive, which I would represent by a rating of ten. The product carries maximum expense points.

I have experience with load distribution testing. Based on the load requirements, I set out the virtual machines and load generator systems according to how many are needed and what capacity they can provide. That dependency is based on the load.

Regarding marketplace specifics like AWS or Azure, I am uncertain whether that applies to AWS Marketplace or Azure Marketplace.

For my project handling, I use JMeter for approximately 90 percent of the projects I manage and OpenText Enterprise Performance Engineering (LoadRunner Enterprise) for 10 percent of my projects. My overall review rating for OpenText Enterprise Performance Engineering (LoadRunner Enterprise) is nine.

I have been using LoadRunner for multiple industries. Initially, I used it for the banking domain, then for the insurance domain, and later, I used it in the manufacturing domain. Recently, I employed it in the healthcare domain. Until 2017, I used it for the manufacturing domain.

LoadRunner has allowed me to provide customers with insights into what issues their projects or applications are facing. With LoadRunner, I was able to offer detailed results and insights that helped explain the problems with their projects.

I've used other tools, such as RPT and JMeter. I find LoadRunner very user-friendly. Particularly, the process of creating scripts in LoadRunner's architecture, though not easy, becomes transparent and understandable, aiding in debugging and repairing scripts. Other tools like RPT or JMeter don't provide the same level of insight. 

Furthermore, LoadRunner's UI is project-convenient, allowing easier simulation of real-time scenarios and producing comprehensive reports that are effortless to read. These reports detail the logs and streamline error debugging.

While I don't see any issues with LoadRunner's functionality, the cost of the tool is a major factor. Many of my customers have had to switch to different tools due to the cost of LoadRunner, despite its advantages, leading them to alternatives like JMeter and RPT. 

Additionally, the packaging of the product, where different protocols come with separate plans, is not very convenient compared to other tools where licenses include all protocols.

I have used LoadRunner almost for 12 to 13 years.

LoadRunner is stable.

LoadRunner Enterprise is also very scalable.

I would rate customer service between seven and eight out of ten. Sometimes I received quick answers, while at other times, especially for some issues, I had to follow up extensively to receive satisfactory responses.

Positive

I have used other solutions like RPT and JMeter. Our choice sometimes depends on cost considerations as JMeter is free of cost, making it a first preference for smaller projects, whereas RPT is cheaper than LoadRunner.

The initial setup of LoadRunner is straightforward. I never found any issues while setting it up.

I handled the setup by myself, but it can easily be done by one of my team members.

While I cannot calculate the exact return on investment, LoadRunner has provided more return in comparison to other tools by offering detailed insights into applications which aid in identifying issues.

In 2019, I was dealing with the costs of LoadRunner. While I don't remember the exact figures, JMeter being free and RPT being cheaper makes them attractive. The high cost of LoadRunner, in contrast, often caused customers to switch.

I have evaluated and used solutions like RPT and JMeter.

If it wasn't for the cost, LoadRunner would be my first preference based on my experience using multiple tools for load testing and performance testing.

I'd rate the solution eight out of ten.

I use it for performance testing from top to bottom at every level possible. For functional user interface performance testing, API performance testing, with all types of tests like load, stress, performance, capacity planning, anything that you might qualify for testing. 

I use it in various domains, in government institutions, in banking, in retail. Actually, my last projects were for government institutions, one on banking recently, and on retail, it worked quite a while ago, but in 2003. So, or some as well.

I use UFT for functional testing in your API testing workflows.

I work with LoadRunner in a distributed environment. 

It performs very well. I have no problem with it. LoadRunner has one major advantage compared to all other tools I know from the market. It's old enough and can test almost every tool in the companies I enter. It supports between 35 and 60 different software applications which need to be tested. 

With LoadRunner, I have one tool that can test everything. If I'm using, for example, JMeter, JMeter can test only web-based APIs and some limited user interfaces, and then I have to use another tool for the others. 

So, this is the main advantage, and this is why I choose to work with LoadRunner. It's not a perfect tool, but it does the job in terms of distributed applications with different technologies in the corporate environment.

Moreover, the integration capabilities are impressive, especially with Dynatrace. It was a significant advantage last week to harvest information directly from Dynatrace, a monitoring system. Previously, when LoadRunner was associated with HP or Microsoft, there was a push to bundle LoadRunner with their monitoring tools. 

If you didn't purchase LoadRunner alongside their monitoring solutions, managing separate logs became a necessity. However, now that LoadRunner integrates with Dynatrace and other monitoring tools, it simplifies the process of integration into a company, taking merely five minutes to set up. 

This ease of integration allows for quick comparison of monitoring and performance results, a feature I highly appreciate.

I find the runtime settings beneficial for creating realistic load scenarios, which allow me to combine different actions into realistic workload scenarios. So, runtime settings and the features in there.

The real-time test analysis and reporting feature has very much enhanced the testing processes for me. In terms of reporting, I have a controller that must be used all the time to make the most out of that license. So, basically, the server that has the controller installed is used window after window of testing. 

I use it for two hours, another colleague's test for two hours, etc. Now, if my test lasts exactly one hour and a half or two hours, I do not have the necessary time to analyze the results on the server. 

So I take it offline on my computer, where I have the Analyzer for free, due to OpenText's license system. And then I can analyze it offline for as much as I want so I can draw my conclusions. 

Features inside the Analyzer that I use are overlapping graphs or correlation of graphs, all these kinds of stuff which allow me to see the results much faster than in other tools.

It would be beneficial if LoadRunner could optimize resource usage, especially for protocols that require significant resources, like TrueClient, which interacts directly with the UI. 

If they could improve resource usage, like ingest or for the load generator, using less CPU or RAM memory, that would be great. That's where I have problems. 

In real time, when they ask for 5,000 or 10,000 concurrent users, I have to provision a lot of virtual machines to define this load.

Then there are situations with certain platforms, especially document management platforms, where the technology is so weird that normal LoadRunner protocols cannot detect it. 

So, in that case, I have to use that special TruClient protocol. I have to use the TruClient protocol, which actually clicks on the object. Despite the SQL technology, I can still create a script and test for performance. 

So what I would appreciate a lot is if this protocol would require less resources on a normal virtual machine. I can use fewer concurrent users with TruClient protocols as opposed to almost one hundred with HTTP/HTML. As opposed to many more with HTTP/HTML from, let's say, JMeter. So, optimization at that level for resource consumption by OpenText would be much appreciated.

I have been using it for many years, since 2003. 

And recently, just yesterday.

I find it stable, there are no issues with its stability.

It's 100% scalable without any problems. 

In terms of usability and volume, everything is satisfactory. However, the licensing cost becomes an issue beyond a certain number of users, as licenses become expensive. 

Pay-per-user licensing limits the tool's use, although scalability and volume handling are otherwise excellent.

My team and I are quite experienced with the tool, which allows us to resolve most issues independently.

We almost don't need technical support. In my team, I have three professionals using LoadRunner, and out of three plus mine, for four people, we can find solutions to even difficult problems. So, I haven't used their support lately.

From the SmartBear tools, we use ReadyAPI. We've been using it for five years, and I've actually been a training partner for IBM for the last three years, delivering training for them using ReadyAPI.

We use it for API testing and service virtualization. I like the interface. It's easy to use. 

Feature-wise, it is the same as other tools on the market. Not something spectacular about it. But usually, it covers all the needs you might have in IT and quality assurance in general.

We also use OpenText LoadRunner Cloud. For API performance testing, I prefer to use LoadRunner.

I've worked with LoadRunner since a long time. I know it much better, it's much more familiar. It's nothing wrong with ReadyAPI. But if I had to choose between the two, I would choose LoadRunner because I have experience with it. It does everything I need, so that's why I work with it.

The setup process is straightforward and easily manageable even by a junior technician.

The installation of LoadRunner takes about an hour, and I can start building scripts on the same day. So, it takes less than a day to deliver something.

I've worked with environments ranging from 10 to 10,000 concurrent users, covering everything in between.

Maintaining LoadRunner itself is not difficult. However, the complexity of maintaining scripts depends on their complexity and the user's scripting skills. So, it is not about the tool, but it is about the user. 

There is an ROI. What LoadRunner does, is it prevents failures when there are many, many concurrent users in the systems of a company. 

For example, what I do is I find in my client's timeline the historical moment where their application crashed, and they actually calculate their own damage. 

Then I come with the performance test, and I take out as many performance bottlenecks as possible. So I tell them, "Look, now it will not crash in production because we have seen this exact situation." And I'm responsible to say for what I'm actually saying. But this is how I prove the value of the tool. 

Another way to prove the value of the tool, even when it's quite expensive in my opinion. But since it can test different applications in the system, I do split the cost of, 30 to 40 projects. And when I split the cost into 40 projects, I will have a return on investment because it becomes a little bit cheaper. It all depends on budgeting and the customer. If I have a customer with an open mind, I can justify the cost. 

If the customer is narrow-minded, then they don't want to use, and they are resistant to change, and they don't want to pay for trained professionals, I rather not sell. 

LoadRunner is easy to use. It's easy to test anything in any big corporation at an enterprise scale. So that's what I appreciate. It is easy to pass the knowledge to newcomers. 

And very heavy resource consumption and expensive. These are the cons.

LoadRunner is user-friendly and adaptable for enterprise-scale testing, with easy knowledge transfer to newcomers. However, its resource consumption and cost are considerable drawbacks.

I recommend it with one remark. Please hire a professional consultant if you don't have in-house resources for the first test. So, with the right consultancy, LoadRunner is a powerful OpenText tool. 

If you don't have the right consultancy, you'll be using it at 10 to 20% maximum, and then it's a waste of money.

Overall, I would rate the solution a ten out of ten. 

We are an e-commerce website. We use it for both website testing as well as API testing. Most of the time, we do API testing, and there is no recording as such. When it comes to UI, we have applications that are pretty rich in Node.js technology and other frontend technologies. There are some specific server-rendered pages as well as client-rendered pages. It is more of recording and then customizing and using it for backend load simulation. For performance testing, the metrics that we capture are mainly based on the frontend. We rely on W3 semantics there.

We predominantly use the Web-HTTP/HTML protocol. We also use it for maintaining our scripts, which means storing and making sure that we maintain the scripts in a proper way.

Performance testing is one of the checklist items for us, irrespective of the release or changes done at the backend. We have customer-focused applications in our business. If the customers are not happy with the way they are accessing the application, it impacts our business. That is the reason why performance testing is a must. LoadRunner is playing a very significant role in completing our tasks.

LoadRunner Enterprise has reduced our workload. We are also using JMeter for some of the applications where API testing needs to be done. JMeter requires a lot of customization using Java, but that is not the case with LoadRunner. It definitely makes our work easy.

LoadRunner Enterprise also saves time. For example, for a script in JMeter, I would have to spend four to five hours in JMeter, including the test data setup and then validating multi-user iterations, whereas, with LoadRunner Enterprise, I can complete that in two to three hours. It saves 50% of the time. LoadRunner Enterprise is also better than JMeter in terms of competency or skills. For example, for JMeter, you need to have a knowledge of Java, but that is not the case with LoadRunner Enterprise. It is as easy as understanding any simple script.

LoadRunner Enterprise has been helpful in simulating the load, but not in the analysis as per our needs. If I can have additional features that will help us in performance engineering, and not just in the testing, it would be great. If we are able to analyze and have exact hot spots during the testing itself, we would not have to depend on any of the APM tools. That will definitely add value.

It is pretty easy to do test execution and results analysis. When it comes to scenario settings, LoadRunner Enterprise has an extra edge over other testing tools in the industry. The scenario setup is easy, and in terms of execution, we have a clear idea of what is happening. When it comes to analysis, it is easy to download the analysis file and see all the analysis. We have many filters available, and it is easy to customize according to our needs.

When we have a new application, recording the application is a pretty tough task. We have tried multiple things. We do scripting or try to record with different settings and on different machines. We try to record multiple times, but we do not know why it is recording and why it is not recording. We do the same thing on different machines. It sometimes records, and at other times, it does not. That is one of the major concerns.

Another issue is that while executing the load test, we recently faced an issue related to the mmdrv process. After the test is complete, from the LoadRunner Enterprise perspective, it does not show anything running, whereas, at the backend, we see that the load is getting generated from the load generator. We got to know that it is because of the rogue mmdrv process that is running even after the test is complete. We have raised this issue multiple times. The patches or solutions that they have given are not working. We have raised a ticket for this one as well. 

For each and every release, one of the criteria that we have is that it has to pass performance testing. It does not matter how big or small the changes are. Irrespective of the extent of changes, performance testing is required. We give a lot of value to performance testing. For us, performance testing is not only the test execution. We consider that as only 10% of the end-to-end performance testing. The analysis portion of it is where we want to spend 90% of our time. Our scripting effort is not too much, but it is still considerable. For a given application, if end-to-end testing takes 16 hours, then out of that, we need to spend at least four to five hours on scripting, customization, test data setup, etc. For the rest of the things, we have to spend about 12 hours. LoadRunner Enterprise can provide value by enabling us to do these analysis and performance engineering aspects within the tool. LoadRunner has a new feature in the cloud for single transaction analysis or single-user testing analysis. It shows all the insights in terms of the response and hot spots. If they can expand that to LoadRunner Enterprise as well, and not just to the cloud, it would be helpful. 

One more issue is that some of the features are enabled in certain LoadRunner versions. They are not available in other versions. To get a specific feature, the problem is that I either have to upgrade to a different tool or completely get rid of it and then use another tool. Instead of that, they can enable the features irrespective of the platform. Each and every tool that they are producing has its own advantages as well as improvement areas. For example, LoadRunner Professional has the advantage of using chaos engineering tools, which is not there in LoadRunner Enterprise. Irrespective of the tool version I am using, I should have the same advantage. I would like to see chaos engineering in LoadRunner Enterprise.

Their support should be more efficient. I do not raise a ticket just to see the answers that are already available on the Internet.

It has been five years.

Its stability is fantastic. Out of 100 test executions, there is usually one case when it is not able to capture and generate the analysis file. We then use some manual methods to generate that analysis file and get the results.

Its scalability is fantastic. We have no issues. We can increase the number of load generators, and we can increase the number of tests that we are doing on a day-to-day basis, so everything is good.

They helped us a lot during deployment, but on a day-to-day basis, whenever we have any issues, there is a lot of opportunity to improve. I have also given this feedback to OpenText. I would rate them a five out of ten because there is a lot of room for improvement.

Neutral

I have used solutions from different vendors. One of them is BlazeMeter, and I have also used NeoLoad in my previous organization. 

In my current organization, we are also using JMeter. With JMeter, we have to spend more time in scripting, test data setup, etc. With LoadRunner Enterprise, there is a 50% time reduction. It is reducing 50% of the man hours that we spend in scripting for JMeter. That is an advantage.

We recently upgraded LoadRunner Enterprise from the previous version. It was definitely complex. There was the infrastructure upgrade as well as the licensing upgrade. We also have to have a different set of load generators, controllers, and other things. It was a complete switch, so it took time and was complex.

It offers flexible deployment options, including on-prem, SaaS, and hybrid, which is very important for us. The reason is that we are migrating our applications from on-premises to the cloud. When we test applications in the cloud, one major aspect or component that we need to consider is network utilization. If your load generators are on-premises and you are hitting from on-premises to the applications hosted in the cloud, network bandwidth consumption will come into the picture, and your response time will include that. It is sometimes acceptable in real-case or production scenarios, but it is sometimes not acceptable because the systems that will access the particular service that is under test may or may not be hosted in the cloud itself. In that scenario, I have to have a solution that I can directly hit in the cloud itself. In that case, I would like to have LoadRunner Cloud. At the moment, we are looking at using the Flex model of LoadRunner infrastructure deployment, which means we will have load generators on-premises as well as in the cloud.

We have our own internal team.

We got a very good deal. We are happy with that. We have 5,000 licenses.

We are currently not using LoadRunner Developer, but we are planning to use it. We also do not use the TruClient feature. It has some disadvantages where it requires a lot of hardware capacity, and we are currently not at that level.

Overall, I would rate LoadRunner Enterprise an eight out of ten. We are pretty satisfied with LoadRunner Enterprise, but it would be great if they also added performance engineering capabilities. It meets our current requirements, but there is scope for improvement. In terms of future growth, we are looking for more contributions from the OpenText side.

It's for performance testing of our front-end UI applications, web service systems, APIs, and for the applications that are developed in-house by our solution developers.

We use it as a standalone performance testing tool, not integrating it with anything else. However, I know other clients may integrate it with tools like ALM, which we haven't done.

LoadRunner Enterprise helps identify bottlenecks in our high-traffic application scenarios. It identifies bottlenecks by monitoring slow response times and exposing errors that might emanate from an application under load. 

It's able to expose these factors to us, allowing us to monitor through LoadRunner and see how the infrastructure is coping during the load test. So, we get a lot of insight into how the application is performing under load.

Moreover, LoadRunner was instrumental in scaling our application for peak load testing. 

There have been situations where we were able to expose some performance degradation, especially in our BI and AIU processes, when a change was implemented to an existing application. We would perform a regression test and sometimes find out that whatever changes had been made actually degraded the application's performance. We were able to expose that and report accordingly for remedial actions to be implemented. 

In terms of scaling up the application, especially for new applications where the business expects it to handle a significant load, we find out through testing if it doesn't meet their requirements. Then, we can advise whether the infrastructure needs to be expanded, scaled out, or up, and we implement solutions accordingly until it meets the requirements.

For me, LoadRunner stands out, especially with its reporting capabilities, the graphs that can be generated, and the unique feature of measuring our application's response alongside our infrastructure metrics, such as CPU, memory, or disk usage, all presented in graph form. This is something other applications struggle to match.

The real-time monitoring and analysis feature improved our test accuracy. It significantly enhances test accuracy by providing a holistic view of the testing process. Real-time monitoring allows us to see not only the front-end response times but also what's happening with the back-end systems or infrastructure, making it much easier to identify and troubleshoot bottlenecks.

The tool is user-friendly and easy to learn, which is great for newcomers to performance testing. The challenge comes with scripting, especially for those without a programming background. Understanding code and modifying scripts with conditions requires some coding knowledge. Without it, it's less likely to fully utilize the tool.

One thing that comes to mind is TruClient. It's great for emulating real user behavior on an application, especially when you need to consider how the application interacts with the browser UI. It's like using a UFT script but directly within the browser.

However, the challenge is that TruClient can be very memory intensive. In most cases, clients' infrastructure simply can't handle the resource requirements for running extensive TruClient tests.

Ideally, there would be a way to address this memory bottleneck. We encountered a situation where a simple request-response test with HTTP wouldn't suffice. The user interaction involved a browser layer that needed to be factored in for realistic testing.

If OpenText could find a way to work around the memory issue, that would be very helpful. Here's why: sending a request and getting a response from the server is one thing. But users also want to consider the additional layer of the browser interacting with the application. 

To make the test as realistic as possible, you need to test it with different browsers. And to do that with multiple browser windows open, you really need a protocol that addresses this capability. It exists, but most clients can't handle the resource requirements for running such tests.

We have been using it for ten years. It has been quite some while. We are still using the version from 2022, we are yet to update the system. However, we are planning to upgrade it soon. 

Stability is alright. I would rate the stability a nine out of ten. The only reason I'm not giving it a ten is that sometimes the controller crashes when a test is about to finish. 

It can be frustrating, especially if you run a long test for an hour and it crashes near the end. You can recover the results, but it requires some effort. That's the main stability issue I've encountered. Other than that, it's pretty stable.

So, it can run stably for an hour on some occasions, but on others, it might crash within fifteen minutes.

For OpenText LoadRunner, it's just my team, so about five people in our organization. 

I'd rate the scalability a six out of ten. The main reason is that it's a very expensive application. Other companies might not be able to afford it. 

For example, if we need to test an application with 10,000 concurrent users, the license can cost a lot of money. That's where OpenText tools shoot themselves in the foot compared to other tools. Because of the price, many companies, like one I used to work for, decided not to renew their licenses and switched to open-source testing tools. 

Even though they might not get the same value, it's a cost they're willing to bear. So, the challenge to scaling up is the price.

We haven't encountered any truly complicated issues. Because we know how to use the tool for the most part. There was one time we couldn't resolve an issue ourselves and called support.

Positive

OpenText LoadRunner was the first tool I was introduced to since I started with performance testing.

I've never used anything else.

The deployment process itself isn't complex for us. Typically, the solution teams handle deployments using their own tools. We just test after the deployment is done. We're not really involved in the deployment itself.

We had a team of ten engineers for the deployment process. 

Moreover, the deployment time can vary between a couple of hours and a few days. The only time it might take longer is if there are external dependencies, where another team's deployment needs to happen first. But usually, deployments are a matter of hours, not minutes.

In South Africa, for a load license with about 5,000 concurrent users, the annual license, not including patches, is around 1.5 million to 2 million, depending on the currency exchange. That's a lot of money, especially for startups. 

Sometimes, you might have the license renewed for support, but if the application is stable, you might not need the annual fee. It can feel like throwing money away.

If they can afford it, then I would recommend it. It's a good product. The problem is the price. So, if they have the budget, then it's the best tool you can go for.

Overall, I would rate the solution an eight out of ten because if it weren't so expensive, I'd give it a nine and a half or even a ten. The price is really steep. 

It's an enterprise-level tool, but there are also smaller companies developing applications that need testing, and they might not have the budget for such expensive tools. So, if the price were more reasonable, it would be accessible to any size enterprise, big, medium, or small.

My team does not use it, but we have other teams that use LoadRunner Enterprise. 

We are an SI, and we have a team that does automated testing. They use LoadRunner Enterprise and similar products, and I am continually looking for ways to improve the start-to-delivery time of changes, upgrades, and releases. We have a content server, and automated testing is up there. I have got Selenium. I have got LoadRunner Enterprise to do the automated testing of the active CI/CD pipeline. Now that LoadRunner is an OpenText product, it will make it easier for us to say that our product of choice is going to be LoadRunner. We can drop it in, and off we go and configure it, and then, hopefully, with the link that we have got, we will get this understanding of content suite, Documentum, and digital asset management, and we will get personalized and more appropriate usage profiles and starting points that we would not get if we were using JMeter or Selenium. We would not have to teach this tool about the OpenText stack.

We just finished the upgrade from one version of the code source to another. We have a whole pile of custom code from other vendors, other third parties, and OpenText as well. Doing any upgrades requires a huge amount of retesting. We have to do functional testing and regression testing. Some of that is being done manually, and some of it is being done through automated scripts. We are looking to try and pull that together so that we can get a faster turnaround time. We can also get more test coverage and get it right. We have a three-month window to get a release out, and we can only do a certain amount of manual testing with people sitting there. With the automated solutions, we can leverage a lot more power to push through and get things done, and that gives me the confidence that it works.

We implemented LoadRunner Enterprise for test coverage. With manual testing or other tools, there are only so many tests that we can run in a given time window. There are only so many tests we can do by manually sitting there and without throwing thousands of people out to testing and having consistent results coming back. When we do an upgrade, there is not just functional testing but also things like load testing and performance testing. We are rolling out Azure Information Protection in the next couple of months. One of the things we have to answer before it goes live is if we turn it on and run it in the way our business works, what is the impact when it comes to user concurrency and user profiles? Is it going to work or is it going to collapse under its own weight? Are they going to notice? What is the operational impact? What is the impact that they needed? Is it an extra ten milliseconds or an extra two minutes to upload a document? Being able to model all that and have controlled evidence available for all the different tests is helpful. 

With LoadRunner now being a part of the OpenText family, there is knowledge sharing between the sibling applications. It means that LoadRunner is better positioned or should be better positioned to pick up their other solutions and support them, and we do not have to start from scratch.

We have various customers who use our solutions. Every time we roll out a new integration, change, or upgrade, there are always SLAs. There are always user expectations for how long things take. There is always bug hunting going on in regression testing and making sure that we have not broken anything. Being able to automate those kinds of tests along with the load and performance testing helps reduce the complexity and risk of those activities to the business. With a complex system, there is a risk that we change something over here, and the butterfly flaps its wings, and we end up with something broken over there. We do not see it because we are looking at that new bit over here, and by having a regression test, we can just get a certain percentage done, but with an automated tool, we come across the entire piece. We can identify a suspect and have a look at it. We can see that if we do this combination of data and this combination of settings or users, it works, and then we change this, and it now does not work. We can then fix it. As we roll out new features and functionalities, we can do performance testing. We can do load testing, and we can make sure that for these new features and functionalities, we are ready with the infrastructure to support them. If we roll it out today, and it struggles, then I know in advance that I can put more infrastructure in so that when we turn it on, things work. I can also take it year by year and slot by slot and compare the data and say that my performance metrics were within this, and they have changed. If it comes up towards the SLA or towards the kind of points where the users begin to spot it, I can take action and ask for some extra storage space. If we have 10 gigs a month and a 100 gigs drive, by month eight, we need to order some more space. We do not want to find out two weeks before we get into the tenth month. By being able to see those trends, graphs, and response times for SLAs, but also for general performance and overviews, we can begin to make sure that everyone is heading in the right direction. 

For me, having a wider test coverage—not just functional testing but performance and load testing—means that I have less to worry about because I have more coverage. I can have five guys for a week pressing buttons, or I can have LoadRunner do a regression test suite of x thousand cases in the same amount of time. For coverage, it is a much more useful tool. It has a history. It is great. It is not a new product that OpenText just built themselves. They have got an industry-leading group of people who really understand things, so it is a great addition.

I do not personally use LoadRunner Developer. All the test automation is done by my test automation team. It is very important for me to be able to turn around changes for customer numbers and have things out there. When I started with the OpenText Stack, they were doing one release every two or three years, whereas now, they are doing quarterly releases. When you have got heavily customized systems, or you have got quarterly releases, and you have got all these integrations with SharePoint, SuccessFactors, Salesforce, and other types of applications, the number of things that you have to certify and accredit for testing every time you live is overall a much bigger headache in the connected world. There is just the nuance of all the different variants, data models, and business cases and scenarios that you see, so it is becoming massively unattainable to do that. Having a tool like LoadRunner to do that for you and additionally, having it as a part of the family is valuable. They are on the inside, and they have deep integrations or deep access to all the other OpenText lines of business groups and vice versa. Both sides are going to benefit and grow from that close integration. It just allows me to be able to say that I can shift left because I have the confidence that automated testing has done all my use cases. I have got my performance data. I have got my load data, and I am more within SLA. Trying to do that manually or with some other tools takes a lot longer to get to that, and we do not always get the complete coverage that we want. There is always some risk around how many test cases you can manage running in a week versus just running a tool to plow through them.

Most of my test automation guys use the scripting engine and some of the drag-and-drop features depending on what they are doing, such as building the usage profiles and the automation for the things that they want to do. The tool is shortening the amount of time it takes us to build those automated test use cases. We have use cases for adding a document as a user, as a records management user, as an admin, as someone who has sysadmin rights, or as someone whose first language is French. We get all of these little nuances of access and permissions and everything else, and it becomes interesting. Being able to use tools to deliver all that makes life easier for me. 

The more I spend time with these tools, the more I understand not just the technical benefits but also how they benefit the end-user community. I understand how they can be used to deliver business goals rather than just technical metrics and help shape what we do and give a reflective user experience. We get to compare what our users do on that client system, and we can quickly build user profiles for that particular client rather than throwing 200 generic requests. We do not work like that because then you end up with incorrect data, and you make wrong designs.

LoadRunner Enterprise has helped streamline our testing processes. My team is primarily focused on the content management space where we have the cadence of quarterly releases. We have our own custom modules, and there are custom builds that we have done for customers. We are quickly able to do regression testing, performance testing, load testing, and new feature testing. We are able to test all the different permutations of user location and object types. We are able to test different variants such as adding a document and then adding an email here or there. We are able to package that part and run it as a part of our deployment pipeline, which allows us to align more closely with clients. We are not a year or two behind of where they were, which used to be the only way of doing it previously. It also assures us that even though we are going faster, we are actually doing more while going faster. We are not going faster and doing less. We are now getting more value.

LoadRunner Enterprise reduced our workload or made our life easier. They are a pain to set up, but we have to invest the time to set up the profiles for the users in terms of what the users do on the system. Do they add documents? Do they read documents? We have to understand what those profiles look like so that we can replicate what the client would do, but once we get that and we run them, I know that they reflect my client. They are an accurate reflection of what the client does. I know that the regression testing is going through all the historic and new things, and everything works. With the performance or load testing, I know that as I turn on any feature in prod, which is always different from every other environment, it is going to work. It is not going to collapse under its own weight. It is going to be performance within SLA and within expectation for end users without causing too many problems, so it becomes a useful armory to be able to say to the business that this is going to work. We are able to model situations where we turn something on, and it does not work because we do not have enough infrastructure. We turn it off again and put more infrastructure in, and we turn it on again and see if it works. Being able to model all of that and find thresholds or breakpoints or being able to prove that we can get the throughput makes the conversations easier. It makes the journey smoother as we do things.

LoadRunner Enterprise has helped improve our product quality.

For me, the test coverage and the performance and load testing aspects are valuable. I have a test management team that works for me. They are the guys who are writing JMeter and Selenium scripts and custom labs to do all the testing. I do not get overly involved in that. My role is more about whether the overall solution is going to work. Is the overall platform going to work? Are we within SLA? Are we within functionality? Also, when we do any upgrades or releases, I have to certify to the client. I need to know how many scripts we have run. If we have done only fifty because we only got two weeks, I need them to do more. Having automated testing that we can run and have it consistent every time, and that can run quicker and more efficiently is very important. I get that level of performance, load, and regression functional testing that makes it easier for me to sleep at night.

After they get over the acquisition, the first improvement is going to be tailoring it for their existing stack of other products. How would LoadRunner work for Documentum? How would it work for Business Network? How would it work for other apps? They can have a pre-package or a guide because they are all in the same family as opposed to being outside. When I spoke to the LoadRunner guys, I said, "You are not a part of the family. You have a simple content server test pack, and you have some accelerators that will get me going with your LoadRunner on another OpenText product." They said, "We just started building those because we have just formally become part of the family." That is going to come out. That is going to add value as I roll out LoadRunner and put it on the mix for other customers and new prospects. When those accelerators are there, I would trust them more. I trusted it previously when they were outside or not inside the box. Now that they are both inside the box, I am expecting synergy to be there.

It is a stable product, but there are always going to be things when you introduce something, when you find a curveball that causes a problem, or when you bump into something that does not work. We just flag it up to them. Most of my guys have been around a while with various of these products, so we are able to dig in and help or support the investigation process that the support guys are doing from outside and feed that information into the support process, the R&D bug-fixing process, or the new feature request process. The guys are getting as much help as they can from us. At the end of the day, the more my guys can do to help out the OpenText guys, the better product or support we get. We get a quicker response, so we are helping OpenText support to help them help us.

Generally, if I say that something does not work and I have an error, I get an email asking for logs. I send the logs, and then they ask whether I did that as a sysadmin or as a normal user. A couple of days go by in me checking that, and it goes back in. After that, they ask about the version of patches that we are running. These are the common things that we get asked, so now, we just tick those boxes when we hand it over to OpenText support on day one so that we do not have this kind of delay in the process. We also keep looking at it because OpenText might get something working, or they might not be able to see anything with it because we might have a customized solution or we might be a slightly different version or something like that. The more we can help them and feed it back, the more information they have to zero in on where that problem is. They can come back and say that the last little bit we gave them got them on the spot, and they have now found the bug, and they can fix it. We will get that in this release, and they can hotfix it back for us. Their help is great. I am an engineer, so I always want more documentation, more support articles, and more architecture models so that I can understand more about the solution. It is good to see some of that stuff coming out as being able to be leveraged.

The products do scale. We have some very big customers. Where we begin to struggle with scalability is generally not within the OpenText purview. The bottlenecks we see when we do large bits of work are in the database or in the network because we are just trying to throw some new requests. It does scale, but we just have to scale sensibly rather than just throw more infrastructure at it. Sometimes, you are not solving the right problem. For example, if a customer wants to create three million business workspaces in six months, I ask them how many they need on day one, or which ones would they work with on day one. We load that set first. We then ask them which is their last set, and they say that they have records for people who are no longer account holders, so they want them available, but their account execs and phone guys are not going to be looking at those on day one, day ten, or day twenty. We then move them last, so we break the problem down, and then we look at the tooling that is there. The good thing and also the bad thing is that there is always more than one way to solve a problem. There are tools. There are REST APIs, and there are prebuilt solutions. We need to work out exactly which is the right approach and the right tool to do it. This is where people at OpenText and other customers come in because you can just ask the question about what you want to do. If someone has done it before, they can give you some input to try this or that or look out for this because they bumped into this. You get that kind of community help and support. You certainly get help from OpenText support as well. 

They are very good when you get the right team and the right person. When I started out, there were one or two products. The first line generally had a good understanding of those one or two products, so you get a lot of things answered straight away, whereas now, sometimes, it takes a while because the first-line guys ask you if you are a Documentum guy, and then they have to get the information to pass on to the right person. Therefore, sometimes it takes a while, but once we find the right person, the support has always been great. They are helpful and supportive.

We use a mixture of products based on industry-leading products and also based on our clients' preferred tool. We are an SI, and we do a lot of work with clients. Client X might say that they are a JMeter shop, and this is what we should use because getting anything else accredited requires a separate process. The next client used LoadRunner or Selenium, so we tend to be testing-technology agnostic. However, now that it is a part of the OpenText family, if I have a say in the matter and if they are on an OpenText program, I can say to clients that it makes more sense to go for LoadRunner. I want to pick the product from the same family rather than picking something that is on the outside looking in and does not have that kind of synergy. 

We have seen an ROI. For me and my teams, using automated testing tools from whichever vendor is saving us time and money and improving quality. It allows us to do all the regression testing and get performance data, which makes us able to spot bottlenecks and problems in what we are rolling out. We can see that something works at this scale, but when we try and scale it up, we start seeing issues or bottlenecks that we have to fix before we give it to a customer.

I do not deal with pricing and licensing. My job is to make the engineering work. 

I would rate LoadRunner Enterprise a ten out of ten. It is a product that I have known as being a market-leading tool in that space for years. Being more of a software engineer and architect, testing tools are not something that I am intimately familiar with, but everything I am hearing from the guys that do that work is that it is the tool to use. It is great to have it in the OpenText family.

My organization uses OpenText LoadRunner Enterprise for anything related to browsers and the web. We have started using shift-left for applications. Costco primarily uses it for testing its various e-commerce applications and websites. We use the product to address challenges in performance testing within the applications.

It is our primary performance testing tool. There was a period when we weren't doing a heavy level of performance testing. It led to a couple of outages of our website, and we had to go ahead and slam LoadRunner into production to address problems with performance. We haven't had an outage since we started using the solution.

We use all the features in the solution. We use it for performance testing and load testing. We have been hearing more about third-party integration with chaos testing. It's certainly something we'd like to do more within Costco. We have also started implementing shift-left and DevWeb.

The tool must improve the ease of gathering and reporting when larger datasets are generated. It must provide better and more direct information. There's a lot of digging around to try and address issues when they arise. Anytime we bring up a window, and it's waiting to do something, it must give a visual indication that it's doing something.

I have been using the solution for four years.

There's a window where LoadRunner is very, very stable, but we're pushing that window hard. For our use case, the tool's stability is a five out of ten.

We struggle a lot with the tool's scalability. When we start using hundreds of load generators and generating tons of data, we find a lot of difficulty in managing the systems. We are wondering what we can do to deal with scaling out the infrastructure as it continues to grow within Costco. The product's scalability must be improved. For our use case, the tool's scalability is a five out of ten.

We have had issue issues with the support in the past. It's been improving, which is nice. I had some wonderful conversations with people at the event, so I hope to continue seeing that. The more communication we get, the better it is.

Neutral

The initial setup is complex. Our implementation of LoadRunner Enterprise is incredibly complex. We use a number of third-party tools to enforce and manage its continued running outside of what LoadRunner provides.

Anything that can stop something that produces a million dollars a day from breaking is good.

There are a lot of negotiations going on right now about the product's pricing. There is a bit of unhappiness on that aspect and how it's currently being presented versus how it was presented previously.

LoadRunner Developer’s shift-left process is becoming more and more important for us. It's a newer initiative for Costco. We are pushing towards it. We already have teams that have started doing lower-level performance testing. It'll continue to get adopted more and more as we push it out.

We are using the solution's TruClient feature. I work in administration. The people who work on scripting have some technical issues with TruClient's scripting. There are a number of scripts out there. LoadRunner Enterprise has helped streamline our testing processes. It is the only method of doing some of our testing processes. We find it very valuable.

Streamlining our testing processes has helped us through further improvements and validation that our systems are performing at the high level that we need them to perform.

LoadRunner Enterprise has reduced our workload. Without it, we wouldn't have performance testing, and things would break a lot more. LoadRunner Enterprise has helped improve our product quality. If it weren't there, things would break. The fact that we're still using LoadRunner is a pretty good sign.

LoadRunner Enterprise has helped us save time. Without it, things would break a lot more. Our primary experience with performance testing is with LoadRunner. I have played around with some other applications, and all of them have ups and downs. LoadRunner provides ease of use and a nice and concise interface.

Overall, I rate the solution a seven out of ten.

We are from a banking background and have a desktop application that follows the complicated .NET protocol. It has a lot of encryptions, and we use the solution to do the scripting. 

We use the Enterprise version because it's not possible to do scripting with open source. We initially started with Professional and then moved to Enterprise. 

OpenText LoadRunner Enterprise has improved the team's productivity. The results that I produce after the test help my team. 

The tool's most valuable features are scripting, correlations, and parameterization. Debugging is also easy. 

The debugging feature needs to include graphs. 

I have been using the solution for two years. 

The stability is good.

The product is scalable. 

The tool's support is good. Though they take time, they always fix my issues. 

Positive

We used JMeter before OpenText LoadRunner Enterprise. We switched to OpenText LoadRunner Enterprise because JMeter doesn't help to test encryptions and decryptions. 

OpenText LoadRunner Enterprise is worth its money. 

We used the Professional version and then moved to the enterprise version. We have subscribed to 1000 user licenses. The tool will be super expensive if we take up 5,000 user licenses. We have to limit ourselves on testing. 

OpenText LoadRunner Enterprise helped us overcome debugging and scripting challenges. It has enabled us to manage projects. We have a place where we can put our results and scripts. It has streamlined our testing process. The tool has improved productivity and result quality. 

I use LoadRunner Developer. The tool reduces and makes our lives easier by recording the workflow and helping us identify the correlation. 

LoadRunner helps us save time. We can create the scripts for dynamic parameterization and automatic correlation in two hours. Earlier, it used to take five to six hours. 

I rate the solution a nine out of ten. 

I use OpenText LoadRunner Enterprise to ensure the stability and performance of various technologies and platforms. We ensure that the platforms are performant and are able to run at the speeds the business requires. We also wanted to ensure stability as the code moves through our environment.

OpenText LoadRunner Enterprise has allowed us to reduce incidents and focus and ensure code and quality are built into our delivery.

OpenText LoadRunner Enterprise supports a number of protocols. The number of automation built into the platform, specifically in the analysis area, is valuable to us.

The process of upgrading LoadRunner can be difficult and time-consuming.

I have been using the solution for ten years.

The solution's stability is fine.

The tool is very scalable. We have had no issues.

Getting support sometimes is a struggle, and it takes a long time to get cases resolved. The support could be improved.

Neutral

Previously, we used LoadRunner Professional.

The deployment was complex. It is very important to us that LoadRunner offers flexible deployment options. I want to be able to deploy the solution in a fashion that is in the best interest of my organization.

We used a consultant to deploy the tool.

We have seen an ROI. The product is worth the money.

The tool is very expensive. It's probably one of its biggest weak points. As a capability manager for performance, it's a battle every year to substantiate the cost. There's an opportunity for OpenText to help us educate people who are not in the field.

We evaluated NeoLoad, Ranorex, JMeter, and the Silk portfolio. We chose LoadRunner because of the breadth of support. No other tool had the breadth of support.

We use the TruClient feature. We use TruClient for understanding webpage rendering. It's TruClient's strong suit. TruClient has been very important to us. Without it, we won't understand the JavaScript rendering the users see on the front end. TruClient is a great option if using standard HTTP is either unsuccessful or time-prohibited.

LoadRunner Enterprise has definitely helped streamline our testing processes. We're using it almost 24/7 to support a host of platforms across the enterprise. If we didn't have it in place, the organization would have hundreds of performance-related incidents every year. However, we don't have them because we're catching them.

Streamlining our testing processes allows us to meet business demands and keep up with very aggressive schedules. LoadRunner has a lot of capability straight out of the box. One of the best things about it is the automation that's built into it. It has a lot of automation and functionality that many other tools don't have.

LoadRunner Enterprise has helped me save time. LoadRunner Enterprise has helped improve our product quality by ensuring transparency between product, delivery, and test. With load testing, we can accurately inform the business of how the platform is doing. By doing so, we can ensure that expectations between product, business, and IT are aligned. If they're not, those issues can be addressed before we go to production.

Overall, I rate the product a nine out of ten.