Apache JMeter Reviews

Apache JMeter is quite flexible and it is also well distributed. It is quite flexible compared to Micro Focus LoadRunner.

JMeter is easy to script. There is less of a problem with doing correlations and parameterization. 

It is not something that can be compared with Micro Focus LoadRunner. It gives the facility too easily; you do things through UI. With JMeter, you really do not have any easy UI to work as, like a Micro Focus LoadRunner.

The stability could be a bit better.

Compared to LoadRunner, it hasn't any proper UI. Recording the script is also not flexible in JMeter. In LoadRunner, we have a couple of options, such as URL-based recording and HTML-based recording. In JMeter, it's not like that. JMeter has a recorder, however, it is not easy to use. It is a bit tricky to configure the automatic recording in JMeter. 

I've been using the solution for four or five years. 

JMeter, stability-wise, is good, however, it is being developed by the community. Therefore, stability is always an open question there.

The solution can scale a bit. It is scalable, however, not like LoadRunner. I have not tested it as such yet. I'm not sure about how fully scalable it is. 

I'm also familiar with Micro Focus LoadRunner.

The implementation process is not so easy. It's difficult to configure.

I'd rate the solution a seven out of ten.

I'm an end-user and a customer. 

We use Apache JMeter for the load generation scripts for performance testing.

The thread groups, samplers, and listeners, which are all determined by the script's requirements, are the most valuable features of this solution.

We use many plugins to customize our scripts, which is its main purpose. We wanted to be able to use a larger variety of customizable plugins to meet our needs. Along with our, JMeter, you would use a variety of plugins.

The number of customization plugins should be increased.

There could be improvements in terms of memory utilization. We are going to migrate away from JMeter in the near future.

The data storage should be improved.

Scalability could be improved.

It should support more protocols.

I have been working with Apache JMeter for three or four years.

We use version 5.0

We are using the cluster mode because one node is definitely not enough. Scalability can be improved.

When compared with other solutions, it's not as good, which is the reason we are moving to another solution.

Scalability must improve.

Generally, we use open forums, to resolve any issues we may have.

I also work with BlazeMeter.

I worked with Apache JMeter, from the beginning.

The initial setup is straightforward.

We have a small team to maintain this solution.

We completed the installation on our own. It was completed in-house.

Apache JMeter is an open-source solution.

We don't use the paid version of this solution.

Everything is included, and there are no additional costs.

I would suggest that instead of using a GUI-based implementation, try to make it code-based. Try to replicate the configuration. The plug should be job-ready, and ready to be integrated as well. Rather than having a UI, and limitations.

Three or four years ago, I would have given it a seven or eight, but now that there are more powerful competitors, I would give Apache JMeter a five out of ten.

I mainly use JMeter to capture the traffic of the most-visited page to see how much load a particular page is getting and how many users are using that page for a particular amount of time. I've also used it to capture APIs for particular pages.

The most valuable features are the ability to capture the entire traffic of particular pages and the proper readability of entire pages and entire APIs.

One of the drawbacks of JMeter is that it can't handle a large amount of load, which forces us to switch to other tools when we need to load more than a 5,000 or 10,000 user load. In the next release, I would like JMeter to be more compatible with other languages in the market.

I've been using JMeter for six to eight months.

JMeter can't be used in the long run, so I'd rate its stability as five out of ten.

I would rate JMeter's scalability as seven out of ten.

Apache's technical support is pretty good, I've had no issues with them.

The initial setup was straightforward and took about five to six months.

I used an in-house team.

JMeter is open source, so there are no licensing costs associated with it.

I evaluated SoapUI, Postman, and Visual Studio. JMeter was more reliable compared to these options.

I would definitely recommend JMeter in terms of usability. If you're using AngularJS as a language for testing UIs, JMeter might not be a good idea. I'd rate this solution as six out of ten.

On-premises

We are in the service industry. We implement it for our customers. We recommend the right tool and set it up for them. So, I've not had any hands-on experience in my current role, but I have a good understanding or a fair idea of the tool's capabilities. I have a team that takes care of the technical aspects.

It is an open-source solution. So, typically when you don't want to make a heavy investment, and you want to do some level of performance testing, Apache JMeter is used. 

It is typically on-premises, and it has also been on the public cloud. It could be Azure, or it could be AWS. It is very rarely on GCP.

It helps them to look into several parameters. For example, when you have certified test cases that are predominantly repeated on an application, you can execute the same thing with increased load. You can see how the application responds and if there is an impact on the response time of the application. You can confine it to certain parameter conditions and then start making changes to see how it performs. You can see where the RAM or CPUs are stagnant and not increased.

The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features.

It is easy to use. If you want to test your application out and not incur a lot of costs, it is probably the best tool.

Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports.

I have been using it for about four to five years.

JMeter is probably good for lower loads. It is not comparable to LoadRunner when it comes to higher loads.

It probably scales up to a few thousand users but not beyond that.

Because it is an open-source community, their support is probably average. It won't be like the support for a commercial product. I would rate it a six out of ten.

LoadRunner is one of the prominent tools. It was formerly HP, and now it is Micro Focus. It has good capabilities and features. It also has decent reporting capabilities. Because of the brand and the capability, it was probably chosen by most of the Fortune 500 clients that we work with. There are also some startup communities or organizations that ventured into other solutions, such as JMeter.

In terms of comparison, primarily, there are three to four parameters. The first one is the ease of use. The second one is about the protocols that need to be tested, whether it's web or API, HTTP, HTTPS, and all the native things. The third one is in terms of flexibility in setting it up and executing, and the fourth one is in terms of monitoring the execution and reporting pieces. Those are the key parameters for pros and cons. LoadRunner gives you a lot more capability and flexibility, but at the same time, it also consumes a lot of resources. JMeter is relatively simpler, cheaper, and easier to use.

I don't have hands-on experience with it, but based on what I have heard from people, it's pretty straightforward in terms of the setup. 

The setup probably takes a week or two, and then the execution is probably a three to four weeks exercise.

We are in the services business. Clients give us access, and we set it up there.

It is certainly good for testing out the applications for performance testing, especially when you have to test them out frequently and make sure that they are good for at least a few thousand users. It has a decent ROI.

It is open source. There are no licensing costs associated. If you need enterprise support, you'll probably end up paying for a license.

You would also factor in the infrastructure cost, but that's not significant.

I would rate it a seven out of ten. It is a decent choice from a small-scale perspective, but reporting could be better. If you want to get some performance testing done without spending money, JMeter is probably the best tool. It doesn't have the best reporting, but it is quite a handy tool.

My main use case of JMeter is for web application performance testing as well as for API performance testing. We are customers of Apache. 

The benefit of JMeter is that it does our performance testing and provides a report without the need to spend money on a licensed tool. It's a significant benefit for us and for the project we're currently working on.

The fact that the solution is open source makes a big difference as we're working on a low-budget project. It's quite user-friendly and easy to use.

There are issues with the plug-ins which you need for reporting purposes as they make the reports quite heavy so you have to run them in non-GUI mode. If you go above the 200 user mark, the application creates a bottleneck and that's one of its major drawbacks. It means you have to run with a master-slave configuration with one system being the master, and multiple slave systems. It's not ideal and I think it could be simplified with a UI that provides direct configuration. In addition, the solution doesn't support SIP applications and some other protocols.

JMeter is not designed for high loads, if you overdo it the tool becomes a bottleneck. Unless you're using JMeter in multiple systems and all the systems are connected through a particular LAN, there is a limit.

Because it's freeware there's no official tech support but you can raise a request on the JMeter site. We haven't had many problems. 

We were previously using LoadRunner for this project but moved to JMeter because it's a freeware testware.

The initial setup is quite easy. There is no deployment process, you download a file from the web application service from your JMeter site, and you can use it. Any user with a little bit of knowledge can do it. The download takes about 15 minutes so you can be using JMeter in about half an hour. From time to time there are version upgrades but they don't affect existing assets. We have four people using JMeter and they don't have any issues. 

BlazeMeter works on top of JMeter and there is a small cost factor to purchase that. It basically gives a slightly more advanced JMeter. 

If you're working on a low-budget project and don't have dependencies of a huge number of users then this is the perfect tool. If you have 2,000 or 3,000 users then it's probably best to look at other options.

I rate this solution eight out of 10. 

On-premises

We are in the financial industry in India and carry out performance testing. We deal with  SWIFT messages and financial messaging systems. A few years ago we had 2 million transactions in five hours; today it's 2 million in two hours. To measure these performances, I use JMeter for regression. We have Exadata, Exalogic infrastructure. We are users of JMeter and I'm a technical specialist. 

This tool is very user-friendly and easy to use. It's open-source so there are no costs involved for the non-production environments. It's easily available to anyone who wants to use it and it has all the features required for performance metrics.

I think it has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems when we are working in certain specific environments that have a proxy setup. When we want it to do a record with some new scripts, there are some challenges there.

I've been using this solution for two years. 

The solution is stable and easily scalable. 

The support is very good, with good technical teams. Whenever we get stuck they support us and provide solutions. 

The initial setup took only 10 minutes but there were some complications with the product initially having less features. That has been greatly improved. It's easy to do maintenance of the test tools which we develop, and it's easy to maintain as a product. We have around 20 users of this solution. 

It's open-source, so there's no cost. If you need some support from the vendors there will be a charge for that. 

I would recommend this solution. Most of the support, guides, and tutorials are available on YouTube. 

I rate this solution an eight out of 10. 

On-premises

I've used JMeter in conjunction with Selenium, Java, and Log4j for logging. I used it before
I ended that contract in August. Its version was up to date at that time.

It was used for an e-commerce site that is specialized in C-PAP or weaving machines in effect. Their max was a thousand people logged in at once. I, of course, pushed the boundaries on that, but it was to test the performance of the website, and of course, I'm had to try subsystems, database interactions, etc.

I'm a total geek, so I liked the fact that I got to program. The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable.

I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this.

If I were going to be Mr. Selfish and say anything I want, I'd say a full feature GUI that lets me drag and drop different modules in line. It could have a simple-to-use GUI.

I have been using this solution for probably a year and a half.

I didn't have any issues with the stability of JMeter itself. There were definitely issues with the program I was testing, but that's why I was testing it.

It was very easy to scale, but I was barely scratching the surface. I have spent 17 years at Microsoft, and for the performance testing that we did there, we had 8.3 million users at once, as opposed to a maximum of a thousand. If I'm scaling, I have to do it quite straightforward and simple, but it was very minimal.

Only I was using it. It was the QA department. I showed it to some of the devs, and they were very interested. A couple of them tried it, but none were actually using it day-to-day for testing out the environment.

I would rate them an eight out of 10 because sometimes, they would take two or three days to get back to me. Of course, at that point, you're like, "I need the answer; I need to answer." So, it was a little bit unrealistic in terms of expectations.

At that particular company, I was the one who was tasked with coming up with the solution, so that was the only one that I looked at simply because JMeter is industry standard, but at Microsoft, they wrote their own custom tools, so I used custom Microsoft tools.

It was rather complex. It is a complex product, but that part of it was very well-documented. I didn't have any problems with it.

Don't be shy in asking questions. Google/Bing is your friend. It is complicated. There's no reason to spend eight hours trying to figure out something, except unless you are trying to learn in-depth. There are a lot of people who've done exactly what you're trying to do, and it doesn't matter what it is.

I would rate it a 10 out of 10 because it is industry standard. It did everything I could've asked. I barely scratched the surface, but what I needed it for, it did well and in a very straightforward-to-implement way.

On-premises

We use Apache  JMeter for API testing. Our latest task combines test cases with our CI/CD pipeline for deployment to Kubernetes. Tests are developed locally and automated for the build using standard Unit and Integration testing techniques. Once deployment is successful to Kubernetes, we develop the JMeter tests for automation in promotion scenarios. Thereafter, once a deployment is successful, we can automate promotion to the next level based on test success.

Test automation moves our organization close to rapidly deploying products. Unit and Integration testing is easy to automate, and most organizations perform these as part of their day-to-day operations. However, end-to-end testing, smoke testing, load testing, and performance testing are much harder to automate. Apache JMeter has aided in that challenge.

While there is a User Interface, the scripting ability is highly beneficial and is easy to use. Tests can be added to a CI/CD Pipeline for integration with testing and deployment scenarios once finalized and operational.

This is a difficult question to answer. On one side, JMeter is very flexible and allows for a high amount of customization. On the other, some tasks are common enough that it merits simplifying the process.

Authentication for API testing could use improvement. Currently, it is a multi-step process to call, extract, and utilize a bearer token securely for API calls. This process is becoming a common enough task that a "wizard" for creating and consuming popular authentication models is merited.

I have been using this solution for about six years.

Apache JMeter is stable, and I personally have not encountered any issues. Depending on the size of test runs, one might need to adjust their JAVA settings to align with the test requirements.

Its scalability works. It is a typically Java run. Therefore, it is limited only by what you can do in Java in terms of scalability.

Developers write tests, verify tests, and maintain tests using version control. They identify and tag each to ensure they are appropriately labeled for test purposes (E.g., unit testing, integration testing, performance testing, and the like). Unit and Integration test coverage is normally high. However, we require testing from outside of the system, and JMeter allows us to create tests automating this process.

Apache JMeter utilizes community support. It is well-documented and has an active community. As far as I know, there is not a "pay-for-support" option.

I have used Postman in collaboration with other developers. However, I prefer Jmeter only out of personal familiarity and not for any technical deficiencies of Postman.

Apache JMeter setup is easy. However, there is a medium-to-heavy learning curve for developing tests and getting started using it for practical uses. Depending on its intended uses, there could be a significant configuration task for a given set of tests.

Apache JMeter is under Apache License, Version 2.0 licensing. Understanding licensing requirements is important for the implementation of any tool.

Understand the use case. Choosing the correct tool for any task is always a challenge. Jmeter offers a significant amount of flexibility and will work for a lot of solutions. Jmeter requires a commitment to learning for optimal operation; without that investment, tests may not yield the appropriate outcomes.

Public Cloud

Amazon Web Services (AWS)

It is specifically used for performance systems. It is used for identifying the areas where we need to improve the application bottlenecks and for load testing. We are using its latest version.

It has helped us to build robust application cater to the learning domain and identify bottleneck prior go live. It helped us refine our deployment strategy and capacity planning.

It is open source as well as relatively extendable. It allows us to extend and add additional functionality and features. Its deployment is also very easy.

It should start supporting the presentation layer. It currently provides performance testing specifically at the application and API level. It can be extended to the presentation layer, which includes mainly Angular and React frameworks. 

It should also be easy to use and easy to train people.

I have been using this solution for more than ten years.

It is stable.

It is scalable. It allows us to extend and add additional functionality and features.

We have around 10 to 15 people who use this solution.

It is open source, so I don't think any support is available.

Load Runner, replaced with JMeter due to lower ROI

Its deployment is easy. It didn't take much time. It took less than 15 minutes.

We deployed it on our own.

1. Scalable Product and solutions

2. Plug and Play with CICD process

3. Reduction in licence cost 

No Licensing cost for JMeter

Yes.

I would recommend this solution. We plan to keep using this solution.

I would rate Apache JMeter a seven out of ten.

Private Cloud

When I was last using JMeter, we were simulating 200 concurrent users and evaluating performance based on transaction times. We were defining SLAs based on the results.

Essentially, we created load scenarios and testing different ones using different workload models.

The recording and playback functionality is helpful.

The reporting is not very good.

When we run with multiple users, it takes a lot of memory.

With respect to the recording and playback functionality, the auto-correlation parameterization is not easy and should be improved.

I have been using Apache JMeter for about four years.

There are issues with stability when running with multiple users because it consumes a lot of memory.

Scalability is fine, although it is important to remember that JMeter doesn't run on its own. It needs to work with load-generations such as BlazeMeter. LoadRunner is the same in that you need a cloud-based infrastructure to run it.

There is no official support. There is a forum where you can ask questions and they respond to you, but the technical support that we have with LoadRunner or NeoLoad is not available.

I have used many similar solutions in the past such as New Relic, AppDynamics, NeoLoad, and Micro Focus LoadRunner Enterprise.

JMeter is not as good as LoadRunner or NeoLoad, and it isn't as easy to use, but it's okay because there is no cost. LoadRunner is too expensive, in my opinion. NeoLoad is cheaper, although not significantly.

From what I have seen, many companies are adopting JMeter because it's free. Especially in Canada, using JMeter seems to be the new trend. Some companies are choosing NeoLoad over LoadRunner because it is easier, faster, and cheaper. Whatever they need to do can be completed quicker. The main problem with NeoLoad is that obtaining resources is harder.

Given all of the choices, my preference would be to implement NeoLoad.

The initial setup is straightforward. I would not say that it is complex and if you already have the file downloaded then it will only take about half an hour to deploy.

I took care of the deployment myself.

I was using the free version of the software.

My advice for anybody who is considering JMeter is to just install it and try it. Creating scripts is a different process when you compare it to LoadRunner or Neoload. There is different terminology compare to these two products, so if somebody has not used JMeter then it may seem difficult at first.

I would rate this solution a six out of ten.

On-premises