Apache JMeter Reviews

I've used JMeter in conjunction with Selenium, Java, and Log4j for logging. I used it before
I ended that contract in August. Its version was up to date at that time.

It was used for an e-commerce site that is specialized in C-PAP or weaving machines in effect. Their max was a thousand people logged in at once. I, of course, pushed the boundaries on that, but it was to test the performance of the website, and of course, I'm had to try subsystems, database interactions, etc.

I'm a total geek, so I liked the fact that I got to program. The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable.

I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this.

If I were going to be Mr. Selfish and say anything I want, I'd say a full feature GUI that lets me drag and drop different modules in line. It could have a simple-to-use GUI.

I have been using this solution for probably a year and a half.

I didn't have any issues with the stability of JMeter itself. There were definitely issues with the program I was testing, but that's why I was testing it.

It was very easy to scale, but I was barely scratching the surface. I have spent 17 years at Microsoft, and for the performance testing that we did there, we had 8.3 million users at once, as opposed to a maximum of a thousand. If I'm scaling, I have to do it quite straightforward and simple, but it was very minimal.

Only I was using it. It was the QA department. I showed it to some of the devs, and they were very interested. A couple of them tried it, but none were actually using it day-to-day for testing out the environment.

I would rate them an eight out of 10 because sometimes, they would take two or three days to get back to me. Of course, at that point, you're like, "I need the answer; I need to answer." So, it was a little bit unrealistic in terms of expectations.

At that particular company, I was the one who was tasked with coming up with the solution, so that was the only one that I looked at simply because JMeter is industry standard, but at Microsoft, they wrote their own custom tools, so I used custom Microsoft tools.

It was rather complex. It is a complex product, but that part of it was very well-documented. I didn't have any problems with it.

Don't be shy in asking questions. Google/Bing is your friend. It is complicated. There's no reason to spend eight hours trying to figure out something, except unless you are trying to learn in-depth. There are a lot of people who've done exactly what you're trying to do, and it doesn't matter what it is.

I would rate it a 10 out of 10 because it is industry standard. It did everything I could've asked. I barely scratched the surface, but what I needed it for, it did well and in a very straightforward-to-implement way.

On-premises

We use Apache  JMeter for API testing. Our latest task combines test cases with our CI/CD pipeline for deployment to Kubernetes. Tests are developed locally and automated for the build using standard Unit and Integration testing techniques. Once deployment is successful to Kubernetes, we develop the JMeter tests for automation in promotion scenarios. Thereafter, once a deployment is successful, we can automate promotion to the next level based on test success.

Test automation moves our organization close to rapidly deploying products. Unit and Integration testing is easy to automate, and most organizations perform these as part of their day-to-day operations. However, end-to-end testing, smoke testing, load testing, and performance testing are much harder to automate. Apache JMeter has aided in that challenge.

While there is a User Interface, the scripting ability is highly beneficial and is easy to use. Tests can be added to a CI/CD Pipeline for integration with testing and deployment scenarios once finalized and operational.

This is a difficult question to answer. On one side, JMeter is very flexible and allows for a high amount of customization. On the other, some tasks are common enough that it merits simplifying the process.

Authentication for API testing could use improvement. Currently, it is a multi-step process to call, extract, and utilize a bearer token securely for API calls. This process is becoming a common enough task that a "wizard" for creating and consuming popular authentication models is merited.

I have been using this solution for about six years.

Apache JMeter is stable, and I personally have not encountered any issues. Depending on the size of test runs, one might need to adjust their JAVA settings to align with the test requirements.

Its scalability works. It is a typically Java run. Therefore, it is limited only by what you can do in Java in terms of scalability.

Developers write tests, verify tests, and maintain tests using version control. They identify and tag each to ensure they are appropriately labeled for test purposes (E.g., unit testing, integration testing, performance testing, and the like). Unit and Integration test coverage is normally high. However, we require testing from outside of the system, and JMeter allows us to create tests automating this process.

Apache JMeter utilizes community support. It is well-documented and has an active community. As far as I know, there is not a "pay-for-support" option.

I have used Postman in collaboration with other developers. However, I prefer Jmeter only out of personal familiarity and not for any technical deficiencies of Postman.

Apache JMeter setup is easy. However, there is a medium-to-heavy learning curve for developing tests and getting started using it for practical uses. Depending on its intended uses, there could be a significant configuration task for a given set of tests.

Apache JMeter is under Apache License, Version 2.0 licensing. Understanding licensing requirements is important for the implementation of any tool.

Understand the use case. Choosing the correct tool for any task is always a challenge. Jmeter offers a significant amount of flexibility and will work for a lot of solutions. Jmeter requires a commitment to learning for optimal operation; without that investment, tests may not yield the appropriate outcomes.

Public Cloud

Amazon Web Services (AWS)

I am using this solution for a banking use case. We have an older version of this solution. We are planning to upgrade to a newer version.

A lot of things are valuable. It is free. It has a lot of features, such as report generation and integration with CI/CD, which makes it very competitive with the other paid solutions available in the market. It is a good solution.

They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. 

In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. 

They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available.

I have been using this solution for almost three months.

It is stable. There is good support, and fixes are also available immediately.

We have not scaled it, but it is scalable. We plan to increase its usage. 

I didn't have to call Apache because if anything comes up, we are able to find a solution in the community.

Its initial setup was straightforward.

We are using the free version, and if required, we can easily switch to the other version.

I would definitely recommend this solution if it suits a use case. It is good, and it also looks promising. We plan to keep using it in the future.

I would rate Apache JMeter an eight out of ten.

On-premises

It is specifically used for performance systems. It is used for identifying the areas where we need to improve the application bottlenecks and for load testing. We are using its latest version.

It has helped us to build robust application cater to the learning domain and identify bottleneck prior go live. It helped us refine our deployment strategy and capacity planning.

It is open source as well as relatively extendable. It allows us to extend and add additional functionality and features. Its deployment is also very easy.

It should start supporting the presentation layer. It currently provides performance testing specifically at the application and API level. It can be extended to the presentation layer, which includes mainly Angular and React frameworks. 

It should also be easy to use and easy to train people.

I have been using this solution for more than ten years.

It is stable.

It is scalable. It allows us to extend and add additional functionality and features.

We have around 10 to 15 people who use this solution.

It is open source, so I don't think any support is available.

Load Runner, replaced with JMeter due to lower ROI

Its deployment is easy. It didn't take much time. It took less than 15 minutes.

We deployed it on our own.

1. Scalable Product and solutions

2. Plug and Play with CICD process

3. Reduction in licence cost 

No Licensing cost for JMeter

Yes.

I would recommend this solution. We plan to keep using this solution.

I would rate Apache JMeter a seven out of ten.

Private Cloud

When I was last using JMeter, we were simulating 200 concurrent users and evaluating performance based on transaction times. We were defining SLAs based on the results.

Essentially, we created load scenarios and testing different ones using different workload models.

The recording and playback functionality is helpful.

The reporting is not very good.

When we run with multiple users, it takes a lot of memory.

With respect to the recording and playback functionality, the auto-correlation parameterization is not easy and should be improved.

I have been using Apache JMeter for about four years.

There are issues with stability when running with multiple users because it consumes a lot of memory.

Scalability is fine, although it is important to remember that JMeter doesn't run on its own. It needs to work with load-generations such as BlazeMeter. LoadRunner is the same in that you need a cloud-based infrastructure to run it.

There is no official support. There is a forum where you can ask questions and they respond to you, but the technical support that we have with LoadRunner or NeoLoad is not available.

I have used many similar solutions in the past such as New Relic, AppDynamics, NeoLoad, and Micro Focus LoadRunner Enterprise.

JMeter is not as good as LoadRunner or NeoLoad, and it isn't as easy to use, but it's okay because there is no cost. LoadRunner is too expensive, in my opinion. NeoLoad is cheaper, although not significantly.

From what I have seen, many companies are adopting JMeter because it's free. Especially in Canada, using JMeter seems to be the new trend. Some companies are choosing NeoLoad over LoadRunner because it is easier, faster, and cheaper. Whatever they need to do can be completed quicker. The main problem with NeoLoad is that obtaining resources is harder.

Given all of the choices, my preference would be to implement NeoLoad.

The initial setup is straightforward. I would not say that it is complex and if you already have the file downloaded then it will only take about half an hour to deploy.

I took care of the deployment myself.

I was using the free version of the software.

My advice for anybody who is considering JMeter is to just install it and try it. Creating scripts is a different process when you compare it to LoadRunner or Neoload. There is different terminology compare to these two products, so if somebody has not used JMeter then it may seem difficult at first.

I would rate this solution a six out of ten.

On-premises

We engage in performance testing using JMeter based on the client's requirements. We decide the endpoints for JMeter based on the domain and application requirements. The requirements, like 50 users per minute or 500 users per hour, guide our performance testing framework.

JMeter has allowed us to effectively verify that, based on pre-requirements, performance standards are met. Since it's an open-source tool, it also allows us to reduce the expenditure on performance testing tools.

The different listeners in JMeter provide various reports and results. It's very useful for performance and load testing. Its compatibility with external tools, such as the ability to import scripts from Selenium and Azure DevOps, makes it a robust and flexible option.

I would recommend the inclusion of plugins for AI in JMeter. Some features still require a certain knowledge level to use effectively, and making these features more user-friendly would be beneficial.

I have not been a direct team member but have managed a team working on JMeter for almost eleven years. In my nineteen years of experience, I have engaged with performance testing, with JMeter being part of our toolkit.

Stability is not a concern for expert users of JMeter. However, stability can be an issue for beginners.

JMeter is widely used and very scalable, rated between eight and nine in terms of scalability. It can also integrate with cloud solutions like Microsoft Azure.

There are many online blogs, forums, and video tutorials available for JMeter, providing substantial support for both beginner and intermediate levels.

Positive

LambdaTest and BrowserStack were part of our working user experience, but not extensively more than that. I initially worked with other performance testing tools like LoadRunner.

The setup for JMeter is very easy. There is no installation required; it is simply a package that needs to be imported and can be set up within half an hour or less, depending on the download speed.

Our clients cover the necessary licensing costs, particularly when using cloud services like Microsoft Azure.

JMeter itself is an open-source tool, so there are no licensing costs associated when running on-premises. Client-covered SaaS licensing, such as with Microsoft Azure, does not involve any costs on our side.

It's important to be very skilled in manual testing concepts before starting with performance testing. Due to JMeter's advanced level of techniques and skill requirements, extensive hands-on experience with real-time projects is needed. It generally takes three to four years of practical experience to master.

I'd rate the solution eight out of ten.

Hybrid Cloud

Microsoft Azure

The tool helps us in the capacity assessment planning for our services. It helps us to make sure that the APIs will be able to afford the traffic that we expect to get from these services in the production environment. 

The product helps me get the expected performance from applications or servers and reduces costs. It also enhances the performance of the services and helped them reach their ultimate capacity. 

The tool needs to have a better Graphical User Interface. Many of the solution's features are difficult to understand due to the complex user interface and user experience. The product needs to add plugins. It should also work on the integration with external partners like IDE and API gateways. 

I have been using the solution for more than three to four years. 

The tool is stable. 

The product is scalable. My company has around four to five users for the solution. 

The product's setup is not complex. The tool's installation was very smooth.  It doesn't have any deployment process. You just need to implement a set of packages to get started with the product. 

I would rate the product an eight out of ten. 

I use the solution for performance testing (Load testing, Volume testing and Endurance testing).

The solution has a friendly user interface and is a very good open-source solution for load testing. It supports both web and mobile network recording via proxy and offers a wide range of protocols. It has strong community support across multiple platforms.

The solution is open-source. 

It is very easy to code and develop the script and it supports web and mobile scripting.

There is a lot of good documentation available. 

The solution is stable. 

The memory utilization in JMeter is poor. Limited in terms of scalability. The system gets hung up for complex scripts and heavy loads which requires improvement.

It needs improvement in analytics and statistics reporting. The reports offer limited information on a load test. I would highly appreciate it if JMeter can provide better reporting capabilities.

I have been using this solution for two years.

It's a stable and reliable product. There are no major bugs or glitches and doesn't crash. 

The solution is scalable. However, it has a lot of memory issues during scale-up. That is the one concern. It's a bit limited in terms of scalability.

I have never contacted Apache customer service for any issues. However, the documentation is readily available online and very useful.

The initial setup is pretty straightforward and easy to implement. It's not overly complex. 

I'd rate it a five out of five in terms of ease of implementation. 

I can do the implementation by myself.

It is an open-source tool.

I'd rate the solution eight out of ten.

Private Cloud