Apache JMeter Reviews

My work is mainly on the testing part with Apache JMeter itself. I work on anything which is testing-related.

Apache JMeter is used for two different tasks: one is performance-related testing, and another one is API testing. It also helps in security-related testing aspects.

I do work with dynamic HTML reporting. It gives the complete insight altogether. We are not using it vanilla, so we have a wrapper around it. We try to present it in a form where our clients or management appreciate it, but this is the base that we use.

It is quite easy to create and manage tests in Apache JMeter, and since I have been used to it, it tends to become easier.

Apache JMeter is easily integrated into our continuous testing in the CI/CD pipelines.

Apache JMeter is mainly used for performance testing, and this is one of the prime features that we use in any of the performance testing aspects. It forms the heart and soul of everything in terms of the performance testing work that we do.

We do have command prompt related or CLI functionality with Apache JMeter, so it is easier. However, since it is open source, much work has not been put into the GUI part. The GUI could definitely be enhanced.

One drawback which I can see when I compare it with other tools is the amount of cache that it uses, the memory that it uses. It becomes very difficult to work with when we have a lot of requests that need to be processed. Anything more than 10,000 requests becomes a bit of an unfavorable task when using Apache JMeter altogether. Otherwise, it does help anyone who is starting off with performance testing.

I have been using Apache JMeter for seven or eight years.

Apache JMeter is quite stable. Going by the same point, if we have more requests, then we do see some latencies altogether from the system perspective or the environment perspective, as well as from the tool itself.

We do have scaling capabilities. When we have a lot of requests to be sent, that is one part where we can scale up Apache JMeter or fine-tune it altogether. We do have some methods where we can distribute the complete load between multiple systems and then try to do our testing. This is one thing where we can really enhance a bit.

I have never had experience working with support from Apache. Since it is open source, we have an open community. We do post our needs, and most of the time it gets answered there. We do not have any support as such.

Locust was one which we used briefly. However, cost and everything adds up, so we fall back to Apache JMeter.

It is quite easy to install Apache JMeter; it is just a stand-alone thing, and you do not need to do much.

I have not used much of the plugins in Apache JMeter, but recently I used Kafka. Other than that, I did not use much. With Kafka, we did just one test case, so I am not completely familiar with it because we had some help from developers, but that is all.

I have started working on Playwright with MCP server, although I still work with the same products with Amazon, Apache, and Microsoft products today.

Playwright is the new product I am working with. Along with the other ones which I was working on already, this is the added thing, so it is not the only solution I am using today.

I do work with Azure DevOps and Cypress today as well. DevOps is the only product from Microsoft that I work with. Currently, I am not using anything from Amazon. I work with Apache JMeter as an end user, rather than having some partnership with Apache.

Apache JMeter does not have a license, so it is 100% open source. I got Apache JMeter from their website itself, and since it is open source, you can directly download and use it.

My review rating for Apache JMeter is 8.5.

We use Apache JMeter for performance testing, and the Locust tool for Python framework for performance testing. We primarily do performance load testing.

Apache JMeter has its own pros and cons when compared to other tools. It is easy to use the tool and it has open-source capability so we can build our custom scripts and execute them. It provides other capabilities, such as integrating a database and connecting to other application servers for monitoring and related functions.

We use dynamic HTML reporting, which helps us in testing analysis by pinpointing the bottlenecks based on the reports. We can identify the specific areas that need attention, troubleshoot them, and report to the development team.

The user-friendly GUI for creating and managing tests makes it very easy to drag and drop samplers. For example, if you want the HTTP sampler, you can drag and drop it and use it. For configurations, we have other samplers. For results, we have the view results samplers that we can also drag and drop. The UI is good in comparison with other tools.

Regarding integration with CI/CD pipelines, we can create Apache JMeter scripts and use the Docker image. From the image, whatever scripting we have done can be connected. We can use the CI/CD pipelines and connect them with Jenkins tools and GitHub. Then we can create the pipelines and automate the end-to-end flow.

For connecting Jenkins to Apache JMeter, JMeter plugins are available, and we have used them. Apache JMeter also has some third-party plugins, which are not native samplers. If we want to use custom test executions, we definitely use all the different plugins available in Apache JMeter.

The capability to simulate users has impacted testing resources and outcomes as Apache JMeter is based on Java, which has a limit to the users in a particular load generator. Apache JMeter provides distributed load testing where you can connect multiple PCs in a master and slave concept, allowing you to pump the load with any number of users. In the past, I have done load testing with 10,000 users by connecting the Apache JMeter distributed network in BlazeMeter. There is a cloud version available, the updated BlazeMeter, and I used that. It is very easy to launch load generators in BlazeMeter, and then we can run the test, scaling up beyond 10,000 users.

There are obviously many areas for improvement in Apache JMeter, as they are providing support only for the web protocol now. They can provide support for other protocols as well, and with AI becoming more prominent, they can implement features where it can generate code by itself based on the results or provide suggestions. Based on the current trends, AI needs to be integrated into Apache JMeter.

I have been using Apache JMeter for more than 10 years, specifically 10 to 12 years.

Regarding stability, previous versions of Apache JMeter were a little unstable, but the new versions are very stable. We use the N-1 versions, which are the stable versions. I have not seen any issues, but occasionally there might be some system hangs depending upon your configuration. Other than that, there are no issues.

As for scalability, if you want to run minimal users, one PC suffices your requirement. However, if you want to scale up to any number of virtual users, you have to set up the distributed network with a master and slave configuration. Then, you can scale based on your infrastructure.

The technical support for Apache JMeter comes from the open-source community. There are many channels available to get support. We can find all the information even in ChatGPT. Just paste whatever error you are experiencing, and it will provide you with a solution in numerous ways. With AI models ChatGPT, troubleshooting issues has become very easy for us.

Positive

The deployment of Apache JMeter can be done on-premises only. We can also do it in the cloud, and it is very simple. You have to ensure Java is the prerequisite for this and then download the new version of Apache JMeter. It is platform-independent, so you can run it on UNIX, Mac, or Windows operating systems. Thus, you can deploy it in the cloud and run it.

Deployment takes just a matter of minutes, with a couple of hours needed to set up the distributed network. As long as the virtual machines are in the same subnet with the same IP address, the connectivity between the master and the slave can be established. If any issues arise, then it might take some time to troubleshoot it.

Apache JMeter is open-source, so there is no pricing. The only concern is to set up your infrastructure to scale up the number of users. It is distributed. You can set up your own virtual PCs, or use the paid version such as BlazeMeter or other options such as OctoPerf. Each vendor has their own pricing details based on users and negotiations. Apache JMeter is open-source. Everything, including scripting executions, can be executed for free, and you can set up your own network and scale up to any number of users without limit.

In my organization, there are two users, me and my teammate, who are working with Apache JMeter.

There is no maintenance required for the solution.

I definitely recommend Apache JMeter to other users because it is an open-source tool. Compared to LoadRunner, you save a significant amount of money. Installation is hassle-free when we compare it with LoadRunner, and there are no additional components compared to the other tools. It is easy to use, and most organizations use it for performance testing, around 70 to 80%.

On a scale of 1-10, I rate Apache JMeter an 8.

We normally use it for testing APIs or scenarios where the user load is very low. We use JMeter for performing online UI performance test execution, particularly for conditions involving only HTTP or HTTPS protocols. 

It's employed predominantly for executing tests on hundreds to a few hundred users. We also use JMeter in our CI/CD integration suite, where it automatically triggers tests when a new build is generated.

JMeter can be integrated with most open-source platforms like Grafana, Prometheus, or even with custom-made tools by extending it and integrating from GitHub. It supports generating comprehensive reports with numerous listeners. 

Although we don't utilize all the listeners, JMeter provides diverse reporting capabilities that we can leverage. Furthermore, JMeter facilitates scripting capabilities, which include options for Groovy scripts.

The tool needs improvements related to client-side metrics, integrating with tools like YSlow or HTTP Watch, and enhancing mobile testing capabilities. Adding security and accessibility features could enhance the tool's versatility. More utility features for workflow modeling, like analyzing logs to create scripts, could be beneficial. 

Enabling the conversion of scripts from commercial tools like LoadRunner or NeoLoad into JMeter scripts would also be advantageous. The AI features in testing and development are completely absent, which is an area of potential improvement.

I have been using both tools, Apache JMeter and LoadRunner, in parallel for over twenty years, wherever needed. 

Although I don't work with JMeter daily, my hands-on experience with the screen was some three to four years back. Nonetheless, we continue to propose this tool to our customers for specific testing requirements.

Now, the product's stability can be rated between eight and nine. However, several necessary features still need improvements, specifically in terms of reports and additional functionalities compared to other commercial tools.

Scalability is somewhat of an issue, which I would rate around five or six. For runs involving a thousand or more users, JMeter presents a problem because of memory and related issues. The entire process operates on Java in the background, leading to automatic heap memory consumption. This restricts the number of users and necessitates increasing load agents or distributing the script across multiple machines.

I have never used any technical forums so far or interacted with NetApp. We typically have internal open-source groups where we engage and resolve our issues.

Positive

I predominantly worked with LoadRunner. The need to shift arose because every customer desires to move to non-commercial, open-source tools. Compared to the earlier days when JMeter was difficult to script and execute, it has now become a stable and significantly improved product.

The JMeter setup is easy. It's an executable file that is installed and Java is configured, and then we run it. If someone can read English, even if they're unfamiliar with new installations, it would probably take a maximum of one to one and a half hours.

It's a cost-effective solution. Not everywhere is there a need to deploy a commercial tool for testing. For tasks necessitating handling below five hundred or of simple HTTP or API calls, JMeter is recommended. Many sectors are interested in extending their capabilities with open-source tools like JMeter.

Using other tools like LoadRunner or NeoLoad allows the integration and execution of a JMeter script in their controllers, but no similar conversion tool is available within JMeter.

I would recommend JMeter as it enables cost-effective quality, negating the need for commercial tools in every deployment scenario. If it's a straightforward HTTP or API-based test, I see no necessity for commercial tools, making JMeter an invaluable resource. 

Regarding ratings, I would rate JMeter as eight out of ten due to the need for improvements in client-side features, mobile compatibility, AI integration, and workflow modeling.

My main use case for Apache JMeter is to generate load and simulate multiple users for an application.

I perform stress, load, and spike testing as my main use case with Apache JMeter.

A quick specific example of how I use Apache JMeter to generate load for my application is that I utilize it for load testing, stress testing, spike testing, and endurance testing to identify performance-related issues in my application.

A recent project where I used Apache JMeter for stress or spike testing is with a travel-based application, Travelist AI, where I executed spike and endurance tests to identify any application performance issues, memory leakage, and any slowness in my application.

The best features Apache JMeter offers in my experience are that it is an open source tool, free of cost, and it can generate multiple users.

Apache JMeter has a straightforward approach that makes it easy to use for me.

Apache JMeter has positively impacted my organization because it is effective and reliable. The specific positive outcomes I have seen from Apache JMeter are that it saves time, although Apache JMeter occasionally hangs, but it still works fine.

Apache JMeter can be improved by enhancing its use for multiple APIs and adding multiple plugins.

While the reporting is good, it is not very detailed in Apache JMeter, which represents a needed improvement.

I have been using Apache JMeter for seven years.

In my experience, Apache JMeter is stable, although it can have occasional issues, but it is overall stable.

Apache JMeter's scalability is used as a load injector, allowing multiple uses for cloud and executing multiple Apache JMeter scripts.

There is no customer support for Apache JMeter.

Previously, I used Apache JMeter and LoadRunner; one is open source and the other is paid.

I have not seen a return on investment right now, as there is no improvement in Apache JMeter and reduction in cost, but I save time and reduce costs with Apache JMeter.

My experience with pricing, setup cost, and licensing is that the cost and license are free because Apache JMeter is open source.

Before choosing Apache JMeter, I evaluated BlazeMeter as well.

The primary use case for Apache JMeter is performance testing. We use it to test the performance of applications.

Apache JMeter offers plugins for reporting and preparing test scenarios. It allows recording to customization, letting you download plugins to connect with databases or external systems. Despite being open source, it offers features comparable to paid tools, and its ability to customize and expand is particularly useful. Additionally, its open-source nature makes it cost-effective.

To improve Apache JMeter, reducing the complexity of load generators and distribution testing would be beneficial. Automating report analysis and supporting customized SLAs for script report generation could also enhance functionality.

I have been using Apache JMeter for six years.

There are few stability issues with Apache JMeter. Sometimes, there are heap memory issues due to customized scripting using Java utilities, which requires increasing heap space. Port usage can also exceed, requiring solutions to increase the number of ports. Overall, stability is rated at eight out of ten.

Apache JMeter can be scalable for limited users but might face challenges with location-based specific requirements. Without location dependency, it's rated ten out of ten, but with dependency, it can be six out of ten.

Apache JMeter relies more on community support, where one needs to do their own POC and find solutions from online resources. We've not had to raise real issues with JMeter within the community.

Positive

Two years ago, I used LoadRunner Enterprise in the cloud before switching to a team already utilizing Apache JMeter. The decision to use Apache JMeter was made by the company.

The initial setup of JMeter is very easy, even for beginners with no experience. There is a wealth of documentation available in the industry.

Apache JMeter is completely free as it is open-source, providing cost-effective customization options.

Before using Apache JMeter, we evaluated LoadRunner Enterprise in the cloud.

I recommend Apache JMeter for smaller and medium-sized organizations without geographical or map-related workflows. It integrates well with cloud solutions and can automate executions via CI/CD pipelines. Overall, I rate Apache JMeter at seven out of ten.

I work with Apache JMeter for multiple projects that are currently running in Standard Chartered Bank. We have integrated Apache JMeter with Jenkins and for other purposes, such as executing testing performance center. We have added several plugins and we are using Apache JMeter for our load test executions.

My usual use cases for Apache JMeter primarily include testing APIs and banking services, such as debit and credit card transaction details and getting other services.

I have found valuable features in Apache JMeter such as the JDBC connections for database connections, and the ability to add more plugins to monitor UI aspects. We can add a large number of listeners that are freely available, and the multiple thread groups allow us to mimic real-time production scenarios with virtual users.

Apache JMeter helps my testing analysis by providing precise reports; the HTML format report gives me the exact transactions, response times, and graphs that show average response times, as well as throughput per second. These graphs help customers understand the behavior of the application.

The GUI of Apache JMeter is not that user-friendly because we have many proxies, and we have to record through the proxy. With the limited SSL we have, we cannot use it for UI, which is a drawback. However, Apache JMeter is really good for REST APIs.

I don't think there are any other areas other than the GUI that I would want improved about Apache JMeter; it is generally good and supports multiple protocols.

I have found multiple plugins in Apache JMeter to be most beneficial; often I use the JSON editor and some listeners to fetch more graphs, especially when running in GUI mode.

Apache JMeter's integration with CI/CD pipelines contributes to continuous testing in my organization through a plugin we have. We need to configure both CI/CD Jenkins and Apache JMeter, so that when the pipeline is triggered through Jenkins, Apache JMeter executes the test automatically to maintain the regression suite.

On a scale of one to ten, I rate Apache JMeter a seven.

I use Apache JMeter for performance testing of web applications. Specifically, I have automated the complete API cycles for performance testing and backend functionality testing. I also generate reports to evaluate the performance of an application, such as download times. Furthermore, I use it for sending end-to-end messages in banking systems and for UI automation.

The valuable features of Apache JMeter include its ability to automate complete API cycles for performance, perform API backend functionality testing, and generate reports that evaluate the performance of applications. It is also adept at handling end-to-end messaging in banking systems and offers UI automation capabilities. I find it highly valuable for these purposes.

Apache JMeter could be significantly enhanced by the integration of BDD frameworks. Although it is strong for many testing scenarios, it lacks seamless integration with BDD frameworks such as Cucumber. Development in this area would make it easier to integrate automation with performance testing.

I have around six to seven years of experience using Apache JMeter.

I rate the stability of Apache JMeter as seven out of ten.

For backend automation and performance testing with web services, web APIs, and queue management systems, I would rate Apache JMeter's scalability as between eight and nine. However, for UI automation, it is limited and therefore rates a one.

Apache JMeter has strong support through its vast Java-based community on platforms like Stack Overflow. The community provides solutions derived from Java, making technical support robust.

Positive

For API testing, many organizations prefer Postman. However, for performance testing and backend technologies, Apache JMeter is preferred because it is a single platform supporting multiple backend systems and is open source.

I always recommend Apache JMeter for performance testing. Although it originally focused on performance testing, it has evolved into a robust tool for various testing types, including queue management and API testing.

With Apache JMeter, I have gained great statistics for performance and server metrics. It offers return on investment depending on how users utilize it. In one instance, I compiled complete stats for server memory and network in a single listener.

Overall, I would rate Apache JMeter as eight to nine out of ten. It is the best solution I have observed and tried, though it is not perfect, as paid tools may offer additional features.

We are using JMeter for load testing and UI performance testing in both GUI and non-GUI modes. We mainly use it for performance testing and rendering purposes across various projects.

With JMeter, we can execute load tests and generate reports quickly in our CI/CD pipeline, immediately identifying and resolving server and client-side issues before deployment. It effectively scales our testing capabilities without incurring additional costs for increased load.

JMeter's open-source nature allows us to create scripts easily, add any number of users during load tests, and handle large volumes of data without additional cost. It offers ample flexibility for scripting and debugging, particularly when capturing dynamic data.

One area for improvement is the ability to decrease load suddenly during tests. Currently, we need to use multiple separate JMeter instances to simulate reductions in load, which isn't ideal.

I have been using JMeter for more than seven years, starting from version two point four until the current version five point six.

In terms of stability, JMeter performs exceptionally well, especially in non-GUI mode, which supports high loads efficiently.

JMeter is highly scalable, easily handling increased loads through the use of multiple servers. However, the inability to decrease load suddenly is a limitation.

The support for Apache JMeter is excellent, as it is an open-source tool with a robust community and ample resources available online.

Positive

Previously, I used other tools like Postman and SoapUI. I prefer JMeter for its ease of use and comprehensive scripting capabilities.

The initial setup for JMeter is straightforward. We can onboard new engineers quickly, although correlation and parameterization might require additional learning.

Using JMeter helps us avoid additional costs for high-load testing since it is open-source and allows for unlimited virtual users at no extra cost.

We evaluated Postman and SoapUI as other solutions, both of which offer similar functionalities but are less flexible than JMeter.

I recommend JMeter for its ease of use. Anyone can understand and utilize it efficiently, making it highly favorable for teams.

I'd rate the solution nine out of ten.

I mainly used Apache JMeter for manufacturing environments at an enterprise level. One specific use case was where we had to send bulk files via API, which we could not replicate with LoadRunner as it might require additional protocol purchases. With Apache JMeter, we could simulate this scenario, uploading millions of files via API.

There is no significant financial investment as Apache JMeter is a free tool, which helps in scenarios where cost constraints are present.

One valuable feature of Apache JMeter is the ability to replicate scenarios where bulk files have to be uploaded via API. This is something we couldn't simulate with LoadRunner without purchasing another protocol.

The user interface is not as user-friendly as LoadRunner. The scripting, which is a record-and-replay setup, seems less intuitive, potentially since I have more experience with LoadRunner. 

Documentation is not comprehensive, making it difficult to find the right answers.

I have used Apache JMeter for around two to three years.

I have not seen Apache JMeter getting crashed in between uses, and it is generally stable without major issues.

Apache JMeter has limitations as it only supports one protocol: the web protocol.

Apache JMeter does not have direct support due to it being open-source, however, there are online communities where I search for answers.

Neutral

I have not used any other open-source tool for performance testing. I have used LoadRunner, but given the choice between LoadRunner and Apache JMeter as a tester, I would choose LoadRunner. However, from a purchasing perspective, I would not go with LoadRunner due to its cost.

The setup of Apache JMeter is straightforward and not complex.

We have a production support team, which is a large team. They handle various tasks besides deployments, supporting all enterprise-level tools.

As Apache JMeter is a free tool, there is no direct investment, making it cost-effective for web protocol jobs.

There were no costs involved as Apache JMeter is a free tool.

I have not used any open-source performance testing tool other than Apache JMeter.

I would recommend Apache JMeter to others if there are cost constraints.

I'd rate the solution six out of ten.

I worked on the performance setting of my product, which is a product with an easy UI element that can be added in. People without forward knowledge will be able to easily understand the tool and use it right away.

Other cloud-based performance tools will allow you to view what's going on during the test. However, with this product, it's not possible to monitor live during a test, such as with BlazeMeter, which is very user-friendly.

It is very user-friendly. We just upload the script, and the dashboards are very informative. It's useful for both the person conducting the test and the higher management, like project managers or senior executives, who may not know about the test. They can easily view the results and gain valuable insights. Additionally, monetary benefits with Apache JMeter are notable since it doesn’t require a licensed version.

While using Apache JMeter, we are unable to view the graph while the test is running because it consumes resources, which is a drawback. With BlazeMeter, you can view the results in real-time.

I worked with it for five to six years.

The initial setup is very easy, just a couple of minutes.

Monetary benefits with Apache JMeter are notable as it doesn't require a licensed version, whereas BlazeMeter involves costs.

I would recommend starting with the basic tool to understand performance settings. I would recommend Apache JMeter first.

I'd rate the solution nine out of ten.