Apache JMeter Reviews

We primarily use the solution to manage our capabilities. Wherever the business goes, we need to work within the expected benchmark. In terms of the user bases we deal with, most of these are products from different locations with different types of functional flow-out. There are environmental integrations also running in the background, which a user is basically accessing while going through a particular workflow. 

We'll also handle the reporting section of our data management responsibilities utilizing this system.

The reports and analysis tools are very good. They are the solution's most valuable features.

The GUI could be improved. When we go into GUI mode, there are occasions where it will not sync with our expectations. There are crashes that happen that will stop the solution from performing. It seems we get minor glitches when we go into GUI mode.  

The data client architecture that we have isn't so great. If we are to consume the data, it won't clear because there is tech running on different agents. 

When I need to pull the reports from different agents, it's not user-friendly. The reporting can be difficult to handle. It's hard to increase it if you are working on a client's architecture.

It's not easy to get the data from one place or to do customizations. 

There are other solutions that allow users to model their load and structure with them. You can't do that on JMeter. 

On other solutions, like Silk Performer, you can do network packeting, which you can't do on JMeter. They should add this to the solution as a capability in the future.

The support management needs improvement. Support is coming from consultants; you will not be able to get on-premise support from all of their agents in one place. On Silk Performer, for example,  they have the capability where you can basically have a summarized report from different agents.

Personally I've been using the solution for the last 10 years. However, in the program we are in, I've only been using this tool for six months.

If you want stability, you will have trouble, because the system is not uniform. You have to do a lot of modeling. 

If you're talking about mobile platforms, JMeter will not allow you to capture the performance of applications on your mobile platform. It's an area where the solution is lacking. It doesn't have the capability to allow developers to check the performance on an AP app.

You can scale the solution, but you will run into issues if you want to run, say, 1,000 users.  You will have to configure multiple agents and then distribute your load. It's quite difficult to manage everything on a single agent.

The initial setup is straightforward. We found it quite easy. For us, it wasn't complex at all. The solution is up and running quite quickly and doesn't take long at all to set up and deploy.

We've looked at Silk Impact and Load Balancer in the past. Each of these solutions offers a few different functionalities that aren't necessarily available on JMeter. For example, on both, there are more network features available that aren't included in JMeter and you can model your load and structure with them as well.

If companies have mobile development, they should use JMeter. If they want to integrate it with the PA pipeline, JMeter will work well. However, there are other platforms and solutions as well.

Other options include Silk Performer and Load Impact. These two solutions have other capabilities that JMeter doesn't have. You can basically model your load and structure with them. You can't do that on this solution. On Silk Performer, you can also do network packeting, which you can't do on JMeter.

Then there are some network features available in Load Impact as well as Silk Performer that aren't in JMeter. 

I'd rate the solution eight out of ten. The solution has certain limitations, but it's still a good product. 

I consider Apache JMeter to be a fundamental tool in the industry for performance testing. Its primary advantage lies in being free, making it accessible to everyone. The platform boasts a wide range of plugins, regularly updated, and has a substantial community. You can find information on almost any topic in the community forums or Q&A sites related to code.

Apache JMeter is well-known and widely used among developers, particularly on popular developer forums. While it may not have the most user-friendly interface, it offers strong support through official manuals and various articles from companies providing load testing services. 

The tool is free, has a substantial community, and serves as a fundamental choice for testers, especially those new to performance testing. While other tools like K6 may be more developer-oriented, JMeter's affordability and accessibility make it suitable for those without extensive performance testing experience.

Given that Apache JMeter is a free and open-source tool, documentation improvement may not be a major concern, as it is mostly contributed on a voluntary basis. The essential information is already available. However, in terms of the interface, there are occasional bugs, and the tool may not address them as quickly as some users would like. Fixing defects and bugs might take a considerable amount of time, with users sometimes having to wait for several months or even a year for the next release to address specific issues.

I have worked with Apache JMeter for years. 

In terms of stability, there have been some instances of unknown bugs, but these are documented on the official pages and tracked in bug repositories where users can find descriptions and potential workarounds. Users with less experience may encounter unexpected behavior, but with online resources, such as Google, they can understand and address the issues. Some known problems, like internal loop issues when launching from the MacOS terminal, can affect stability, but they are not critical and usually have workarounds. While these issues may slightly impact the user experience, they do not significantly detract from it. On a scale of ten, the overall user experience with JMeter might be rated around seven or eight.

Scalability is possible for both small and large companies. In smaller companies, JMeter can be scaled across multiple machines, utilizing them as load generators. For more complex network setups, collaboration with the DevOps or system administration teams may be required to handle firewall configurations. In large enterprises, options like Docker Swarm or Kubernetes can be employed. Additionally, using JMeter images or containers within these frameworks allows for quick scalability.

While I haven't personally engaged in direct interaction with support, I have encountered challenges and utilized forums and official support channels to address and resolve those issues. So, yes, I've had experiences leveraging these resources for problem-solving.

A few years ago, I found Apache JMeter to be a reliable tool, and while it may have changed since then, I haven't used it recently for a direct comparison. Around five years ago, I had a positive experience with Neotys, which was an excellent tool, albeit expensive. However, as its pricing is now contact-based, I'm not sure about the current costs. Additionally, I've recently explored K6, which seems promising. K6 is actively developed and evolving rapidly. It caters more to a coding-centric experience, making it favorable for developers who appreciate JavaScript-based coding without extensive installations. This tool might see significant growth in functionality in the coming years.

If someone is completely new to Apache JMeter, it might take around one hour to half an hour to set it up initially. This includes spending time reading manuals, watching YouTube videos, and getting familiar with the tool. For those with some experience, the setup process could take as little as ten minutes. This involves downloading and installing JMeter, configuring Java if necessary, and adjusting path variables. The overall time may vary based on the machine's readiness and the user's familiarity with the setup process.

One piece of advice I often provide to those venturing into performance testing with Apache JMeter is to carefully consider the expected load. Due to its utilization of the Java Virtual Machine, JMeter can be memory-intensive. For extensive testing scenarios with a high volume of virtual users and transactions, it's crucial to meticulously configure JMeter, considering each element in the scenario. However, for simpler yet intensive testing needs, individuals might find less resource-intensive tools like Gatling or Locust more suitable.

If it were a commercial tool, I'd rate it an eight, but given that it's a free and non-commercial tool, I'd lean more towards a nine. While it's not perfect, I believe it's one of the best options available for now.

We primarily use the solution for performance and delivery testing.

It's a free tool. It's a very common tool. There's a lot of support in the community for this. It mainly supports web API testing.

The UI could be better. It can have some Reach UI also, which would be helpful, and maybe a relatively simpler way of using it. It needs simple modules. There are quite a lot of things which are kind of abandoned, so they can definitely improve on it.

Integration with some of the other features should be managed. However, it's open source, so there is not much to complain about there.

It's an open-source tool; we cannot ask for additional features really.

The product could use some kind of filtering and monitoring and different degree of dashboards and analysis. If that can be provided, that would be very, very helpful.

I’ve used the solution since 2009.

The solution is relatively stable. There are no bugs or glitches, and it doesn’t crash o freeze. It’s reliable.

It can scale. However, what others say about scalability is that you need to have some proper calculations to be done first.

We have dealt with support. Technical support is found in more technical forums. It’s open-source, and communes are where you go for information. We have received good responses when we have inquiries. There are quite a lot of forums in general available.

Now, as we have grown as an organization or as a team, there are still questions such as, "What are the limitations of this tool?" And we put that to JMeter so we can learn what is best for the maintenance.

The installation can be a bit complex. There are quite a lot of things and issues if you go deep into it and if you're setting up JMeter. For example, direct script captioning is slightly different. It's a bit more complex, the correlation parametrization. Setting up the workload model can be complex as it is based on a Java service mechanism.

It becomes a bit challenging to manage. If I want to put a 50,000 user load, I have to be very careful how the memory is utilized. I must be very aware of the underlying system's capability to execute this. I have to make it into multiple nodes and run them in parallel. There are some calculations, and there are some good power processes that will be required when you are using JMeter. You need to understand its limitations and load work them on.

The solution doesn’t really need maintenance. They're open-source tools, so we don't expect any maintenance. What we typically do is we pick up a particular version and understand the limitations of it. We then play within those limitations.

We handled the initial setup in-house. I handled the setup myself.

The solution is free to use.

We may be using the latest version. I don’t know the exact version number.

It is on-prem. However, we can put it on the cloud as well. We install it on any machine, so it can be a local or cloud-based machine.

I’d recommend JMeter. One critical piece of advice is to plan properly. For JMeter, planning and having an understanding of the limitations are important. If you play within those limitations, it is a really great tool. There are trade rules that apply to a lot of things that we use. However, there's plenty of material available so far. Whoever is doing it, can plan it accordingly. They can create those tests, and execute those tests. That said, at the same time, be aware of the scalability of the JMeter. For example, a single JMeter line, which is running, can take 1,000 to 2,000 threats, not more than that. Again, it depends on the available hardware. If you want to scale to, say, one million or something, different parts of planning are needed.

I’d rate the solution eight out of ten.

We primarily use it for conducting different types of performance testing, such as load testing, spike testing, and endurance testing.

JMeter is basically the art of the entire performance testing process. We generate load on our application using JMeter and then monitor various metrics like CPU with different monitoring tools. It's the essential foundation for our performance testing.

JMeter is doing some good things with upcoming releases, but the main area for improvement is the extensions available. 

Another area of improvement is the reporting part, specifically regarding report generation. 

There are certain things like we can't merge custom metrics into the JMeter reports. We're limited to JMeter metrics, and other server metrics can't be integrated with JMeter dashboard. This forces us to rely on another tool. 

We should be able to add or custom-configure server details directly in JMeter reports.

We have been using it for three years.

I would rate the stability a nine out of ten.  It's generally quite stable, it hardly has crashes or issues.

Scalability is good for my use cases, but to generate a large load, you must go into distribution mode, which is more complex to configure and requires powerful machines. So, while it's fine for my needs, but the scalability wouldn't be a perfect ten. I would rate it a seven out of ten, as there are some limitations for large-scale testing.

I'm the one who uses it most extensively. And the other we have four to five people are using it just for their local testing. It's development testing kind of stuff.  

It's an open-source community. So we can post our queries there. We generally get good responses from the forums. So it's good enough.

I used another tool like HP LoadRunner. And now it's offered by some different companies, ownership-wise, but it was long back. 

I would rate my experience with the initial setup an eight out of ten, where one is difficult, and ten is easy. Even for new users, the installation is straightforward based on the documentation. 

However, customizing and building something extra can be complex. But overall, it's easy enough to download and start working with.

Our main system is on the cloud, so we primarily use JMeter on the cloud. However, some use cases require on-premises deployment, and we use it there as well.

One person is enough for deployment. The deployment won't take much time. It is very fast, typically within five minutes.

We use the open-source version. 

I'll definitely advise that you should at least give it a try. If it serves your initial needs and meets your expectations, you should go for it. 

It's quite an old and up-to-mark tool with a proven track record in the industry, and there's a strong community behind it. So it's definitely worth giving a shot.

Overall, I would rate the solution an eight out of ten. 

We use the solution for performance and security testing.

The solution's most valuable feature is its ability to install and automate functions with ease.

They should improve the solution on its UI front.

 We have been using the solution for six years.

I rate the solution's stability as a seven or eight.

We are an enterprise business using the solution. It flips a bit whenever we add more than 10,000 virtual users to the solution. It becomes impossible to get the expected result. We have to use distributed testing to deal with the issue. I rate the solution's scalability as a six.

We check with the solution's community support forum in case of any queries. It is very active.

We tried out solutions such as LoadRunner and NeoLoad. As these were commercial tools, we settled for JMeter. Also, the reviews from those who have used the solution helped us switch to it.

I rate the solution's initial setup as an eight. It is easy.

Regarding deployment, you have to download the installation file and save it in the system. If it has specific prerequisites, such as installing Java, you need to ensure it is compatible with JMeter's current version.

The solution is free of cost as it is open source.

The solution works well for small or medium applications but not with 10,000 or more footfall. I rate the solution as a seven.

I use JMeter for concurrency and some backend services. We are also using JMeter to apply our tests within our framework by sending some requests to JMeter to assess our application's performance. My company is developing software based on data analytics that produces insights and graphs like Tableau or Power BI. We use JMeter to test the graphs, how the tables are rendered, and how long it takes to render some applications. 

For instance, we have a problem with one of our clients that use some complex scenarios or queries that take too much time to retrieve from the application or render to the end user. We ran a couple of tests on the application using JMeter and spotted the service or request that was taking too much time.We alerted the developers about this, and they took the appropriate action to fix this problem. Afterward, we run another cycle of JMeter to ensure everything is working as expected.

There are two modules. The analytics module is the interface for the stakeholders and company decision-makers. The number of users isn't that big, unlike the retail applications websites. It's 500 users at most. The other module is deals with data volume. We are currently doing some POCs to check whether we will benefit from JMeter in this area because it's not a concurrency issue. It's a data volume issue.

We have a hybrid deployment because we are using JMeter to assess the performance of our products. If the product is deployed on the cloud, we use JMeter on the cloud. If it's on-prem, we are using it on-prem. At my previous company, we mainly used JMeter on-prem.

I expect that we will continue to rely more on JMeter, and we have multiple DevOps pipelines using JMeter to test another module in our application.

I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance.

The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter. It's working on this, actually.

It's an excellent plugin that you can use to record the scenario from Google Chrome, and it integrates easily into JMeter. They could also make it easier to generate the built-in report. Now, you run the tests and generate the charts in a separate column. The graphs and charts that display the test metrics could be better.

I worked with another tool called Web Performance Tester, and its interface is better than JMeter's. They have intuitive graphs while you are running the tests, so you can see how things are going. It shows you the number of concurrent users logged into the system, the number of failures, response times, etc.

I worked in performance testing from 2011 until 2019. I was working with another tool, but in the last few years, I started using JMeter for a couple of projects.

It's hard to say definitively. When there's something wrong with JMeter, we're unsure whether it's something in the device because there are lots of requests coming from the machine. Maybe we didn't use it that many times from our local server. There are some bottlenecks, but in many cases, we only need to restart JMeter, and it works fine.

JMeter is scalable. I did a test using it in a distributed environment with more than one client. We used JMeter to load from the server and also the plugin tool. It works fairly well in a complex environment. I still need to research the maximum concurrent JMeter can handle. Is it 1 billion or 10 million? 

In my work experience, I need to load maybe 1 million users. For example, most of my work is on the backend of e-commerce websites with a maximum of 1,000 users. We have many clients, but the load is not that big. 

I don't think JMeter has technical support because it's an open-source application, but there is support for third-party apps that use JMeter. For instance, BlazeMeter has good technical support.

I have more experience in a tool called Web Performance Tester, but it has some limitations. For example, it doesn't support some forms. A long time ago, their support was better, but lately, their support hasn't been so good. They always ask us to switch to the consultancy service and they don't give us the solution. If we face problems, they always tell us, "Okay. You can consult us." This is not a good practice, actually.

Also, Web Performance Tester isn't well-known, so some of our clients aren't confident in it because it doesn't have a reputation.  They were more familiar with JMeter, and it's something I've used.  However, I have no problem with Web Performance Tester. It's hard to compare the two tools because it's not apples to apples. 

I joined this company in October, and they had just started using JMeter. Previously, they were using a tool that they had developed in-house. They found that JMeter has more capabilities and specificity than the tools they were using.

Setting up JMeter is straightforward. It doesn't need an installation like other tools. There is a batch file for Windows. Around four people in my organization are responsible for maintaining and managing JMeter.

JMeter is a free open-source tool. There are some third-party tools built on top of JMeter that have a license or something like BlazeMeter. I think you can also purchase some additional services.

I rate Apache JMeter eight out of 10.  Before using JMeter, you should find some tutorials because you need to be trained to use it. It's not a plug-and-play tool. This is what I did. Spend some time researching JMeter's capabilities. We met a lot of people who knew about JMeter, but when you tell them that it can do something, they're not aware of it. 

For instance, through taking some courses, we learned that JMeter has distributed performance capabilities. Some people still know that you can't make these recordable scenarios. This information hasn't reached everyone in the IT market.

Apache JMeter is quite flexible and it is also well distributed. It is quite flexible compared to Micro Focus LoadRunner.

JMeter is easy to script. There is less of a problem with doing correlations and parameterization. 

It is not something that can be compared with Micro Focus LoadRunner. It gives the facility too easily; you do things through UI. With JMeter, you really do not have any easy UI to work as, like a Micro Focus LoadRunner.

The stability could be a bit better.

Compared to LoadRunner, it hasn't any proper UI. Recording the script is also not flexible in JMeter. In LoadRunner, we have a couple of options, such as URL-based recording and HTML-based recording. In JMeter, it's not like that. JMeter has a recorder, however, it is not easy to use. It is a bit tricky to configure the automatic recording in JMeter. 

I've been using the solution for four or five years. 

JMeter, stability-wise, is good, however, it is being developed by the community. Therefore, stability is always an open question there.

The solution can scale a bit. It is scalable, however, not like LoadRunner. I have not tested it as such yet. I'm not sure about how fully scalable it is. 

I'm also familiar with Micro Focus LoadRunner.

The implementation process is not so easy. It's difficult to configure.

I'd rate the solution a seven out of ten.

I'm an end-user and a customer. 

Any front-end technologies like Angular, React, or others might access the services. Therefore, you must ensure that using SAP with JMeter works fine. Unless you ensure this, the front-end team may not access the services properly.

You can save the state of your tests and test plans after conducting tests, allowing you to showcase the results to others later. This feature makes it easier to demonstrate what testing was done and share those test cases.

From an improvement perspective, consider a scenario where your application needs to retrieve data from PayPal and store it in an Excel file in an external network. Connecting to such a source using JMeter might take some time. Previously, you might have linked your application to a server to obtain these results. However, achieving this kind of integration directly in JMeter can be challenging. If it were possible to configure JMeter to handle such server connections seamlessly, its capabilities would greatly be enhanced.

I have been using Apache JMeter since 2004-05.

It is a stable tool and has been stable for a long time. Organizations have been using it extensively since 2017. However, it's important to note that while JMeter is excellent for testing the performance of your service, you cannot directly forward the results to the end user.

You can increase the testing scale. When deploying a microservice in a production environment, performance testing is essential. You should test to handle at least one million users, a minimum benchmark. 

From a developer’s perspective, if your service can handle one or two million users with a response time of less than one second, it is considered acceptable, provided the data is accurate. In production environments with multiple servers, response times might vary, but it is crucial to perform thorough testing in a local environment to ensure the system meets performance criteria. JMeter helps verify that the service can handle the expected load and deliver accurate results.

I am using Azure and GCP.

JMeter is easy to set up, taking about half an hour to install. However, the real effort lies in configuring and developing the tests, which can be time-consuming. 

If there is an update, it doesn't typically take long to come. They may release an upgrade within one or two years. If you have a test suite, it can be easily imported, and you can start using it. This process hardly takes more than half an hour.

Apache JMeter is configuring it properly, especially when dealing with SSL configurations. It can be tricky and time-consuming if you're unfamiliar with handling SSL certificates, keys, and JMX configuration. However, the process becomes straightforward if you know where and how to pass the necessary keys and configurations. Without this knowledge, you'll likely need to spend time researching and Googling how to set everything up. In contrast, if you're familiar with these configurations, setting them up in JMeter is quick and straightforward.

Overall, I rate the solution a nine out of ten.