Postman vs Runscope comparison

"The solution collects scenarios and allows for testing SOAP API, REST API and API automation at the same time."

"The solution is stable."

"The product is easy to implement."

"The user-interface is very good."

"It is stable and does not lag."

"The solution has a great user interface and collection of features."

"In Postman, we have an option to directly import a call, make it a Postman collection, and execute it in a batch. This feature is very useful. It saves a lot of time. The manual effort is also reduced when we can just pick it in a Postman collection and then run and execute it to get the results."

"The scalability is good."

"Environment initial variables allow our team to run tests against multiple environments on the fly."

"It helps us verify that production deployments do not break the features our customers use."

"As we have the APIs scheduled, we use the notifications. We know the exact moment it is breaking something that might impact the customer system, that might result in a feature not working. We know "in advance" and we can fix it before the customer notices. It helps us to be more proactive."

"The feature of automatically executing tests at a predefined time interval is great as we can be continuously testing our end-points before and after builds are deployed."

"When it's a problem on our side, the server is down, or there is a lack of memory, the first thing to break is the API, because the calls can't be processed. So we know exactly at that moment because we receive notifications, and we can take action. It doesn't only notify us of the type problem, but it also links it to our servers."

"We have the ability to populate environment variables using scripts. This allows us to keep the configuration mostly dynamic and avoid a lot of manual updates when something changes."

"This solution has improved our responsiveness to issues in both production and non-production environments. Through integration with Slack, our development and DevOps teams are quickly aware of issues related to new changes deployed to non-production environments, which tightens our feedback loop during testing. Through integration with PagerDuty, our DevOps resources are able to reduce the adverse impact of a given production issue."

"Valuable features include: Being able to add tests as sub-tests in a parent test for regression testing; notifications for tests that pass/fail; being able to customize expected results with assertions; being able to set tests to run daily."

"I have recently noticed that, for some reason, I am not able to import collections in JSON. It is weird because I was able to do it before. When you create a new collection and documentation, in tests, you usually have to click Save. If you don't click Save, your collections don't sync with other devices, which is something I don't like. It is not clear for the users in a shared workspace, and I usually forget to click Save and end up losing all the collections that I created. I create a lot of routes, and clicking Save all the time isn't efficient. Instead of clicking Save all the time, there should be an option to save everything in one go."

"To get this email reporting function, we have to integrate Postman with Newman. If the two are not integrated, we won't be getting all these things in one group."

"Postman does not have database validation available, which affects its scalability."

"There are definitely features that could be improved; for example, if you derive it from Ready API, maybe Postman can be a good contender and replicate some features such as more scripting and control over API calls."

"I would like that if you click on one of the fields you should be able to see the entire row of fields, names, and values."

"The tool needs to have a reporting feature."

"If we have a certain build on one machine it won't work with another build even if we are using the same URL. It would give us a connection refuse. So developing environment compatibility would make it better."

"Initially, the setup is a little bit complex. Whenever a user comes to Postman for the first time, it is just a bit complex since they do not know where to start and what is the structure."

"Reliability. I would like to see more money invested in disaster recovery testing of the application. The product should be hosted across several AWS regions so if there are issues with an EC2 instance in a particular region, the whole application won't be affected. Once or twice a year there are issues with AWS that causes the reliability of the application to suffer."

"There are not many options for scalability."

"File upload is a big part of the products that we test. The lack of file upload in Runscope requires us to still use UI (Selenium) automated tests for these scenarios."

"The user interface requires a lot of clicking around."

"The initial setup included a lot of repetitive manual work."

"If triggering test bucket via trigger API, there is no option to see if the tests which were triggered passed or failed."

"We definitely make use of historical test results, especially when tracking more difficult issues such as timeouts. Per our test frequency (15 minutes) we often need to go back further than the last 100 test runs. We are aware that results further back in time are available through the Runscope API, however, it would be great if the dashboard allowed easy access to a larger amount of historical data."

"We would like to be able to select the actual time a test will run automatically. It appears that you can set up the automatic schedule, but the test runs based on the last time it ran and not a time you select. Since we want these tests to run in the middle of the night, it would be nice to be able to select that."

"We are currently using the free version, but we have plans to buy a license."

"Its price is moderate as compared to other competitors. The version that we are using is not open source, so it is not free."

"I use the tool's free version."

"We do not pay any money for Postman."

"We use the open-source version. They introduced a new licensing model where your company must subscribe to their services if you're using it for business purposes. However, I think they charge too much if you're only using the basic features. It limits what your organization can do if you're not a paying customer."

"It is free."

"The solution has a licensing fee which might be prohibitive."

"Unlike APIs, the solution does not require a license."

"Pricing is fair."

"Having the different plans organized by team size, buckets, and requests per billing period works for our needs. Larger companies may need to take a close look at these elements."

"Runscope pricing is unbeatable"

"The price for the product is very reasonable for a team of more than four to five members."