Perfecto Reviews

We use it to test the mobile apps that we develop.

The Perfecto device lab has allowed us to run tests on multiple devices at the same time and across multiple platforms as well. That increases the efficiency of our testing and saves money, time, and effort. It would take us six times longer to do what we now do.

We are also able to run tests without needing to monitor them. It delivers comprehensive testing that we otherwise wouldn't be able to achieve.

In addition, it has allowed us to maintain high app quality. By testing the app thoroughly, which Perfecto allows us to do, we get that high quality.

The automated test reporting functionality is the most valuable feature. We use the CI Dashboard. It's very important as it is the main reporting tool for our automated tests.

Another important feature is being able to manually control a device.

When it comes to executing cross-platform testing, we only use two platforms, but it's still pretty good.

We also utilize Perfecto’s cloud-based lab to test across devices, browsers, and OSs, and that is really important as well. The lab provides us with same-day access to devices.

The flakiness, or the accuracy, of the test execution can be improved. Also, the responsiveness of their cloud lab could be improved as well.

I have been using Perfecto for a year and three months.

It's stable most of the time but we do come across bugs and we have to reach out to support.

It's very scalable, although it's unlikely we will increase our usage of the solution.

It is used across at least 10 teams in our organization. 

We get pretty good response times and I have someone I can contact very quickly if I have a problem.

Positive

We did not have a previous solution

It took us about three months to realize the benefits, from the time of deployment.

It's a pretty fair investment.

It's definitely on the higher end of prices for this type of service.

Look at the cost of the product and check its feasibility in the context of your projects, and what you want to use it for.

The maintenance involved with the solution can be done by one person and the main duty is to ensure that the scripts are running correctly.

With Perfecto, we are trying to automate our mobile test cases. These are test cases that run on mobile devices including Android and iOS phones, as well as Android tablets and iPads.

We are a financial institution and we rely on dedicated devices within a private cloud. Due to the sensitivity of our data, it is mandated that we use a private cloud.

I am the director of automation and analytics within our digital portfolio. We use Perfecto for the visualization and reporting of test results, including the relevant numbers.

We leverage Perfecto for reporting and analytics, and in that regard, it provides good reporting features. In terms of analytics, Perfecto still doesn't have predictive modeling or similar analytics features in the tool. In terms of the features that it does have, we use them to the fullest.

We make use of the Perfecto Reports dashboard. Within the framework, we use the Allure reports for automation execution.

We are using the Perfecto cloud for both real and virtual devices. For example, we have some cradles for real devices that we have licenses for, we have virtual devices by means of emulators and simulators, and we have browsers. These are the three types of licenses that we have.

The physical devices are sitting in the Perfecto device lab. The emulators and simulators work to simulate Android and iOS devices, and the browsers are separate again. As the main testing platform for our operations, the device lab is very important to us. We used to do everything in-house, which required on-premises infrastructure to do so. We got rid of the VMs and our physical machines, including our Macs. This is, in fact, one of the reasons that we chose Perfecto.

The fact that the device lab provides same-day access to new devices is a good thing for us. We have an SLA for procurement and if we need a new device, our SLA stipulates that Perfecto has a certain number of days to procure it. We are happy because when a new device becomes available, we get instant access. 

Another example of how Perfecto has helped to improve our organization happened during COVID. Sometimes during the pandemic, when we were working away from the office but still had the on-premises infrastructure, somebody would have to travel to the office only to restart a mobile device or a Mac machine. We no longer have to do that and that's a big relief for us.

When it comes to procurement, when Apple or Samsung launches a new device, all we have to do is pass the request to Perfecto. This is an example of where we save on infrastructure but it's not the only advantage. We are able to execute all of our tests in parallel on different cloud devices. Previously, we were challenged in terms of scalability with local infrastructure. Now, scalability is not a challenge and we can add as many virtual devices, running in parallel, as we need. Previously, we were limited to running a few devices in-house and using a few different browsers.

Perfecto has helped to speed up our testing effort. This is true both in terms of time to market and in terms of execution time. 

The most valuable feature is cloud-based automation and testing. 

In terms of Perfecto's ability to perform cross-platform testing, I would rate it a ten out of ten. The first advantage is that we can take the entire execution to the cloud. We don't have to maintain a lab in-house. Second, we can have parallel test execution, which was tricky when we were doing everything in-house. Third, we have better browser automation and device test coverage.

When we maintain a physical inventory in-house, we cannot get all of the operating systems or different types of phones that are available on the market. Even when it comes to browsers, for example, if I have to run multiple versions of different browsers, whether it is Chrome, Edge, Firefox, or Safari, it is much easier when we can take it to the cloud. Perfecto provides that with all of the options for all of the browsers, as well as the real devices.

When we onboarded Perfecto, the test cases and the framework were already designed using open-source technologies. We used TestNG for the framework and Appium for the automation. As Perfecto supports Appium, it was easy for us to implement and we are still running these technologies.

I would like to see the inclusion of machine learning features. If we can have that, it will be a better tool.

On the analytics side, Perfecto is in need of improvement. The product has a cool dashboard, no doubt about that. But that dashboard is limited to visualization and slicing and dicing the numbers, as well as getting the reports in multiple formats. What I would like, beyond that, is if we can have some predictive intelligence capabilities. This includes machine learning and modeling capabilities. I would really like to see that from Perfecto.

As an example, I would like to be able to predict our failure rate in an upcoming release of our software.

The reliability of devices is very good, at perhaps 99.9%. However, sometimes, we find that some devices are offline and not available. I would like them to achieve even closer to 100% reliability. I know that 100% is not achievable but they can still push that number.

The timing of the software is an area that can be improved. For example, there is a new Appium release and Perfecto should be able to get it to us more quickly. They take their own time for internal testing before they roll it out to customers but we would like to use it faster, so it would be helpful if they can increase the speed of that process. It is somewhat related to improving their time to market.

I have been working with Perfecto for more than five years.

The reliability of the access to cloud-based devices is close to ten out of ten. We totally rely on cloud devices and don't have a fallback plan. We don't have any in-house infrastructure anymore, which means that we don't test it on any devices that are on-premises. Everything is done from the cloud. As such, it has to be reliable. Performance-wise, I would rate it a ten out of ten.

Overall, I have good confidence in the stability of the solution.

Using the device lab has helped us to scale our day-to-day testing operations and in turn, it has helped to scale our business overall. 

Using virtual devices, such as the Android emulators and iOS simulators, is a feature that has definitely been a big help. Testing on virtual devices is a major chunk of our test cases, and it's important because we are not deepening any local infrastructure. We are totally reliant on virtual devices.

With respect to how hard it is to scale, we have a certain number of licenses and we've been told that we can scale up to whatever number we want to. Initially, there was a hiccup when we scaled to testing many devices in parallel, but I'm pretty comfortable with it now.

We generally plan to increase our usage of this product. More teams are being onboarded to the cloud and although we haven't chalked out a plan as of it, we do plan on purchasing more licenses. 

The technical support is good. Our service-level agreement states that if we open a ticket, it has to be resolved within a certain number of days. This depends on the criticality or the gravity of the defect. There is a proper process in place for that.

I don't think that the support has slipped on their deliverables, so I would rate it a nine and a half out of ten.

Positive

We did not use another cloud provider prior to Perfecto.

Initially, we were doing our testing in-house. To do this, we were maintaining an inventory of mobile devices. Now, with the use of this tool, we have taken everything to the cloud. That means we are not maintaining the inventory anymore, but we are achieving good results. In addition, we are benefitting through cost savings.

The product is a black box to us, and there is minimal setup. Perfecto set up the cloud and they give us access through a browser and URL. Once we get into their cloud, we can pull the devices and do what we need to do.

As the setup is done on their end, it's very easy for us.

Comparing the cost of the product and licensing to what we get for our investment, of course, there are benefits. There is a measurable ROI and we conducted an analysis. In fact, the anticipated ROI is one of the reasons that we chose Perfecto.

Getting rid of our local infrastructure has translated to cost savings. We got rid of VMs, Mac machines, and other infrastructure that was needed. Of course, we do still maintain some devices in-house, but otherwise, we save the money spent on infrastructure. This is from the perspective of purchasing the infrastructure, as well as paying to maintain it.

Overall, it has definitely translated to cost savings and the reason is multifold. We don't have to have a local infrastructure, saving on procurement, support, and maintenance costs. Also, being able to execute tests in parallel leads to additional cost savings.

The additional coverage we get is for the same cost. Specifically, I used to only test using the Chrome browser, whereas now with Perfecto, I get a lot of options. I can test using Chrome, Edge, Firefox, and Safari. This takes the same amount of time and we incur the same cost.

Pricing is an area where Perfecto can do a little better. When we obtain additional licenses, we enter into negotiations with them. 

For customers like us, who have been with Perfecto for many years, it would be helpful if there were more discounts available. 

Before choosing the product, we did an evaluation and the good part was that, in another area of the company, in the wealth management department, they had onboarded Perfecto before we did. This gave us some insight into the product before we began our proof of concept.

After the PoC was complete, we procured the solution.

To sum up, the advantages that we get from this product start with the infrastructure. We save on procurement, support, and maintenance of it. Next, we were limited to only a few devices that we had in-house. We were also limited to only a few browsers. We now get more coverage on the browsers and we have real devices. Finally, the parallel execution translates to cost savings.

My overall feedback on this product is good. My advice for anybody who is considering it is to evaluate and assess the product, as well as conduct an ROI analysis, before procuring it. Individual needs might be different. You might realize your return on investment after four or five years. Or, the break-even point might be longer for you.

To conclude, this tool is pretty stable and robust, so there should be nothing to worry about on the technical side.

I would rate this solution a nine out of ten.

We have all these devices and we test our mobile applications on them. We also use it for web testing, as a backup, when the other web browser testing tools that we use are down. We have combinations of mobile devices on which we do manual testing and we use it for automation testing as well.

Our organization mostly focuses on the front-end. We're a "digital factory" for all the UX and UI development testing. We design and develop front-end solutions for our bank. The solution is used by testers to test any changes related to mobile. 

Having Perfecto to test all the various changes to our applications on virtual mobile devices allows us to complete our testing and regression testing within the scheduled time. It enables us to do pretty much everything that we could do in the actual devices. It's really helpful in both automation and manual testing.

The Perfecto device lab has helped us scale our testing operations. It has become embedded with the success of our business and our projects. It would cripple our productivity if we didn't have it. We're very dependent on it. Our testing has been done utilizing Perfecto a lot lately.

Because we don't have a lot of physical devices within our team—we probably have one or two—and because our team is composed of about 50 resources who do testing, we cannot send them the devices, now that we're all working virtually. Perfecto is a perfect tool for allowing everyone to work remotely.

We have 42 devices in our Perfecto cloud base and all of them are being utilized by the testers. Our testers can access one of those devices at any time. As soon as that device is available, they can use it. They don't have to go to the office to use it. They can sit in their houses and perform their activities.

It has also helped to speed up our testing efforts because testers don't have to go to an office. However, the actual testing requires the same steps that you would run, and takes the same amount of time, if you were using the actual device. The savings come from having all your documents or screenshots saved by clicking one button. That's something that helps a lot. Also, the flexibility of changing from one device to another, just by clicking a button, is very efficient.

In addition, Perfecto helps affect our product's quality because it mimics real device behavior; everything is similar. Productivity-wise, we're able to test all the features and we're able to see what we need to see. For example, we can see the behavior of our application on every device due to the screen resolution of the device. If it's an iPad, the size of the screen is different than if you're using iPhone 12 or 13 Mini. When you go to Android devices, it's also different. Having all of these devices inside our cloud within the Perfecto tools, allows us to perform testing and deliver a better quality product.

The mobile testing is the most valuable feature. Having the devices available virtually to test for all the changes we have enables our team to function. We're all working virtually, as my team is spread out globally. I have team members in India, Charlotte, and Toronto, as well as in Dublin. It would be impossible for all of them to have the actual devices to test all the changes, developments, and new applications.

In recent days I have been exploring the license keys we have with Perfecto and I found out that we have five license keys for web browser testing. That's when I started exploring that area and I'm planning to use it a lot for web browser as well.

In terms of cross-platform testing, they offer all of it, every device available in the market. It covers real scenarios that mimic production so that we don't miss out on any devices that our clients might be using to run the applications we develop. It's been great and very helpful.

The cloud-based lab, with all the devices that we need, is very important. If Perfecto were to go down, it would block our testing, as all testing of our mobile-related changes is done in Perfecto. That means it enables us to complete the tasks assigned to our team. It's 100 percent important for us.

Also, having access to new devices same-day enables the testers to test them right away. That makes everyone productive and efficient.

Because I'm a project manager, I don't use Perfecto’s reporting and analytics, but our business support manager presents me reports where I can see how many devices were used or how much testing was done. Our technical managers use the reporting and I know it has been a very useful feature of Perfecto.

And having the Perfecto tools accessible from a browser is convenient, efficient, and fast. It's easy to go to the browser and launch the application, log in with your account, and you should be good to go. You just select the device that you actually wanted to perform your testing on.

I'm hoping that Perfecto will come up with browser testing as well, because it would be easier to access it.

I've been a user of Perfecto for quite a while now. Even before my current placement, I was a tester elsewhere as well. I have used it for around five years. Now, I don't use it day-to-day, my team uses it. I'm still a user because I do admin stuff and maintain all the devices in our cloud and do acquisitions if we need more devices.

It's very stable. Even before I worked in my current company, I had never heard of it being down. It's always been there and available to use.

It covers device availability. Every time there are new devices in the market, they have it in their cloud. It depends on us if we want to purchase them. 

We're planning to increase the number of devices we have because there are always new devices on the market. We'll have to take a look at the devices that we may not need and acquire new ones, to make sure we test all the latest devices. We want to make sure that no users are left out and that our projects are up to date with the new devices.

We have 500 users signed up in Perfecto, including developers, and I'm sure they're using every different feature available in the solution. It's used by about 20 different teams. It's used for UA testing and regression testing. It's used for every level of testing in our environment.

The technical support has been very helpful. Whenever we raise a ticket from the support website portal, they reach out to us within a couple of hours. That's pretty fast for a turnaround. We don't have to revisit and ask for follow-up.

Positive

I wasn't involved in the initial setup. It was already in place when I arrived.

In terms of maintenance, once in a while we have to do failover testing to make sure we're able to continue business as usual if Perfecto were to go down. Our Perfecto business account manager is always there and she's very helpful. We have another tech guy in their support team who helps us. I and another tester perform the failover testing on the URLs that they gave us, to perform a dry run. The Perfecto team has been very helpful and approachable for any questions that we have had.

I've hardly seen Perfecto down, so far. You only lose the benefit of it when you're not able to access it. So the return, for sure, is 100 percent.

I got a quote for one device the other day. It was $1,300 for two years. That's a little expensive, but it's okay.

There are no additional costs. Support comes with it by default, other than if you need additional devices. Then you have to have that device credit, which is the $1,300 I mentioned.

It's a great tool to use if your work involves mobile development. It replaces actual devices with ease of access. It's very convenient to use. It's very user-friendly and that's something that I like about, for a user, a tester, or an admin like myself. It's very straightforward. You don't need a lot of automation knowledge to do testing. From manual testing to automation, and even for developers, it's an all-in-one package if you're looking for a mobile testing tool.

I love the idea that they have a one-stop shop for Perfecto and BlazeMeter. I saw that presentation yesterday. However, my portfolio doesn't deal with backend stuff.

Perfecto is part of our testing life. When we test, we do it in Perfecto. Without it, our team would not be able to perform testing on mobile devices. It's becoming the "backup app" for our web browser testing tools. It's the life of our testing. It's part of it. It's embedded in our testing life cycle. It's the tool that we use whenever functional testing, usability testing, and regression testing are needed. I don't hear of any other tools that we use for testing mobile changes.

On the mobile side, I'm amazed at the solution. I'm amazed by how they actually came up with the idea and all that it can do for us. It has been good. It helps us a lot in every possible way that we need it. I love Perfecto. I would recommend it, always.

We are mainly using it for automated testing for some devices that we're not able to have physically. We are also using the Perfecto service to do manual testing.

We deploy builds daily, and we have an automated pipeline that goes directly to Perfecto. We do it on a daily basis.

If anyone who doesn't have company-issued devices wants to do testing of our mobile application, we can always provide them access to Perfecto, and they can play around with the app there. That has been really nice.

We recently set up a couple of test cases for automated testing. Being able to run them automatically and being able to see the success and failure rate of those runs every day has saved us a lot of time. It has also been helpful in identifying issues, and it has also been helpful in terms of the timeframe and being able to find them a lot sooner.

It works well when it comes to executing cross-platform testing. I would give it a positive rating in that regard. We're able to test on Android and iOS devices pretty seamlessly.

We are utilizing Perfecto's cloud-based lab to test across devices, browsers, and/or OSs. The device lab is very important for our testing operations. If we have issues with the devices being offline, we wouldn't be able to run our automated tests. 

Perfecto's device lab has helped us to scale our testing operations. Currently, our company doesn't issue Android devices to us. I have an iOS device that's company-issued on which I can do manual testing. For Android testing, we've been leveraging the Android device in Perfecto. It is pretty important, and it has been helping us to determine what our baseline is with the Android builds. Because we're not able to physically see it, having it in Perfecto is super helpful.

It has saved us the cost and effort of having to maintain our own virtual test environment. It is hard to quantify in dollars, but in time, it has probably saved up to 100 hours, not more than that. As we build out our test suite more, it would save us even more time. Running those automated tests running as compared to someone manually testing the application and finding issues would definitely save time. Being able to find issues a lot quicker results in us being able to fix them a lot quicker.

It has helped to speed up our testing efforts, and it has also affected our software quality. It has improved the quality of our software due to us being able to find and fix the bugs a lot sooner.

Manual testing is definitely valuable. The insights feature is also valuable. I just started looking into that to get more data around the results of the testing. I'm looking to explore that one a bit more, but definitely, the manual testing and the insights are the top ones.

Their team is really great to work with. They're very flexible, and they always show care. They prioritize our work, our company, and our working relationship. I appreciate the ad hoc sessions that they schedule to provide help with troubleshooting, provide the information that we're looking for, or do a demo of a new feature that they have. They're always willing and very quick to get that scheduled for us. I appreciate that a lot.

If we could run an accessibility test in Perfecto against builds, it would help us a lot. Currently, that's a very manual process for us. We haven't found a tool that can do accessibility scans for iOS and doesn't depend on engineering effort. Having a feature related to that would be really awesome for us.

I've been using Perfecto for about a year and a half.

We have had some issues in terms of our environment, and it's hard to say whether it's because of Perfecto, or it is actually our app build that is introducing some of those errors that we have to maintain. So, overall, I would rate Perfecto a seven out of ten because sometimes, we also have lagging with some of the devices. We get error messages, but we're not quite sure what's causing them. However, whenever we do submit tickets for them for support, they're resolved pretty quickly. So, I would rate it a seven out of ten for stability.

We have an end-to-end (E2E) environment that we deploy for testing in Perfecto. Currently, it just supports our product team, which has about 10 people. In the Credit organization or department, we're the only mobile team. The web team, with about 50 people, is evaluating using Perfecto, but currently, our product team is the only one that I'm aware of in our organization. In the main organization, there is another app team that uses Perfecto. They use it a ton more than we do just because their product is much bigger. Their team roughly has 200 people.

There is a ton of potential for scalability because the other team that uses it is huge as compared to ours. They use it pretty heavily. They run thousands of tests a day, and I'm able to see their metrics as well. So, it definitely has a huge capability for scaling.

Their support is amazing. We have a technical consultant from Perfecto with whom we work. Whenever we need help with troubleshooting, he's very responsive over email. He is also very responsive in terms of scheduling a working session with the engineering team. It is easier to troubleshoot that way. Their customer support is great. I would rate them a ten out of ten.

Positive

We didn't have anything else in place before.

I wasn't involved in its deployment. In terms of maintenance, across the enterprise, I'm sure everyone has their own scenarios, but in our product team, we do have to have maintenance. We run into scenarios a lot where there are issues with our end-to-end environment, and we have to invest some time to make sure that everything is running as expected.

It is not a scheduled maintenance per se. Usually, if we have a line of sight to some updates that are coming from Perfecto, we can account for them, but a lot of the time, they are the changes within the app, which as such wouldn't be considered as maintenance. So, we do have to maintain it, but it is not scheduled maintenance, and we do have to put in effort just to make sure everything is running. Usually, just a pair of engineers do that, and it's more on an ad-hoc basis as opposed to on a scheduled basis.

We started seeing its benefits in the last three months. Last year, we were still doing a lot of the initial setup, and we weren't utilizing it as much for testing, but this year, we are getting a lot more usage out of it. It is just because of the way the team has been working. It is not a limitation of the tool.

I am not sure about its pricing, but from our perspective, licensing has been easy. Anytime I have new users or requests for users that want to get added, it's a very simple process. I just give the architectural owner of the product the name and email address, and they're able to easily add a new user. We don't have any issues in regards to getting licenses, but I don't have any insights into pricing.

I personally didn't evaluate other options. This is our enterprise tool. When they selected it, they did evaluate other options, but it was done way before we got the tools. So, I'm not sure what the other options were.

Perfecto has a lot of taped demos and webinars and informative videos to help you discover the features of the tool. They are very good to get more familiar with how to use it and how to find the best features that work for your team. I would also recommend working closely with their account management team. Their customer support has also been very helpful. When we get stuck and we don't really know which direction to go, we have a contact, as opposed to just trying to figure it out ourselves. It is always easier to just talk to someone from their team to get a more clear direction. So, the informational videos that they offer as well as leaning on the actual representative from the company to help you define the best use of the tool for each product are the way to go.

At this time, we are leveraging Perfecto's reporting and analytics very lightly. We're just using the default dashboards that come with the tool. We're just getting into having more useful data and using it more. I expect to be doing more with that. These analytics will be important for us because we'll be able to understand our failure versus pass rate for a lot of the automated testing. We won't be able to understand why those failures are occurring, of course, but at least, it will give us an indication of fixes that we need to address which might have got missed with manual testing.

Perfecto's cloud-based lab provides same-day access to new devices, but it is not too important for us. It depends on what the time window is. Usually, we get it within a week of our original request, and that seems to work fine for us. So far, we haven't had an immediate need to have the device available on the same day.

Perfecto supports a range of open-source technologies, but I don't have enough experience with that. Currently, we're using Appium by default, and I don't know if that's a limitation of Perfecto or an industry limitation.

I would rate Perfecto an eight out of ten.

Our main use case for Perfecto is for our testers to test their mobile or PC applications.

One of the benefits of Perfecto is that it helps us keep our devices up to date with the right OSs, because we test across different mobile OSs. The solution has been instrumental in helping us to keep our devices at the right levels for our testing.

The solution also helps us to eliminate defects. Any application defect is a bad thing. I'm the "application police." If something is wrong with it, I'm going to give you a ticket and ask you to get it fixed. As a bank, we cannot have defects in the applications and Perfecto is instrumental in that process. That's how it helps us, organization-wide. It really stinks if one of our end-users opens up a mobile app, expecting they can do something but they can't because of a problem with the application. Testing with Perfecto, we find those defects and we get the developers to get them fixed before the end-user sees them.

When we retest and they pass, we're achieving the quality results that we're looking for.

And it has saved us tremendously on the effort and the costs of maintaining our own virtual test environment. We don't have to hire so many personnel to take care of those devices. Perfecto has people employed in its lab to physically work on devices when needed. We would have had to do the same thing if we didn't use the solution. As a high-profile bank, hiring an employee includes doing extensive background checks, and those kinds of things cost the company. Using Perfecto saves time, resources, and it saves on physical intervention with the devices. Overall, it results in huge cost savings for our company. 

There are a whole bunch of things that I like about the solution, but I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. The reports are very detailed. They provide all of the information needed for the testers and developers to really look at their applications and know if there are any issues or not.

Perfecto is excellent when it comes to executing cross-platform testing. They offer support, and their support is excellent. They're very responsive. With a lot of those cross-platform applications, we have to work together so that we can set up the configurations between the two interfacing applications, but it's very good.

When it comes to reporting, we take a look at how many users are using Perfecto, and for which devices. That helps us to narrow down exactly which devices are being used the most for the applications our teams are testing. I find the reports and the analytics to be extremely helpful. I cross-reference the user reports with the users' lines of business because it's helpful for us to know which users and which lines of business are using Perfecto.

The most widely-used reports are the individual testers' reports. When they run their automation scripts, it gives them a lot of the details about the test run, such as which components passed or failed. And when something fails, it points that out with an error so that the development and testing teams can look at what that error is pointing to.

And the cloud-based device lab is extremely important to our testing operations. The mobile device testing is one of the main things we use Perfecto for, although we do have a couple of teams that use the PC platforms as well. On a scale of one to 10, the importance of the mobile testing is a nine or 10.

Our testers have other means of testing our PC applications. They can test some of those applications from their local machines. They may or may not need access to Perfecto's PC resources, but for mobile devices, we have users who are offshore, and they do not have access to physical mobile devices. Having Perfecto's cloud-based application, users in India can access the cloud and do their testing. It's extremely important for our organization to be able to provide mobile devices for offshore testing.

We mainly use Appium among the open-source technologies that Perfecto supports. A couple of our PC teams may be using Selenium, but for the most part, the mobile users are using Appium, and the integration between Perfecto and Appium is excellent. If there is any issue for our Appium users, a lot of the time it's not on the Perfecto side, it's on our side; usually configuration issues. 

I can't really say anything negative about Perfect, but if we talk about feature enhancement, there are a couple of things that I'm working on with our customer support manager. I'm hoping they can support on-premises instances. We have been working on a JIRA integration with Perfecto—and I'm extremely impressed that they have that—but at this time they're not supporting onsite JIRA instances, which is what we have.

I love the idea that the testers will be able to run either their manual or their automation scripts, and that the test report will automatically go into that team's JIRA instance. That will save some time. The testers won't have to extract the report, save it, and bring it over to their JIRA instances. Then, when the scrum master reviews the test results, if there are questions, they have all of the details from the report right there.

One other enhancement that I've already spoken with them about is that they don't support Android for ADA (Americans with Disabilities Act) testing. They mentioned that they're going to try to begin supporting that later this year.

I have been using it off and on for at least seven to 10 years.

I support our teams and help them with their test environments and mobile device configurations. It meshes perfectly with my job role and what I like to do. Perfecto is my favorite tool.

I would rate the stability of the solution as an 11 out of 10. There has not been one moment that Perfecto has not been accessible. They schedule their maintenance windows during the weekends, and on most weekends, nobody in our company is working. And even on most weekends the platform is up. 

I never receive an email from a user saying, "Hey, Perfecto is down," or that there's a problem. That is not to say it won't happen, but if it happens we will address it at the time. But since I've been using Perfecto, we haven't had any issues like that.

We have open cradles where we can add devices and shift things around as needed. The scalability is 10 out of 10. All we have to do is specify which devices we want, where we want them, and what OSs we need those devices on.

As a ballpark figure, on both of our clouds, there may be 1,100 to 1,200 users.

We're talking about getting a Perfecto ADA demo scheduled. If our teams like how ADA works for them with Perfecto, I anticipate a lot more usage of the application.

Perfecto's support goes above and beyond, with its automation support. We have biweekly meetings with some of Perfecto's support staff and they help us with automation issues. If there is some problem that Perfecto can help with, they are there. Their support is excellent. They're very friendly and great to work with.

From time to time, we may find a URL that needs to be whitelisted or allowed with Perfecto. The support team has been great as far as making sure that IPs are allowed, and that things are up on the Perfecto side for any integration.

Positive

When I started, we were using an in-house solution. Later, I was moved to Perfecto as the admin, and the more I started using Perfecto, the more I really liked it. Using Perfecto, there weren't as many issues as we saw on the other platform. Perfecto is better than our other tools, and I've been trying my best to move others off of other tools they're using and onto Perfecto.

The switch to Perfecto was an internal process. I was an admin on the other solution, and I had to do all the device upgrades to that platform. But there was a security issue with that process and those upgrades were moved to a secure location within our company. We still have the other solution, but we don't let the word out about it too much.

I was not involved in the initial setup, but I was extremely involved with the configuration to comply with our security requirements.

We're accessing Perfecto through private clouds that they host. We have two instances, enterprise and wholesale, and we configure our users here for activation and utilization in both of those clouds. We have an internal solution in which we add the users to Active Directory groups. If a user leaves the company, they're not able to access Perfecto through our clouds. That setup complies with security policies.

Perfecto is only used in our test environment. We cannot place any actual production data outside of our company so we use fictitious data. It's not data about real people or real accounts. Those devices are configured for our test environment only.

I go beyond what the normal Perfecto admin would do. Normally, the admin would just help facilitate the accounts and get users logged in. But I go a step further, because some of our users are unfamiliar with the environments and how to set things up. I help them with their test environments, on our side, for the mobile devices. That's not something Perfecto support would be able to do.

We had to involve a consultant from Perfecto because certain changes were needed on the Perfecto side, as well as on our side, to comply with our security regulations. That worked great. The consultant was great to work with, and we got everything matched up, configured, and working. He was extremely helpful and instrumental in our success in meeting our security requirements. Our security team is happy.

We have another platform that supports some limited ADA testing, but it has been having intermittent issues. Perfecto offers some ADA support and offers pretty much 100 percent uptime. That saves time because testing with the other platform has to wait until those intermittent issues are resolved. Because Perfecto is up, the testing can go on. The other platform can become unavailable for three or four days and that could be brought down to zero if we can use Perfecto for ADA.

There is another team within our company that went off on their own and got an application called BrowserStack. However, my manager and our team are trying our best to get that team to stop using BrowserStack and over to Perfecto. That would also result in cost savings for our company by saving the cost of that platform. From my comparison of BrowserStack and Perfecto, I like Perfecto a lot better because of the UI, reports, and everything else.

I've seen a lot of new functionality added over the years, a lot of performance and speed enhancements, and I think the product is very good. It's one of the top solutions for device simulation and for users to perform testing. I'm very happy with it and I really like the product a whole lot. It's a very good application.

Perfecto is doing everything that I need it to do, and I enjoy using it. Sure, we have a few open issues, but we're seeing timely updates.

We have a cross-section of mobile devices and web devices, and we use those for browser compatibility and device compatibility testing, as well as functional testing of several apps and websites. So, we have testers that pull up the appropriate device, whether it's an Android or iPhone, and validate that all the functionality of the app or website is working as expected.

It is hosted by Perfecto. They take care of all the hosting.

Benefit-wise, by being able to share all the devices across 800+ users, we get a much better cross-section of testing across a wide platform of devices versus having to buy one or two phones for each tester. To be able to share a pool of devices is a huge saving. It is also a huge benefit to be able to test on a wide variety of devices. So, it saves us from having bugs go out that would only be found on, for instance, two versions back of OS or a device that's not really common but is used by our customers. If our app is having trouble running on a Google Pixel 5, we're going to find it because we have access to a wide variety of devices in our Perfecto cloud.

The reporting feature is really tough to find in some of the other products that are competitors. Having your CITB-type dashboard, where we can see the test results and see recordings of each test that passed or failed, is probably one of the distinguishing aspects of Perfecto. We also recently started using SSO for login which has streamlined our onboarding process and improved security.

Its performance should be improved. Anything to speed up the user interface would be a great help. On the admin side, it would be useful to be able to filter/sort any page with a list of items and export it to a CSV file.  

In the web testing area, we discovered when you select Windows 10 or 11, the VM is running a version of Windows server that emulates the desktop experience. We would like to have the ability to run on the actual desktop version of Windows 10 and Windows 11.

I have been using this solution for six years.

I would give it pretty high marks. It doesn't have a hundred percent uptime, but it is pretty good. I would probably rate it a four or a four and a half out of five.

Its scalability is very good. We're a fully agile organization, so most of the users are testers, but we also have developers, product owners, and maybe a handful of business analysts who use it. It is currently being used extensively by a team of 800+ users. 

I commend their tech support. Their support is very efficient, very helpful, and very personalized. They just go above and beyond. I never had any complaints about their tech support. I am definitely satisfied with their support.

Positive

The initial setup was done before my arrival at the company.  But since then I have been an administrator and have found it to be easy to manage the device pool and user base. 

There are four of us who do the admin for Perfecto. Mainly, it is a liaison with Perfecto on a routine basis. We don't do any updates to the software since it is all in their hands. We're paying for software as a service. So, our main duties are adding users, troubleshooting problems that our testers encounter, and managing which devices are going into the cradles. I and the team of three others are responsible for those admin duties. Everyone is a tester.

ROI can be measured in terms of the number of mobile devices we DON'T have to purchase since we can share our device pool across 800+ users. It also provides a solution for our offshore testers who otherwise would not be able to test on a mobile device since we cannot issue company phones to our offshore sites. Aside from that, the ROI is the broader coverage we gain by maintaining a device pool that has a mix of devices that closely matches what our customers are using, as well as a range of OS versions that allow us to test against versions N, N-1 and N-2 of IOS and Android.

The pricing is competitive when you compare apples to apples. We have seen cases where comparisons are made between the cost of public cloud vs private cloud. Private cloud is more expensive but it is required in some cases where security is of utmost importance. I advise taking advantage of Perfecto's Black Belt service. The personalized service you get is worth the extra cost.  Perfecto's support team is amazing. We normally get same-day resolution of issues we report, and in some cases, they are resolved within the hour. Consider this when looking at price and comparing against other tools.

We continually evaluate test tools for use by our QA team. In the past, we have reviewed SauceLabs, BrowserStack, Kobiton, and others. We believe we get the best value from our investment in Perfecto.

I would recommend trying it. Go through the trial period. I would imagine that anyone who does so would compare it favorably to the competition. It may not be the lowest-cost product out there, but you get a lot of value from the tech support and from the feature set that is out there.

I would rate it an eight out of ten. It is a great solid product for mobile testing and web testing. It has got great features, and we've utilized pretty much all of them.  We use it heavily for both manual/exploratory testing as well as automation testing.  

We use it for manual mobile testing and a little bit of web testing.

Perfecto makes it easier for us to do mobile testing, rather than shipping mobile devices. They always have the newest versions.

Although I can't comment on cost savings, it is useful for us because of our offshore-onshore model. We are able to utilize the devices even if we don't have a physical device available to us.

Mobile testing is the most valuable feature as it has reduced dependency on physical devices. We are located offshore and we don't have the physical devices, and shipping physical devices after every new release would be a difficult task. But with Perfecto, it is easy.

Perfecto’s cloud-based lab to test across devices is very important because as a banking and financial project, we need to ensure that our app is working on all devices, irrespective of version.

It is slow compared to physical device testing. The interactive speed could be improved. 

And sometimes we have issues with our app not working properly with Perfecto. Even though it is working perfectly on the physical device, in Perfecto we find issues. That is another thing we'd like to see improved. Perfecto support said not to use the instrument option so that you do not face that issue. But unless we use the instrument option we cannot do the inspection. We haven't had this issue with all versions of our apps, only with a few.

I have been using Perfecto for almost five years.

The stability of the solution is good, around eight out of 10. And the new features they're bringing out are also good. They are useful for us. 

The scalability is a seven out of 10.

Whenever we have an issue we raise a ticket and it is resolved pretty fast. The technical support is great and it's quicker now than when I first used it. The follow-up is also great.

Positive

The initial deployment is not that complex, and the solution doesn't require any maintenance.

We don't have to carry physical devices from one place to another. They are virtual, so we can always use them without any issues. It has been very helpful from that perspective for us.

Overall, it's a great solution. They just need to improve on the slowness. I would definitely recommend it.

We have a website as well as a mobile app, Android and iOS. We do a lot of functional, UI-related testing. And we have automated a lot of test cases through Perfecto, and we have even started to automate mobile now.

Perfecto's device lab has helped us to scale our testing and it has made testing easier for us. We've been able to run our regression testing. We have hundreds of test cases, the majority of which are mobile-related, and we use Perfecto for them.

We've also been able to get our testing done a lot faster, especially during COVID when we weren't going to the office and didn't have our physical devices. We ended up using and being dependent on Perfecto a lot.

We have access to 15 or 20 devices. Even if some of them are being used by other testers, I can utilize another one. That was not stopping my testing when I was doing manual testing. That has helped us a lot

And for automation, we run our jobs at night when these devices are not being used by manual testers. Just the fact that we can automate our tests on Perfecto and run our jobs and have test results in the morning has given us a lot of flexibility and has helped a lot.

We've been able to automate test cases that are mobile-related. That's helping a lot with our automation. Obviously, you can't automate on a physical device, it's something that we do on a virtual machine. The fact that we can automate on mobile, Android and iOS, is working well for us.

We use the CI Dashboard. Perfecto's analytics are very important because that is where we have our test results, which we later input into Jira as well.

We use the device lab in our testing operations and mobile is one of the things we use. We have multiple devices, about 20, including Android and iOS. That's what we are running our automated test cases on. The device lab is extremely important because we are doing our automatic test cases and even our manual testing there. We don't have physical devices. We go into the Perfecto website and use all the mobile devices available on the cloud.

Also, we're working in Agile and we need results ASAP. The fact that the lab provides same-day access to new devices is extremely important to us.

We are mostly using Selenium and we are slowly putting our hands on Appium as well. That's working out well for us too.

When using devices on the cloud, it lags quite a bit at times. I know that these are real devices that are being projected on our laptop screens and monitors, but if the speed could be improved, that would be good. For example, if I open up a Google Pixel on my laptop, using Perfecto, just navigating on Google Pixel is quite slow. Improving on that would help us a lot.

I've been doing manual testing with Perfecto for a while, but I started automation a few months ago. That's when I really got to work with Perfecto properly.

My main area of concern when it comes to improvement of the solution is that it lags often. If that could be improved, then it would make our testing a lot faster and more efficient. Apart from that, I'm pretty happy with Perfecto. It's going in the right direction.

It's definitely scalable. For mobile testing, if Perfecto gets faster then we can utilize it in our bigger test cases. We do have some data-related test cases that really slow it down. If Perfecto can find a way to handle that, it can definitely grow with us.

The technical support gets back to us within a few days. I have not used it that much and I don't have complaints, nor have I heard complaints. My team lead is the one who uses the technical support more often, and he's quite happy with it.

Positive

It's a great tool for running functional, UI-related test cases. But I would give a heads-up about the speed that the mobile devices run at on our laptops or monitors, because it can slow us down a bit.

But in the bigger picture, it helps us because, obviously, even when doing manual testing, you're not going to be testing on your own personal devices. With a lot of people working from home, you cannot give cell phones to everyone to test on. Overall, it definitely helps.