What is our primary use case?
We use it to automate regression test cases for the mobile app of our application. We use Android and iOS for execution, in both English and French. We also use the solution to execute tests in different browsers.
How has it helped my organization?
It has really helped us speed up our testing process. We are able to execute tests overnight and see the results quickly.
Our devices are in the Perfecto cloud and that's extremely important because we have monthly releases and a lot of OS upgrades in our mobile application. We always have to validate our app for both forward and backward compatibility, as well as in the browser, to make sure that all the features of the application are working. Perfecto facilitates delivering the product on time, through the cloud devices, so that there is no impact to our end-users. That's extremely important for us.
The Perfecto lab also provides same-day access to new devices, which is another very important aspect. If we want to do some tests to make sure the features work in the latest Android 11 version, we don't have to contact the Perfecto team and ask them to provide us with access to that particular device. We can get that access and execute tests against the devices.
What is most valuable?
The CI dashboard tool is very good, as is the Live Stream monitoring. Whenever I want to monitor execution, I can open multiple tabs in Perfecto and it is easy for me to refer to the CI dashboard and the Live Stream.
It's also very user-friendly. I can see and locate any issues that come in and it's easy to see which test case has failed. Under the Perfecto job name, if you click on it, it will show how many passed, how many failed and due to what issues, such as "element not found" or "object not found" or a Java error.
It also has video recording with the screenshots which is really good.
Another very good feature is the cross-platform testing. We are able to execute in multiple browsers and multiple Chrome, Safari, and Microsoft Edge versions. We have the capability in Perfecto to develop scripts. We just need to capture the capabilities of the different browsers and update them in our Quantum framework and we are able to execute. It's very easy.
There is an option in Perfecto to capture the capabilities using the tool itself. If we want to run it in, let's say, a particular Android device, we can capture it from there and put it into the code and we can update the information and execute.
The range of open-source technologies that Perfecto supports is really good. I don't see any issue in using Perfecto for that.
In terms of its reporting and analytics, whenever I'm doing any execution for a new release, I just put in my release number and it's easy for me to look for what happened in the test case execution I ran for the previous month's release. I can easily locate it and I can get the information.
In addition, I can schedule the devices I need in Perfecto and I can block those devices so that other users on the team do not try accessing them.
What needs improvement?
The reporting can be improved due to the scrolling you need to do. We have an option for filtering and generating reports, but we are limited to three months of history. After that, we are no longer able to see the information. If they would maintain the history for a minimum of two years, that would be helpful.
Another issue is that sometimes, when we have to locate a button through physically accessing a mobile app, versus through Perfecto, we feel that Perfecto is a little slow. If they could improve on that slowness in accessing the app, when we want to click a button, that would be great because we feel the difference. An improvement in the connectivity speed is required.
For how long have I used the solution?
I have been using Perfecto since 2019.
What do I think about the scalability of the solution?
I would rate the scalability as an eight out of 10.
How are customer service and support?
I work closely with my Perfecto vendor and he is very supportive. Every time there is a new update he will share it with us and get our feedback to help enhance the tool. They help update the core into the Quantum framework. We have had some key, latest, knowledge-sharing sessions with their team regarding these new updates.
We also have the Perfecto support portal, where we'll raise incident tickets. If, for example, a device is not working, it's not connecting, or a tool is not supporting what we're doing, we will raise a defect ticket. We can indicate the severity and wait for updates from the Perfecto customer solution team.
In terms of objects, we had some challenges initially and we told our point of contact about them. They updated the solution as a result. Also, sometimes we were not able to access iOS devices. Every time we wanted to go to iOS devices, we had to trust the app, otherwise the lab would not be launched. And we had some challenges accessing the iOS app from the Quantum framework. We worked with the Perfecto team and we got solutions to those issues.
That process is ongoing. We ask them about whatever ideas we have for them to improve. One of the ideas I asked about is because there are a lot of things we need to export into ALM Test Lab. It would be great if Perforce could create a plugin that could be added to the Perfecto portal so that we could just provide the ALM URL domain name and password to connect it to Perfecto. Right now, we need to write code to integrate it with ALM and export the results once the execution is done. An integration with Jira would also help.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
I have not used any other tool for a long time.
How was the initial setup?
I wasn't involved in the initial setup but there is some ongoing script maintenance involved.
What other advice do I have?
It's user-friendly, adaptable, extendable, and scalable. We are able to understand everything through the UI. They just need to enhance it further to adopt the kinds of changes we would like to see.
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.