Sauce Labs Reviews

I'm on a team called the "Quadrant Three Mobile" team. Our focus is mobile testing and mobile browser testing. Sauce Labs provides iOS simulators and Android emulators that we use in our tests. It's running on our private cloud, and we have a Sauce tunnel set up to communicate between our environment and the Sauce Labs environment.

We don't often use the Sauce Labs UI to track the tests we need to do. We have a UI on our end to do that. However, we log in to the web interface to view the dashboards to see the test strength and what was triggered or to search for specific tests exact testing. The Sauce Labs dashboard has links to videos of the actual test runs, which helps us deal with debugging issues and test failures. 

Quality control engineers are the main users of Sauce Labs at the company, but developers also use it for making the CRM code base. They run tests on Sauce Labs mid-development to ensure their piece of work is going well.

Sauce Labs helps us identify the root causes of bugs. The solution offers a lot of flexibility by providing the latest iOS and Android emulator versions, and even the Appium updates when it comes out in the market. 

Those have been helpful because the apps we test on Sauce Labs are used by customers across the globe, and we want to be sure we're testing on the latest iOS and Android versions as well as older versions that our customers still use heavily. Sauce Labs has done a decent job of giving us all the versions we need. 

Sauce Labs' dashboards are a good supplement for our dashboards and UI. The web interface is useful when we want to triage a specific test run or overall across different jobs. It's also handy when we want to look at what results we get when we run a test across a whole account.

Sauce Labs' ability to test the visual aspects of our apps' UI has been crucial. At the end of the day, mobile testing is not only about functionality but also making sure your UI looks good in different languages and works well. Sauce Labs supports screenshots whenever test failures happen, giving us a good idea if something is broken on the visual side.

Sauce Labs' logging features could be improved. For example, when we run a mobile test on iOS, we want a way to capture the logs generated. It has some drawbacks there. Sauce Labs isn't able to give us everything we test for. 

Sauce Labs' dashboard could be improved by adding more filters and allowing more customization options. There was one instance where the dashboard on the Sauce Labs UI didn't meet our requirements, so we had to use the Sauce Labs API to create some apps and dashboards on our own. The API endpoints could be a little more robust and customizable. 

We have been using Sauce Labs for four to five years now.

Sauce Labs' reliability is one of the most important aspects of the platform. They have a status page to notify if there is any downtime. We rely on Sauce Labs because we have a continuous development process here, and we are testing code that comes out through Sauce Labs' emulators. 

However, we've had occasional performance issues. For example, there are times when devices need to be restarted or the UI takes longer to load and things like that. The virtual emulators and simulators have been good, and their hardware and infrastructure have been standing up well.

I think Sauce Labs has proven that their platform is pretty scalable because they give us a broad selection of versions available for iOS and Android. They also support their emulators with good virtual machines, so Sauce Labs can scale their infrastructure to their customers' needs quite well.

I don't know exactly how many users are currently working with Sauce Labs in my company, but it's in the hundreds. We'll certainly expand usage in the future because we're planning to offer new kinds of mobile apps, and when those come out, we'll need to test them using Sauce Labs. As the number of products we offer increases, the usage will too. We use it for nearly all of our functional testing.

I rate Sauce Labs' support at about seven out of 10. We put in certain requests with Sauce Labs, and sometimes it takes a while to get solutions implemented, depending on the priority.

My team didn't deploy Sauce Labs in our environment. A different team handled that. We did the initial installation back in 2016, and I don't recall it being very complex. 

They had some binaries and decent instructions on how to set up Sauce Labs' tunnels. They also give instructions on how to capture logs of those tunnels to see traffic between your environment and the Sauce environment. All of that is pretty well documented, and support teams have been quite helpful. Whenever we needed something out of the box, we've been in touch with them for it. 

I can't really quantify a return on investment, but testing on Sauce Labs has helped us catch issues and fix problems much sooner than we would have without it. That has saved us money.

I rate Sauce Labs eight out of 10. I recommend it for mobile testing. Sauce Labs is a reliable vendor, and you can trust their network. They have a good platform, with broad support for various mobile OS versions. Their availability is also high in the region, so we seldom see outages. The dashboards on their website for tracking your mobile testing strength are helpful. Overall, it's a good product for anyone wanting to do mobile testing. 

The main aspect I've learned from working with Sauce Labs is that it's not easy to maintain a range of devices or virtual environments and provide it for mobile customers. We have tried to do that at my company, and it's challenging. I definitely like what Sauce Labs is doing in terms of providing that reliable network for us. We're also learning more about our own product through testing on Sauce Labs. The sooner you test, the quicker you identify issues in your own application. 

We mainly use it to run our test cases for different platforms. We're able to run it for multiple browsers and multiple devices. We use it for about 90 percent of our CI/CD test cases.

We are using it for automated testing and not for visual testing or performance testing.

It's important that the solution gives us access to automated functional testing, as well as browser/OS combinations, mobile emulators and simulators, and real mobile devices, all in a single testing suite. That's particularly so for our client-facing applications so that our customers are able to access our products and offerings through any device and platform. For those types of applications, it is a huge help. The number of browser OS combinations, mobile emulators and simulators, and real mobile devices it offers is really good. There has not been a case where we say we want a given combination and we don't have it.

One of the most valuable features is that we do not have to have the cross-platform testing vehicles in-house. Sauce Labs gives us the ability to test across platforms and that really helps give us confidence in our products.

One of the challenging areas for us is the reporting and the matrix. It should be based on roles, but right now it is only available for the admin role. The admin role can really do a lot of stuff, but our infrastructure team, which holds the admin role, is not ready to hand it over to us on the testing team. If Sauce Labs had permissions associated with roles, where this role could do this activity and that role could do that activity, it would be easier for us.

We have raised this suggestion with our product manager and she has put it in the product backlog, but, of course, she cannot guarantee when that will be looked at.

I joined this company seven months ago, so I've been using Sauce Labs for that long. But the company has had it for about three years.

From a solution stability point of view, it's good.

We are able to run some of our tests in parallel on the solution, within the constraints of how many licenses we have. We have some 50 testing teams across our organization, and we use them. There has never been an issue with not being able to execute tests.

Latency, due to Sauce Labs being a cloud-based solution has not been a concern.

We did have downtime in late September last year, but it was because of our lack of knowledge in terms of the configuration. Sauce Labs support was really quick in responding and triaging and fixing it, which was really great.

With scalability, they are going in the right direction. I have attended some of their new product webinars on API testing, and I really found that to be cool. We may use it, or evaluate it at least, once we start going into that direction. That goes for visual testing as well. If they are able to deliver on some of their promises I really look forward to that and seeing how we can utilize them.

We could use Sauce Labs even more. While we are using it in our testing teams we are not yet there for things like API testing and visual regression.

Our next step, in terms of our exploration of the solution and how we want to use it as part of the CI/CD, is that we are moving into GitHub Actions. We were using Jenkins and it worked well with that. We are moving to GitHub Actions and trying to figure out how that will work.

Overall, technical support has been really good. They have responded to our questions and have triaged pretty quickly and followed up.

The reports that our account manager at Sauce Labs provides are helpful to us to see where our gaps are. She also helps us evaluate where we could be from an industry-standard perspective. She gives us an idea of what others are doing, what she's seeing out there, and in which areas we can improve.

In terms of using Sauce Labs’ technical expertise to help integrate automated testing into the CI/CD pipeline and DevOps toolchain, we have not done that yet. That is more due to the way our organization is structured. We, as the testing team, do not have access to how the tool is managed or how it was set up. There are probably gaps in terms of our understanding of how the tool can be used effectively. But we have discussed this with Mandy, our account manager, and she has offered to do a free session for our associates, which is scheduled for next Friday, and we are looking forward to that. It should help us understand what the best practices are and whether we are using it right.

From the perspective of our organization's roadmap, we are new to testing automation in general. Our focus is on automated functional testing. There are certain use cases for visual testing, and there have been talks about performance testing, and an evaluation will be going on for that. As part of the workshop next week, Sauce Labs will also be presenting some of their offerings, so we'll be looking at those as well.

Positive

Sauce Labs is a tool. It can fit the needs that you have, but you first have to figure out your needs. Once your team has been able to successfully use it, and things have stabilized, go to the next need you have. That's exactly what we are doing. Once we get our functional testing to be stable, we'll look into the next performance test case.

Don't try to do everything together. When you're going with a new solution, it takes time for people to adopt it.

The process could be different for a large company like ours, versus a smaller company where there are just a couple of products. They may be able to move faster. For a financial institution like us, there are so many roadblocks that we have to go through. I do understand that the licenses are expensive, and you do not want to get these licenses and have them sitting idle.

I would rate Sauce Labs a 10 out of 10 because I have not been able to find any use case that Sauce Labs does not provide.

Our healthcare company has a QA team who uses Sause Labs to review various metrics.

For me, as a software engineering manager, I am able to see who from my team is logging on, when are they logging on, and other things in regards to activity within the environment.

The platform is optimized for automation and integration with major CICD platforms and developer tools. This allows us to speed up our production without burning up all of our resources.

It also provides access to automated functional testing (this couples nicely with our other functional testing that we have), visual registration testing (which is as easy as a click of a button), and front end performance testing (which provides a clear understanding of our current state of performance).

Sauce Labs provides us with an all-in-one solution that saves us time, money, and human capital.

Since this is an all-in-one testing site, we are able to take advantage of the browser OS combinations, mobile emulators and simulators, and real mobile devices. This is important to us since we have a variety of users, browsers, OS, etc.

I would like for there to be more detail in regards to the quality of our code i.e. how many failures occurred, how many passed based on industry standard metrics, etc.

I have been using this solution for a few months.

This is a stable solution, I have not had any stability issues.

As told by our rep, this is a fairly scalable solution.

I was not a part of the initial setup though I have all the reason to believe that it was straightforward.

You will save a lot of time thanks to this solutions ability to run parallel testing. My team saves about a week and a half on our deployment schedule. Just be sure that you have settled on your design prior to implementation.

Our company has different divisions, like research and development, manufacturing, and research laboratories. There's also a department called Global Infrastructure Services. In total, we have five or six areas. In some cases, Sauce Labs is only used in a segment of a department. Take, for example, research and development. They might not use it in the laboratories, but they have a digital health section that works on apps or digital solutions for medical diagnostics.

In this example, the primary use case is developing different software applications for medical diagnostics. They use Sauce Labs for testing applications in different operating systems and environments. We do regression testing across various ways someone might use an application. Instead of securing physical machines, they use Sauce Labs to do that. Many of the applications are internal to the company, so we use Sauce Connect to link up to the internal network to test those applications.

We do a combination of parallel and sequential testing in Sauce Labs. Whenever you log in, you see all the tests on the dashboard. You have concurrent and parallel tests running, but some people will also use a sequential one.

Before we started using Sauce Labs, they needed to use physical machines to test applications. We required a Mac or Windows laptop to try different combinations, so it used to be a cumbersome process. Sauce Labs made testing more straightforward, so we could run more elaborate tests. 

Sauce Labs provides us with more combinations to test, so we can keep adding platforms and devices to our network. That's been a very seamless experience. If there's an iOS or a private device we need, Sauce Labs has helped get all those set up when needed.

As platform administrators, we get requests from different departments. Project teams using Sauce Labs come to us and say, "We have these three or four devices, but we might need another one that is private to us with this combination of operating systems and browsers." We get those spun up in Sauce Labs and have customer support. 

The ability to run tests in parallel has cut down our test execution time for every application we use. The more we can reduce testing time, the easier it is to deploy an application, so running parallel tests is obviously an excellent way to do that.

It offers a decent range of browser-OS combination simulators and mobile device emulators. Our company tries out lots of different applications and solutions. It's not just for product development but for any application we use internally. For example, we use JIRA for project management, but we tried many solutions before deciding on that particular one. 

We've been using Sauce Labs for a long time, even though there were other candidates we could have used. It's a robust platform that has met our needs, especially with Sauce Connect. We have internal applications that aren't available externally, and Sauce Connect makes that possible.

Nothing comes to mind right now, but this might be because I'm an administrator and not an end-user who is familiar with the application's use case. If we have teams asking for something that isn't already there, we communicate that back to Sauce Labs. That hasn't happened so far in the time that I've been in this role. 

I may not know what should be improved on the platform, but I think it could offer a greater variety of testing solutions. I know there might be competing solutions that cover more aspects that I'm unaware of, and it could expand a bit more. We've done a lot of automated testing scenarios, and that's all the rage these days, so I think Sauce Labs could look at potential ways to improve and cover other scenarios.

I work for the IT team of a pharmaceutical company, but different departments use Sauce Labs. I belong to the team that manages all applications the company uses, so we provide Sauce Labs for other users but don't use the product ourselves. I support it from the administrative side, but I've only been in this role for a few months. 

I haven't seen any downtime in the few months that I've been in this role, and we haven't heard any complaints about latency, but we've always used a cloud-based application. We wouldn't know the difference. We could compare if we had an on-premise version, but we've always used the cloud version, and it's the fast version.

We have purchased a consistent amount of licenses over the years, and we are looking to grow with a focus on automated testing this year.

Sauce Labs' support has been awesome. We have monthly support calls to catch up on any requests that we might have. When there is a new update or release, somebody from Sauce Labs comes to talk to us. It's not just us, but all users. It's like an open house. We have been planning that for some time now, but COVID has put a damper on those plans. When we meet monthly, we share any findings on our site, and they come up with best practices to share with us as well. We are satisfied with Sauce Labs' customer service.

A supply management team deals with pricing and licensing. We look at the utilization of licenses every month and see if we need to purchase new ones then shoot them an email saying we need more. We don't get into the inner workings of licensing and procurement.

I rate Sauce Labs eight out of 10. There are more and more different testing scenarios for various machines and environments we use every day. Sauce Labs provides a robust platform to test combinations of what a user might encounter when accessing any application.

Sauce Labs reduces the testing time and covers all the possible scenarios that could come up, so we don't have to worry about any scenario that we might not have missed testing manually. It's a great platform to cover all your bases.

We have several healthcare products across the healthcare continuum, and we use the Sauce Labs platform to test our applications across different browser and OS combinations. We also use it to do mobile testing across different mobile devices that we may not have. It's not easy to set up these different configurations, so the cross-browser and cross-OS platform in the cloud gives us the ability to test across these different configurations without having to set them up or maintain them.

At this point, we are trying to focus on API testing.

My role is with a central team that helps other teams. If another team is struggling then we reach out to them and offer assistance. Because of this, I am familiar with how some teams are using the product. If on the other hand, a team is doing well and doesn't reach out, then I don't have any insight into how they are using the solution or how well things are going. 

We've had an event where I worked with the vendor to coordinate an automation day at my company. They worked with our QA engineers through a training process where we had a test repo that we were able to run across different browser and OS combinations using the platform.

Technical support is very important because it helps us get through issues faster. When we are trying to integrate with a new tool, as is the case now because we're trying to move to GCP, we have a dedicated solution engineer to work with us. The engineer is helping us as we ramp up with GCP, getting our automation to work.

We went through this a few years ago with Azure. Learning a new tool is challenging enough, and then trying to make it work with our existing processes is even more difficult. For example, we need to enable our test automation to run on the new platform and have everything work together. It can be challenging, not to mention that mobile testing is still evolving. This is why support is important. They have helped us with our questions and to get through any blockers.

For the most part, the product is optimized for automation and integration with the major CI/CD platforms and developer tools. That said, our constant complaint is that it is kind of slow. We are trying to figure out ways that we can work around the slowness, and what things we can do better.

We have a lot of products here in the company, and not everyone is leveraging the cross-browser testing platform that is available at Sauce Labs, but we have more and more teams adopting it in the last three years. We are trying to collect more insights into how our users are accessing our applications and using that insight to cover all those different testing configurations using the Sauce Labs platform, so we are trying to leverage the platform more in our internal testing.

For now, Sauce labs covers our needs with respect to the number of browser/OS combinations and mobile devices. However, having more availability and looking into different strategies for making things faster are important. Our goal is always to get faster feedback on a build, which means if it takes a long time to run an automation suite then it's less likely that we'll use it. We're always seeking ways to add capabilities and expedite things.

Having the ability to test our application on additional browser/OS/mobile device combinations opens up additional revenue streams for us because we have a larger user base. It gives us the confidence that the application we are putting out will generate a positive user experience because we've covered all of those testing combinations.

We've gone through a lot of OKRs and have had a general goal to reduce the time it takes for us to test applications. This product enables us to run tests in parallel, which helps to make things faster. I don't have any concrete or documented examples of how long it would take to complete our testing before using the tool because we've evolved the way we write automation.

There were times where it would take half a day to run an entire suite of tests, and through some of the improvements we've made, with and without the tool, our goal has been to be able to run a smoke test in 30 minutes. Some teams are able to achieve that, whereas others cannot.

There is really no one answer that fits because we have teams that are across the spectrum. Some are working with mature products and some are just getting started. I know of one of our teams that optimized their automation and with Sauce Labs, they have been able to take their automated test time down from three hours to approximately 30 minutes. This is only one team out of many but it demonstrates a significant change.

The most valuable feature is cross-browser, cross-OS, cross-mobile device testing.

The parallelization of automated tests is a best practice that I recommend. Some of the teams are able to do that, although some cannot because of the way the tests are formatted. If their configuration allows for it then it is definitely something that I recommend they take advantage of.

Running testing in parallel has worked well for some teams. In one instance, a team was able to split their tests so that they ran across two platforms. Running these two configurations, they initially cut the three-hour testing time in half. Within each of those configurations, we were able to parallelize even further within our test framework, and ultimately, we were able to get the test time down to 33 minutes.

The process can be a little slow to configure and get started. It's a cloud platform and you're trying to run automated tests. It first has to be configured, then the test downloaded, then you have to configure the environment that you're going to run on, then kick off the test based on what your needs are. All of this takes time.

We have faced challenges with the availability of mobile devices. There was once or twice where there were no mobile devices available. These instances may have been due to a service outage.

I have been using Sauce Labs for approximately five years.

We run our testing on a daily basis and I don't think we've run into any stability issues. Nothing that I am aware of has been a concern.

As we run into issues with anything that we are trying to do or integrating with to run our test automation, we reach out to their support team. Anytime that we need something and we need a quick answer, my first recommendation to anyone that reaches out to us for help is to call support and get it logged. This way, we can get someone to start responding.

Some of the support team members are better than others. Overall, I would rate the support between six and seven out of ten.

We did not use another similar solution prior to Sauce Labs.

Also, personally, I have not used any other cross-browser testing platforms.

I was not involved in the initial setup or getting it integrated with our automation test framework, but I think it was fairly easy.

The initial setup was done in-house.

As a SaaS product, it is always up to date.

The pricing is definitely on the higher end, and there are other options that are more cost-effective. One of our teams that was using Sauce Labs separately decided to go with a different solution that was less costly.

Prior to choosing Sauce Labs, the company evaluated BrowserStack. I was not part of the decision process but the differentiating factor may have been the support.

We have our internal framework and we basically use Sauce Labs for cross-browser capabilities and mobile device capabilities. Since we have our own framework, we have not explored any functional testing capabilities that the platform has.

The vendor has acquired API Fortress, and it is available within our account for free to try out. That is something we're going to have a demo next week to look into because, at this point, we want to start focusing on API testing. It's helpful that we are able to run our UI tests across different platforms but we would like to focus more on the other layers of the application. We would like to test under the hood, the business layer, more.

I would rate this solution an eight out of ten.

I am part of the QA team. I am implementing this solution. Right now, we are also trying to implement this solution in order to gather results in the testing process.

We have several stores around the world, more specifically around Latin America. We are trying to automate many tasks for the mobile applications that we are building. We are also trying to automate many web tasks in order to upload items to the tool.

The device level of the platform is the most valuable aspect for our team.

So far, the stability has proven to be quite good.

The solution seems to have interesting automation capabilities that we still need to understand. 

The testing process is difficult. I need to prove the complete competency of the tool, and I am finding that challenging. 

Right now, we are in the initial phases when it comes to actually using the product. We are testing some tools in order to implement them in the future. We are in a very early stage.

While we are still in the early testing phase, from what I have seen so far, I can say the solution is quite stable. There are no bugs or glitches. It doesn't crash or freeze. The performance appears to be very good.

That said, we still need more time to really dig into its capabilities. 

We've been in touch with technical support as we've run into some issues and we want to better understand how the product will work in practice. We've been in touch to try to get a hold of someone that can walk us through a customer journey so that we have the knowledge we need to use the full potential of the product. We're still working on making this happen.

The solution has not been implemented fully yet. We are still trying to uncover if this is the right solution for our company. We seem to need assistance with support and with the setup process before it will actually go live. We're not far enough into the process to really discuss the deployment and if it is difficult, or complex, or very straightforward. Those details are still to be determined. 

I'd rate the solution at an eigh tout of ten overall. 

We had an issue in the past and we're looking further into the issue.  

The primary use case for this solution is for automation testing on mobile and web on our testing environment so we can implement continuous integration and continuous delivery into the workplace. With the automation practice being used we are able to deliver more and possibly deliver daily after each build is created in theory. With the automation, we can now focus time testing the higher traffic areas or higher risk areas that could possibly crash or cause a bad experience for the end-user of either the web or mobile application. 

Sauce Labs has improved the organization with the ability to test more internet browser configuration, more mobile testing devices that are not a flagship phone, and more operating systems. With the internet browser testing, we are able to config multiple test scenarios and pinpoint which internet browser we want to test on and what operating system we want to run the test on. So with Sauce Labs, we are able to cover multiple Chrome, Firefox, and safari variations to support and test for all users. 

The feature that I have found most valuable is their device bank. With this, I am able to test more niche phones that is not a flagship and have the ability to support the end-user. This has been useful because when older devices from Android and iOS are running older OS and they experience a crash, it has been hard in the past to replicate their issues without purchasing that device and hoping it matches the phone specs. With the device bank, it seems like all a major of phones from android 5.0 are covered up to the current. 

A service that could be improved is some of their Wiki documents are either outdated or point to broken URLs; however when contacting the customer service they are able to respond in a quick manner with either an updated document or with the steps to fix the issue. 

Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code. 

The company has been using Sauce Labs for 8 months.

No we were not using a different solution. 

I think a longer trial period would be beneficial. Instead of 14 days look at 28 days due to in part the company that is on the trial period could be busy or possibly busy during the trial period. 

We looked at other options but found that Sauce Labs was the best option to support both Web and Mobile.

My company uses Sauce Labs to run all smoke and regression tests for our application.

We primarily run our tests on Chrome 83, but we occasionally run on other versions (81 or 84) to check to compatibility. Our smoke suite runs on a nightly build, and regression on a bi-weekly basis.

For building out new automation features, we have to have a passing Sauce Lab run before creating a pull request. Doing this ensures that our new automation features will run on other environments and not just locally.

The insights section provides a great overall state of the automation suite and can identify trends relatively quickly. If we see a dip in our passing rate over time, we can look at what changed when the test started failing to find the root cause rather than doing a quick fix to find that the test fails a short time later.

The visual data is helpful for all levels within the organization from the QE engineer level up to the director level.

The ability to change the browser version has also ensured that our application stays compatible with previous and updated browser versions.

One major feature that I like about Sauce Labs is the recording feature. I love the option to watch a video playback and command and view log when I'm trying to figure out why my test was passing locally but may fail on Sauce Labs. It's helped to make sure of my automation.

Another feature I find valuable is the ability to choose browser versions. This feature allows us to test on our minimum browser version but also check/verify that our application is working on the most recent browser version as well.

The insights tab is also very valuable, as I’ve discussed early; it provides a great deal of data and determines trends in our automation sooner.

From a company use standpoint - I have no complaints. The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended. Personally speaking - I like to maintain my automation framework and would like to occasionally test that it continues to run on Sauce Labs after releases without stealing from Sauce Labs.

I've been using Sauce Labs consistently for a year now since I've started with my company.  After being introduced to Sauce Labs, I've incorporated it into my framework that I maintain.

A previous solution I had used was running a VM from a NAS server to run our automation scripts.

Be sure to plan out your solution before starting the free trial; 14-days goes by rather quickly.

Sauce Labs was already integrated into our framework and SDLC process when I started with the company.  After using it for a year now, I would highly recommend this to anyone.