Sauce Labs Reviews

We attempted to host and run an in-house Selenium grid in 2012, but we had several failures and false positives that arose from our unstable Selenium grid versus their code. While some failures were due to normal challenges associated with a grid running hundreds of nodes, others were due in part to low familiarity with Windows, since our company’s primary focus was UNIX and Linux that year.

After initially experimenting with Sauce Labs, I automated a handful of tests. I couldn’t believe it. I didn’t have to manage Windows boxes, browser versions, and security settings for Windows machines. Because virtually no one on my team had Windows experience at the time either, I knew Sauce would be well received.  

In a startup environment, often everyone reinvents the wheel with their favorite tool, and then you have to support multiple tool chains. We found no backlash when it came to Sauce, however. Every developer and QA person said, ‘this is amazing.’ Within a week everyone was using Sauce. I turned off our Selenium grid and deleted the VM that it was running on. After moving more tests to Sauce, the team was able to reduce testing time from 30 days to 3 days, a time savings of over 99%.

Our test and development environments are highly automated. QA testing is complicated because of its large matrix of supported operating systems. We use 400 to 600 UNIX, Linux, and Windows VMs to run more than 1,000 UI tests in parallel. In addition to these VMs, we use 20 machines from Sauce to run 20 test suites in parallel continuously with Jenkins, a popular CI system. We use Sauce Connect to work behind our secure firewall, and Sauce is the final check for quality across all browsers.

I do not want my team spending time on browser or image maintenance. It’s too expensive and it changes too quickly. I love that Sauce handles that for us. 40 engineers currently use their Sauce Labs account. Our large team finds the video debugging tool extremely helpful as it aids communication. With Sauce, we experienced an epiphany. It works right out of the box by recording video so we don’t have to look at log files. It helps us debug instantly.

Sauce immediately allowed my team to scale our testing from literally the day we rolled it out. It also allows me to access browsers I hadn’t thought about using previously, such as Safari and older versions of Internet Explorer, which would have required the team to manage infrastructure they were not equipped to do. We’re not browser savvy. Avoiding this, thanks to Sauce, is a huge win.

To achieve the same results in-house we get by using Sauce, we’d need to employ several systems engineers to set up and maintain the grid, for a total cost of approximately a half million annually in headcount. Lastly, avoiding the hassle of managing unfamiliar systems has helped retain top engineering talent within my team. If you tell your team of UNIX and Linux people they’ll have to maintain Windows systems for IE use, that could be an issue for some folks. So I think I saved people from leaving because Sauce handles this for us. Sauce allows me to deliver a robust and sophisticated UI testing infrastructure with very little work.

What is most valuable?

How has it helped my organization?

What needs improvement?

For how long have I used the solution?

What was my experience with deployment of the solution?

What do I think about the stability of the solution?

What do I think about the scalability of the solution?

How are customer service and technical support?

Which solution did I use previously and why did I switch?

How was the initial setup?

What about the implementation team?

What was our ROI?

What's my experience with pricing, setup cost, and licensing?

Which other solutions did I evaluate?

What other advice do I have?

Valuable Features:

Improvements to My Organization:

Use of Solution: