SmartBear TestComplete Reviews

We use this solution for test automation. A few months ago we began working on a project, and this solution is part of an evaluation phase. We are trying to include the whole suite from SmartBear including HipTest and SoapUI Pro.

Our primary use case is a web solution that we are trying to get working on the mobile solution, based on Flutter. All of our apps are built on Flutter, which is still very new for us. There is no support right now, so we are working closely with SmartBear TestComplete to integrate it.

We have an on-premises deployment.

The most valuable feature is the integration with Azure DevOps. That has been, by far, the best thing for us. We can use it to integrate everything that we are running in the test automation tool, which is linked directly to the test cases. This is the only tool on the marked at actually has this functionality, and it has only come about since 14.2, the latest version.

What is currently missing from this solution is better support for mobile testing. This is what we are currently negotiating with them for.

Specifically, we would like to see Flutter support in the next release of this solution.

From my experience at this point, I would say that this solution is stable. Even with the earlier versions, as long as you set it up properly, it was stable.

In terms of scalability, I think that it works pretty well if you have agents.

We have not spoken directly with technical support. Currently, we are talking to the product manager.

We are using the SmartBear suite, which meant that setting up TestComplete was pretty straightforward. As we are still evaluating, it has not been fully deployed.

We haven't fully evaluated this solution, although I know that they have made a lot of updates since I worked on it the last time. The results of this evaluation will set the standard for other projects. We are creating the basis for everything. As things progress, it will be me creating the test, but it will be executed on several machines.

I would rate this solution an eight out of ten.

Evaluated for automating regression suite for windows desktop application and found testcomplete suits well in the aspect of identifying most of the objects and in few other parameters.

Good test coverage through automation and provides unique solutions to the most of automation challenges (e.g. comparison of images).

- Object spy eases the object identification method.

- TextObject recognition feature of this product exposes most of the objects to perform the actions.

- Test items, project variables helps in managing automation suite and scheduling execution.

- Jenkins integration to schedule executions.

- Integrated with selenium webdriver.

- Conversion of JScript project to Javascript project

- Integrated to Environment manager

- Support to read text from PDF

- Increased performance with less memory and cpu usage.

- Supports various and different versions of browsers in web testing.

- Support to Windows mobile application. (Native iOS and Android apps are supported currently.)

Customer Service:

Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

Technical Support:

Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

In-house team is used for implementing framework for automation through TestComplete. Tool must be explored completely and knowing its unique features is more important before implementation as it has lot of inbuilt features.

TestComplete now have come up with three modules (Web, Desktop & Mobile), so based on the type of product for automation, it is adequate to purchase the required module. Node-locked and floating license are available which could be decided based on the team composition and work strategy. Current license pricing is available at http://smartbear.com/product/testcomplete/pricing/

The most valuable feature is the ability for a non-developer to develop intelligent, robust, data-driven tests.

TestComplete has greatly improved our organization functions by allowing us to develop, manage, and execute many forms of testing in one central repository. Being able to develop, manage, and execute JUnit, NUnit, PHPUnit, PyUnit, Selenium, functional, and nonfunctional tests has given our company great insight as to the health of our testing efforts.

I have used this solution for four years.

No issues.

No issues.

The level of technical support leaves a lot to be desired. The only way to get support is via email, therefore problems that could take a matter of minutes to solve by phone or web conference can sometimes take several days.

We previously used IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, HPE Mercury Quick Test Professional, and IBM Rational Robot.

We switched because of the ability for a non-developer to develop intelligent, robust, data-driven tests.

The initial setup was very straightforward and took very little time.

No advice. I did not deal with the licensing and pricing.

I did compare the tool to IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, and HPE Mercury Quick Test Professional.

TestComplete is a great product. It is a perfect fit for an organization that has a hard time finding testers with a development background and want to centralize the development, management, and execution of their tests.

Name Mapping and finding objects at run time. I have no issue because its IDE is simple to use.

• License is less expensive than the other products available in the market.
• I have automated the regression test suite of my company's product.

It has memory leaks. I have seven complete test projects, all of them loaded in one project suite. I don't know why memory increases when I just write the code and check-in/out files from the VSTS.

For last five years.

Often crashes with large projects. File check-in takes too much time in TFS.

No idea.

Community is very helpful.

I didn't switch from another solution.

I would say straightforward, and really easy to work on TestComplete with its simple IDE.

They are changing the licensing structure which is really bad. TestComplete 11 supports all TestComplete modules in one license. TestComplete 12 has separate licenses for each module, Web, Desktop, etc. I hate the new licensing structure.

I have no idea.

It's a really good product with minimal issues.

It has easy recording, understanding, and use without too much programming.

Should improve parts for desktop products.

I have used it for four years and still use it today.

Yes, some.

Good, I only needed it once.

Yes, I had already used it. I changed because I did not have any solutions as close to the real thing as this. Free tools mostly do not resolve the problem.

The first was difficult only in learning how to do it. Going forward, it has been easy and intuitive. We scale projects the best way for the company as we find that the simpler way to do it.

It is expensive, and it is difficult to acquire living in Brazil. They should facilitate this. There are few representatives for this solution with little interest in selling it here.

Yes, JMeter.

It is a great product. Just get advice before installation and everything will be much easier for you. I strongly recommend it.

• SoapUI/ReadyAPI integration
• Selenium integration
• We run all of tests using TestExecute, this is just the executer for all TestComplete tests – command line version so can be integrated with any CI tools
• We used to these tests seamlessly on Jenkins (TestComplete has a plugin for Jenkins)
• The best part is TestComplete gives choice of selecting any of the six languages it supports
• List of checkpoints available in TestComplete is one of the strongest available in the market
• We also make heavy use of version control integration and JIRA integration
• Data driven testing feature of this tool is one of the best

We did a POC for number of tools in past and eventually decided on TestComplete. We started with one tester and one test which we used to trigger manually. As the product matured, it added cross browser, mobile testing functionality, integration with Selenium/ SoapUI and Jenkins plugins, so we started using it more and more. Currently, the automated test team size is 20 and we run hundreds of tests automatically at the end of build process without any human intervention. For us, the automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers. The ROI has improved significantly and all the regression tests for our product are automated. We have plans of using TestComplete in the test driven development approach where developers can make use of our tests as part of their unit testing and I am sure we are going to achieve a lot more from this tool in coming years.

• Error handling features in the tool are a little limited
• The tool currently is Windows only, would like to see it being implemented for Mac and Linux
• Once the tests are created in one languages, you can’t change it use other languages

There have been no issues with the deployment.

The tool sometimes seems a little unstable and crashes sometimes on Windows 10.

There are no issues with the scalability.

Average

Good

No. We did a POC on multiple tools and TestComplete was our first selection

StraightForward

In-house

QTP, TOSCA, RATIONAL, RANOREX

The tool has great capability but also has great potential to demoralize if you implement it in incorrectly. Automated tests complement manual testing and doesn’t remove manual tests completely so I would suggest keep your manual tests, they will come handy as reference when the automated test fails or needs an update. I would suggest following before you start implementing the tool in you automated test environment

• Choose the language to be used and stick to it. TestComplete supports six languages (VBScript, JScript, Python, DelphiScript, C++Script, C#Script)
• Choose the naming convention for objects in the name-map and keyword tests so that you can make use of these tests in several projects
• Always try to reuse the tests with parameterised values wherever possible rather than duplicating the tests
• Version control the tests so that you can maintain multiple versions for multiple branches of your tests
• Define and document a standard process for automation and communication
• Execute the tests frequently so that you can get the best out of the tests and defects are identified as early as possible

It's cross platform automation capabilities specially ranging across web, UNIX (via putty), and other systems.

TestComplete replaced QTP as the preferred choice of tool for the organization. It is much faster, works better across technologies (esp. Flex based UI) and is better compatible with newer technologies directly out of the box. We could have our automation engineers create tests more easily. Also, we were able to set-up lab machines to enable distributed runs for more applications in a shared environment.

The code editor, though following eclipse-style, is still a work in progress and gives a very poorly formatted code once viewed via other editing tools like Notepad++. Performance is another aspect which can be improved.

I've used it for one and a half years.

TestComplete was fairly easy to scale once we had the licenses in place for the organization.

We used tech support for some specific third-party grids we had to automate. The support was average and we ended up creating our own automation solution for that piece of automation.

We previously used QTP, and later moved to TestComplete as it provided more out of the box support for newer technologies like Flex and AIR.

Initial setup was very straightforward. We did face some hiccups in license procurement however, once licenses were procured the process from there was quite smooth.

We had a mixed team. The implementation was smooth overall and requires a few skilled automation experts to oversee the transition/initial implementation.

We achieved ROI in eight months from the start of implementation. Get a good automation architect to implement a good ROI directed framework. It is very easy to lose direction during a mmigration.

• QTP
• Selenium

The features of TestComplete that were most valuable to me were:

• The ability to run a whole suite of tests automatically (which we did overnight)
• The ability to create test scripts that were easy to modify;
• The ability to easily review a log of software errors, if any

It would have been a huge task to have to run the test scripts manually, probably 12 hours straight. We were able to run tests on daily promotions of software so all software was tested every night, making any bugs instantly visible the next morning. It gave us quick confirmation that software changes worked, and without breaking any existing software.

We were testing handheld barcode scanners running WindowsCE with many menus of warehouse functions, and our biggest problem was the timing between input and responses. This was because sometimes the Windows PC feeding the scanner script data was faster than the scanner could process it, so we had to constantly tweek the wait times so the script wouldn’t bomb out. I have since used software with a “wait for” function that would wait for a response, but if TestComplete did, we didn’t know about it.

One of the main reasons I was hired was to use TestComplete with green screen applications, which was possible, but not easy. You had to know the exact column and row position of every text line on the screen so you could verify the string being tested and analyze exactly where and what the response would be.

I used TestComplete as a QA Automation consultant for about six months.

No

No

No

We never needed customer service.

We never needed technical support.

Nothing previous to TestComplete.

That happened before I was hired, so I don't know.

TestComplete was implemented with the help of contract software developers who created the test scripts, which was invaluable to be able to use it for testing, and to use them as templates so we could copy and modify to make new scripts.

unknown.

unknown.

TestComplete was installed before I was hired, so I know nothing about this.

Desktop and web application support. TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications. It's an amazing feature for companies that want to automate UI tests on each application built in-house.

Before using TestComplete on our projects, we used to use a lot of tools to automate our applications, such as Sikuli or Java Robot to automate desktop apps, and Selenium WebDriver for web apps. After starting to use TestComplete, we were able to centralize all scripts in only one tool and technology.

TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services. In a microservice world, this is a big flaw. Another thing is that the cross-browser support has a lot of different traits between browsers. It should be improved.

Five years.

Sometimes the app crashes during test execution based on the amount of code that is running. It's recommended that you modularize scripts, but that is not a justification for not being more stable.

TestComplete has a test executor app that can be used in a distributed test execution environment. The problem is that this is a paid product. Thus, it was pretty expensive to scale this architecture.

Eight out of 10.

I used to use HPE QTP but TestComplete has the same features, low price, and support for handling a lot of stacks. In addition, I don't need to use hundreds of plugins.

To start working with TestComplete, we only needed to install (next, next, finish flow) and then start using it. There are some configurations to do to help increase efficiency, but I do not consider that more than a nice-to-have.

Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day. If they will each use it all the time, buy licences for all of them.

No, I did a PoC and discovered the pros and cons.

Do a PoC and try to understand if TestComplete fits your context and requirements. Use the script-driven approach instead keyword-driven, because the former is more efficient.

One of the valuable features is object identification. It provides the different options to users so they can uniquely identify the objects.

By automating the regression test cases, it has increased effective testing process and reduced the cycle time.

I would like to see recovery scenarios.

I have been using this solution for over four years.

I did not encounter any issues with stability.

I did not encounter any issues with scalability.

I would give technical support a rating of 8/10.

When we started using this tool, only TestComplete provided the support of the automation for WPF objects.

The initial setup was easy.

This is cheaper compared to other testing tools.

We evaluated other options QTP.

This is the best tool for GUI and Web Automation.