SmartBear TestComplete Reviews

I work as a tester for a company.

I use TestComplete in SmartBear and the open-source version of SoapUI.

We use TestComplete for our regression test case distribution.

For Windows application automation, we use TestComplete, and for API automation, we use SoapUI. When compared to TestComple, I find SoapUI to be extremely useful.

TestComplete is extremely efficient when it comes to scripting. In a single day, you can automate more than 100 test cases. If you write the code, Selenium allows you to automate only 10 to 15 cases.

When compared to other tools, it is very simple.

When we try to automate and keep the same script in a few cases, it fails due to control changes. You must have a session open when running in TestComplete, otherwise, your script will fail.

When compared to the competitors, it is a little more complicated.

During the distribution of our regression test cases, the control IDs are not always recognized correctly.

Our frequency control IDs are getting changed. For example, if I develop a script in one mission and then try to deploy the same script in another mission, the script will not be executed properly.

Selenium is more efficient than TestComplete at the point of execution, in my experience.

In terms of execution, attempting to run a script from multiple missions will result in the script being deployed and failing to work in some of the missions.

It would be beneficial if they provided a new open-source library that could be integrated with Selenium. Currently, we must create Java bridge libraries in order to integrate with Selenium. It would be preferable to have a simple APA for consuming the Selenium feature console.

I have been working with SmartBear TestComplete for two years.

We've also phased TestComplete, a slightly lower version, and are looking into TestComplete 14, using the trial version.

This solution is used by myself and two other members of the same team.

I have not contacted technical support because I am using the trial version.

The initial setup is very easy.

We have a TestComplete 12 license.

I would rate SmartBear TestComplete a six out of ten.

The primary use is to run regression tests on a call accounting system as a web application, which is running on a Windows operating system. The tests simulate user actions using UI and database verification.

The solution has saved a lot of human resources by running full regression tests prior to each release. In addition, during the years, many major bugs have been detected by the tool, and it has saved us the big expense of fixing problems after a release. Our confidence with the tool increased as releases were performed successfully, without any rejections from the field.

The database checkpoints detect problems which are difficult for a human resource to find. In addition, verification of UI items in all screens is also important task that consumes too
much manual resources.

Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work.

Eleven years.

Stability issues occurred only when connecting to the SourceSafe. Sometimes, after getting the latest version, the tool hangs and it should be reopened in order to recover.

No issues.

Customer service and technical support responsiveness are high. Everyone is very professional.

We did not have a previous solution.

The initial setup was straight forward. No issues at all during the setup stage.

The implementation was in-house.

Our ROI is about $10,000 a year.

The license price for a physical machine is cheap, and for virtual machine, it is very expensive.

We did not evaluate other solutions.

• SoapUI/ReadyAPI integration
• Selenium integration
• We run all of tests using TestExecute, this is just the executer for all TestComplete tests – command line version so can be integrated with any CI tools
• We used to these tests seamlessly on Jenkins (TestComplete has a plugin for Jenkins)
• The best part is TestComplete gives choice of selecting any of the six languages it supports
• List of checkpoints available in TestComplete is one of the strongest available in the market
• We also make heavy use of version control integration and JIRA integration
• Data driven testing feature of this tool is one of the best

We did a POC for number of tools in past and eventually decided on TestComplete. We started with one tester and one test which we used to trigger manually. As the product matured, it added cross browser, mobile testing functionality, integration with Selenium/ SoapUI and Jenkins plugins, so we started using it more and more. Currently, the automated test team size is 20 and we run hundreds of tests automatically at the end of build process without any human intervention. For us, the automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers. The ROI has improved significantly and all the regression tests for our product are automated. We have plans of using TestComplete in the test driven development approach where developers can make use of our tests as part of their unit testing and I am sure we are going to achieve a lot more from this tool in coming years.

• Error handling features in the tool are a little limited
• The tool currently is Windows only, would like to see it being implemented for Mac and Linux
• Once the tests are created in one languages, you can’t change it use other languages

There have been no issues with the deployment.

The tool sometimes seems a little unstable and crashes sometimes on Windows 10.

There are no issues with the scalability.

Average

Good

No. We did a POC on multiple tools and TestComplete was our first selection

StraightForward

In-house

QTP, TOSCA, RATIONAL, RANOREX

The tool has great capability but also has great potential to demoralize if you implement it in incorrectly. Automated tests complement manual testing and doesn’t remove manual tests completely so I would suggest keep your manual tests, they will come handy as reference when the automated test fails or needs an update. I would suggest following before you start implementing the tool in you automated test environment

• Choose the language to be used and stick to it. TestComplete supports six languages (VBScript, JScript, Python, DelphiScript, C++Script, C#Script)
• Choose the naming convention for objects in the name-map and keyword tests so that you can make use of these tests in several projects
• Always try to reuse the tests with parameterised values wherever possible rather than duplicating the tests
• Version control the tests so that you can maintain multiple versions for multiple branches of your tests
• Define and document a standard process for automation and communication
• Execute the tests frequently so that you can get the best out of the tests and defects are identified as early as possible

1. Object Browser is the most powerful feature. This has helped me tear apart every single component of the application under test. It's an awesome feature.
2. Test Editor and Debugger: This has really made my life easy as I can trace memory and stack values very easily.
3. Data driven testing has been made very easy. I am comfortable using any sort of backend to support data driven testing such as Excel/databases/etc.
4. It supports using third party DLLs has given me an edge on several occasions.

Previously, we had to wait for the end-of-sprint to make a stable release. Now, rolling out a release is not a problem. As soon as a build is deployed, the scheduler quickly executes User Acceptance Tests, and verifies that the build is fit to be delivered to the clients.

1. Support for working with COM and ActiveX objects should be more powerful. Currently we have to drill down a lot to find solutions for certain tasks.
2. Cross browser testing is a bit complex when it comes to scripting. Developers have to take care with it care, or else issues may arise at deployment time.

Overall, I have used this for more than two years, including v1040. Currently I have been using it hands on for the last year.

There often are issues with web test execution. Web pages often do not load on the first attempt.

Customer server is absolutely robust. The team respond within 12 hours.

I never needed to use paid technical support as their forum is extremely helpful. The community members respond to issues within 24 hours. It’s really amazing that within the past twelve months, I have never had a technical question I asked remain unanswered.

I am an expert on the following:

• Selenium RC and Servers/Java - but this has multiple limitations
• IBM Rational Functional Tester - it's expensive, and not as powerful as TestComplete
• AppPerfect - is not in competition with any of the above, it’s pathetic

It was straightforward. Both the node locked and floating licenses were easily deployed, and now the whole team is using it.

At one point, we had to update the hardware of the servers where the licenses were deployed, and this was very easy. We just disabled the licenses from the account and deployed them onto the upgraded servers.

I did it myself in-house. It’s very simple. No rocket science needed. Just read the manuals and you can do it.

We are saving a lot in terms of client satisfaction. We are not yet in a phase where we have savings in terms of a reduced QA team, but our clients are happy that they get bug free software. Moreover, our testing team spends more time testing new features, and we are confident enough at build roll out time that all previously developed modules are stable.

I like the cross browser compatibility. It saves a lot of time re-writing scripts to accommodate different browsers.

We are consultants. So we simply provide an automated solution to a client, then move on. We don’t use the product in our day to day work.

The way objects are added and used when utilizing descriptive programming could be improved. It is a little unwieldy, compared to UFT.

In UFT, using descriptive programming for a web page you can use.

Browser(description).WebList(description).Select anything.

Regardless of how many panes, frames, panels etc are in the hierarchy before the Weblist object.

But in Smartbear you have to store every frame, panel etc.

So that if you didn’t use the ‘Alias’ functionality you would have an object description miles long.

But even having to use the Alias, you still have to add each and every frame, panel, etc. whereas in UFT you can just use page.object and it will find the object on the page (as long as you’re using unique descriptions!!) without worrying about frames, panes, etc.

We have used this solution for about two years.

I did not encounter any issues with stability.

I did not encounter any issues with scalability.

I didn’t require any technical support.

I routinely use SmartBear, UFT, and SilkTest. I fit the application used to whatever my client requires.

The initial setup was very straightforward. Even the mobile testing side was easy to setup

My advice so far, is that while it’s not quite as powerful and easy to use as UFT, its price tag more than makes up for it. It makes it an excellent cost saving alternative.

I use several tools.

As with all tools, verify that it will do what you need for a reasonable price.

We primarily use the solution for our automating our test scripts.

The record and replay aspects of the solution are quite useful for people. With them, you don't have to write any scripts. Basically, you can record your actions and play them back later.

The initial setup is very easy.

Right now, the product only supports Windows. We'd like to see it work with other operating systems as well. They should definitely be working with iOS, for example. That should just be a given.

The product is quite stable. There are no bugs or glitches. it doesn't crash or freeze. It's reliable. 

The scalability is quite good. If a company would like to expand it, it can do so. 

Technical support is excellent. If you reach out to them, they will get back to you within 24 hours. 

Positive

The implementation process is hassle-free. I'd rate it a five out of five in terms of ease of installation. It's a very simple process. 

The amount of maintenance needed probably depends upon your project and the number of test scripts you have. It is based on the complexity of your application. If it is straightforward, one person can manage it once it is automated. A larger amount of scripts may require a bit more maintenance, however, something that is low to medium is easy to handle. 

While it's my understanding that clients don't seem bothered by the pricing, I can't speak to the actual cost of the product. That said, it is, as far as I know, competitive in the market.

I'm a reseller. I'm not coming from any organization as a consumer who is consuming TestComplete features. Actually, I'm a solution architect who is demonstrating the features of the tool to others.

While I tend to look at client requirements before recommending any solution if it did fit in with a client's need I would definitely recommend the product. It's very reliable. 

I would rate the solution eight out of ten. While there are some gaps, we've been mostly happy with its capabilities. 

I use this solution for functionality testing, API testing, and performance testing.

We are moving away from this solution to use Eggplant Functional.

The most valuable feature of this solution is regression testing tools.

The SoapUI tools are very good, as well as cross-browser capabilities.

The GUI is very good.

There is a problem with usability because the speed decreases, which could be an issue with scalability because of too many hits on the site. It depends on how distributed their systems are, and how well they can handle multiple connections.

The artificial intelligence needs to be improved.

I have been using this solution for between six and twelve months.

This is a good product in terms of stability.

The technical support for this solution is always available. They have an online chat that is quick.

The initial setup of this solution was quite smooth. It was not complicated and within a week it was working. It did not take long.

We performed the implementation in-house.

This is a pay-per-use service that is not expensive, and cost-efficient if you have a small team.

We have also been using Eggplant Functional, and the decision has been made to move forward with that solution, instead of SmartBear. It is not for technical reasons. I also find Eggplant to be a little bit pricey.

This product is quite mature, able to compete with other products in the market, and I would highly recommend it. Overall, we're very pleased with the implementation.

My advice to anybody who is considering this solution is to do a PoC and try this solution out. They can always reach out to me for help, and I can assist.

The biggest lesson that I have learned from using this solution is that there is a lot of hard work going on behind the scenes with this tool, to make each customer's journey easier. This tool is simple to implement and easy to use. If you don't have much time to do a lot of reading then it is still easy to take each case and adopt it quickly.

I would rate this solution a nine out of ten.

Desktop and web application support. TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications. It's an amazing feature for companies that want to automate UI tests on each application built in-house.

Before using TestComplete on our projects, we used to use a lot of tools to automate our applications, such as Sikuli or Java Robot to automate desktop apps, and Selenium WebDriver for web apps. After starting to use TestComplete, we were able to centralize all scripts in only one tool and technology.

TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services. In a microservice world, this is a big flaw. Another thing is that the cross-browser support has a lot of different traits between browsers. It should be improved.

Five years.

Sometimes the app crashes during test execution based on the amount of code that is running. It's recommended that you modularize scripts, but that is not a justification for not being more stable.

TestComplete has a test executor app that can be used in a distributed test execution environment. The problem is that this is a paid product. Thus, it was pretty expensive to scale this architecture.

Eight out of 10.

I used to use HPE QTP but TestComplete has the same features, low price, and support for handling a lot of stacks. In addition, I don't need to use hundreds of plugins.

To start working with TestComplete, we only needed to install (next, next, finish flow) and then start using it. There are some configurations to do to help increase efficiency, but I do not consider that more than a nice-to-have.

Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day. If they will each use it all the time, buy licences for all of them.

No, I did a PoC and discovered the pros and cons.

Do a PoC and try to understand if TestComplete fits your context and requirements. Use the script-driven approach instead keyword-driven, because the former is more efficient.