OpenText ALM / Quality Center Reviews

Most valuable to us is the ability to have the system organized into distinct roles and sections. That way, we can grant different users access to the specific section they need to access. We have business users that only need to run tests, so they only need that small section of the application. We have the BA's, product trainers, who only care about the requirements.

It has made our development process more professional. The whole interim process is a lot more professional. You can align it with the development life cycles, get the developers to buy in, and try and get it all linked in to the TFS Visual Studio.

Integration is also important to us. You've got Sprinter, which is quite nice for those that aren't familiar with what they've got to do. It's a nice little guide. Also, you can link it in with performance and automation tools, and kick things off with the push of a button.

New development methodologies, such as continuous integration and kanban boards, are being implemented by Microsoft and others to try to get their tools into the testing profession. ALM's got to push back and think more about the overall end-to-end development process. It's very much still a testing tool. We have a few awkward links rather than being a full solution.HPE ALM lacks a few of these features, but for a testing focus tool, helping to ensure quality, I think it's really good. It's good at its core necessities.

It's very stable at the moment. We're not on the most recent version. We have been using version 1201 for 2 ½ years. I did the upgrade, and I found it easy for me to do, because I'd done the previous upgrade as well. The documentation from HPE isn't that great if you don't know what it means. It’s been stable, but I say that, because I did the install.

Scalability is good.

Technical support is good. We've had to get quite deep down in some incidents, so we've actually managed to get through to third level support and speak to the developers. At that point, you're both talking the same language. They can understand your issues and you get good resolution if it gets to that level.

I was not involved in the initial setup back in those days. A couple contractors did it. It was called TestDirector in those days. I'm going to have a look at the new HPE ALM Octane later.

Their licensing model is expensive. We could scale it up and use it everywhere, but then, you look at how much it would cost for the licenses and you really think, "Is it worth it?"

I was not involved with the decision process, but I did put a case together to continue using it. Our parent company was trying to push us to use Microsoft TFS. I was basically showing how much better ALM is over TFS. For what we were using it for, it's just much better than TFS. It was the testing tool of choice.

Try and have a play with it and don't be afraid to customize. We've got this big workflow in ours, so you can control the rules a lot better as to who can do what, who has access, and what they can see. Out of the box, it's a bit vanilla and there's the risk that someone could be given wrong permissions and accidentally do something they shouldn't.

1> Intuitive GUI: Fairly easy to use and follow. For using QC, deep knowledge of the tool is not required. 2> For each test case, a test script with detailed steps can be created. This makes it easier to run the test script. 3> Provides interface with other test management systems like JIRA. 4> Excellent reporting process including customizable reports and charts. This is very useful for monitoring the progress of QA cycles and communicating the same to the higher management. 5> It stores test cases, test scripts, and requirements in a modular fashion, which can be easily copied and modified to create new test cases. 6> An extremely useful feature of QC is that it allows linking defects with higher level artifacts e.g., a defect can be associated both with a failed test script and the unmet high level requirement. It allows traceability of a defect with varying granularity of information. 7> No extra form required to perform searches on the defect list. Search is available for each field right on top of the list.

1> High licensing cost. 2> QC lacks a "watch" feature thus disallowing independent actors, such as managers / leads, to track the progress of issues. For example, for each defect, only the assigner and the assigned receive any updates / notifications. Everybody else has to employ external means e.g., e-mail to get these updates / notifications, thus introducing humans in the loop.

I have used HP Quality Center (QC) for over 3 years in an industrial setup. QC is a versatile Quality management tool that offers test case and defect management capabilities along with a customizable reporting process. It also integrates with other defect and requirement tracking tools, making it a good fit in multi-team environments, as well as integrated application environments. QC does have an involved initial setup, but once it's done, it is fairly easy to use by testing and defect management teams. However, as QC comes with a high license and maintenance cost, it is more suitable for large projects in terms of cost.

The overall task management. Managing all the assets and metrics.

I'm not familiar with all the changes, but they definitely have to be more DevOps friendly. They have to certainly be more open source friendly. That's the world we live in, where we can cut costs away from large-scale vendor contracts and service contracts. The ability to seamlessly integrate and provide more capability for those, managing those infrastructures and solutions, is going to be critical historically.

A lot of the vendor products - not just HPE or, in this case, Micro Focus, or whomever that I've dealt with over the years - were much more proprietary, much more exclusive. And what we're finding now is that the world doesn't work like that. Particularly as you move left and shift towards DevOps, application teams now don't consume from a central resource, they consume based upon decisions made internally to that application team.

Ultimately, what they need is flexibility. So any vendor product needs to have that intrinsic in its fiber, to be able to adopt open source, and integrate basically into almost anything, to expand out the choices available to an application; to make the decisions that need to be made independently at the time that they need to make them.

Not having looked at the latest, ALM Octane, just coming from the old world, at the time that it was necessary to implement a test management system to gather more information, metrics across different teams, different platforms, it served the purpose.

Things change constantly these days. There's a lot more going on. There are a lot more integrations available. I think if we're looking at the legacy owned product, I think its kind of come and gone as far as its ability to do what you need to do in a DevOps world. Any solutions in the future - I know ALM Octane is the heir apparent to the old infrastructure - it's going to have to be more DevOps friendly. It will need to be able to enable the consumers, the application's users who ultimately become the developers, to see the value in a more organized test management practice, versus more of a kind of hidden, under the sheets unit testing.

It's actually a whole trajectory of different solutions, different tests, that need to follow the pipeline for those folks. Anything that's not DevOps friendly, that's not DevOps easily consumable, to make the case for a more formal test management practice, is really going to end up by the wayside at the end of the day.

11 years.

My experience with the solution is that it has been fairly stable. What lies underneath is what creates the instability at the end of the day, the architecture that you are providing the solution on top of. I think once you figure out a viable, scalable approach to it, then the software itself, at least in my experience, has been very stable in running a test management operation.

It has met our needs. Just as long as you have the right architecture from the old days of physical server hardware to more of the newer stuff, which is VMware within datacenters - more virtualized.

And certainly the next rage for everybody is moving into Cloud infrastructure. So things are becoming much more self-service. You're getting model scaling. You're getting the things that are making the system more maintainable. But from a scalability standpoint, you want to be able to scale to the needs at the time that you need them. The Cloud certainly provides that capability.

I think like every company, they're changing the landscape. Support, in my experience, has been pretty good. There are always challenges based upon the routing/tier structure of who gets the issue first, how it gets routed, how it gets filtered down to the specific expertise that you need. That depends on your acumen as far as knowing your tooling, knowing your approach, what that's going to be.

Somebody who is very savvy, will obviously have frustrations coming into a tier-one support desk. Who they really need to go talk to at the end of the day may be somebody, and it will vary by company, like a tier-three, real low-level, very experienced resource support tech who fixes those issues. So it's going to vary based upon the customer's competency versus how they are routed through a support desk.

Testing is going to be testing. And the same challenges that you have in any of the different industries are going to be the challenges that you have in the ours, the insurance and financial industry, as well.

You know from DevOps to Agile, to Shift Left to Cloud, to managing your test assets efficiently and effectively, industry is really not going to make a difference.

I've been in a number of different sectors over the years. I've been in QA about 25 years, and having been in the natural gas industry, financials, insurance, HR systems. They are all pretty much the same challenges around testing. So I don't see a discrepancy in terms of the application you're testing. It's almost agnostic to the challenges that are innate with trying to test, within any type of development environment. Now, it just happens to be a more self-service DevOps model, where application teams make those decisions. But there's still always going to be those QA challenges.

Test management and reporting. Those are the two most important things. I tell my customers that the two main reasons they have ALM:

• Quality management, project management from a QA perspective - testing, defect management, how testing relates back to requirements.
• And reporting to make good business decisions in the future.

When I was a customer, it improved my organization because I was able to manage, to enforce standards on building tests, executing tests, and manage centralized reporting.

Now, I translate that over to my customers from various levels of the spectrum from complete, "We have no idea what to do to, we're doing stuff but we know we need to change," to "We've got some stuff and we just want to tweak what we're doing now."

I'd like to see the idea of users being flushed out more, so not just, "This defect is now assigned to a particular person," or "This person is assigned to execute a test."

I want to see the users expanded out to teams where you have five users and the sixth user is the manager, so the manager can roll the idea of somebody being responsible and accountable. The idea of things being assigned to a team of users and users belonging to that team. There are ways of getting around this in the tool because it's very customizable, but I'd like to see that separate from the idea of using security groups, which is one way of getting around that.

I'd like to see the concept of teams put into it.

ALM has gotten more stable over the years. It's a stable app. Like any other large, complex application, you run into things every now and again. We have a system to report things and get them taken care of.

I have customers that are small and customers that are enterprise-wide. So I'm able to deploy it in both kinds of environments and customize the tool, depending on size and level of maturity, for any kind of customer. Also within any vertical as well.

I have used tech support. Mostly because I'm with a consulting company and we also support ALM. We have our own internal support organization that people can get into.

In terms of Micro Focus support, because I'm a more advanced user - I've been using this tool since version 7 - I typically don't get a whole lot from first-level support. I tend to want to go right up to second, third, or even directly with the development organization. So I'm more the outlier, edge-case kind of person compared to most customers out there.

Once I get to the people that are at the level that I know I need to deal with, they're good. I'm also dealing with the people on the other side of the ocean, working directly with people who may have actually coded ALM to begin with.

When I became a customer in 2000/2001, when I first started, I was involved in the decision to purchase the solution. Now, as a professional services consultant, that decision has been made and I'm going in there to either deploy, upgrade, or help them use ALM to best suit their needs. In some cases I help them figure out what it is they need to have ALM do for them or how to customize it best.

When I was a customer, we were not using another solution. We were completely manual and I was a department of one. I was the QA organization for a small development company and the two company owners said to me, "We want to invest in this, go look and see what's out there and show us what our options are and what you think the best option is."

What caused us to switch to this solution was the customizability. The fact that we could make it give us the information that we needed to get out of it. The support organization seemed very top-notch. I actually learned a lot from the support organization when I was getting started in it. And I found it more intuitive then the Rational solution.

I've deployed it in many organizations because I'm a consultant. I've deployed it, upgraded it, customized it, in various ways for different customers.

In terms of complexity, it really depends on the needs of the customer.

When I was a customer in a small development organization that only had 20 people in the entire company, I deployed it, I did the customization - that was way back in the day.

Now, I have customers along the entire spectrum from small to large enterprise. Some customers are okay with near vanilla, out of the box. And some customers have very complex sets of business logic that they feel, for whatever reason, need to be enforced as far as how their defect management lifecycle is going to go. How their test construction, test execution lifecycle is going to go, how they want to manage requirements, and that can require significant customization.

Some of my customers have compliance concerns, they have digital signatures and they have FDA 21 CFR Part 11 compliance. They have all of these rules that they have to follow and some of them are subject to interpretation, so with one particular rule I have one customer who says, "This is how we interpret the rule," and they have me customize it one way; and I have another customer who says, "No, we're not going to interpret it that, way we interpret this way," and it's a completely different set of customizations.

Back then it was Mercury Test Director, which is now ALM. We were also looking at the Silk products, and we were looking at the Rational, now IBM, products.

When selecting a vendor to work with, I want to see that the technical people are really knowledgeable of what they're talking about. I want to know that the tool can give me what I need, not just, this is a standard proof of concept. I want to see what I need to see, and I want to know that, down the road, I'll be able to either get out of it what I need or be able to learn or have somebody come in to help me get out of it what I need. Because if I'm not getting out of it what I need, then I've wasted my money.

I give it a nine because nothing is perfect, there's always room for improvement, especially when you're talking about an app system as large as ALM is. I've been using it for so long it's kind of second nature for me to think about where its strengths are, and know that if I can't get something done one way there's always another way around it. Or I can integrate something into it or build work flow to make the UI behave the way I want it to.

Regarding advice to a colleague about ALM, remember that your process and your methodology should be driving what you need out of their tool and not the other way around. Tools can do some really cool stuff. You may look at it and say, "Okay, maybe we could get some value out of this feature that we're not doing today." But don't make that the driving force. It really needs to be able to support what you're doing and force the things that you want to get out of it. Because there's a truism in reporting: If you don't capture the data you can't build a report that's meaningful. So make sure it can get you what you need.

ALM: You cannot just say one feature is most important. You get the most value
using all modules from Management to Defects. When you use the tool end-to-
end, you can pull efficient project reports (especially scorecards) from the
Dashboard. So everything is integrated and only then you can evaluate the tool
fairly. ALM is very flexible and each module can be used independently, but
when you do that you are only using the tool as storage, not as a test
management tool.

UFT: It became much more stable tool in terms of object recognition over the
years. It is easy to use as long as the user has basic software development
knowledge and understands that the software automation process is not just a
record/playback.

ALM: We currently successfully manage all testing projects due to ALM’s invaluable capabilities, which are listed below:

• Built on best practices with a flexible structure, organization, and documentation for all phases of the application testing process.
• Serves as a central repository for all testing assets and provides a clear foundation for the entire testing process.
• Establishes seamless integration and smooth information flow from one stage of the testing process to the next.
• Supports the analysis of test data and coverage statistics, to provide a clear picture of an application’s accuracy and quality at each point in its life-cycle.
• Supports communication and collaboration among distributed testing teams.
• Reduces time needed to create test execution summary reports.
• Reduces the time needed to write and execute manual tests with HPE Sprinter tool.
• Users can capture their actions automatically as steps in a formal test.

UFT: We save time executing smoke and regression tests. We also use UFT to create test data.

I would like to see better Reporting functionality especially more sophisticated graphs, for example Actual vs. Planned or high level progress graphs using indicators like traffic lights. I would like to see more sophisticated or flexible Dashboard views, such as editing and resizing. I use scorecards and pull them into the Project Reports using customized templates. Scorecards can only be refreshed from the Site Admin, which then test leads has to depend on the ALM Admin to refresh the reports if needed after the scheduled auto run. There should be ability to refresh scorecards (execute KPIs) from the project itself or give more frequent auto refresh option like even every 5 min. This is a really burden on the team.

I would like to see Requirements mapped to test steps so we can combine multiple requirements validation in to one test case but map the verification steps to the associated requirements, so if the step fails only fails one requirement not all. When we are operating in an Agile world we do not have time to write test cases to capture one-to-one coverage. I know ALM allows many-to-many mapping but we cannot get true requirement pass/fail status if we use many-to-many option. Test configuration option kind of on the right path, but can only be use for data driven test cases, I cannot add design steps. If we can add design steps to a subset of a main test using Test Configuration option, I think we may be able to differentiate individual requirement that was failed without failing all the requirements mapped to the main test case.

We have used this solution for 17 years.

I did not encounter any issues with stability.

I did not encounter any issues with scalability.

In terms of technical support, I usually get solutions to my issues. I did not have any issues to call technical support for a long time.

If you follow the instruction, the setup is straightforward. It definitely requires an experience user to do the installation and setups, especially for upgrades.

I always used ALM and UFT. However, I had training and evaluated IBM JAZZ tools.

HP ALM is a good tool for a centralized and coordinated view of requirements, tests, defects, and iterations.

The main barriers of entry are cost and implementation, especially if an enterprise implementation is the best solution

Being able to have one place to review defects, testing progress, and defects was very useful.

Merging 40 different streams, just for defects, into one solution that had good search and reporting capabilities saved a significant amount of time in coordination, defect management, and by consequence, there was better control of the quality of delivered software.

The main barriers of entry are cost and implementation, especially if an enterprise implementation is the best solution

I have used this solution for eight years in a variety of versions and companies.

The biggest challenge was finding the appropriate resource balancing for the enterprise release. It is not very clear how that was going to be implemented due to documentation in 2010.

If there is a need for third-party integration, the documentation is not very good. We were able to integrate with FIT, but it took a very capable programmer to figure out how to do it. Again, this was in 2010. Hopefully, the documentation has improved.

I did not encounter any issues with stability.

I did not encounter any issues with scalability.

Customer Service:

Customer service was adequate.

Technical Support:

Technical support was adequate

We used a lot of home grown 'tools' and spreadsheets in one location and Lotus Notes in another.

In one instance, it was straightforward because anything was better than spreadsheets.

In the location that used Lotus Notes, there was a significant amount of resistance because of loss of control.

Neither instance was due to the tool, but it was due to cultural issues.

The implementation was done in-house.

At a Fortune 100 company, we achieved a reduction of 30% of defects in the first year and decreasing percentages the subsequent years.

The dollar figures were proprietary, but were significant even for an $11 billion dollar revenue company.

The main concern is that there is a significant dollar investment, so do good research to make sure the tool will meet your needs.

We evaluated IBM tools, as well and a couple of Open Source tools.

• Requirements sync and traceability: This allows us to see how many requirements have been tested and to show auditors this information easily.
• Test Execution (Test Lab): This allows us to track our manual tests with date and time and enter actual results and screenshots.

QC has been invaluable in the past for documenting our testing process, especially when needed for audits.

The Active-X technology requires client-side installations that are difficult to manage in environments where the tester's PCs are locked down to prevent installs. Test management is too rigidly dedicated to older ways of testing with scripted test cases. More support for newer approaches, such as exploratory testing or behavior driven testing would make QC more relevant to the way testing is done in many current contexts.

ALM requires that you install client side components. If your organization does not allow admin rights on your local machine, this means you will need someone to run the installation for you with admin rights. This client side install is also limited to Internet Explorer and does not support any other browsers.

As far as the test structure goes, you are limited to to a step-by-step test case with description, expected result, and actual result for each step by default. This makes it difficult to support an exploratory testing approach with ALM. Of course, much of this part of the tool can be customized, but it still pales in comparison to something like the Test and Feedback tool that Microsoft provides for exploratory testing.

My understanding is that the newer Agile Manager product is more friendly to exploratory approaches, but I have not used this product yet.

We have been using this solution for 16 years.

In terms of stability, the QC client crashes often when attempting to expand a node on a tree. Upgrades are a nightmare and documentation is difficult to understand.

There were no issues with scalability, but I have never managed a large user base.

Technical support has gotten better than it was a few years ago, but Tier-1 seems to just go through the motions of asking questions I've already answered.

I have used other solutions, but many do not have the traceability requirements that ALM does.

The initial setup was highly complex, mostly because of the database setup. Upgrades are even worse, especially if you need to migrate to a new server, since the repository needs to be copied over.

Make sure you get the correct license for your needs. The full ALM license lets you use the requirements tab, along with test automation and the Performance Center. You can also just buy the Quality Center edition (Manual testing only), or the Performance Center version (Performance Testing only). I have no idea where they get their pricing numbers from, but they seem to always be negotiable.

I have used Oracle's Test Management solution, Zephyr, Bugzilla, TestTrack, JIRA, and others.

Be sure to have a DBA available when you install. There have almost always been changes needed to the DB when I have installed the application.

Being able to manage tests as this is something very difficult to find in other products. There are a few open source ones that handle test management, but right now HPE ALM is still the best solution to handle tests.

It helps us to keep track of everything happening. When you test the software you've got results. Results can be OK or not OK. If you just get the results in Excel or things like that, you cannot work as a team because just one person at a time will be able to access it. With ALM, we can have several people working on the same product at the same time. Then we use it a lot for trustability, so we can add trustability to the facts, to requirements. It's very useful for that to verify everything that happens.

As soon as it's available on-premises we want to move to ALM Octane as it's mainly web based, has the capability to work with major tests, and integrates with Jenkins for continuous integration. This is lacking in the standard ALM which was great a few years ago but it did not evolve enough, and that's why we are waiting for Octane.

We've used it more than 15 years, so it's very stable. There is a new version coming, ALM Octane. Octane is new so we don't use it yet.

We have plenty of projects with the current ALM, so it scales well. I'm not afraid of an issue with Octane and believe it will be the same.

I'm disappointed with the support as they're not reactive enough. They don't know the product very well, and to have things changed we need to access R&D directly and then things move. Otherwise, it's kind of difficult.

Beforehand we were using just paper and Excel, and things like that. As soon as ALM was tested at the time we began to use it and sensed it's presence in the company and now every tester is using it.

For ALM it was complex because it's not fully web based so you need to install a client on your desktop and with all the Windows security stuff you need to be an admin on your desktop so it's a very complex set up. On the service side it's kind of complex but we have tech experienced people to do that and to set up the database and everything, so it's OK. With Octane it should be really much simpler because for the user because it's just a web application so you've got nothing to do.

I'd rate the pricing as 3/10 as it's very expensive.

The first criteria we look at is functionality. We have plenty of different projects so we need a full spectrum of functionality. The problem we have today is the price. It's a very good solution but it's expense so we are challenged by our finals and everything but the price.

If you have the money then you can go with ALM, as it's a very good product. You won't have any surprises with it so that's good. Otherwise, there are some open source solutions that are a little bit less functional, but you can play with them and get them to work, products like Squash TM or things like that.