What is our primary use case?
After using various dynamic scanner tools such as AppScan, Micro Focus Fortify, Contrast, and Checkmarx, we discovered that the number of false positives we were picking up began increasing over time, while the number of valid issues was very low. Thus, we decided to use Seeker (an interactive scanner) for our security scans instead, because it allowed us to lower the amount of false positives while simultaneously improving our security coverage.
Since Seeker is an agent-based tool, we can integrate it in any area, such as our automation area or functional test area, meaning that whatever gets covered as a part of automation or functional tests will also be reviewed by Seeker. In this way, our coverage is dramatically increased, and at the time that we procured Seeker, we found that the rule sets were at least 75% as comprehensive as the rule sets found in any other dynamic scanner.
Seeker's R&D team gave us a very good view of what their plans were and they have continuously added new rules and checkers to the tool. This was another of the main reasons why we went for Seeker as our IAST tool in the first place.
In my company, there are around 20 staff members from our security team who use Seeker directly. We also have what we call "security satellites", each with their own ID, across our applications and products, and counting these would put the total users of Seeker at around 45 people.
What is most valuable?
A significant advantage of Seeker is that it is an interactive scanner, and we have found it to be much more effective in reducing the amount of false positives than dynamic scanners such as AppScan, Micro Focus Fortify, etc. Furthermore, with Seeker, we are finding more and more valid (i.e. "true") positives over time compared with the dynamic scanners.
Another major advantage is that Seeker can be used to greatly improve our security coverage overall, which is one of the main reasons why I requested that we try an interactive scanner in the first place. Due to it being an interactive agent-based tool, you can integrate it in any area that you need it. This increases the coverage by a large amount, because each of these areas will subsequently be reviewed by Seeker.
What needs improvement?
One area that Seeker can improve is to make it more customizable. All security scanning tools have a defined set of rules that are based on certain criteria which they will use to detect issues. However, the criteria that you set initially is not something that all applications are going to need. The purposes for which applications are designed may differ in practice in the industry, and because of this, there will always be tools that sometimes report false positives.
Thus, there should be some means with which I can customize the way that Seeker learns about our applications, possibly by using some kind of AI / ML capability within the tool that will automatically reduce the number of false positives that we get as we use the tool over time.
Obviously, when we first start using the scanning tool there will be false positives, but as it keeps going and as I keep using the tool, there should be a period of time where either the application can learn how to ignore false positives, or I can customize it do so. Adding this type of functionality would definitely prevent future issues when it comes to reporting false positives, and this is a key area that we have already asked the vendor to improve on, in general.
On a different note, there is one feature that isn't completely available right now where you can integrate Seeker with an open-source vulnerability scanner or composition analysis tool such as Black Duck. I would very much like this capability to be available to us out-of-the-box, so that we can easily integrate with tools like Black Duck in such a way that any open source components that are used in the front-end are easily identified. I think this would be a huge plus for Seeker.
Another feature within Seeker which could benefit from improvement is active verification, which lets you actively verify a vulnerability. This feature currently doesn't work in certain applications, particularly in scenarios where you have requested tokens. When we bought the tool, we didn't realize this and we were not told about it by the vendor, so initially it was a big challenge for us to overcome it and properly begin our deployment.
For how long have I used the solution?
I have been using Seeker for two and a half years now.
What do I think about the stability of the solution?
We have not experienced any issues with the stability of Seeker.
What do I think about the scalability of the solution?
We haven't found many issues with scaling Seeker, although we have noticed one or two problems with the logs taking up too much space when we have had certain storage space constraints. However, this is more of a cleanup and maintenance issue rather than a scalability issue.
Talking purely about scalability, we have deployed Seeker in more than 60 different applications and we have yet to encounter any issues so far. We also have about 45 people using it right now.
How are customer service and support?
We have used their support before and we are happy with both the response times and the answers they provided us with.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We have used a variety of different dynamic scanners before going with Seeker, which is an interactive scanner. The other tools we tried include AppScan, Micro Focus Fortify, Contrast, and Checkmarx.
The reason why we switched to Seeker is because, over time, the number of false positives in these dynamic scanners kept increasing, while the number of true positives was limited. Because of this, we didn't find any of these dynamic scanners as effective as an interactive scanner like Seeker.
How was the initial setup?
The initial setup is quite quick and easy. In fact, I was the one who did the deployment, and it only took about one hour.
Once you have deployed Seeker on an enterprise server, the second step is that you have to deploy the agent on the application that you intend to monitor, which is where Seeker will identify any security vulnerabilities. This step is also mostly easy, though how easy it is depends on the type of application. If it is a plain vanilla kind of installation, where you have some kind of Java or NodeJS application, it generally takes about an hour.
However, if it's a Docker-based or container-based deployment, or an application based on something like .NET, then we sometimes find that the agent installation fails for various reasons. When this happens, we have to open a support ticket and wait for help in order to resolve it.
All in all, the enterprise server installation is very easy and straightforward, but with the agent installation you might face problems up to 50% of the time for a variety of reasons, depending on what type of application is involved, the type of deployment used, and so on.
What was our ROI?
When it comes to ROI, it depends on the use case. Generally, you should be using the tool only when it is absolutely necessary to be run every week or every other week.
When you are using it for over a year in this manner, even if it identifies only one issue per week, then I would regard that as a worthwhile benefit, and this is something we are seeing right now. We use it continuously and consistently over a long period of time, in and out, and this is where we have begun to see our ROI.
On the other hand, if you just want a scanner that you can turn on and off every now and then, Seeker would be a waste of investment.
What's my experience with pricing, setup cost, and licensing?
The licensing for Seeker is user-based and for 50 users I believe it costs about $70,000 per year.
What other advice do I have?
The biggest piece of advice I can give to others is to thoughtfully define your use case before using the tool. For example, you should think about questions such as: what do you want to use it for, exactly? Where do you want to use it? Do you want to use it in the sprint life cycle, or perhaps the automation area? What is your purpose in using it? Do you want to replace your existing dynamic scanner, or improve your security coverage?
And it's important that you define not only your use cases, but also your compliance rules and policies that will be applied to the various projects that you create, because Seeker comes with all the capabilities to do so. Taking these factors into account will help you a great deal in taking advantage of what Seeker has to offer. And this means nailing down the right kind of issues you intend to remediate, track, and tackle using the whole project going forward.
If you just integrate it and start using it without answering these kinds of questions, you are going to be faced with a huge amount of work when it comes to eliminating false positives, and clearly, that is not the right approach.
I would rate Seeker a seven out of ten.
Which deployment model are you using for this solution?
On-premises
