Skybox Security Suite Reviews

We use it to verify firewall compliance with NIST best practices for access and that our firewalls are configured correctly. We're also getting ready to roll out their Vulnerability Management package.

We mostly use Firewall Assurance and we're getting ready to start using Vulnerability Control.

What we have done is found a lot of misconfigured stuff on firewalls. Our company, Verisk, is a company that buys other companies. We have 70 or so companies at last count and most of them are founder-based companies we bought. They had little to no idea of how to actually secure a firewall correctly. Using Skybox, when we bring them on we take a look at how their firewalls are configured and then make recommendations as far as what they need to do to tighten it up. That is the main function we've been using it for and that is where we have gotten the most benefit out of it.

From Firewall Assurance, the only other real benefit you get is eliminating shadowed rules and redundant rules. You can optimize a little bit based on real usage to move the rules that are used more towards the top of the access lists so that the firewall processes them a little faster. It's a small benefit but it's definitely something that, depending on your business, may be important to you.

The most valuable feature is the compliance, whether it's access compliance or the configuration compliance, to make sure that all of our devices are configured as they're supposed to be, to limit access as much possible, to follow least-access guidelines.

Reporting. A lot of the reports, out of the box, are limited to a certain number of either configuration violations or access rule violations. So when you first set up a new firewall to be monitored by Skybox, you don't get a real full report. You have to really tweak it to get everything.

In our business, our company buys a lot of other companies and a lot of them manage themselves. Unfortunately, for Firewall Assurance in particular, if you need a group of people to be able to manage their firewalls and only theirs, it's almost impossible because to add a new firewall you have to be an admin, and you can't limit what an admin sees. If I want a particular company to be able to add their firewalls, they're going to see everybody else's firewalls as well, which is much more access than they need. That is one thing I would love to see fixed.

Stability is good. They do come out with a lot of patches and the updating process, while not a pain, is pretty frequent.

We had to separate our initial appliance into the server and a separate collector just because we have, at last count, about 120 firewalls in there. Collecting all the firewall information in all the logs daily during off hours, it started to get a little choked up. When we separated the server and the collector onto two different machines that fixed the problem.

On a scale of one to 10, I would rate Skybox technical support at about eight. It's not perfect, but good. They are not always able to answer questions on first contact but the questions always get answered. The answer is not always what I want to hear, but they do get answered.

I used the AlgoSec. AlgoSec wasn't broken up into modules, it was one solution. It was good; again, not perfect, but then their prices just got ridiculous. The fact that Skybox is broken up into modules and you only have to pay for what you're actually going to use, that was the main reason for switching. The pricing was secondary. AlsoSec doesn't do everything that Skybox does, but they were charging a lot more.

Setup is relatively straightforward. There were a couple of things that I found a little difficult. They have an Add Firewall Wizard, but if you want to create a task list or a task group that runs on a certain schedule, it's almost easier to import the firewall as a task rather than using the wizard. You almost have to do the work twice if you do use the wizard. 

The other difficulty was, it really wasn't made clear that separating the server and the collector, for a certain number of firewalls or over, was a best practice. Having to go back and redo that was a little bit of a surprise.

But overall, it's relatively easy to use. There is a little bit of learning curve to figure out how to get the right information out of the reporting. But once you do it, it works.

As with anything else, I would love it to be less expensive, but do I think pricing is a good value? Sure.

I've had issues with licensing where, when they were expiring and I asked for the updated licenses, I would the wrong ones. I think their process needs to be straightened out a little bit - I don't know if they fixed it already, it has been awhile. It wasn't as straightforward as it could have been. When you get the licenses you just put in the license numbers so it's working. That part is easy. It's getting the correct licenses that can be a little cumbersome.

We looked at AlgoSec, but their pricing was too high. And previously I had looked at Tufin but they just didn't have the wealth of features that either Skybox or AlgoSec have. Overall, we evaluated other stuff. It's just that Skybox made the most sense for us.

• Determine what your needs are.
• Buy only the products you need, when you need them.
• Make sure that your sales engineer goes over best practices with you so that you do it right the first time.

We have been reselling Skybox for probably about five years now, so I'm pretty familiar with it. I've done numerous POCs and I've had hands-on with it quite a bit.

Because I get to work with a bunch of different customers, I get to see just about every use case for Skybox. The first one, which is pretty simple, is auditing firewall rule sets; taking a look at all the configurations that are on the firewalls and ensuring that they're locked down. What we run into a lot of times are firewalls that are set up with excessive permissions, meaning they allow a lot more traffic than they should. Skybox is essential to tearing that down.

Network visibility is another big use case, learning where all the assets are located on the network and how they can talk to each other. 

The last one that I deal with quite a bit is the vulnerability/exposure-monitoring piece. Looking at those vulnerabilities that are on the network, providing the context of network-based mitigation, and then reprioritizing or recasting those vulnerabilities.

Specifically, in the Vulnerability Management piece, vulnerability management products are very noisy and they provide this arbitrary score called the CVSS that rates the criticality of the vulnerability. How bad would it be if somebody were to exploit this vulnerability? That doesn't matter if I have something on the network that prevents that vulnerability from being exploited. What Skybox does is to allow organizations, including three of my largest customers, to reprioritize the vulnerability they attempt to patch and mitigate, based on the contextual awareness of the network.

Also, for the vulnerability, it's the operational efficiency of the patching team. Patch management programs are very expensive to run from a headcount cost, and also from a potential downtime cost, and there is a never-ending stream of vulnerabilities. The ability to contextualize those and recast them in a meaningful way to my organization, and to all my customers, has been very valuable in increasing the efficiency of the patching process.

With the Firewall Assurance, that changes the way applications are introduced into the environment. So instead of asking for firewall rules which may or may not be relevant, or could already be there, or could be over-permissioned, Skybox can be used to map out the resources that that application is going to use and provide the exact rules that an application would require to function correctly. If the traffic isn't able to flow for the application, if it's erring out, Skybox can be used to troubleshoot that and say, "All right, where is the traffic being stopped and why, and how do I fix that."

The Vulnerability Management module is among the ones we talk about the most and the one that customers are biting off on quite a bit.

Skybox, in general, has quite a few features that are particularly useful to large clients, but their scalability is unparalleled in the space. They have massive scalability, thousands of devices that they can pull from, hundreds of thousands of IP addresses for the vulnerability results and casting; that in itself is very unique. The way they do vulnerabilities, providing the additional context of the network mitigations is fairly unique and valuable.

The only place where Skybox has room for improvement, and they're working on releasing this, it's just a slow-go, is the UI. The user interface has historically been via a locally installed thick client. They are moving to a web-based console and it's slowly coming out. It looks really good right now. I've seen the previews. I've seen what's going GA. Really, it's just building in that feature parody, to take all the features that are currently in the thick client and move them into to the thin client of Web-based GUI.

Skybox is in three of my largest clients and they have hundreds of thousands of IPs and thousands of devices reporting into it. It has never been unstable for them. It's always available.

It scales just fine. The way that it's built with three-tier architecture, it makes it very horizontally scalable, so I can have multiple fallbacks. If one machine does fall offline, there are four other machines that are doing the exact same job to pick it up. But I've never had a problem where fault tolerance was necessary. It's just an available option that makes everything a bit more robust.

I've only had to call in twice, and the first-line support was able to resolve the issue within around 10 minutes. It was a pretty quick phone call, and it was immediate. Their tech support has been phenomenal.

I'm a reseller of this product but I represent a hundred security products to my customers. The other ones that I've looked at or used, or I have seen used in the past, are Kenna Security, FireMon, AlgoSec, Tufin. There are a couple others too, but these are off the top of my head.

Setup is not complex, but it is a little bit more time consuming because of the three-tier architecture. It scales really well, but that means there are more pieces to install during the setup, although it's not hard. Everything is just "click, click, click, next." You get through it really quickly. It's just a lot to do.

It also depends on how you deploy it. If you stand it up bare metal, it's a lot to do, but it's not exceedingly difficult. If you stand it up as an OVA, it's a five-minute installation. 

So it depends on which route you go on the installation.

In terms of licensing, it's about defining use cases. If somebody were to say, "Hey, how should I go about the licensing?" I would say, "Define what use cases you're looking for. Look at Skybox's entire portfolio and decide what is important, or what would improve your organization and then just license accordingly."

I have some customers who only purchased Firewall Assurance. That was all they're interested in, and they eventually grew into the Vulnerability Management. Then I had the exact opposite where they started off with Vulnerability Management, looking to improve their operations efficiency, and then they eventually branched into the Firewall Assurance module.

The only piece of advice I would have is, feed it all of the data sources. Skybox can take in a lot of information; structured, unstructured. It has a ton of integration partners. Even if you don't know if you'll need to use them all, just integrate everything you can into Skybox as a centralized platform, because it does quite a bit more, the more data you feed it. You increase its capabilities when you give it more data sources to look at.

I'd rate Skybox at 10 out of 10. I'm the Director of Security Architecture, so I'm very customer-facing and senior when it comes to product management and security architecture development. I tend to develop a baseline of programs whose capabilities I feel every organization should have. The ability to appropriately prioritize vulnerabilities inside the environment, and then to have visibility into the traffic and rule sets of an organization, are two of the top capabilities that I recommend. Skybox is the only one that does both of those in a single platform.

When I go into an organization, especially larger ones that are 5,000 or 10,000-plus employees, the first things I'm looking for are: How are you doing your vulnerability scanning and what visibility do you have in your firewall traffic? Typically, the answer to both of those is, "We don't have a lot there," and Skybox is one of the first things I'll recommend because it's almost imperative to get operational efficiencies. Firewalls are very basic. Firewalls are the front line against inbound traffic. If you don't have something like Skybox inline, able to see what's going on with your traffic flows, you can't appropriately implement those firewalls. So Skybox is typically one of my first three recommended products for just about every client I step in front of.

We use it for Vulnerability Management.

Standard scanning solutions are not able to give any priority in terms of associated risk from identified vulnerabilities. Understanding the real exposure from vulnerabilities and associated assets can reduce the time and investment needed to mitigate risks.

Also, by reducing the number of vulnerabilities that have to be analyzed and managed we have the chance to create a process of management.

Identifies direct exposed vulnerabilities.

Firewall Change Management has to be improved with rules provisioning on firewalls because that is where the competition is going and is what customers need.

No issues with stability.

No issues with scalability.

Technical support is good.

Initial setup was simple because we approached the project in small steps.

The pricing is okay.

We evaluated Tufin, Algosec for Firewall Assurance. We did not evaluate other products for VC.

Involve network, security, and operations at the same table for smooth project startup.

Auditing firewall changes on a weekly basis. We use the Network and the Firewall modules. Firewall as I said, and we use the Network and Firewall for PCI compliance reporting.

It has automated things. What was a manual process is now just running a report and delivering it to the people who have to mitigate the issues. A better workflow.

It's the firewall change audit every week. Also, being able to track firewall ACL usage, so that we can produce semiannual reports on ACL usage and on shadowed and redundant rules on the firewall.

It's tough to say, because the areas of improvement, I understand the difficulty. For example, they pull configs from thousands of types of devices, and it's difficult for them to stay on top of when vendors change the way their commands work. If anything, it would be staying on top of the collector scripts, but I understand that's a very tough challenge.

We had an issue one time, but it was related to a major release upgrade. But that happens every now and then with vendors. 

We've had no need to scale it.

 Excellent. They're right on top of it. Very reactive.

Straightforward.

The product's pricing is excellent value.

In terms of licensing, make sure you understand your network components, all your hops through your network, thoroughly, before you decide on the total cost. If you want to do point-to-point flow analysis and such, you need to have the configuration of all the devices in between point A and point B. A lot of people don't realize all their network components until they start using this product.

We evaluated FireMon versus Skybox when we selected Skybox - they were really the two that were best at doing automated reporting for PCI. It was a compliance issue. We thought Skybox really fit our needs best.

Other than what I said - ensuring that you have a really good understanding of all the network components that you have to ingest configurations from - definitely take it out for a proof of concept for 30 days. There are a lot of features in here that we don't use, Change Management and stuff like that, that you want to take a look at and see if they fit your needs.

I would say the reason I can't go higher than eight out of 10 is that their major release announcements aren't always straightforward. You usually discover that there is a new major release when going to their website and you discover it on your own. So they're not really good at major release announcements. 

The use case is firewall rule lifecycle management.

This is something we are on the way to doing.

Network path analysis is the most valuable feature. There is a lot of work in my team to support internal and external customers, to answer their questions and difficulties with connectivity that doesn’t work. Mostly, the problem is missing orders. So if the customer can do the network path analysis for himself, the customer is able to write the request for the missing connectivity, without any support from my team.

Aside from Firewall Assurance, we are using Network Assurance and Change Manager for an overview of the whole network and for documenting requests and the recertification of the ruleset. 

At the moment we have a lot of work in implementing the tool in our workflow, so we are not looking for new features. But this could change in the next six to 12 months.

We have had some issues implementing the tool, but we are in contact with Professional Services to fix them.

No issues with scalability at the moment.

We are still in project mode, and we have direct contact.

No previous solution.

It was not so straightforward, but we bought onsite support from Skybox Professional Services.

The pricing is high, and the licensing model needs more flexibility.

In my case it was important to know the workflow, and then to look for a tool that could support this workflow to make it easier.

Primary use case for us is configuration management and configuration compliance.

It's given us more visibility in terms of what are the kinds of configurations that are on these devices, and how many of these are stale rules. So it's helped greatly in terms of cleaning up of rules, for sure. And it has definitely given a more secure way of backing up the configuration on these devices.

For us, it's more important for our firewalls, to maintain the configuration compliance, to look at duplication of rules; clean up functionalities on the firewall and compliance of the firewall. That's where it's most important. We're still looking at making use of this tool for other purposes, but it's still a work in progress at this moment.

We are using Network Assurance, primarily for our devices like routers and switches.

Change Manager is still a work in progress for us. While we have that module, we're still working on customizing it. It's understandable and it works well, right now. We are looking at automating that whole change management procedure using a third-party API integration along with Skybox. So that's still a work in progress at the moment.

The primary room for improvement would be to enable a web interface, which is not something which is there in the product. This is supposed to have come a year, a year and a half ago, but still has failed to come out. It still needs a client application to be installed on a workstation to be able to access that server and then run these reports. So I cannot extend that access to anybody. It has to be one administrator all the time. So unlike a web interface, where you can give multiple users simultaneous access and generate the various reports, that isn't a possibility at the moment.

The stability is something that is questionable. I don't know whether it is because of the kind of infrastructure we have or because of the product in itself. We're running it on a virtual machine right now. Maybe once a month, or once in every 45 days, it requires a restart because the application fails to connect. So I have to restart the whole Skybox Manager itself, the Skybox server itself, and then connect to it from our Skybox Manager. As I said, I don't know whether it's because of our environment or if this is primarily how the application behaves on a VM environment. That's still a question mark at the moment.

I don't think scalability is a problem area, to be quite frank. As to how much you can run, it primarily depends on the type of licenses you have. And they also have a hardware appliance which can take on any workload. So I don't think this is a problem area.

The technical support is only telephoning. While I do have a local contact with whom speak, more often it's via the telephone. They do have a fairly decent turnaround time. I wouldn't rate them as the best people around in terms of turnaround time, but it's fairly decent. I haven't had too many negative experiences with them so I would rate them fairly satisfactory at the moment.

We were not using any other solution, not something similar to Skybox at least. Our principle company was using a competitor product from Tufin. Their use case was fairly limited to firewalls. We wanted something beyond firewalls and also wanted an ability to manage and automate the change on the firewalls, etc. That is why we went for Skybox.

It was definitely complex. It wasn't straightforward by any means. It's not something that a person who doesn't know the product can deploy. It's not a simple double-click on an MSI or an EXE and it starts running. The documentation needs a lot of improvement, the setup documentation. But we had brought in our Professional Services at the time of installation, so someone was here to do that installation. But if you were to ask me to do it from scratch, I would hesitate to do so because it was rather complicated even just looking at it.

I think for the cost, what we got definitely is worth it. We're looking to expand it come next financial year, that will start from April of 2018. We'll need to look at how we will get better pricing this time around. So we're looking to expand the scope four-fold or five-fold, beyond what we have actually done initially.

The only caution I would give is that whenever you license for Firewall Assurance at the same time, for some weird reason, you have to get one Network Assurance license, just to manage one firewall. That's a little bit of overkill. But otherwise, in general, the pricing is fairly okay. But I would prefer that they change that licensing model for a firewall not having to consume a Network Assurance license.

We did a proof of concept with Skybox and Tufin, both. In terms of setup times, in terms of the interface in itself, Tufin was much better. But Skybox had better features and we felt it fit in more for our use cases, which is why we went with Skybox.

If you choose Skybox, then I would say spend a lot of time making sure that your network and your IT segment and all your devices and scope are properly documented. Make sure everybody fully understands how each of your networks are interconnected and exactly how your deployment happens. Because without that documentation, you will have a real hard time even explaining to a Professional Services guy how this needs to be set up. Because, like I said, this is not something which is straightforward.

It does need some time, especially in an enterprise environment where you're primarily using an RFC 1918, which is a private address space. Most of the time you will see that address space being used across different LAN communication technologies, so you will see a lot of conflict. You might see a lot of duplication. That is where the real problems start. So I would rather spend more time analyzing the whole setup, sitting and making sure it's well documented, before even getting into documentation. That's what I would tell the prospective buyers of Skybox.

Right now, based on my experience of having gone through, of having used it for the last one-and-a-half to two years, I would rate it about a seven out of 10. The reason for that is because of the lack of a web interface which is a big no-no for most companies. In today's world, nobody wants to use a fixed client to manage a security appliance. And the second one is because of the complexity of the whole setup itself. Otherwise, the product in itself is fairly good.

• Potential attack vector discovery
• Network troubleshooting
• Security check
• Compliance

Aside from Firewall Assurance, we use Network Assurance for network visualisation and troubleshooting. Currently, we are not using the Change Manager Module.

User interface. A web interface would be better.

• Change Tracking (audit logs)
• Access Analyser (access checks)
• Compliance

No issues.

No changes to check.

It is sufficient.

No.

The initial setup was easy.

No.

Check product compatibility. In our case, during implementation, we realised approximately 30 devices were not supported by the Skybox platform.

I am impressed with the tool's change management, firewall and network assurance. 

The solution needs improvement in firewall configuration checks. I would also like to see more configuration checks for Forcepoint and for other non-supported firewalls. 

I have been using the product for three years. 

My company has 20 users for the product. 

I would rate the solution's setup a ten out of ten and is straightforward. The product's deployment takes one week to complete. 

I would rate the tool's pricing an eight out of ten. 

I would rate the product a ten out of ten.