What advice do you have for others considering CRITICALSTART?

Trust the CRITICALSTART team. For the products that they resell and support, they know them very well. As you go down that path, you have a good heap of knowledge to rely on. Do not try to build it out or figure it out yourself. We have since transitioned Cylance and Carbon Black over to CrowdStrike. We still use them for that service and also use them for our SIEM, because they host and manage Splunk for us. That all integrates into ZTAP. Using that and any new products that we bring in-house, we work with CRITICALSTART to see if they have already gotten an integration connector built. Typically, we'll use theirs. If there's already something built, or they have the appetite to build it, we'll use that service as we onboard it internally as well as into CRITICALSTART. The biggest lesson is transitioning from alert overload to being at a point where we do have eyes on alerts, where every alert is truly possible. It's something that a lot of people sell and not a lot of people do very well. Being able to come into this relationship, then where we're at today, it kind of opened my eyes to: There is the opportunity and the possibility to do this. Stuff is not going to get dropped or missed by our operations group. I would give them a nine (out of 10). They are right there at the edge, probably a leader in the market. That's kind of why we chose them. Of course, there is always room to improve, but they're doing a lot of things right. We appreciate their team.

If you have people who already do this at your company, and they're paid well and they know what they're doing, and you have multiple products like this that they can manage, then you don't really need CRITICALSTART. But if you are a small group of IT people trying to support an entire company and you have a crazy, complex product like CylancePROTECT or Carbon Black defense or Palo Alto Cortex XDR, or anything like that, then it's probably better to leverage an expert company like CRITICALSTART. The only data source we are using them to manage is our antivirus and they integrate with that. I don't know if they would have been able to integrate with our other data sources. We didn't try that. I have used CRITICALSTART's mobile app but I haven't used it lately because we get so few alerts that I don't really need it. A lot of people use the mobile app for when they're home on the weekends and they need to get stuff remediated quickly. We don't have people working on the weekends, usually, so it's not a huge issue for us. If my company is working, I'm at my office and at my computer already so I don't need the mobile app for that. The mobile app has the basic features that you need to use their service. I don't remember if it lets you link to the service they're managing; for example, I don't think there's a link to the Cortex XDR app from CRITICALSTART's mobile app. So you can't really dig deep into anything on there, but that's not their fault. It's just because you can't do that, period. But for quick remediation or quick alerting, it's perfect. I haven't spoken to CRITICALSTART's analysts lately. During implementation, we had weekly meetings. Usually I only talk to them when things aren't going well, so the fact that I haven't talked to them in a while means we're good. But they were always available when I needed them. If I needed them quickly, they could join a meeting within a day. Out of all the service providers I've had to work with over the years—I've been here six years—CRITICALSTART is my favorite to work with. I see them at almost every convention that I go to, no matter what city I'm in. I'm always happy to see them and they always recognize me. I feel like that's worth something when you're looking for someone to work with. They have a personal touch.

I love the fact that they were local to the DFW area because I know them and they know me. When I've had to have some heart-to-heart conversations, it's simple enough to have a face-to-face meeting with their leadership, break bread, and have some pretty direct conversation. And they listen. They express why they handle things a certain way, but they are willing to listen and see how they can integrate, modify, and change, not to just accommodate the customer, but also to make it consistent amongst all of their customers. That's the other thing that I'm very big on a proponent is, if I'm doing something, I don't want to do it just for me. I want to make it better for all the other customers that use that product. After a year of using the service, our expectations have been met in terms of services delivered on time, on budget, and on spec. I'm ready to take it to the next level. I'm ready to do the endpoint protection integration. Unfortunately, that costs more money so I've got to get that approved. My advice would be to make sure that you know what it is that you really want done. Understand what your use cases are as an organization before you get a jump in with anybody. Ask very direct and hard questions to those that you're meeting with. Take it beyond the sales engineer or the sales guy. Ask for meetings with the leadership of the MDR Service, they're willing to meet with people, to have those good conversations about what the services are. When I first went into it, I thought it was machine learning that was handling Splunk integration. I found out after the fact that it wasn't. It was use case build-outs that they built as alerts within Splunk that did correlation. And then based on those correlations, or use cases as they call them, they are ingested into Z-TAP, and Z-TAP then looks at filters. If it doesn't meet a filter, then it gets populated down to an analyst. If the analyst finds that it needs to be further investigated by the client, then they escalate it down to us. Whereas, with an endpoint integration, that is machine learning. I think that was the misconception in the way that it was described and explained. That was one of the direct conversations that I had with them. Was that going into it, I thought that Splunk was machine learning as well but then I found out after we integrated it and asking some very direct and hard questions to their implementation people, that it wasn't. They explained to me why it can't be or why they're not there yet. Needless to say, that was one thing that I wish was better explained and articulated, and they now know that. Unfortunately, machine learning is the future for this type of service. The way that technology is progressing and the more and more the bad guys are utilizing machine learning themselves on how to build out malware and attack situations, if you're not using machine learning in certain aspects, you're behind the game or you're doing it the old school way. Which is not saying that the old school way is bad, it's just slower. I would rate CRITICALSTART an eight out of ten.

I would suggest using a phased approach, instead of dumping everything in from the beginning and then trying to sort it out, triage-wise. If you add types of sources or tools to it one at a time, instead of "everybody into the pool" right away, that really helps you. That way it allows you to get your handle on the smaller piece of the pie first and then work your way forward. As for what to start with, it depends on what you're pushing to them. I didn't start necessarily right away with the MDR, but I did have my endpoint protection being looked at by them, at least. Then I added in my SIEM, which added to the overall complexity level. Unfortunately, I didn't have one completely finished before I added the next and that slowed me down a little bit. That was too much for one person to try to handle all by himself. The biggest lesson is that even if you have a small team and limited resources, you can actually be effective as a company, from a security program standpoint, by using their service. My expectations have been more than met in terms of service delivered on time, on budget, and on spec from CRITICALSTART.

The new web portal they implemented is quite robust. It's very next-generation, but it does need small tweaks. You have to get used to it and learn a little bit about it. That's why I prefer the mobile app. The mobile app seems to be more straightforward. The new UI has more advanced features but you would have to click around and learn a little bit more. It's not as intuitive as the mobile app, but the functionality is there. As for their contractually committing to paying a penalty if they miss a one-hour SLA to resolve an escalated alert, we have never run into that situation. They haven't missed an SLA in two years. They offer a very personal, connected experience. I don't know of any other company that has that kind of a personal touch to either its services or its MDR solution. That was the decision-maker for us. This has been a positive experience and money well spent. If we had to do it again, we would gladly choose the solution that CRITICALSTART provides, versus going with other solutions or using something in-house where we would probably have to spend double what we are spending now.

In terms of advice, I don't feel that implementing this service is any different than implementing any other system into your environment. A lot relies on your project management skills. I would attempt to test your MDR choices against a framework. The framework that comes to mind is the MITRE ATT&CK framework, which everybody is familiar with. Have realistic expectations about what vulnerabilities your MDR partner is really going to mitigate. That's the lesson I have learned. In terms of CRITICALSTART's Trusted Behavior Registry and the way it resolves things that are known as trusted, so that the focus is on resolving unknown alerts, I'm obviously not looking at all of the alerts that they work on. But what they escalate to me, only the alerts that I'm seeing —which is a small percentage — if I were to rate them on a scale of one to 10, I'd rate this aspect at eight. There are a few things that slip through, things that they'll escalate that I know should not have been escalated, but it's a very small percentage of what they actually escalate. It's a very small percentage where I'll have to just say, "Hey, did you mean to do this one, because we've been through this before," or a virus total shows that it's 100 percent clean, so why did it get escalated? It's not common but it does happen. The service missed a pen test, but I still have a high level of confidence with the data and the actions they take. We had hired a red team, so the situation was a red team test. Red teams are generally 100 percent successful, or very close to it. With them, you always expect to uncover the unknown. But I do have confidence in the tool and the data that they are looking at. The number of escalated alerts we receive, compared to the number the service's Trusted Behavior Registry resolves, is probably less than 5 percent of the total.

Our expectations have been met in terms of services delivered on time, on budget, and on spec. The implementation went as expected. The pricing hasn't been an issue. Everything went as was decided at the beginning. Everything has gone through as I would expect. I would rate CRITICALSTART a ten out of ten.

So far, I'm very happy with the service. However, we have no comparison. This is the first ever MDR service that we have used. We have not had enough time to really verify the protection that the service offers is enough because we haven't suffered any attacks. We don't know whether we're lucky or if the service really does work. You can never do enough to stay safe. It has helped me to see a lot of things going on with our network that I didn't see before. We were just not equipped with the right tools to really have a clear view of our network, and now we do. For smaller companies, in order for them to grow, they have to trust the professionals. Sometimes, we tend to save every dollar possible and do everything on our own, either by reading a book or taking a course. It's a good thing to learn new things but I learned that no one can cover every aspect of a company's IT needs. When the time is ready, you need to leave certain things to the people who are really good in that area, freeing up yourself to do things that you are really good at. I would give it nine out of 10 because of the pricing. So far, that's the only downside that I can see.

The biggest lesson I've learned from using CRITICALSTART is that you don't necessarily need an internal SOC to make your customers happy. We get asked all the time on questionnaires, "Do you have a SOC?" We're able to say, "No, we use an external SOC to manage alerts for us." I've really only been pushed on that a couple of times. And at other times I've had companies that are larger than you would think come back and say, "Hey, we do the same thing." They may have an internal SOC too, but they still leverage a similar company to triage stuff before it even gets to their SOC. I use CRITICALSTART's mobile app occasionally, although not as much as I did when I didn't have a dedicated person really looking through the alerts. It's mostly good. I don't have any major complaints about it. There are a few things here or there that need to be polished, but I think it's come a long way. The rest of the team is like me. They use it occasionally to pull up an incident that may be a higher risk, when they're running around doing things. But for the most part, we use the web browser. On a daily basis there is only one person using CRITICALSTART. He's a security analyst for me. I'll occasionally jump in and my architect will as well, to help on the more advanced things or to adjust the filters and to do things that the analyst doesn't really do. I would rate CRITICALSTART at eight out of 10. There's room for them to improve, but overall it's a good value and we're happy with them.

Do your homework. Compare the big boys, the larger managed service solutions, with some of the more boutique ones, like CRITICALSTART, and ask yourself: What is it that you want? Do you want to be a small fish in a big pond or a big fish in a small pond? You always need more logging space than you actually think you need. They monitor our endpoints. I would definitely give them a nine (out of 10). They are extremely effective in combating alert fatigue. They're creative in the way they do business. They are also very approachable and very customer service-oriented.