What Questions Should I Ask Before Buying SIEM?

Some areas and questions for evaluating a SIEM solution. These are some common things that come up from customers that we deal with. But there can also be a lot of others based on specific business needs.

It helps if they have a clear objective of what it is you are wanting. So review questions like the following:
* Is it just logs from a select few systems or all systems like servers, databases, applications, and desktops?
* Are all the users and systems internal or are they also mobile and could be working from home or over the Internet from a café?
* What operating systems need to be covered ie Windows, Linux, Solaris, Mac OSX, etc.?
* Do you want to collect syslogs from other devices like firewalls, routers, switches, wireless APs, etc.?
* There can be some discussions on Agents vs Agentless so there can be discussions on the pros and cons of these needs.
* Do you have compliance issues they need to manage ie like PCI DSS, ISO27001, HIPAA, etc.?
* What is it that you are wanting from the SIEM in reporting, there can be a lot of options on static reports, dynamic dashboards, PDF reports, correlation of log data with other systems like ticketing systems?
* Do you want to run a SOC or just get reports if and when they want to look at something, do you have the resources to monitor things or do you need to also work with an MSSP.
* What sort of alerting and threshold reporting do you want to get?
* Do you have complex network segments with multiple zones to collect and aggregate logs that they need to centralize to keep the logs away from the systems generating them and away from potential hackers?
* In general for those that are starting out for the first time, just stick to the core critical systems and collect logs from those systems until you understand more of what you are wanting to do with a SIEM collection and reporting. This helps to keep the project scope more controlled and confined so it's easier to manage. As you learn more then you can grow the scope later on.

Once you have a clearer idea on what you are wanting then it's looking to the vendors to download the software and see how well it works in your environment.
* How easy was it to get an eval license, did the sales and presales support help you get going quickly.
* How easy and quickly can you install the software and start to collect logs and then start to get some reports and visualizations on the data.
* How easy was it to identify problems and security issues, and what sort of value is that to the business.
* How easy is it to roll out, many large corporate environments can have complex change control processes and can the software easily fit within these processes.
* Cost is always a component to any solution so how well does it scale for your business, does it have known costs or are there variable costs like per GB of storage which often bites customers as there is always more data than you expect.
* How well can the solution scale out to hundreds or 10s of thousands of systems as the business needs change or the business grows.
* Can upgrades and license changes be done with minimal effort?
* What are the futures of the company do they invest in R&D to keep enhancing the product as there is always something new to do or OS version to support.
* How well does the vendor do support, do they only do internet only or do they allow you to talk to a real person that can understand you.
* Does the vendor play nicely with others, almost all customers have a mixed environment so being able to integrate and work other SIEM vendors always helps.

So having a clearer understanding of what they are wanting makes it easier to see “ yes this was a success” or ”no this was a failure” and did not meet the business objectives. Some use a scoring system in a spreadsheet to rank various areas from a scale of 1-10 with 1 being poor and 10 meets all needs. By doing this in a matrix it often helps to sort the good and bad more easily and the good from the very good as part of the review process. So having a bit of structure to the evaluation process helps with finding the right fit for the business.

Discovery questions you should ask any SIEM vendor:

-Would you like more insight into what’s going on in your network?
-Are your security-related compliance efforts manual and time-consuming?
-Would you know if an advanced threat went after your customer data or employee data before it was too late?
-Do you feel confident you're protected against stealthy, long term attacks that use social engineering tactics?
-Can you detect all the threats and risks taking place across mobile computing, social networks, and cloud environments?
-Do you find it difficult to keep up with constantly evolving threats, using limited staff and budget?
-Do you have a clear sense of what the risks are, associated with any vulnerabilities in your network, so you can build a prioritized plan of addressing the vulnerabilities?
-Are there any devices you’ve recently added or network changes you’ve made that impact your ability to ensure security and demonstrate compliance?
-Old ways of protecting networks can't keep up, and many organizations are looking for help in improving their security and risk posture. Is this a priority you are considering today?

The eight features of a modern SIEM based on an open, big data architecture:
-Leverages real-time behavioral analytics including machine learning.
-Enriches data with additional context to facilitate accurate prioritization of threats.
-Easy access to pre-packaged security content, relevant security use cases, and a support library with dynamic security content.
-Predictable cost and low TCO with a pricing model that is aligned with your business.
-Automated incident response capabilities through automated playbooks.
-Cloud-based SIEM deployment options for cloud or hybrid IT environments.
-UEBA, NTA, and SOAR capabilities available in the SIEM platform.
-Legacy SIEMs require a lot of manual work. Security analysts need to spend a lot of time switching between solutions and screens while hunting down threats, manually remediating breaches, and writing and tweaking the manual rules the SIEM relies on to find threats. A modern SIEM uses integrated SOAR to drive security response through automated case creation and management, ending swivel chair investigations and freeing up security analysts to focus on security.

Compared to a legacy SIEM, which struggles to meet today’s security challenges, a modern SIEM improves your security posture through improved detection, investigation, and response capabilities.

Before buying a SIEM solution first ask yourself the following question: For what purpose and for what requirement will I purchase a SIEM?

The scope:
- Will it only be for compliance (but then it could be sufficient to a good Log Management tool)?
- Does the scope also for security monitoring (correlation, investigation, analysis, and reporting) and then also SIEM make sense?

If you are in the second case you need to ask yourself a second question:
- Who will use your SIEM? Anybody thinking that the SIEM produces alone results and benefits (then you must abandon the idea of buying a SIEM)?
- Will there be a service/SOC outside?
- Will there be an internal SOC?

If you are in the last case (the one that justifies the purchase of a SIEM and not an MSSP) you need to think about the best purchase to maximize its potential that you have in terms of the number of operators/analysts and their automation and competence (*).

- How and in what time does the SIEM vendor support you in the post-sales phase for software issue (numbers and real cases)?
- How does the SIEM start to collect first logs and visualizations (numbers and real cases)?
- How many days of additional vendor professional services should serve for an average deployment (up to 5,000 EPS) and one large and complex (up to 10,000 EPS)?
- What is the vendor best practices for the roll-out of SIEM in an IT environment complex systems and processes (real cases of implementation)?
- How much do I have to consider me (*) independent in changes to configurations and evolution of SIEM finished roll-out?
- How to scale the license of SIEM to the increase of my IT environment to monitor (an example)?

I would stress about the importance of obtaining from the vendor real numbers of real cases.

That is correct, you don't just install it and that is it. There is quite some work to do after installation:
* You need to get events into the system, they need to be normalized, this is dependent upon the vendor and how they offer support for it. Again this is also important where there is a version upgrade of the source device where log types change.
* You need to configure correlation content and tune it to fit your environment – remove false positives, add assets to the SIEM and so on
* Monitor the system what kind of alerts are generated
* Keep the system up to date with vendor-provided updated software

What questions should someone ask before purchasing a SIEM?
* Do you have an existing library of use cases?
* What kind of content is available?
* Is this content updated regularly?
* What kind of event sources do you support?
* What If I need to add a custom application?
* What is your license model? If I have a surge, will the system accept it – will anything be throttled as a result of license violation?
* How can I monitor for the availability of elements within the system – usually the collection layer and analysis/storage layer are separate – if the collection layer does not work that means the analysis layer has nothing to analyze. So how can I monitor that?
* Can I upgrade the system just by changing the license? Will the proposed solution limit us at some point and it will need to replaced as a whole – this is usually true with SIEM that is delivered as an appliance?
* Does the license limit me in any way as to how many different sources I can collect?

What am I using for SOAR?
What am I using for Ticketing?
What am I using for communication?
What am I using for ML/UBA?
How quickly do I need to be operational?
Will I be staffing my own SOC or farming that out (MSSP)?
What is the bandwidth required for all of the data that I need to process?
Am I going to use in-house bare-metal, data-center bare-metal, my cloud, or somebody else's cloud offering?
How well is the company that owns the product going to support/extend it (i.e. DO NOT BUY ARCSIGHT)?
Would I rather pay for a product or for people (this is important because many cheap products are admin/staff-heavy)?

-Ease of operation including patching and upgrades.
-Should ensure that all related suspect data (network traffic, user behaviour, ..) are gathered and presented as one suspect security incident to significantly reduce the analyst work.
-Provides an easily understood summary of each suspect security incident with prioritization and important details and drill down for all details to ensure more efficient handling of suspect security incidents.
-Broad out of box support (collect/receive, parse) for devices, applications including from cloud, os, security solutions which should be continuously and automatically updated (versions and new).
-Extensive out of box support for detecting suspect network traffic, suspect user behaviour (user behaviour analytics), continuously updated.
-Easy support for or builtin continuously updated threat intelligence.
-Out of box support for vulnerability scanners to provide better prioritization of suspect security incidents.

What questions should someone ask before purchasing a SIEM?

-Ask about and understand the ease of use.
-How long to implement and make the SIEM operational based on use cases?
-What compliance functionality is included for alerts, rules, and reports?
-Does the SIEM have a fully integrated and easy to implement UEBA component?
-Is the reporting tool native or is it an OEM solution?
-Can the SIEM run on-premise, in the cloud or in a hybrid mode?
-Is the solution sized accurately on both hardware and cost perspectives?
-Is the SIEM vendor-independent or from a multi-product company where additional components may be needed for full visibility across the network?

Help your peers ask the right questions so that they'll make the best decision.

When moving ahead with a SIEM purchase you need to have clarity on your goals and requirements. Create a list and prioritize it in terms of importance:
Reasons for looking at a SIEM?
Key features you'd like to have.

Some reasons you might see:
Need to meet new compliance laws on logging and reporting.
Need to centralize all my security technologies to better access threats: Firewalls, Anti-Virus, End Point, etc.
Company execs are looking for use to beef up our security posture and we are unable to keep up with all the event logs and potential threats.

Key Features:
Support for existing technology: Firewalls, End Point, EDR, Anti-Virus.
Support for Network flows, User Behavior Analytics, Forensics, AI, etc.
Need to run in the Cloud. on AWS on Azure. Host on-prem in a virtual environment.
Need the option to start on-prem with ability to move another platform.
Offers 24/7 365 Managed Services for your SIEM.

This is just a good starting point. You can dig much deeper with building out a full requirements list by googling sample SIEM RFPs. Be careful not to get lost in the feature functionality loop. I've seen companies crippled by this as all vendors start to look the same on an RFP reply. Stick to your main reason above and then create a shortlist. Look to Gartner and Forester analysis to help get started on your shortlist.

All the previous answers are excellent and certainly should be part of the due diligence. 

I frequently run into the question about the SOC and who is going to monitor the SIEM and respond to incidents 24x7/365. All that aside, the one main question I ask is "Why do you want to purchase, deploy, tune, and monitor your own SIEM when today, you can purchase this as part of a holistic security solution As A Service?" 

I think today people understand the need but fail to understand the amount of time and expense it is going to take to actually implement. 

That's why MSSPs are generally better suited to help companies deploy this As A Service. This answer might be a bit of a left turn from the spirit of your question but still begs to be answered in every SIEM opportunity. 

You are right! SIEMs do take some time to implement, especially if you have a lot of sources to monitor and integrate to the SIEM. Give the SIEM a few weeks to set a baseline and clean out false-positive events.

Before purchasing a SIEM you have to consider how it should be implemented. On-premise SIEMs are usually very expensive and needs expertise in using. You can also opt for VM and cloud-based options to save. And don't forget the Managed SIEM option especially if you don't have the people to manage it.

Very useful with support for flows as it is easy to implement, will provide a lot of automatic use cases, detect other uses cases than log-based.

Don't expect to just plug any SIEM system into your network and everything will work automatically. You will have to adjust some settings according to your needs. Depending on the size of your network maintaining a SIEM system can be a full-time job.

What are your reasons to implement a SIEM system?
Do you:
a) Just want to collect logs?
b) Want to perform a search from time to time?
c) Want to deploy an automatic reporting and alerting?

Does it support analysis of the logs from your specific environment. (especially network and security devices)?
What are the total costs (including all necessary add-ons and licenses) now and also in the near future?
Don't buy a SIEM system just by reading the specs and data sheets, instead request a PoC (Proof-of-Concept) so you can test it live in your environment.

The following questions should be asked:
1- How the product is licensed?
2- What integrations can be done with proposed SIEM solution?
3- Is it equipped with Behavioral Analytics?
4- How Threat Intelligence performs?
5- How to perform forensics using SIEM solution?
6- What are the built-in reports? And can it be customized?
7- How can real-time alerts be generated? Can we create our customized rules to make alert us?
8- What are the dashboards and can they be customized?
9- Does it identify Zero-Day attacks?
10- Can we vulnerability assessment using the tool?
11- How the incident response is handled using this tool?
12- What will be the cost of the solution?
13- How the support services are provided?

As you can see from the answers that you have received, it all depends on what your goals are, objectives you are trying to meet and your current infrastructure. I think the responses you have received have all pointed out some good information that you can use to guide you in the right direction. Each manufacture will have their way of implementing their version of SIEM. SIEM solutions do work if it is deployed/implemented correctly.