What needs improvement with Microsoft Security Copilot?

I believe Microsoft Security Copilot could use performance improvement based on what I have seen. The UI is sluggish, which is a specific area where performance could be better.

The effectiveness of Microsoft Security Copilot's natural language interface is sometimes slow and it occasionally has hallucinations, but other than that, it works well. I do not know when it is slow for us; it is random. I ask a question and sometimes it takes time to think. Sometimes it does not give the accurate answer. This is very typical of large language models, so I do not get too excited, but that happens sometimes. I assume Microsoft Security Copilot can be improved; the context and the training that the agent has is limited to security, and it should be. This is why I would expect it to be faster than other agents. Slowness and hallucinations are detractors, so I think this should be improved.

The flexibility to have more integrations with other available security tools would improve Microsoft Security Copilot, particularly with third-party integrations to enhance ease.Regarding pricing, it is a bit high, as each time it is discussed with customers, pricing becomes a challenging topic. This is an area worth investigating for potential improvement.

Governance is an area where Microsoft Security Copilot can be improved, and it should be included as an additional feature in the next release.

I think it can be improved by having more users feeding it information because the more things it pulls from, the more we teach it and it builds on it. More users means more ways to fix and build upon it.

One thing that we heard not from our engineers but from our procurement team was that they are not very clear on the licensing and pricing for Microsoft Security Copilot. We have been trying it out for the last two months as we are a Microsoft shop and have been given early access to it. However, there has not been a lot of clarity for our procurement team on what will happen once this trial period or this early access is over. If there could have been a bit more clarity on whether this is based on a per seat per month basis, or if it is based on APIs, or on the basis of how many calls to action you are taking, that would help. When they have tried to reach out to the sales team, the response they have gotten is that they should try it out first and pricing will be discussed later. Just a bit more clarity on pricing would be nice to have.

The natural language interface is extremely approachable because inputs are as good as outputs, which is a universal understanding of natural language models and LLMs with that interface. The real value is in the tools that sit underneath it. The question is what the agent can actually do once asked that question. Can it pull information from areas that previously it could not? Can it actually action on these? Can one determine, via MCP or something of that nature, what the security posture should become and what steps need to be taken next? It is truly dependent on the underlying infrastructure and technologies, less about the interface itself.AI features have not quite been used to automate tasks yet. Having a human in the middle is still proving to be the best method for adoption because not many people are ready to give AI the wheel. Human involvement with enough refinement and inputs from that trusted human, and then eventually, companies should allow AI to fully automate their security processes within about a year. Agent-to-agent is how Microsoft Security Copilot can be improved. Agent-to-agent is the next logical step in creating an automation chain where RBAC can be enabled to ensure that each agent only has a specific amount of permissions through least privilege and zero trust principles. The capability to then understand, detect, determine, and respond can be reduced even further.

The main concern relates to security and data privacy. When storing data in Microsoft environments such as Azure or Microsoft iCloud, customers express concerns about data security. Even though Microsoft is a major technology company with insurance coverage, customers worry about potential data leaks, especially in sensitive industries such as banking and semiconductor manufacturing. Customers who are not technically proficient often rely on consultants for guidance regarding security integration. They either trust our recommendations or hire internal consultants familiar with Microsoft products to evaluate security layers before making decisions. Regarding automation, customers are seeking alternative solutions because they are not utilizing all features of Microsoft Security Copilot due to expensive licensing. They predominantly search for low-cost, open source options. A suggestion for Microsoft would be to develop more customized solutions that can integrate with different platforms or open source solutions. Rather than requiring exclusive use of Microsoft products for security layer services, they could consider implementing multi-layer integration with various open source platforms.

To make it a perfect 10, Microsoft Security Copilot would need to take decisions independently and reduce hallucination. The issue with all AIs, not just Microsoft Security Copilot, is that when I do something and feel it's wrong, I need a tech lead to verify.

For many customers, it's too expensive. It comes at a price. What we end up doing is activating it for the purpose of extracting an action plan, then we turn it off. So, unfortunately, for most customers, we're not using it constantly. We're using it sporadically or while investigating a particular incident. We are turning it on and off because of price constraints. More transparency is needed. The pricing model should also be different. To improve Microsoft Security Copilot and make it the best solution in the market, the main hurdle today is cost. If a way could be found to make it more cost-effective and streamline the licensing mechanism, they have the technology to succeed in the marketplace. It's potentially better than what's out there.

I believe Microsoft Security Copilot could be improved in terms of the UI and the process. The UI needs improvement because it can be distracting with some windows on your right and left sides. That's how I feel; maybe some other users have different opinions.

It would be beneficial if Microsoft Security Copilot could work across multiple systems. For example, I would like it to automatically transfer information from Outlook to PowerPoint. At the moment, this feature is not available.

Currently, Microsoft Security Copilot is not very intuitive. I still need to figure out how to operate some features, so more intuitive methods would be beneficial.

One area for improvement is ensuring that personal information is not exposed through Copilot. For example, salary information of executives should remain confidential. Currently, I do not see this feature. Additionally, there should be better natural language processing to eliminate the need for prompt engineering.

The solution does exactly what I need, but sometimes the prompts required to get the needed information need to be rephrased several times. This might improve as I become more accustomed to using the system.

It would be beneficial if Copilot's security features were available to roles other than just the top security personnel. Other roles could use some of the functions, perhaps with limited access, allowing for greater flexibility within an organization.

At the moment, I have not identified any areas for improvement. I am aware that security features are planned for further development next year.