Artificial intelligence is everywhere in cybersecurity conversations right now. From phishing emails that read flawlessly to automated reconnaissance at machine speed, AI is already reshaping how cyberattacks are launched and defended against. Yet alongside real progress comes real confusion. Headlines shift between promises of fully autonomous security operations centres and warnings of unstoppable AI-driven cybercrime.
Based on insights shared during Eye Security’s recent webinar “AI Agents in Cybersecurity”, this article unpacks what AI, and especially AI agents, mean for organisations today. It separates practical reality from hype, explains how threat actors are already using AI, where defenders benefit, and why human expertise still matters. Most importantly, it outlines how small and mid-sized organisations can prepare for what comes next without getting lost in buzzwords.
Watch the full on-demand webinar here:
AI is a force multiplier for threat actors and defenders alike
At its core, AI acts as a force multiplier. Large language models (LLMs) do not invent entirely new attack techniques but they dramatically increase speed, scale, and efficiency. Tasks that once required specialist skills or significant time investment can now be automated, templated, and executed faster than ever. Threat actors benefit from this just as much as defenders.
On the offensive side, AI enables adversaries to map attack surfaces and conduct reconnaissance far more rapidly, craft highly tailored phishing and social engineering campaigns at industrial scale, and prototype exploits quickly after new vulnerabilities become public.
Defenders, however, benefit from the same acceleration. AI systems can process and summarise vast volumes of security telemetry, eliminate repetitive manual tasks, and streamline documentation, reporting, and investigation workflows.
In both cases, the technology amplifies human capability rather than replacing intent, creativity, or accountability.
The rise of AI agents: scaling the threat actor workforce
The term refers to systems in which multiple AI agents operate simultaneously, each with clearly defined responsibilities. Rather than a single operator switching between tasks, these AI agents allow tasks to run in parallel, effectively functioning as a coordinated virtual team.
In a practical attack scenario, one agent may scan for exposed services while another maps vulnerabilities, a third crafts phishing lures, and a fourth prepares exploit code, all without human intervention at each step.
This concept has gained widespread attention following reports that state-aligned groups are experimenting with fully agent-driven attack chains. The technical capability is noteworthy, but the strategic implication is more fundamental. Agentic AI enables a single attacker to scale their operations far beyond what was previously possible.
Yet, despite this acceleration, the underlying mechanics of the attacks remain familiar. They still depend on exploiting known vulnerabilities, abusing credentials, and leveraging classic social engineering techniques. In other words, AI amplifies operational speed and efficiency, but it does not reinvent the core playbook of cyber intrusion.
What AI attacks look like today and what they don’t do (yet)
Despite frequent dramatic headlines, fully autonomous, end-to-end AI-driven cyberattacks are not yet the reality. The capabilities demonstrated today remain powerful but are largely confined to specific stages of the intrusion chain rather than replacing the need for human direction or decision-making.
The areas there AI is already effective
At present, the most visible and operationally relevant impact of AI appears in the early phases of an attack. Phishing and broader social engineering campaigns have become more convincing, better localised, and trivial to generate at scale. On the exploitation side, proof-of-concept code now emerges far more quickly after vulnerability disclosures, often within hours, accelerating the window in which organisations are exposed.
AI also excels at reconnaissance, automating the information-gathering and target-profiling steps that traditionally consumed significant manual effort. Much of this progress is intentionally subtle. Recipients may not be able to distinguish whether a phishing email was authored by a human or generated by an AI system, and that uncertainty is precisely what threat actors are aiming for.
The areas where AI still falls short
Equally important is recognising the limits that persist. AI systems do not yet conduct fully autonomous intrusions without human oversight, nor can they reliably make nuanced judgement calls during live operations. They lack an understanding of organisational context, business priorities, and the real-world consequences associated with operational trade-offs.
Even the most advanced models are vulnerable to hallucinations, misinterpretations, and blind spots. And these are risks that remain unacceptable in high-stakes environments such as incident response. Human expertise continues to play the central role in orchestrating complex intrusions and in defending against them, with AI serving as an accelerator rather than a replacement at every stage.
Why fully autonomous SOCs are still a myth
The vision of a fully autonomous, AI-only Security Operations Centre is appealing. It promises faster detection, lower operational costs, and significantly reduced reliance on human analysts. However, despite rapid advances in automation and machine learning, this concept remains far beyond what is possible in real-world security operations.
A functioning SOC does far more than identify suspicious events or generate alerts. Effective cyber defence depends on capabilities that current AI systems cannot consistently provide, including:
- Correlating signals across fragmented systems to form a coherent picture of an incident
- Understanding operational and business impact, not just technical symptoms
- Communicating clearly with stakeholders during high-pressure, time-critical situations
- Making judgement calls under uncertainty, where incomplete information demands expert interpretation
For this reason, Eye Security treats AI as a highly capable junior colleague who is fast, helpful, and able to take over repetitive tasks, but never operating without supervision. Human analysts remain accountable for:
- Final decision-making and validation
- Interpreting ambiguous signals and edge cases
- Coordinating communication and escalation paths
- Orchestrating the full incident response lifecycle
AI enhances operations but does not yet replace the critical thinking and contextual awareness that define effective security operations.
The human-in-the-loop model
The most effective operating model in cybersecurity today is neither human-only nor AI-only, but a tightly integrated collaboration between both. In this model, AI accelerates the work and expands analytical capacity, while humans ensure contextual understanding, judgement, and accountable decision-making.
In practice, AI contributes by:
- Taking over repetitive and time-consuming groundwork
- Preparing concise summaries of complex data
- Generating first drafts for reports or investigations
- Identifying patterns and correlations across large data sets
- Reviewing and validating AI-generated findings
- Refining outputs based on operational, business, and threat context
- Making the final decisions during investigations and incidents
- Providing oversight, explainability, and accountability
For details, watch the on-demand webinar "Insights from the SOC: Why Analysts Still Matter in the Age of AI":
The overlooked risks of shadow AI and data leakage
And still, one of the most immediate and underestimated challenges organisations face today is not malicious artificial intelligence. Rather, it is the uncontrolled use of AI within their own workforce.
Employees frequently upload internal documents, emails, and sensitive data into publicly accessible AI tools to streamline their daily tasks. Much of this happens outside the visibility of IT or security teams, creating what is increasingly referred to as shadow AI.
The security implications are significant:
-
Sensitive information may be inadvertently exposed or processed by external systems.
-
Data may be retained or used for model training without the organisation’s knowledge.
-
Businesses lose control over where their information is stored, transmitted, or replicated.
-
SMBs are disproportionately affected, often lacking the resources to enforce enterprise-level governance.
A related and rapidly emerging challenge is prompt injection, a new class of vulnerability inherent to large language models (LLMs). Because LLMs interpret embedded instructions within written text, threat actors can manipulate AI behaviour by inserting hidden commands, much like SQL injection exploits in traditional applications.
Eye Security has explored an innovative defensive application of this concept: Prompt Injection for Good. By embedding invisible instructions into documents, AI tools reading them can be instructed to warn users or block certain actions such as uploading sensitive data. Although this method is not a complete or foolproof safeguard, it demonstrates a conceptual shift. Here, data itself becomes an active control surface in AI security.
👉 Find out more about the Prompt Injection for Good project in our research article "Battling Shadow AI: Prompt Injection for Good" and try out the free open-source prototype on GitHub.
Guardrails matter and they won’t always be there
Although today’s commercial AI platforms include guardrails that restrict harmful use, these protections are only a temporary barrier. Current systems deliberately limit the generation of malware, fraud techniques, or automated abuse, making it harder for users to misuse them for malicious purposes.
As open-source language models continue to improve and as hardware requirements decrease, threat actors will increasingly be able to run high-performance AI systems entirely on their own infrastructure with no restrictions, no monitoring, and no safety layers.
Once these models operate without guardrails, the barriers to large-scale abuse fall rapidly. Phishing campaigns can be launched with far greater speed and precision. Deepfake voice and video fraud becomes easier to produce and more convincing to victims. Impersonation attacks can be tailored to specific individuals or organisations with unprecedented accuracy. This is an emerging reality that organisations must prepare for now.
Where AI brings real defensive value today
Despite the very real risks associated with AI-enabled attacks, there are also clear and immediate areas where AI delivers meaningful defensive value. These strengths appear most prominently in structured, repeatable workflows that benefit from automation and consistent pattern recognition.
Today, AI significantly enhances threat exposure management by helping organisations identify and understand the vulnerabilities and assets that define their attack surface. It also accelerates attack surface discovery, enabling teams to map external-facing systems more quickly and systematically than before. Similarly, AI contributes to more accurate vulnerability prioritisation by analysing risk factors, contextual signals, and exploit likelihood at scale.
In addition, AI is increasingly effective at supporting automated penetration testing. It can generate test cases, analyse responses, and highlight misconfigurations, thereby reducing the manual workload for security teams. Beyond technical testing, AI improves policy and compliance support by drafting documentation, interpreting regulatory requirements, and flagging gaps that require human attention.
Practical first steps for organisations
For organisations wondering where to begin, the advice is intentionally pragmatic:
Centralise AI usage
Organisations should begin by consolidating their AI tooling. Rather than allowing teams to experiment with multiple disconnected applications, it is more effective to select a primary, managed AI platform that integrates with existing security and productivity systems. This reduces fragmentation, ensures consistent controls, and gives IT teams the visibility needed to monitor and govern usage responsibly.
Build awareness first
Employees need a clear understanding of both the benefits and the risks of AI tools, particularly when it comes to data handling. Many incidents arise not from malicious intent but from well-meaning staff uploading sensitive information into public services.
Implement basic guardrails
Warnings, access controls, and usage guidelines help set expectations and create boundaries for safe AI interaction. Even lightweight governance mechanisms reduce the likelihood of accidental data exposure.
Use established frameworks
Established standards such as the OWASP LLM Top 10 offer a structured way to identify and mitigate risks associated with large language models. These frameworks can be adapted to organisations of any size.
Work with experienced partners
For most small and mid-sized businesses, building an AI governance model is not realistic. Partnering with specialists ensures that organisations receive the expertise required to implement effective controls, integrate AI securely, and respond to emerging threats.
Looking ahead with optimism and concern
There is genuine optimism around AI's potential to improve cybersecurity. It can make security teams more effective, enhance preventive controls, and reduce manual overhead, allowing analysts to focus on higher-value tasks.
At the same time, there is justified concern about the rapid evolution of open-source models, the potential loss of existing guardrails, and the growing risk of AI-enabled impersonation and fraud. The organisations best positioned for the future are those that act early by developing measured, human-centred AI strategies that balance innovation with accountability.
Final thoughts and outlook
AI will not replace security professionals. However, those who integrate AI responsibly into their workflows will significantly outpace those who do not. The objective is not autonomy for its own sake. Rather, the aim is to achieve better outcomes, including faster detection, clearer decision-making, and more resilient organisations.
When used wisely, AI becomes a powerful ally, enhancing human expertise and operational efficiency. Used blindly, however, it introduces additional risks. The key distinction lies in the combination of expertise, robust governance, and clear intent.
Frequently Asked Questions (FAQ)
What are AI agents?
AI agents are autonomous AI systems that can perform specific tasks independently. In cybersecurity, multiple agents can work in parallel, scanning for exposed services, mapping vulnerabilities, creating phishing campaigns, or drafting exploit code, effectively forming a coordinated virtual team.
Are AI-driven cyberattacks fully autonomous today?
No. While AI accelerates certain attack steps such as phishing, reconnaissance, and exploit prototyping fully autonomous, end-to-end attacks without human oversight are not yet common. Human expertise remains essential.
How does AI help defenders?
AI enhances defensive operations by processing large volumes of security data, automating repetitive tasks, generating summaries, supporting reporting, and identifying patterns. It amplifies human capability rather than replacing analysts.
Can AI replace a Security Operations Centre (SOC)?
Not yet. AI can automate certain tasks, but SOCs require human judgment to understand context, correlate data, communicate with stakeholders, and make decisions under uncertainty. Human oversight remains critical.
What is the most effective AI-human model?
A human-in-the-loop approach is most effective. AI handles repetitive groundwork, while humans validate findings, make decisions, and remain accountable. This preserves efficiency without compromising trust.
What is shadow AI, and why is it risky?
Shadow AI occurs when employees use AI tools without IT oversight or governance. This can expose sensitive data, reduce organisational control, and introduce vulnerabilities.
What is prompt injection, and why does it matter?
Prompt injection is a vulnerability in large language models where hidden instructions in input data can manipulate AI behavior. While it can be exploited maliciously, organisations can also use it defensively, for example, to prevent sensitive data from being uploaded.
How should organisations start using AI securely?
Start by centralising AI usage, building awareness among employees, implementing basic guardrails, using established frameworks such as the OWASP LLM Top 10, and partnering with experienced cybersecurity providers.
