AI Cybersecurity and the Governance of Digital Trust

AI lowers costs for both cyber defenders and attackers alike
Verification automation benefits defenders, but human judgment remains crucial
AI-generated impersonations challenge identity verification, favoring attackers

In 2025, the global average cost of data breaches declined to $4.44 million—the first reduction in five years—primarily due to accelerated detection enabled by artificial intelligence and automation, according to IBM and the Ponemon Institute’s annual analysis. Concurrently, Verizon reported that AI-generated text in phishing emails had doubled over a two-year period. These contrasting statistics encapsulate AI’s dual effect of lowering operational expenses for both cyber defenders and attackers. However, it is critical to acknowledge that these figures represent only officially reported incidents. The true incidence of breaches is likely substantially higher, as many organizations choose not to disclose events due to reputational risks, some lack the capability to detect such incursions, and numerous individuals remain unaware or dismissive of suspicious activities. This underlines the systemic challenge for public governance in managing cybersecurity risks that extend beyond isolated corporate boundaries.

Cybersecurity as a Complex Public Governance Challenge

The increasing dependence on digital services, spanning online banking, e-commerce, cloud business systems, and Industry 4.0 innovations, has markedly expanded exposure to cyber risks. Central to this transformation are high-tech manufacturing and semiconductor industries whose operations now rely heavily on connected robotic systems, IoT sensors, AI-driven automation,and integrated supply chain software. While these digital linkages enhance efficiency, they also introduce vulnerabilities that transcend individual organizations. For example, a ransomware attack on the supplier MKS Instruments in 2023 disrupted Applied Materials’ operations, leading to an estimated $250 million loss in quarterly sales. This interdependence underscores how underinvestment in cybersecurity in one firm can create cascading losses across interconnected industrial networks, financial systems, and public services.

In this interconnected environment, artificial intelligence emerges as both a powerful tool for cyber defense and a catalyst for novel threats. The pivotal policy question is not whether AI is inherently beneficial or harmful in cybersecurity, but rather how its dual-use nature impacts the balance of power between defenders and attackers. Because AI lowers operational costs and scales capabilities for all parties, the decisive factor lies in the governance frameworks that guide its deployment, alongside the role of human oversight. Current evidence underscores that AI is neither an unequivocal solution nor a solely detrimental force; instead, its ultimate influence hinges on institutional controls, regulatory approaches, and informed human intervention.

Artificial intelligence is transforming cybersecurity beyond mere tool enhancement, supplanting traditional reliance on fixed malware signatures, static rule-sets, blacklists, and threshold limits that functioned when threats evolved incrementally and attackers faced technical constraints. This prior environment is rapidly vanishing as cyber threats accelerate, diversify, and traverse complex interconnected digital infrastructures, rendering human manual detection insufficient. Consequently, AI now serves as the indispensable central nervous system of contemporary cybersecurity operations, continuously processing vast data to identify anomalies and emergent threats. Nevertheless, AI cannot be regarded as a panacea for cybersecurity challenges,since adversaries also harness AI to refine the precision, ingenuity, and scalability of their attacks. This accelerating arms race thus concurrently advances both defensive and offensive capabilities, underscoring the need for vigilant governance and comprehensive institutional oversight.

Enhancing Defensive Capabilities Through AI

A persistent challenge in cybersecurity is the disparity between the speed of cyberattacks and the human capacity to detect them. Attackers may compromise systems in mere minutes, while organizations often require hours or even days to identify such intrusions. Artificial intelligence addresses this gap by continuously scrutinizing network traffic, scrutinizing user activities, and analyzing system logs to establish baseline patterns of normal behavior. When deviations occur—such as an employee account initiating an unusually large data transfer during off-hours, or an industrial machine executing unexpected commands—AI-powered tools flag these anomalies instantly. The efficacy of this rapid detection was quantified by IBM in 2025, revealing that organizations extensively employing AI and automation in security experienced average breach costs that were $1.9 million lower and resolved incidents 80 days faster. This capability is especially critical in manufacturing sectors, where operational interruptions carry substantial financial consequences. Factories rely on interconnected industrial control systems, including robotic arms, automated quality inspection apparatus, and intricate assembly lines; breaches in these systems risk halting production or inflicting physical harm. AI’s ability to monitor these complex environments at scales beyond human capacity enhances resilience and operational continuity.

AI also enables a shift towards more anticipatory cybersecurity governance by analyzing extensive historical attack data, long-term vulnerability patterns, and global threat intelligence to predict which weaknesses are likely targets. The decreasing costs of such analyses make this approach increasingly accessible. For example, the DARPA AI Cyber Challenge in August 2025 demonstrated that autonomous AI systems could efficiently scan 54 million lines of code, successfully identifying and remediating most intentional vulnerabilities while uncovering previously unknown flaws—all at relatively low cost per task. Furthermore, in July 2025, Google reported that its AI agent, Big Sleep, intercepted exploitation of a critical SQLite database vulnerability before any public disclosure, marking a notable precedent of proactive AI defense in real-world settings. This preventive capability is especially crucial for smaller enterprises without dedicated security teams, which often depend on managed service providers to integrate predictive AI monitoring within a broader institutional framework that balances innovation with responsibility.

Artificial intelligence is fundamental to the cybersecurity frameworks that protect modern organizations. Financial institutions employ AI-driven analytics to scrutinize transaction patterns in real time, enabling the immediate detection and blocking of fraudulent attempts, such as a suspicious nominal charge used to test stolen credentials before larger thefts occur. Telecommunications providers utilize AI to monitor network traffic continuously, identifying and mitigating denial-of-service attacks designed to overwhelm service availability. Similarly, e-commerce platforms rely on AI algorithms to detect fraudulent listings, automated fake reviews, and unusual seller behaviors, while cloud service providers incorporate AI-based threat detection within their offerings, extending sophisticated defenses to smaller enterprises that otherwise lack access to such capabilities. Notably, defenders maintain an institutional advantage rooted in intimate knowledge of their own digital environments; AI models trained on proprietary data can identify subtle anomalies imperceptible to external actors. This localized intelligence underpins more effective defense strategies. Overall, AI-enabled cybersecurity has become indispensable, particularly for small and medium-sized organizations that would face significant challenges in safeguarding their systems without access to these advanced, embedded technologies.

The Escalation of Sophisticated AI-Enabled Attacks

The dual-use nature of AI in cybersecurity significantly empowers attackers as well as defenders. Cybercriminals increasingly deploy AI to craft highly personalized phishing campaigns written in fluent, natural language that no longer exhibits the telltale errors of traditional scams. By mining data from social media and other digital sources, attackers glean detailed insights into a target’s professional role, hobbies, and social connections, enabling the generation of tailored messages that greatly increase the likelihood of deception. According to Verizon's 2025 Data Breach Investigations Report, approximately 60 percent of intrusions analyzed involved a human factor, reflecting the persistent vulnerability of human judgment. Compounding these risks, AI-driven voice cloning technology enables scammers to imitate the voices of family members using only brief publicly available audio samples, resulting in fraudulent calls that convincingly solicit urgent financial assistance. These sophisticated attacks circumvent many of the emotional signals individuals traditionally rely on to verify caller identity, thereby posing profound challenges for identity authentication frameworks and underscoring the need for regulatory standards that address AI-enabled impersonation while balancing technological innovation and privacy considerations.

The emergence of AI-generated synthetic videos has complicated the task of verifying identities and preventing fraud, as illustrated by a costly case in Hong Kong. In January 2024, an employee at Arup’s Hong Kong office took part in a video meeting in which the chief financial officer and other participants were entirely fabricated digital avatars created from publicly available audio and visual materials. Exploiting these highly convincing digital replicas, fraudsters authorized 15 financial transfers amounting to approximately HK$200 million (US$25.6 million) across multiple accounts. Crucially, Arup confirmed that none of its internal IT systems had been breached, and traditional intrusion detection systems did not register any alerts. This incident underlines how AI has lowered the technical barriers to sophisticated deception, posing significant institutional challenges for cybersecurity governance aimed at detecting and managing AI-facilitated identity fraud without compromising operational trust or privacy.

AI also dramatically expands the scale and speed of vulnerability detection, automating reconnaissance and scanning activities that previously required extensive human effort. By autonomously probing thousands of websites and connected devices for weaknesses, AI systems can carry out comprehensive surveillance of potential targets with unprecedented efficiency, shifting cybersecurity from reactive to proactive postures. For instance, in November 2025, Anthropic reported that a Chinese state-affiliated group, designated GTG-1002, exploited the Claude Code AI platform as an autonomous agent to conduct cyber espionage across roughly 30 organizations spanning technology, finance, chemicals, and government sectors. This AI agent managed 80 to 90 percent of tasks such as target reconnaissance, vulnerability identification, and data extraction at speeds unattainable by human teams, although only a limited number of intrusions succeeded. Furthermore, researchers document malware that mutates its code upon replication, evading conventional signature-based detection. Consequently, operations once demanding large expert teams may now be directed by a handful of supervisors, fundamentally altering attack economics and complexity in ways that challenge existing cybersecurity governance frameworks.

AI-facilitated cyberattacks pose acute risks to critical industrial sectors where digitalization has integrated historically isolated control systems into expansive networks. The increased connectivity, combined with AI’s capacity to rapidly identify and exploit vulnerabilities, incentivizes targeting less-protected suppliers or network nodes as entry points. The 2025 cyberattack on Jaguar Land Rover exemplifies this challenge: its British production facilities were incapacitated for approximately five weeks, resulting in a £1.9 billion estimated economic loss predominantly from disrupted manufacturing and supply chain interruptions impacting over 5,000 organizations. Although AI involvement in this specific incident was unconfirmed, it illustrates the systemic vulnerabilities and accumulating damages possible when a single breach occurs within an interconnected industrial ecosystem, highlighting the critical governance imperative to enhance collective resilience without imposing disproportionate burdens on smaller network participants.

Balancing AI Reliance: New Vulnerabilities and Oversights

Despite its considerable strengths, AI remains susceptible to intrinsic limitations rooted in its dependence on historical training data and statistical pattern recognition. Cyber adversaries, constantly innovating with new tactics, techniques, or sequences of attack events that diverge from previously observed behaviors, can exploit these blind spots, enabling attacks that evade AI detection. This constraint highlights a fundamental boundary of AI: it cannot anticipate or identify threats beyond the scope of its learned experience. Conversely, AI systems may generate excessive false positives, misclassifying legitimate behaviors as threats, unnecessarily isolating systems, or triggering alerts that flood security operations centers. In these high-pressure environments, persistent false alarms contribute to analyst fatigue, reducing vigilance and obscuring genuine threats within the noise. Furthermore, generative AI models introduce a distinct challenge, occasionally producing exaggerated or fabricated outputs. Notably, during the GTG-1002 campaign, an AI system was documented to overstate findings and occasionally fabricate data. Additionally, the maintainers of curl—a widely deployed internet data transfer tool—suspended their bug bounty program in early 2026 after an influx of AI-generated vulnerability reports, many of which were unfounded. These factors underscore the imperative for rigorous human oversight, validation mechanisms, and institutional accountability frameworks to mitigate risks from AI-driven errors in cybersecurity operations.

Excessive dependence on AI-driven automated detection systems can induce a false sense of security among cybersecurity teams, leading to diminished vigilance toward subtle or novel threat indicators. This complacency may cause personnel to overlook nuanced anomalies, misclassify suspicious activities as benign, or defer critical evaluative decisions entirely to algorithmic outputs without sufficient human scrutiny. Such overreliance neglects the complexities of cyber threats that often involve human-centric cues—like voice inflections, atypical phrasing, cultural context, or behavioral subtleties—areas where AI currently lacks nuanced understanding. Furthermore, institutional governance deficiencies aggravate these risks. For instance, IBM’s research highlights that 97% of organizations experiencing AI-related security breaches lacked robust access control mechanisms, and only 37% had formal policies overseeing AI usage or detecting unauthorized 'shadow' deployments. Entities with extensive unmanaged AI use faced breach-associated costs averaging $670,000 higher. These findings underscore that human expertise remains indispensable; cybersecurity professionals contribute critical intuition, contextual awareness, and holistic interpretation beyond AI’s present scope. Effective governance frameworks accordingly emphasize AI as a complementary augmentation, with human analysts actively interpreting ambiguous or unprecedented situations, posing thoughtful inquiries, and synthesizing diverse signals to uphold organizational security and accountability.

Privacy Concerns from Continuous AI Surveillance

The integration of AI-driven monitoring in cybersecurity presents significant privacy and governance challenges. Such systems require extensive access to detailed user behavior data—including login locations, device identifiers, communication metadata, typing patterns, browsing histories, and subtle behavioral cues—to effectively detect threats promptly and accurately. While this comprehensive surveillance enhances security responsiveness, it simultaneously raises critical questions about the proportionality and legitimacy of monitoring in workplaces and digital platforms. Moreover, the data collected under these regimes becomes an attractive target for attackers; IBM's research indicates that incidents involving unregulated or "shadow" AI monitoring revealed personal information in 65% of breaches, surpassing the 53% average across all breaches. This dynamic underscores the need for policies that carefully balance effective cybersecurity with strict privacy protections, clear data governance, and robust safeguards against misuse or overreach.

The intersection of AI-powered cybersecurity monitoring with existing data protection frameworks presents significant regulatory and operational challenges. Most data privacy laws, including the EU's General Data Protection Regulation (GDPR), were established prior to the widespread deployment of pervasive AI surveillance technologies, resulting in interpretive ambiguities for organizations employing these tools. Key issues include defining thresholds for what constitutes excessive behavioral data collection, determining the extent and circumstances under which users must be notified about surveillance expansions, and implementing safeguards to prevent repurposing security-collected data for unauthorized uses—such as employee performance assessments without informed consent. Addressing these questions is critical to ensure that AI-enabled security monitoring respects individual privacy rights while maintaining effectiveness and transparency in institutional oversight.

Cross-border data flows intensify governance complexity, especially as cloud services increasingly store security-related data internationally. Organizations must strike a balance between comprehensive, uninterrupted oversight and transparent governance frameworks that ensure access controls, clearly defined data retention periods, and purpose-specific processing. Absent such safeguards, AI cybersecurity tools risk evolving into instruments of excessive surveillance, undermining the trust essential to secure digital systems. Additionally, overly aggressive monitoring may disrupt operations and damage user relationships, offsetting security gains.

Integrating Human Judgment and AI in Security Governance

Careful analysis reveals that the reality of AI in cybersecurity defies simplistic judgments. AI alone cannot guarantee a secure digital environment, nor does it exclusively empower malicious actors. In automated verification tasks—such as code vulnerability assessments or detecting anomalous system behaviors—defenders retain an inherent advantage, benefiting from AI’s capacity to process large data sets and validate patches or deviations efficiently. Conversely, in domains requiring human trust—like authenticating voice communications or video identities—the burden of verification shifts to recipients confronted with convincing forgeries frequently generated at low cost by attackers. This asymmetry means the balance of security depends fundamentally on where verification costs are concentrated and how organizations allocate these costs through technology deployment and governance practices.

Effective governance of AI-enabled cybersecurity requires embedding these technologies within comprehensive institutional frameworks that emphasize not only advanced tools but also sustained human oversight and organizational accountability. This entails establishing clear governance protocols, continual system monitoring, stringent access control policies, ongoing professional training, and frequent auditing to ensure operational integrity. Skilled cybersecurity professionals remain essential for interpreting AI-generated alerts, conducting thorough investigations, and making critical decisions where automated systems may fall short. The UK's National Cyber Strategy 2022 illustrates this approach by prioritizing investments in workforce capabilities as a foundational element of national resilience. Practical governance measures include mandatory independent verification of significant or sensitive transactions through pre-registered communication channels, obligatory secondary approvals to prevent unauthorized video conference authorizations, and targeted user education programs — evidenced by Verizon’s findings that trained individuals report phishing attempts at over four times the baseline rate. Furthermore, for large-scale manufacturers, securing supply chains has become integral to maintaining production continuity; thus, extending AI-powered defensive measures via shared monitoring facilities or enforceable security requirements in supplier contracts can effectively elevate protections for smaller enterprises that lack the capacity to deploy these technologies independently.

At the individual level, enhancing digital literacy is vital to counter evolving AI-enabled cyber threats. Traditional indicators of fraud—such as poor grammar, unfamiliar phone numbers, or suspicious email addresses—are increasingly inadequate. Users must adopt rigorous verification practices, including confirming information across multiple communication channels, exercising caution when receiving urgent monetary requests via phone, regularly updating passwords, and maintaining a healthy skepticism toward even highly realistic AI-generated messages or videos. Cultivating such cybersecurity awareness is poised to become as indispensable as financial literacy, equipping individuals to navigate the complexities introduced by AI-assisted deception and contributing to broader societal resilience in digital environments.

Artificial intelligence undeniably enhances cybersecurity by enabling earlier detection of threats, accelerating incident response, anticipating emerging risks, and scaling protection efforts across complex digital infrastructure. However, this same technological power facilitates more persuasive scams, perpetually evolving malware, and large-scale vulnerability discovery that benefits malicious actors as well. The initial metrics presented—IBM's $1.9 million average savings for organizations integrating AI within robust security and governance frameworks contrasted with Verizon's reported doubling of AI-generated phishing texts predominantly affecting unsupervised individuals—highlight the critical divergence in outcomes based on institutional choices. Ultimately, the role AI will play in securing or undermining digital trust hinges less on its intrinsic capabilities and more on how governments, organizations, and society govern its deployment with balanced safeguards ensuring protection, privacy, and accountability.

The views expressed in this article are those of the author(s) and do not necessarily reflect the official position of Gordon Institute or its affiliates.

References

Anthropic (2025) Disrupting the First Reported AI-Orchestrated Cyber Espionage Campaign. San Francisco: Anthropic.
ARPA-H (2025) 'Challenge showcases AI's power to secure America's health care', Advanced Research Projects Agency for Health, August.
Bloomberg (2023) 'Applied Materials' sales shortfall linked to cyberattack at MKS Instruments', The Edge Singapore, February.
European Parliament and Council of the European Union (2016) Regulation (EU) 2016/679 (General Data Protection Regulation). Brussels: Official Journal of the European Union.
HM Government (2022) National Cyber Strategy 2022. London: Cabinet Office.
IBM and Ponemon Institute (2025) Cost of a Data Breach Report 2025. Armonk, NY: IBM.
South China Morning Post (2024) 'Arup confirmed as victim of HK$200 million deepfake scam', South China Morning Post, 17 May.
Stenberg, Daniel (2026) 'The end of the curl bug bounty', daniel.haxx.se, 26 January.
TechInformed (2025) 'Jaguar Land Rover cyberattack to cost estimated £1.9bn', TechInformed, October.
Verizon Business (2024) 2024 Data Breach Investigations Report. Basking Ridge, NJ: Verizon.
Verizon Business (2025) 2025 Data Breach Investigations Report. Basking Ridge, NJ: Verizon.
Walker, Kent (2025) 'A summer of security: empowering cyber defenders with AI', The Keyword, 15 July.

Adapted from [Howie Chang Series] Is AI in Cybersecurity a Blessing or a Curse?.