# Security in the age of AI

> AI agents are finding old vulnerabilities and making complex attacks easier. A short history of automated hacking, what comes next, and how to prepare.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-10-21 | Topics: [AI](https://flaviocopes.com/tags/ai/) | Canonical: https://flaviocopes.com/security-ai/

AI agents are going to find a lot of ways into software we thought was reasonably safe.

The [critical PHP vulnerabilities disclosed in October 2026](https://flaviocopes.com/php-security-update/) are a warning. Researchers used AI to help turn old problems in a language runtime into working attacks against real applications.

If you run PHP, patch it. That post covers the immediate work.

Here I want to talk about what comes after that. Because PHP is one example of something much bigger, and we've had enough examples already to stop treating each one as a surprise.

I expect agents to make serious security research available to far more people, including people who want to break into your app. Systems that nobody had the time or budget to examine closely will get examined.

That includes the little app you shipped and haven't touched in six months.

## We've been automating attacks for a long time

The internet was dangerous long before anyone connected a language model to a terminal.

The [Morris worm](https://www.fbi.gov/history/cases-and-criminals/morris-worm) spread across networked computers in 1988. Software could already move between machines and exploit weaknesses without a person typing each command.

We've since had scanners, password attacks, exploit kits, and tools that throw enormous numbers of unusual inputs at software to see what breaks. That last technique is called **fuzzing**, and it has helped find bugs for years.

We've also had repeated reminders that a problem in one shared component can affect applications everywhere. [Log4Shell in 2021](https://www.cisa.gov/news-events/cybersecurity-advisories/aa21-356a) was a remote code execution vulnerability in a Java logging library. Organizations had to work out where that library existed inside their systems while attackers were already exploiting it.

So the idea that attacks can be automated, or that one dependency can put a huge number of sites at risk, is familiar.

The history of machines *finding* the weaknesses is worth looking at too.

### 2016: machines competing to find and fix bugs

DARPA's [Cyber Grand Challenge](https://www.darpa.mil/news/2016/mayhem-winner-cyber-grand-challenge) put automated systems into a competition to find vulnerabilities, exploit them, and patch software. Mayhem won.

This happened years before the current coding agents. The competition used a controlled environment, so winning it didn't mean a machine could break into any company. But the research goal was already clear: automate work that had required security specialists.

### 2024: an agent finds a real SQLite bug

Google's Project Zero and DeepMind [reported a vulnerability found by Big Sleep](https://projectzero.google/2024/10/from-naptime-to-big-sleep.html), their AI agent, in SQLite.

It was a memory safety bug in a widely used database engine. Google said it was found before reaching an official release, so SQLite users weren't affected.

That's a useful example of this going well. An agent found a real problem, and the developers fixed it before users received the vulnerable code.

### 2025: more real bugs, plus reports of malicious use

At the finals of DARPA's [AI Cyber Challenge](https://www.darpa.mil/news/2025/aixcc-results), the competing systems found 54 planted vulnerabilities across 63 challenges. They also discovered 18 real vulnerabilities that the organizers hadn't planted.

These were specialized systems with substantial engineering behind them. Still, the results went beyond solving a prepared exercise.

Later that year, Anthropic [reported disrupting an AI-assisted espionage campaign](https://www.anthropic.com/news/disrupting-AI-espionage). According to its account, attackers used Claude Code to carry out substantial parts of the work. Humans chose targets and remained involved.

That's the provider's account of what it observed, rather than an independent measurement of all AI attacks. It gives us a concrete example of malicious use alongside the defensive research.

### 2026: Firefox, then PHP

In March, Anthropic [reported finding 22 security vulnerabilities in Firefox](https://www.anthropic.com/news/mozilla-firefox-security), including 14 rated high severity.

In April, [Mozilla reported fixes for 271 vulnerabilities](https://blog.mozilla.org/en/firefox/ai-security-zero-day-vulnerabilities/) identified during its initial evaluation of Claude Mythos Preview. Mozilla later [explained the engineering work behind those results](https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/).

Those numbers aren't counts of successful attacks against users. They describe vulnerabilities identified and fixed in a defensive collaboration. They do show that heavily tested software can still contain a lot for these tools to find.

Then came PHP2Shell.

The [PHP security release](https://www.php.net/archive/2026.php#2026-10-21-5) fixed two critical vulnerabilities that can allow an attacker to execute code without logging in. Some demonstrated paths run before your application code, which means your login screen cannot protect that part of the runtime.

In [Wiz's account of the research](https://www.wiz.io/blog/php2shell-php-rce), AI helped with discovery and with adapting working exploits across different applications and memory layouts. Wiz also credits Calif, OpenAI, and the PHP team for their work on discovery, coordination, and patch verification.

This was a coordinated research effort. The part I find worrying is how much of the difficult adaptation work can now be handed to an agent.

## An agent can keep working when the first attempt fails

Think about how you use a coding agent.

You give it a problem. It reads the code, runs something, gets an error, and changes its approach. Sometimes you have to intervene. Sometimes it works through a long sequence of failures on its own.

That same loop is useful for vulnerability research.

An agent with the appropriate tools can examine a code path, form a hypothesis, and test it in a lab. If the result doesn't match its expectation, it can inspect what happened and try again.

```mermaid
flowchart TD
  accTitle: How an agent iterates on a suspected vulnerability
  accDescr: In an authorized test environment, an agent reads code, proposes a weakness, and tests it. Failed attempts provide feedback for another hypothesis. A reproducible result goes to human validation and remediation.
  A[Read code in a test environment] --> B[Propose a weakness]
  B --> C[Test the hypothesis]
  C --> D{Reproducible result?}
  D -->|No| E[Inspect failure and revise]
  E --> B
  D -->|Yes| F[Human validation]
  F --> G[Patch and regression test]
```

Running a scanner against a list of known vulnerabilities is useful. An agent can also investigate why something unusual happened and decide what to examine next.

It will make mistakes. It can spend hours chasing something that doesn't exist. But the attacker gets to decide how much time and money to spend on those attempts, and can run several investigations in parallel.

The cost of that work matters even when the success rate is far below perfect.

## Being small will protect you less

If a custom attack requires a specialist to spend weeks on your app, your app needs to be worth that effort.

If parts of the investigation become cheaper, more targets become worth trying.

That's my expectation for what comes next. Mass scanning will continue, with more automated investigation behind the interesting results. A strange response from your forgotten admin tool could become the beginning of a longer attempt.

An attacker doesn't need to know who you are. Your server might have credentials for another service. It might hold customer data, send email, or run workloads someone else would rather you paid for.

Even an unsuccessful intrusion attempt can cost you money if it consumes paid API calls or expensive background jobs.

I would stop using "nobody cares about this app" as part of a security plan.

## Old software will get a lot of new attention

We have decades of deployed software. Much of it was written when today's tools didn't exist.

Some of that code has had excellent security review. Some has barely had maintenance. An old internal service might become reachable through a new proxy, or gain access to data it was never designed to protect.

I expect agents to uncover more problems in this accumulated code. They'll also help people turn published vulnerability details into attacks against installations that haven't been patched.

We already had urgent patching before AI. My concern is that more of the work between reading an advisory and producing a usable attack can be automated. Depending on the vulnerability, defenders may have less time to react.

This doesn't establish a date when everything gets hacked. A bug, a working exploit in a lab, and a successful compromise of your deployment are different things. Permissions and deployment details still affect what an attacker can do.

It does make long delays harder to justify when an exposed service has a serious fix available.

## Our own agents create another way in

There's also the agent you invited into the system yourself.

We give coding agents repository access, shell commands, and credentials. Some can deploy applications or change cloud resources.

An agent may then read a web page, an issue, or a file containing instructions written by someone else. If it follows those instructions as though they came from you, that's **prompt injection**.

The attacker is trying to redirect software that already has permission to act.

I would treat an agent with production access as another privileged application. Give it a separate identity where possible, restrict its permissions, and keep credentials short-lived. Enforce limits outside the conversation, through the tools and infrastructure it uses.

Telling it "never reveal secrets" doesn't remove its ability to send a secret somewhere. Keeping a password out of the model's context helps, but an agent allowed to use that password can still perform harmful actions through the authenticated session.

I've written more about [how agents can log in without seeing passwords](https://flaviocopes.com/ai-agent-passwords/). The permissions after login deserve just as much attention.

## How I would prepare

For a small app, I would start by finding out exactly what is running.

List the domains and services you expose. Include abandoned previews, old containers, and the server you only use for one background task. Record who patches each runtime and how you would take it offline.

Then remove the things you don't need. A deleted test deployment gives an attacker fewer opportunities and gives you one less system to maintain.

The PHP release is a good rehearsal. Could you identify every PHP runtime you operate? Could you tell whether your hosting provider had patched it? Would you remember the copy bundled inside a container?

### Make updates routine enough to do urgently

If deploying a patch is frightening, you'll postpone it.

Keep the build and deployment process working. Test the important user flows, and have a rollback procedure. For a security update, check that rolling back won't put the vulnerable version back online.

Use supported runtimes and follow your providers' security notices. Prioritize exposed vulnerabilities according to their impact and evidence of exploitation. You need a way to interrupt normal work for a critical update.

Managed services can reduce the maintenance you own. Check where that responsibility ends. A managed database doesn't update the runtime inside your application container.

### Limit what a compromised app can reach

Assume one application process gets compromised. What can it read or change?

A database account for one app should have access to the data that app needs. A deployment token should have the smallest useful scope. Backups should be protected from deletion by the same credentials used by the running application.

Keep databases and administrative interfaces off the public internet when public access isn't necessary. Separate development credentials from production credentials.

These decisions reduce the damage when a vulnerability gets through your other defenses. They still matter if the attacker is much better at finding bugs than you are.

### Test the rules your application depends on

Updating dependencies won't catch every mistake in your own code.

Can one customer read another customer's invoice? Can someone call an admin endpoint directly? Can repeating a request issue a second refund?

Write tests for these rules and enforce them on the server. An attacker doesn't have to use your interface, and an agent can help investigate the routes behind it.

My free [Security Fundamentals course](https://flaviocopes.com/courses/security-fundamentals/) covers how to identify these boundaries and test them.

### Use an agent to help defend what you own

Give an agent a limited, authorized task. Ask it to review an authentication flow, trace where secrets go, or propose tests for access control failures.

Require evidence for its findings. It should identify the affected path and explain how to reproduce the problem safely. Review the patch and keep a regression test for the behavior it fixes.

I [used AI to help secure a server](https://flaviocopes.com/securing-a-server-with-ai/) after moving it. That was useful work. An agent can help you get maintenance done that you would otherwise keep postponing.

Start with a disposable environment or a read-only review. Give it more access when the task requires it and you understand what it can change. A generated report saying the system is secure doesn't establish that it is.

### Know what you would do after a break-in

Keep useful logs outside the application server. Decide which events should alert you, such as unexpected administrator creation or changes to deployment credentials.

Test restoring a backup into a separate environment. Check that the application works with the restored data. Keep recovery credentials somewhere you can reach if your main account is locked out.

Have a short response procedure that covers isolation, preserving evidence, revoking access, and rebuilding from trusted inputs. If someone has already taken over a server, installing the security update doesn't remove what they left behind.

The [Linux Server Security course](https://flaviocopes.com/courses/linux-server-security/) goes through this operational side in more detail.

## There is useful work we can do before the next disclosure

The SQLite and Firefox examples are encouraging because developers used these tools to find and fix problems. More capable agents can help defenders examine code that has waited years for attention.

But finding a bug creates work for someone. Maintainers have to validate it, develop a compatible fix, and get that fix to users. Funding the projects we depend on matters here too. Faster discovery won't help much if the people maintaining the software can't keep up.

For the apps we run ourselves, we can improve that last part now.

Pick one deployment. Find out what's exposed, who can change it, and how it gets patched. Restore its backup. Remove the old admin tool you've been meaning to delete.

When the next serious advisory arrives, you should already know where to start.