Finding vulnerabilities has become the cheap part of security testing, and most of our industry still charges for it as though it is the expensive part.
On 5 November, OpenAI published the results of an experiment that I expect to be quoted for years. In a single run in mid-October, a group of AI agents working through the Linux kernel produced 2,308 candidate flaws, reproduced 1,724 of them and wrote 1,169 working exploits, which trace back to 948 distinct vulnerabilities. Each one lets somebody who already holds an ordinary account on a machine take full control of it. The computing bill was US$1,213,430, or roughly US$1,280 for each vulnerability. Most of the commentary stopped at that figure. The more instructive one sat in the same report and received far less attention: after ten days of reporting, 61 of the 948 had been fixed in the kernel itself, 43 more had fixes waiting in a staging queue, and 299 were still being reported and discussed.
Picture an emergency department that takes delivery of a scanner that can image every person who walks through the door in seconds, for almost nothing. By the end of the first morning, the waiting room is full of people holding accurate imaging results and the diagnosis. The department has the same surgeons it had yesterday, the same operating rooms and the same twenty-four hours. Nobody in that waiting room is healthier yet. The scanner has not shortened the surgeon’s day; it has lengthened the line outside the operating room, and the person whose judgment now matters most is the nurse at the triage desk, who decides who is seen first.
I want to be careful and honest about what those figures prove. The kernel is maintained by a community with its own review culture, much of it volunteer, and ten days is a snapshot of one project. A 400-person insurer with a managed patch cycle is a different patient altogether; it does not develop a popular OS kernel and will never receive 948 high-risk findings from one run. These particular flaws also require the attacker to be inside already. And cheap discovery is good news for defenders, because a flaw found by a researcher is a flaw no longer waiting for a criminal. I was among those who doubted, some years ago, that AI would get this far in security testing, and I am glad to have been wrong.
What travels from the kernel to the insurer is the ratio. The cost of producing a confirmed finding is falling quickly, while the hours that an IT team of six has each week to test patches, change configurations and negotiate maintenance windows have not changed. The imaging became faster while no new surgeons were brought in.
A second signal arrived a week earlier from a different direction. On 29 October, Google added a US$250,000 reward to its cloud bug bounty program, reserved for a researcher who can demonstrate complete control of a customer’s cloud environment starting from nothing: no prior access, no stolen credentials, no help from a careless user. The program’s lead explained why. The number of valid reports, he wrote, “has remained stable over the last 18 months” while “the total number of reports has grown tremendously”, and automated scanners “generate a high volume of low-impact noise”. A company that buys vulnerability findings at very large scale looked at its own waiting room and decided to pay most for the rare case that needs a surgeon.
Two points make a line, and a line is not yet a trend. What persuades me the line is real is that I hear the same thing from the other side of the table. In most conversations I have had with security leaders this autumn, nobody asked how many findings we could produce, because they already have tens of thousands. They asked which handful sit on a path an attacker could walk, and what to fix on Monday morning.
This is where I believe our profession is doing something wrong. We still write proposals that promise coverage and deliver reports judged by their weight. We score weaknesses against industry averages rather than against the damage each could do to this particular business, and in doing so we push the problem down the execution chain to an administrator who will lose a week chasing something that never mattered. A report of four hundred findings handed to a team that can fix forty is a full waiting room with nobody at the triage desk.
The expensive part of testing, and the part worth paying for, is triage with proof. That means showing that a weakness is on a direct attack path to your crown jewels and not merely that it can be exploited or appears in a CVE database, ranking it by what it would cost your business if someone used it, explaining how to fix it in your systems, and returning to confirm that the fix worked. Machines do a great deal of this work and will do more. At Parabellyx, an offensive security firm, automation carries between 60 and 90 percent of the testing work depending on the type of test. An automation rate is not an accuracy rate, however, and in our measurements, expert validation that removes about 90 percent of the noise before a customer sees it. OpenAI’s own researchers reviewed every exploit and every patch before sending it to the kernel’s maintainers, and their time is not in the US$1,280. Even in that experiment, the scarce resource was a person deciding what deserved attention.
A penetration test is just a piece of paper unless it is acted on. Here is something you can do this week without buying anything. Take the most recent test report or scan export your organization received and count three things: how many findings it contained, how many were fixed, and how many of those fixes were retested. If the first number is large and the other two are small, a faster scanner is not what you are missing. Before your next test begins, ask the people doing it how they will decide what to fix first and how they will prove it. If the answer is a severity score from a public database, you are being offered another scanner when what you need is someone at the triage desk.
None of this is cause for alarm. Finding has become cheap, which puts the knowledge of where you are weak within reach of organizations that could never afford it before, and what remains is the work of choosing well. And that is why we are here for you.