The alerts had been red for so long that nobody opened them any more. That is how Priya described it when she inherited her team’s security tooling — a DevSecOps scanner bolted onto the pipeline by a former architect, configured to block on every finding rated medium or above, with a backlog past eight hundred items and a skip button everyone had learned by muscle memory. The tool was expensive. The team’s respect for it was zero. This is a diagnosis story, and like most diagnoses, the treatment only worked once the actual disease was named.
DevSecOps symptom one: the bypass became culture
Watch what happens when a gate blocks everything. Engineers do not become more secure; they become better at routing around the obstacle. On Priya’s team the skip button had a keyboard shortcut. Reviews merged with known red builds because a red build meant nothing — the signal was permanently saturated. This is the classic failure the field keeps rediscovering: gating on severity alone trains bypass. The scanner was not wrong often, which sounds like a virtue, but it was wrong often enough that people stopped asking whether any given alert deserved attention.
Symptom two: ancient findings held new work hostage
Half the backlog predated the current codebase. Merge a one-line fix and the gate would surface four-year-old issues in files nobody had touched since — and refuse the fix until they were resolved. Nothing about that arrangement protects software. Old debt and new changes are different clinical pictures, and treating them identically guarantees both get handled badly. The baseline had never been separated from the flow of incoming work, so the pipeline treated history as if it were happening today, forever.
The diagnosis, stated plainly
The disease was never “we lack a scanner”. The team had one, and a costly one. The disease was that the gate’s rules bore no relationship to trust. DevSecOps practice reframes gating as a credibility contract, and the contract has exactly three clauses:
- The finding must be new — introduced by this change, not inherited from history.
- The finding must be certain — the tool’s confidence in it is high enough to act on without a human doubting.
- The finding must be fixable now — the repair is small enough to land with the change itself.
Everything else converts into recorded work: an owner, a note, an expiry date. Nothing on the old baseline ever holds a release. Priya’s predecessor had bought a tool without ever writing the policy that should have governed it.
Treatment begins with measurement, not deletion
Before touching rules, the team spent two weeks in report-only mode, counting instead of blocking. The method descends from Google’s Tricorder programme, and its numbers are refreshingly concrete. A rule earns enforcement only when false alarms approach zero in practice. Machine-written review comments should stay below one in ten. Results should attach to the lines a change actually touched, not to the repository at large. And a thirty-finding sample tells you quickly whether a rule deserves anyone’s attention. Rules that flunk their sample get demoted or retired. In one documented case, a single rule showed a 43.6 percent false-alarm rate and was demoted accordingly — demotion rather than deletion, because a noisy rule still teaches you something about the codebase.
The numbers that told Priya the truth
The Atlas-style worked example reads like her team’s diary. Across 1,240 pull requests, a gate blocked 96 and was overridden 22 times — a 22.9 percent override rate, which is not a discipline problem but a design confession: engineers trusted the gate so little that nearly one block in four was manually cancelled. Above roughly five percent per month, the override figure means the team is telling you the rules are wrong. Below it, the gate is doing real work. On the outcome side, the same example tracked 412 deploys with 398 carrying verified evidence, and nine escaped defects of which three would have been catchable upstream. That gap between nine and three is where placement decisions get argued.
Why grading had to come first
Notice the pattern underneath the fix: findings needed classification before they could be trusted, on two axes at once — how sure the tool is, and how much the flaw matters. Medicine does this instinctively. A Class IV heart-failure grading changes the conversation immediately because the classification encodes severity and urgency together; likewise murmur grading scales exist so a description becomes a decision. A scanner output that says only “medium” has failed to classify anything. Confidence, novelty, and fixability are the axes that turn an alert into an action.
The humans DevSecOps quietly depends on
Rules and numbers carry the month-to-month work, but two human arrangements keep the system alive. The first is the exception review: every override and every expiry gets a named person and a date, so debt ages visibly instead of silently. The second is the champions model — engineers given roughly ten to twenty percent protected time to carry security context inside their own teams. Research keeps finding that culture, not tooling, predicts whether any of this sticks, and teams that treat security as one department’s job tend to rebuild Priya’s situation within a year.
Six months later
The backlog did not vanish, and that was never the goal. What changed: the gate stops only what its three clauses cover, every bypass request turns into a tracked exception — someone’s name on it, ninety days before it lapses, and the monthly review reads three numbers aloud — override rate, median fix time for new findings, and escapes. The skip button lost its shortcut. For the full framework behind this approach, control placement, metrics, and the supporting evidence, the Atlas write-up on DevSecOps is the best starting point the team found. Priya’s lesson is not about tools at all: a control nobody trusts is worse than none, because it spends credibility you will need later.
