Speaker
Description
AI patch review is already here. Sashiko, the reviewer bot now running on many of the kernel's mailing lists, finds real bugs. On a hard subsystem it also invents them: an architectural detail that does not exist, reported confidently as a bug. Marc Zyngier described this on the list recently. An agent that gets "creative about specifications" is particularly annoying, he wrote, because the report is close enough to real that a maintainer has to spend time disproving it. A finding like that costs more than a missed bug, and enough of them will get the tool ignored.
Why does it happen? The model sees a subsystem through its review context: the per-subsystem prompts that state the facts and invariants the code assumes. Where that context is thin, the model fills the gap by guessing. A better model would guess less, but the context is the part maintainers can actually fix, and today nobody has to: the prompts live outside the tree, no one is named as responsible for them, and nothing checks whether they are right.
The proposal is to treat review context the way the kernel treats code. Each subsystem's context gets an owner of record, named in MAINTAINERS, separate from the code maintainer. That owner keeps the context correct in whatever way fits the subsystem, with the same leeway maintainers already have over their code. The methods are familiar: cite each claim to the specification or the source, and pin each rule to a real patch with a known verdict, so the rule carries its own regression test.
I wrote the arm64 and KVM guides in the review-prompts project Sashiko draws its context from, have maintained them since, and have contributed to Sashiko itself. In live review those guides catch real bugs. They have also failed in both of the ways context can fail: a claim that was wrong from the start, and a claim that was right until the code moved. The talk walks through both and shows what citing and pinning did about them. One part is not solved: there is no mechanical check that a model given correct context still reaches the right verdict. That is an open problem, and a working session is the right place for it.
At LSFMM earlier this year there was support for moving these prompts into the kernel tree, but what standard they must meet, and who answers for them, was left open. The decision this session puts to the room: should review context enter the tree as code, owned and verified, or stay as it is? If the answer is yes, I would take the first turn, on KVM/arm64.