Article 74NXP Exelbierd: What's actually in a Sashiko review?

Exelbierd: What's actually in a Sashiko review?

by
daroc
from LWN.net on (#74NXP)

Brian "bex" Exelbierd has publisheda blogpost exploring follow-up questions raised bythe recent debate about the use of the LLM-based reviewtool Sashikoin the memory-management subsystem. His main finding is that Sashiko reviews arebi-modal with regards to whether they contain reports about code not directlychanged by the patch set - most do not, but the ones that do often have severalsuch comments.

Hypothesis 1: Reviewers are getting told about bugs they didn't create.Sashiko's review protocol explicitly instructs the LLM to read surrounding code,not just the diff. That's good review practice - but it means the tool mightflag pre-existing bugs in code the patch author merely touched, putting thoseproblems in their inbox.

Hypothesis 2: The same pre-existing bugs surface repeatedly. If a knownissue in a subsystem doesn't get fixed between review runs, every patch touchingnearby code could trigger the same finding. That would create a steady drip ofduplicate noise across the mailing list.

I pulled data from Sashiko's public API and tested both.

External Content
Source RSS or Atom Feed
Feed Location http://lwn.net/headlines/rss
Feed Title LWN.net
Feed Link https://lwn.net/
Reply 0 comments