Best of…: Best of 2023: A Single Bug
It's New Year's Day, and it's a great opportunity to look back at our story from last year that is in the most dire need of a New Year's Resolution: A Single Bug. --Remy
Matt's team had a party after their last release. It was a huge push, with tons of new features, that came at the end of many months of work. On the Monday after the party, they came back into work for unsurprising bad news: nothing is perfect, so there were several issues and defects that needed to be patched, quickly.
Since QA is the team responsible for signing off and approving any work, QA is the team that also owns the defect tickets. Matt and his team can't do any work without a ticket, which meant they spent almost an entire day knowing there were bugs to fix, but without any idea of what bugs to fix.
The next day, QA finally finished triaging the issues. There were a slew of low priority tickets, none of which were bugs, but enhancement requests- this screen is confusing, this path through the application requires too many button presses, no one can find this option. There was, however, only one bug ticket.
"Hunh," Matt thought, "that doesn't sound so bad."
Upon opening the ticket, Matt discovered that it was indeed bad. There were almost a dozen serious bugs, but for whatever reason, QA had bundled them into a single ticket. This made everybody's life much harder. Every change in the code had to have an associated ticket, every bug ticket had to have an attached test plan, every ticket has to have a single owner and assignee (but Matt's entire team would be splitting this work). Everything about getting this fixed was harder because QA had created a bottleneck by tying together unrelated bugs into a single ticket.
So Matt went over to Bruce, the QA manager who'd created the ticket. "Could you please split this ticket?"
"No, I can't."
"Yes, you can. Just abandon this one and make new ones."
Bruce shook his head. "You don't understand. These are post release bugs. Which means we released software with bugs in it. Which means the QA process failed. When the management dashboard shows a dozen high priority bugs post-release, management thinks that someone wasn't doing their job properly. That looks bad. So, we make one ticket, roll all the issues under that one, and it looks much better on the dashboard."
Matt was offended that anyone would try to game the system like that. QA was making everybody's job harder and trying to conceal issues from management. Well, the joke was on QA- Matt went straight up the tree to the management team.
He sent an email, laying out what was happening, and most important, why it was happening. QA was trying to hide the failures in the QA process. Later that day, one of the directors set up a meeting with Matt to discuss the email.
"So, I understand you have some concerns," the VP said, "and I just wanted to show you how we view that." The VP pulled up the management dashboard, and flipped back to an old release, from a few years ago. Many of the metrics showed red stoplights. "So, here's a release that went badly. Too many tickets for bugs." They flipped to the most recent release. Here, all the lights were green. "And this is a release that went well."
"Right," Matt said, "but this release only looks like it went well because they only opened one ticket for many bugs!"
The VP nodded without listening. "Right, but this dashboard tracks open tickets. We like releases with only one open ticket. The lights are green, see?"
"But there are more bugs than there are tickets. They're hiding the fact that there are more bugs!"
"But this dashboard doesn't track bugs, it tracks tickets."
There are few things more immovable than a manager with pretty green lights on a dashboard. Goodhart's Law struck again. Matt admitted defeat and fixed the ticket the hard way.
[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!