The Collapse of Complex Software

Nolan Lawson, writing in a blogpost: Anyone who's worked in the tech industry for long enough, especially at larger organizations, has seen it before. A legacy system exists: it's big, it's complex, and no one fully understands how it works. Architects are brought in to "fix" the system. They might wheel out a big whiteboard showing a lot of boxes and arrows pointing at other boxes, and inevitably, their solution is... to add more boxes and arrows. Nobody can subtract from the system; everyone just adds. This might go on for several years. At some point, though, an organizational shakeup probably occurs -- a merger, a reorg, the polite release of some senior executive to go focus on their painting hobby for a while. A new band of architects is brought in, and their solution to the "big diagram of boxes and arrows" problem is much simpler: draw a big red X through the whole thing. The old system is sunset or deprecated, the haggard veterans who worked on it either leave or are reshuffled to other projects, and a fresh-faced team is brought in to, blessedly, design a new system from scratch. As disappointing as it may be for those of us who might aspire to write the kind of software that is timeless and enduring, you have to admit that this system works. For all its wastefulness, inefficiency, and pure mendacity ("The old code works fine!" "No wait, the old code is terrible!"), this is the model that has sustained a lot of software companies over the past few decades. Will this cycle go on forever, though? I'm not so sure. Right now, the software industry has been in a nearly two-decade economic boom (with some fits and starts), but the one sure thing in economics is that booms eventually turn to busts. During the boom, software companies can keep hiring new headcount to manage their existing software (i.e. more engineers to understand more boxes and arrows), but if their labor force is forced to contract, then that same system may become unmaintainable. A rapid and permanent reduction in complexity may be the only long-term solution. One thing working in complexity's favor, though, is that engineers like complexity. Admit it: as much as we complain about other people's complexity, we love our own. We love sitting around and dreaming up new architectural diagrams that can comfortably sit inside our own heads -- it's only when these diagrams leave our heads, take shape in the real world, and outgrow the size of any one person's head that the problems begin. It takes a lot of discipline to resist complexity, to say "no" to new boxes and arrows. To say, "No, we won't solve that problem, because that will just introduce 10 new problems that we haven't imagined yet." Or to say, "Let's go with a much simpler design, even if it seems amateurish, because at least we can understand it." Or to just say, "Let's do less instead of more."

Read more of this story at Slashdot.