Better parts, worse system

There's a rule of systems thinking that improving part of a system can often make the system as a whole worse.

One example of this is Braess' Paradox that adding roads can make traffic worse, and closing roads can improve traffic.

Another example is the paradox of enrichment: Increasing the food available to an ecosystem may lead to instability, and even to extinction.

Richard Hamming put it this way in what he called the first rule of systems engineering:

If you optimize the components, you'll probably ruin the system performance.

For more variations on this principle, see this Twitter thread.

I've been thinking about this lately in regard to software tools. If you're just starting out and ask around for the best way to do this and that, you might get individual bits of good advice that add up to bad advice.

You're still using X? You should be using Y. Y's better."

When someone says Y is better" they probably mean that it has more features. It may also be objectively better by several other criteria. But that doesn't mean it's better for you, at this point in time, given your experience, your temperament, and your project.

The Dreyfus model of skill acquisition says that beginners want context-free rules: Y is better than X. That's an understandable desire, and people are all too willing to give context-free advice. But context matters. A lot.

Yesterday I needed to do a little search-and-replace text munging, join two data sets, and compute some basic statistics. I thought about how I could go off on tangents for each task, using the best tool for each task individually, and take forever to get my job done. Instead, I cobbled something together quickly at a command line, with each step being far from the most general solution.

If you choose the best (i.e. biggest) tool for each task, you'll get affirmation, or at least avoid criticism, for each choice. And you'll likely end up with an unwieldy hodgepodge of tools, none of which you're confident in using. If instead you think carefully about your abilities and your needs, you'll gradually assemble a collection of tools that work together for you, making you more confident and productive.

The post Better parts, worse system first appeared on John D. Cook.