The Search for Truth

Every time you change existing code, you break some other part of the system. You may not realize it, but you do. It may show up in the form of a broken unit test, but that presumes that a) said unit test exists, and b) it properly tests the aspect of the code you are changing. Sadly, more often than not, there is either no test to cover your change, or any test that does exist doesn't handle the case you are changing.

This is especially true if the thing you are changing is simple. It is even more true when changing something as complex as working with a boolean.

Mr A. was working at a large logistics firm that had an unusual error where a large online retailer was accidentally overcharged by millions of dollars. When large companies send packages to logistics hubs for shipment, they often send hundreds or thousands of them at a time on the same pallet, van or container (think about companies like Amazon). The more packages you send in these batches the less you pay (a single lorry is cheaper than a fleet of vans). These packages are lumped together and billed at a much lower rate than you or I would get.

One day, a particular developer saw something untidy in the code - an uninitialized Boolean variable in one of the APIs. The entire code change was from this:

 parcel.consolidated;

to this:

 parcel.consolidated = false;

There are some important things to note: the code was written in NodeJS and didn't really have the concept of Boolean, the developers did not believe in Unit Testing, and in a forgotten corner of the codebase was a little routine that examined each parcel to see if the discount applied.

The routine to see if the discount should be applied ran every few minutes. It looked at each package and if it was marked as consolidated or not, then it moved on to the next parcel. If the flag was NULL, then it applied the rules to see if was part of a shipment and set the flag to either True or False.

That variable was not Boolean but rather tri-state (though thankfully didn't involve FILE_NOT_FOUND). By assuming it was Boolean and initializing it, NO packages had a discount applied. Oopsie!

It took more than a month before anyone noticed and complained. And since it was a multi-million dollar mistake, they complained loudly!

Even after this event, Unit Testing was still not accepted as a useful practice. To this day Release Management, Unit Testing, Automated Testing and Source Code Management remain stubbornly absent...

Not long after this, Mr. A. continued his search for truth elsewhere.

[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!