Getting Lost in the World

Unit tests are important, but unit tests alone don't guarantee a good code base. Sandra, still suffering at InitAg brings us a bug that was sitting in their project for months, undetected.

In this case, Sandra's team needed to work with geographic information. Now, this is hard. Geography is hard. Maps are hard. Coordinate systems are hard.

These complexities can express themselves in surprisingly simple ways. In common communication, many of us likely describe the coordinates as "latitude and longitude". It's how textbooks and even Wikipedia describe locations. But Geographic Information Systems do it the opposite way- lon/lat. Longitudes first.

In fact, there's a standard for it, called GeoJSON, which defines a point thus:

{"type": "Point","coordinates": [lon, lat]}

The problem Sandra's team needed to solve was "simple": pull data from one set of databases, and dump it into a different set. In the end, it would get stored as GeoJSON, though each system might have a different way of actually handling the storage.

Different developers worked on it. And two different developers found two different ways to solve the problem. They wrote unit tests, based on their assumptions of how things worked. And thus, they managed to get things wrong, twice.

def lat_lon_position2point(position: Dict): return {"type": "Point", "coordinates": [ position["lat"], position["lon"] ] }

 # Gets converted to GeoJSON by the DB engine my_thing.location = [thing.position.lat, thing.position.lon]

Both of these approaches are transposing the coordinates. Under this scheme, the residents of New York City would find themselves competing for subway space with the penguins of Antarctica.

But a simple coordinate transposition isn't the WTF. The WTF is that this actually got deployed. It was fortunate that the first few locations processed had "latitudes" that were absolutely invalid (greater than 90 degrees) which let Sandra catch the bug before it actually mangled any real-world production data.

Through the whole process, everyone was testing their code, but the tests were driven by their assumptions about how the data should work- they expected the data in lat/lon, so they tested that the data was written (and retrieved) in lat/lon. All the lights on the dashboards were green, everyone was patting themselves on the back and celebrating a successful release.

[Advertisement] Otter - Provision your servers automatically without ever needing to log-in to a command prompt. Get started today!