CodeSOD: Parse, Parse Again

Sometimes, a block of terrible code exists for a good reason. Usually, it exists because someone was lazy or incompetent, which while not a good reason, at least makes sense. Sometimes, it exists for a stupid reason.

Janet's company recently bought another company, and now the new company had to be integrated into their IT operations. One of the little, tiny, minuscule known-issues in the new company's system was that their logging was mis-configured. Instead of putting a new-line after each logging message, it put only a single space.

That tiny problem was a little bit larger, as each log message was a JSON object. The whole point of logging out a single JSON document per line was that it would be easy to parse/understand the log messages, but since they were all on a single line, it was impossible to just do that.

The developers at the acquired company were left with a choice: they could fix the glitch in the logging system so that it output a newline after each message, or they could just live with this. For some reason, they decided to live with it, and they came up with this solution for parsing the log files:

def parse(string): obs = [] j = "" for c in string.split(): j += c try: obs.append(json.loads(j)) j = "" except ValueError: pass return obs

This splits the string on spaces. Then, for each substring, it tries to parse it as a JSON object. If it succeeds, great. If it throws an exception, append the next substring to this one, and then try parsing again. Repeat until we've built a valid JSON document, than clear out the accumulator and repeat the process for all the rest of the messages. Eventually, return all the log messages parsed as JSON.

As a fun side effect, .split is going to throw the spaces away, so when they j += c, if your log message looked like:

{"type": "Error", "message": "Unable to parse JSON document"}

After parsing that into JSON, the message becomes UnabletoparseJSONdocument.

But at least they didn't have to solve than newline bug.

[Advertisement] Forget logs. Next time you're struggling to replicate error, crash and performance issues in your apps - Think Raygun! Installs in minutes. Learn more.