CodeSOD: Wear a Dunder Cap

In the Python community, one buzzword you'll find thrown around is whether or not an approach is "pythonic". It's a flexible term, and something you can just throw out in code reviews, even if you've never written a line of Python in your life: "Is that Pythonic?"

The general rubric for what truly is "pythonic" is generally code that is simple and code that operates explicitly. There shouldn't be any "magic". But Python doesn't force you to write "pythonic" code, and it provides loads of tools like decorators and metaclasses that let you get as complex and implicit as you like.

One bit of magic is the "dunder" methods, which are Python's vague approach to operator overloading. If you want your class to support the [] operator, implement __getitem__. If you want your class to support the + operator, implement __add__. If you want your class to support access to any arbitrary property, implement __getattr__.

Yes, __getattr__ allows you to execute code on any property access, so a simple statement like this"

if self.connected: print(self.coffee)

"could involve twenty database calls and invoke a web service connected to a live coffee machine and literally make you a cup of coffee. At no point does the invoker ever see that they've called a method, it just happens by "magic". And, for extra fun, there's also a global method getattr, lets you wrap property access with a default, e.g., getattr(some_object, "property", False) will return the value of some_object.property, if it exists, or False if it doesn't.

Whew, that's a lot of Python internals, but that brings us to today's anonymous submission.

Someone wrote some classes which might contain other classes, and wanted them to delegate property access to those, which means there are a number of classes containing this method:

def __getattr__(self, item): return self.nested.item

So, for example, you could call some_object.log_level, and this overridden __getattr__ would walk through the whole chain of objects to find where the log_level exists.

That's fine, if the chain doesn't contain any loops, where the same object appears in the chain multiple times. If, on the other hand, it does, you get a RecursionError when the loop finally exceeds the system recursion limit.

Our submitter found this a bit of a problem. When the access to log_level failed, it might take a long time before hitting the RecursionError- which, by the way, isn't triggered by a stack overflow. Python tries to throw a RecursionError before you overflow the stack.

You can, but very much shouldn't control that value. But our submitter didn't want to wait for the thousands of calls it might take to get the RecursionError, so they wrapped their accesses to log_level in code that would tweak the recursion limit:

 # Protect against origin having overwritten __getattr__ old_recursion_limit = sys.getrecursionlimit() n = 2 while True: try: sys.setrecursionlimit(n) except RecursionError as e: n += 1 break try: log_level = getattr(origin, "log_level", None) except RecursionError as e: object.__setattr__(origin, "log_level", None) log_level = getattr(origin, "log_level", None) sys.setrecursionlimit(old_recursion_limit)

So, first, we check the current recursion limit. Then, we try setting the recursion limit to two. If the current recursive depth is greater than two, then when we try to change the recursion limit it throws a RecursionError. So catch that, and then try again with one higher recursion limit.

Once we've set the recursion limit to one greater than our current recursion limit, we then try and fetch the log level. If we fail, we'll just default to None, and finally return back to our original recursion limit.

This is an impressive amount of effort into constructing a footgun. From the use of __getattr__ in the first place, to the misplaced effort of short-circuiting recursion instead of preventing cycles in the first place, and finally to the abuse of setrecursionlimit and error handling to build" this. Absolutely nothing here should have happened. None of it.

Our submitter has confessed their sins. As penance, they'll say twenty Hail Guidos, and fix this code. Remember, you can't be absolved of your sins if you just keep on sinning.

[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!