CodeSOD: A Small Closing
Dario got a call that simply said, "Things are broken." When he tried to get more details, it was difficult for the users to pin it down more clearly. Things would work, then they wouldn't. The application would run, then it would hang, then it would suddenly start working again.
He knew that his co-worker, Bob, had been doing some performance tuning, so that was probably the right place to look. "Bob, have you made any changes that might cause weird behavior?"
"No, no. I've just been optimizing the file-handles."
"Are you just using jargon to sound like you know what you're talking about?"
"No! I mean, I've just been optimizing how we manage our file-handles so that we don't leave them open."
"Oh, that makes sense. Show me.."
Proud of his clever solution, Bob showed him:
def close_all_files(): for fd in range(resource.getrlimit(resource.RLIMIT_NOFILE)[1]): try: os.close(fd) except OSError: pass
The function resource.getrlimit() returns some resource limit that controls the behavior of the Python runtime. This could be the size of the call stack, how much CPU time it's allowed to use, or in this case, the RLIMIT_NOFILE- the number of allowed file descriptors for the current process.
On a typical system, this number could be quite large. For example, 9,223,372,036,854,775,807. Then, for each one of those integers, the code then asks the operating system to pretty-please close any files with that file descriptor. If anything goes wrong, catch the exception and ignore it.
Now, the good news is that this can't close any files that weren't currently opened by the Python process. The bad news is that it can close any file opened by the Python process. This triggered a cascading series of Bad file descriptor errors, and every time the function ran, it would grind the system to a halt while it tried to close every possible file ever opened.
[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!