Single-User Mode

It was a common occurrence for JarosAaw F.'s brother, Andrzej, to call him for help. Andrzej worked as a data-entry specialist for the Polish Ministry of Employment, while JarosAaw was part of the ministry's IT department and moonlighted as a Python programmer.

One day, JarosAaw answered another call from Andrzej. "No sane person can do this job," Andrzej started. "There just aren't enough hours in the day."

"'Hours in the day?' You enter data and run reports using web software. It's not that difficult."

"No, you don't understand. The ministry's starting a new labor force program, and I'm in charge of the data entry. But there are too many records to enter by hand."

"Andrzej, I'm really tired of this," JarosAaw said. "I'm going to write you a script to help with your work, and in exchange you don't call me at work for six months. Deal?"

Eternity in a Cell

JarosAaw had never actually seen the software Andrzej used to enter data. Just getting to Andrzej's department required signing an authorization form at the front office, then a half-hour wait until his supervisor gave the go-ahead. But JarosAaw knew writing the script for his brother wouldn't take very long.

Andrzej showed JarosAaw to his desk and opened a browser. The web application looked current circa 1998, as if someone mocked up a Windows GUI in Internet Explorer 4. There were rows and rows of grey tabular data, text set in black Tahoma. Next to each cell was a button marked Edit.

"You press the button to edit each cell, which takes a few seconds to load," Andrzej said, demonstrating. "Each cell, and every row, must be entered one at a time"

"Surely there's a 'bulk import' somewhere." JarosAaw scrolled down the page, looking through buttons labelled Income Report and Demographic Report. There was nothing marked that looked remotely like a bulk import feature. "How long does it normally take you to enter a row?"

"Five minutes. I usually get about 10 records an hour. There are 400 records, and my boss needs them done by the end of Sunday." That left five days. "I still have reports to run while I do this. That's why I need your help."

Parallel Punishments

"Okay," JarosAaw said, "can't you get someone to help you with this? Five people entering data would mean it'd get done in a day."

"Even if we had that many people to spare, there can't be more than one person entering data at a time. The system would crash."

JarosAaw chuckled. "Oh, come on. No web application forces a single user to enter data."

"A year ago, we tried using two people at a time," Andrzej whispered. "A whole dataset was corrupted. It set back the department's mandate by six months, and my last supervisor was fired because of it. My new supervisor won't tolerate any further delays."

A CRUD-like web interface, with no parallel data entry allowed, it seemed deliberately designed to make one person's life miserable " and that person was Andrzej.

"It's a good thing I work in IT," JarosAaw said. "Because what you need is a tailor-made import script, and I'm the one who can write it for you."

Automated Manual Entry

First, JarosAaw installed Python on Andrzej's computer. If JarosAaw were about to exploit the ministry's applications, he wanted to use a language he was familiar with. Next, he entered a few sample rows from the dataset Andrzej was burdened with. He logged every HTTP request the web application made, making a list of calls his script would need to make. Finally, he converted the Excel file the data came in to .csv so his script could consume it. It was tedious, but not as tedious as Andrzej's burden.

"Let the script run overnight," JarosAaw said. "If anyone asks, tell them I did this on my own initiative. It should be done by morning."

The next morning, Andrzej himself went to see JarosAaw in the IT department. He looked relieved. "You just saved me four days of manual data entry."

"You can use it on other datasets," JarosAaw said. "Just convert the files to .csv and pass in the filename as an argument from the command line. And make sure no one else is entering data while it's running."

"Thanks," Andrzej said. "So, call you in six months?"

JarosAaw shook his head. "This one's on me. I had no idea what torture they put you through until you showed me."

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!