The Record Setting Score

A few years ago, Joel Spolsky wrote his simple, 12-point test to measure how "good" a software team is. Stan was thinking about this test quite a bit, when he started a new job at a company which set a new record for failure.

Stan's journey started after an excited recruiter told him, "I have a great role for you, working for a cutting edge online retailer in London!" Stan became a Senior PHP Developer on a team of three others. Each day there, he asked another question from the Joel Test.

Let's review Stan's experience against the 12 questions from the Joel Test.

1. Do You Use Source Control?

Stan asked about source control during the interview, and the interviewer replied, "No, we don't have any at the moment. But we will soon!" When Stan started, he found that the only copy of the source code was the live website. Any changes meant downloading the site, altering the files, and re-uploading the changes over FTP. He very quickly got used to hearing the question, "Is anyone changing users.php?", or "Hey, where are all my changes from yesterday?!" Having four people work on a live website using only FTP to synchronize work caused friction, like throwing sand in an engine.

Joel Test Question 1: Fail

2. Can You Make a Build in One Step?

Building should be as easy as possible. The best solution is to have a script which can handle the entire build process from start-to-end. Check everything out from source control (if you have one!), compile binaries, and create the installation packages. Without this, building will be error prone.

Without source control, and without any concept of development servers or test servers, "builds" were deployed via FTP. The team made database schema changes by uploading specific PHP scripts with the required SQL and loading them as a page while praying that nothing breaks. When things did break, it was extremely difficult to debug or rollback the changes. I'm not sure "build" or "release" are even valid terms for this company, any more than "architect" describes a kid who lobs globs of mud at a wall.

Joel Test Question 2: Fail

3. Do you make daily builds?

Daily builds make sure your repository trunk is in good working condition. At least once a day, obvious errors like incomplete check-ins or missing dependencies are easily discovered, and discovered early, before any software enters production use. The term "daily build" implies some sort of integration and testing environment, not ad-hoc FTP uploads to a live site.

Joel Test Question 3: Fail

4. Do you have a bug database?

You need a good way to track defects. What's the error? Who reported it? How can it be reproduced? What's the priority? When was it fixed? Of course, tracking bugs is meaningless when one developer's FTP upload can overwrite another developer's previous efforts, causing the bug fix to be lost for all eternity.

Joel Test Question 4: Fail

5. Do you fix bugs before writing new code?

Stan's company fixed bugs all of the time, as any software firm must. However, those bugs regressed all the time too, since one developer's fix could easily be rolled back when another developer uploads their version of the PHP file a few hours later. This methodology ends up like this.

Joel Test Question 5: Fail

6. Do you have an up-to-date schedule?

Having a schedule would mean having a list of features, a timeline by which they need to be finished, and priorities to help determine what gets in and what doesn't if the timeline slips. Instead, Stan had a constant stream of emails, memos, and Post-It Notes handed down from management.

Stan might be half-finished with a "top priority" feature when a different director would storm into his cube. "Did you fix my issue yet? Get on it! Now! NOW! It's an EMERGENCY!"

Joel Test Question 6: Fail

7. Do you have a spec?

Good software has some kind of list of requirements, or at least a description of what the software should do. For Stan, his "spec" was determined by which director was currently in his cube, screaming about their personal fire that needed to be extinguished. It's only by the grace of good fortune that his cube was too small for two directors to be badgering him at once- that situation might have come to blows.

Joel Test Question 7: Fail

8. Do programmers have quiet working conditions?

"Are you aware of the exciting new opportunities available to you, with our products?"

One of the many "top priorities" for their company was marketing, so they hired a stable of telemarketers. Short on office space, they stuffed them in the only place that was free: next to the development team. Within a few days, the developers wanted to chop off their own ears, just to quit hearing the marketing script on endless repeat.

"Are you aware of the exciting new opportunities available to you, with our products?"

Some offices invest in cheap white-noise generators to cut down on that sort of cross-talk. Stan's wasn't one of those.

"Are you aware of the exciting new opportunities available to you, with our products?"

In fact, the office had other infrastructure problems. When winter arrived, the building's heater broke down. Working from home was out of the question- how could directors storm into your cube to "clarify" priorities? Instead, everyone was forced to work in the sub-zero office.

"Are you aware of the exciting new opportunities available to you, with our products?"

Developers huddled in their cubicles, wearing coats, hats and gloves. Imagine the quality software written by a team-

"Are you aware of the exciting new opportunities available to you, with our products?"

-wearing gloves, in the freezing cold, while listening to the steady mumble of marketing copy.

Joel Test Question 8: Fail

9. Do you use the best tools money can buy?

Developers need to have powerful PCs. If you give your development team a stack of Pentium M laptops with single-core CPUs and 512MB of RAM in 2015, they'll spend 90% of their time standing on their rolly office chairs and fencing with makeshift swords, or just watching Chrome cry while rendering web pages.

When Stan first started, he was assigned a nice, shiny MacBookPro, stuffed to the gills with RAM and an SSD. Shortly after he got it set up for development use, the sysadmin informed him that this was a requisition mistake. "This machine was actually for one of the new graphic designers. You get this one."

The replacement was a 5 year old machine that was suitable for email, Solitaire, and virtually nothing else.

Joel Test Question 9: Fail

10. Do you have testers?

Stan's company almost wins this one on a technicality. While they didn't employ any testers, they had customers. They were abused as testers, since they would naturally call in and complain when ~a software release~ big ball of mud was FTP'd to the live server.

A generous grader might be tempted to give partial credit for this, since they at least recognize that testing needs to happen. But let's repeat this: their customers were their dedicated testers on a production website.

Joel Test Question 10: Double Fail

11. Do new candidates write code during their interview?

During Stan's interview, he was not asked to write any code. In fact, his interviewer didn't speak English terribly well, so the interview was mostly the two of them smiling and nodding at each other. It probably didn't matter, since most of Stan's day wasn't spent writing code- it was spent rewriting the code that got deleted after the last FTP-based screw-up.

Joel Test Question 11: Double Fail

12. Do you do hallway usability testing?

A good way to test is to grab a random person as they walk through the hallway, and force them to use the software you just wrote. They don't work on your team, and may not be familiar with the project or its requirements. Can they use it? Is it intuitive? Does the user interface make sense? This is a great way to discover usability issues that developers easily overlook.

In Stan's company, they sort of did the opposite. Random directors would storm into Stan's cube, not to try the software, but to dictate a new mandate without any thought to whether or not it made sense.

Joel Test Question 12: Reverse Fail

Total Score: -2.0000000000000000001420001, thanks to two double fails, and the floating point rounding error is caused by the Reverse Fail.

As the new Senior PHP Developer, Stan fought to clean things up somewhat. He got a Subversion server set up on a spare box, and managed to get a development server with a clone of the live system. In a few months, they started working almost like a "real" software team. Productivity went up, bugs got patched before users saw them, and FTP-induced rework was almost entirely killed.

Sadly, after all these changes, management called in the team and complained that they were all overpaid, lazy, and a drag on the company. Stan quit on the spot, and over the next few days, the rest of his developers followed him. Weeks later, he heard through the grapevine that they'd brought on a new team, and were undoing Stan's "wasteful" changes.

Apparently, the highest priority directive was to ensure their record-setting score on the Joel Test.

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!