The Contract Access Upgrade

Microsoft Access represents an "attractive nuisance". It's a powerful database and application development platform designed to enable end users to manage their own data. Empowering users is, in principle, good. But the negative side effect is that you get people who aren't application developers developing applications, which inevitably become business critical.

A small company developed an Access Database thirty years ago. It grew, it mutated, it got ported from each Access version to the next. Its tendrils extended outwards, taking over more and more of the business's processes. The ability to maintain and modify the database decayed, updates and bugfixes got slower to make, the whole system got slower. But it limped along roughly at the speed the business required... and then Larry, the user who developed, retired.

And that's where Henrietta comes in. She was hired on contract to take this ancient, crufty, Access database and reimplement it in C#, with a WPF front end (because "web application" sounded too scary a shift), with a SQL Server backend. The project was already in-flight, under the sober guidance of internal developers who had analyzed the Access database in detail.

There was already a source control server set up- an SVN server. Henrietta found that odd, but odder still was the change history: 100,000 commits from fewer than 20 developers, in only six years. Now, that's not ridiculous- but it's a steady cadence of two commits per developer per day, including weekends and holidays.

There was, fortunately, a lot of documentation. None of it was about the code, but instead about the organization. Who works for who, when a given management position was created, how long someone had been in that position. Nothing about the software internals. Definitely nothing about the custom UI framework someone had bolted on top of WPF.

When Henrietta noticed she couldn't find documentation about coding standards, or code review processes, she went to one of the other developers and asked: "What's our coding standard? And how do we handle code reviews?"

"Our whats? I don't know what those are."

Well, Henrietta finished up her first ticket, had her commit, and then did what all the other developers did: committed it right into the trunk of the repository.

Now, that was her first commit, and it was a training commit: she just needed to add some validation to the UI to make sure it didn't allow empty form fields. With that under her belt, her boss assigned her a new, more complex task. It needed her to make changes in the database, add new workflows to the application, a few screens, and so on.

"So," she asked, "is there a spec for this somewhere?"

"Oh," her boss said, "we don't write specs before we develop. Develop the feature and then write specs to describe its behavior."

Well, Henrietta didn't like to work that way, so she started by drawing mockups in a diagramming tool. This, as it turned out, was completely new to the organization. No one had ever done a screen mock up before. The handful of diagrams that did exist all were drawn with the same tool: Microsoft Paint.

Once Henrietta had decided what her feature was going to look like, she made a feature branch to start her work- and discovered that the way the application was architected, you couldn't conveniently develop in a local branch. In fact, you couldn't even get it to easily point at a development database. Everything had to go through trunk and get pushed to a dev server for testing- one dev server which all the developers had to share.

When Henrietta's code didn't work, she found out why: there was a "convenience library" developed by her boss that contained critical functionality for the application. If you didn't call certain methods in that library, the application wouldn't work. These methods were undocumented, and also, no one knew where the code lived. They only used the binary, compiled version of the library.

Once Henrietta had reshaped her code around the arcane bondage that the library demanded of her, she had reached the point where she didn't understand her own code anymore. Before she can get into the work of testing the code, a new issue rises to the top of the priority list and she's told to stop what she's doing and tackle that.

This was meant to integrate into a 3rd party SOAP-based web service. It transports sensitive data... over HTTP. There's no encryption at all. The WSDL file contains overlapping definitions of two different versions of the API, and the contradictions mean it's possible and easy to send malformed requests with unpredictable behavior. And when it does catch an error, it simply responds with "Error".

At this point, months had passed. So it was time for the organization to change their tooling. Everyone was commanded to update to the newest version of .NET Core, a new version of the IDE, and now a new code review tool. Crucible was rolled out with no instructions or guidance, and developers were expected to just start using it.

This delayed Henrietta's work on the 3rd party interface, so she went back to the complex feature with database changes. She discovered there are no foreign keys. Also, because there weren't any foreign keys, the data can't have foreign keys added, because the columns that should enforce referential integrity don't match up correctly.

Meanwhile, the Project Owner, frustrated by the slow development progress, started writing code themselves. They used the wrong set of project files, pushed it directly to a customer, and caused multiple crashes and downtime for that customer.

Back to the database, Henrietta discovered that there's really no abstraction around it, implementation details of the database have to be reimplemented into the UI. She built a UI control that encapsulated at least some of that functionality, and added it to the global UI library. Her boss noticed that change, and told her, "no, that's specific to your module, put it in a local library." Her boss's boss noticed that change, and said, "that UI control is very useful, put it in the global library."

Neither boss could agree on the correct location for it, so as a compromise, they created a new "global" library for "accessory controls".

Frustrated by all of this, Henrietta decided that she should try and get a local development environment set up. She ended up spending a few days on this, only to discover that certain stored procedures call out to other databases via hard-coded connection strings, and if she tried to run a local copy she'd simply start mangling data in other, production databases. Her boss noticed her spending time on this, and complained that she was wasting her time.

When Henrietta finally finished her big feature, she deployed it to the test environment. It blew up, but for reasons she could easily understand, and it only took a few days to fix it. The customer tested the feature, and it wasn't what they thought it was going to be. Once they understood the requirements, which weren't their original requirements, they were happy with the feature, but wished they'd gotten the feature they asked for. With this sign off, the Henrietta pushed the change to production, manually (because why would you automate deployments?). The customer's application immediately crashed because their database was incompatible with the current version of the code. There was, of course, no rollback procedure, so Henrietta was expected to spend a weekend combing through the customer's database to figure out which field contained a value that crashed the application.

After that, frustrated, Henrietta went to her boss. "Why are we doing things this way? We're spinning our wheels and making no progress because we have no process, no organization, and everything we do is fragile and we're not doing anything to fix the fundamental problems."

"That's the way we do it," her boss said. "Stop asking questions about everything, don't question anything, we're not going to change that. Just do your work or find a new job."

Henrietta took that advice to heart, and found a new job. All in all, she spent 8 months fighting her way upstream against a river of crap. It wasn't worth it.

[Advertisement] Otter - Provision your servers automatically without ever needing to log-in to a command prompt. Get started today!