We return to Virginia N’s refactoring efforts (previously: here, and here).This code is elegant in its stupidity, combining two anti-patterns in one glorious, “No, don’t do that, whyyyyyyyyyy!â€. To wit, the for-switch, and the “build SQL statements through string concatenationâ€:
Becky is not a programmer, but a physicist. She works in academia, alongside other scientists. Modern science generally requires some sort of heavy computation, which means scientists write code. It’s often not very good code, but that’s just the nature of the beast. The code exists to provide an analysis, not to be deployed as an app to the masses.Most of the time. A few civil engineers were working on a brand new Android app for traffic analysis, with plans to distribute it. Unfortunately, they had some problems, and wanted more experienced eyes. Becky set aside the Fortran77 she was working on to trace through their Java code, and found this:
This is a follow-up to With the Router, In the Conference Room, revealing the… STUNNING CONCLUSION!How It Really EndedDarren took the case up to his boss, and then to their boss, up the management chain. No one was particularly happy with Cathy’s tone, and there was a great deal of tut-tutting and finger-wagging about professional conduct.But she was right. It was Mr. Green who failed to follow instructions, it was Mr. Green who cost the company thousands, along with the customer relationship problems caused by Cathy’s sudden emergency trip back to the home office.In what can only be considered a twist ending by the standards of this site, it was Mr. Green who was escorted out of the building by security.The killer was Cathy, in the issue tracking system, with the snarky bug report. [Advertisement] Universal Package Manager – store all your Maven, NuGet, Chocolatey, npm, Bower, TFS, TeamCity, Jenkins packages in one central location. Learn more today!
One of the most important aspects of software QA is establishing a good working relationship with developers. If you want to get them to take your bug reports seriously, you have to approach them with the right attitude. If your bugs imply that their work is shoddy, they are likely to fight back on anything you submit. If you continuously submit trivial “bugsâ€, they will probably be returned right away with a “not an issue†or “works as designed†status. If you treat any bug like it’s a critical showstopper, they will think you’re crying wolf and not immediately jump on issues that actually are critical.Then there’s people like Mr. Green, a former coworker of submitter Darren A., that give QA a bad name. The Mr. Greens of the QA world are so incompetent that their stupidity can cause project delays, rack up thousands of dollars in support costs, and cause a crapstorm between managers. Mr. Green once ran afoul of Darren’s subordinate Cathy, lead developer on the project Mr. Green was testing.Cathy was en route to the United States from London for a customer visit when her phone exploded with voicemail notifications immediately upon disabling airplane mode. There were messages from Darren, Mr. Green, and anyone else remotely involved with the project. It seemed there was a crippling issue with the latest build that was preventing any further testing during an already tight timeline.Instead of trying to determine the cause, Mr. Green just told everyone “Cathy must have checked something in without telling us.†The situation was dire enough that Cathy, lacking the ability to remotely debug anything, had to immediately return to London. Mr. Green submitted a critical bug report and waited for her to cross the Atlantic.What happened next is perfectly preserved in the following actual bug report from this incident. Some developers are known for their rude and/or snarky responses to bug reports that offend them. What Cathy did here takes that above and beyond to a legendary level.
You may remember Virginia N from An Extinction Event, where she struggles to refactor a legacy project with some… unusual design principles. ReSharper still continues to choke to death on their codebase, but her management has let her know, this won’t be a problem going forward.“You see,†her boss explained, “we’re going to move the logic into stored procedures. That way, we can more easily re-use the logic between the Windows Forms client and the Web app.â€â€œOh, there’s going to be a web app, now?†Virginia asked.“Yes! They’re going to use best practices, like unit testing, so that they don’t end up with the same kind of mess we have,†her boss said.Virginia was halfway to filing a request for a transfer when she heard more about the web project. Then she heard about their plan. They weren’t going to simply build a web-client for their backend, but instead were going to build an inner platform. The page logic would be a simple template, and all of the rules, styling, data and display logic would be stored as data in the database. “It’s gonna be really flexible!â€Virginia decided to stick with the fetid field she knew, instead of the “green field†which was going to be a fetid swamp in a matter of weeks.In Virginia’s swamp, she has many, many 40,000 line classes. That much code means the code has no real cohesion, so it leads to Virginia finding variables named thus:
Our Anonymous submitter's first job was helping to support a distributed system running in a low-energy embedded platform. Interesting on its face, the platform was actually a bloated, outdated monstrosity made worse by the decision to use C++ in conjunction with a homemade (read: unsafe) binary data format.The platform had one mysterious, catastrophic flaw in particular: once a cluster was up and running for a few weeks, it would sometimes fail with a series of random segfaults, with several nodes crashing at once. Serial port debugging showed that many of these failures were proceeded by an ominous log message, a single line with the number 10001 and nothing else.After pondering the mystery for some time, someone thought to search for the number 10000 within the code base. Thus they found their culprit within the very core of their application: the code that read each record stored in their homemade binary files and copied it to memory.
Sometime back, our friend Fred told us about his experiences with homegrown PK/FK relationships. Today, he regales us with a tale of trying to get users to use their new-and-improved sso mechanism, even if they don't want to.His company currently runs a legacy reporting portal service that has an old-school sso which is used by several third party systems. This mechanism stores user names and passwords as clear text in the DB. It also passes them in clear text in a hidden HTML form. The third party code would create the hidden form with the user name and password in clear text and JavaScript-submit it to the login page - without HTTPS. OK, it was the way things were set up way back then.Nowadays, his employer knows better and has created a new and improved system with slightly more sophisticated security. They encourage their end users to make the switch and use the shiny new platform. Unfortunately, the third party systems still have hard-coded embedded links to the old system, and although switching to use the new system would improve security, the users are in no rush to make changes to their software.When you have lemons, make lemonade.Fred was unwilling to succumb to the laissez-faire attitude of lazy customers, and so wrote a URL-rewrite rule for the IIS server that was running their old VB6-era system. It redirected posts to login.asp to /hijack/login.aspx. The new page contains logic to check if the user is configured on the new platform. This magic was accomplished by using the 307 Temporary Redirect, which maintains the POSTed payload; something which is not done by a standard 302 (perhaps this is a Dark Secretâ„¢ of IIS - who can say?) If the user was found in the new application, an access token was fetched from IdentityServer3 and the incoming user was redirected to the new platform. Otherwise, /hijack/login.aspx output the same old fool-proof hidden HTML form with clear text user name and password and JavaScript-submitted it - without SSL - to the renamed login2.asp page of the old system.Upon pondering his creation, Fred winced at the plethora of Biblical-Level WTF in this set up, but in some curious way the solution is kind of cool and he thought the world should know about it.[Advertisement] Release!is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!
After some “miscommunications about coding standardsâ€, Nicolas’s company decided that they should have one of their internal developers do code reviews on anything that came from their external, third-party developers. Nicolas drew the short straw on the most recent project.The specific problem in play is that they had config-data, structured as nested dictionaries. You know the kind of data-structure- something like:
“I logged into the admin app.â€Julie’s boss had the username and password for the admin app, so that wasn’t too surprising.“With my regular username and password,†her boss added.That was a bit more of a problem. The app in question was an internal, home-grown CMS. The admin portion of it was secured by a single username/password combo, controlled by a config file. It wasn’t the most secure thing on Earth, but it should at least be secure enough that you needed to supply the correct values.Julie checked the code, and found that wasn’t the case:
A great many breakfast cereals promise some sort of health benefit. This brand is good for your heart, that brand has 11 essential vitamins and minerals. This one’s got bran! Just because there’s a promise of health benefits doesn’t mean they actually exist- most of these cereals are lightly fluffed sugar held together with a smidge of starch.Object-oriented languages promise a lot of code-health benefits, and used properly, they can certainly deliver. In this somewhat tortured metaphor, the Lucky Charms marshmallow of OO features is Inheritance. It’s tasty, it’s easy to explain, but it’s not really good for your code. A little bit, here-and-there, from time-to-time is great. But some folks buy the three pound bag and that’s just not good for anybody.Daniel took over some code left behind by a retiring co-worker. The code was seeded through with abstract classes- though, because this was Python, “abstract†was more of a convention than a statement of fact. This predecessor took abstraction to its “logical†conclusion, and when testing abstract classes, they wrote abstract tests:
At the very beginning of my career, I was a junior programmer on a team that developed software to control an electronics test station, used to diagnose problems with assorted components of jet fighters. Part of my job was the requisite grunt work of doing the build, which entailed a compile-script, and the very manual procedure of putting all the necessary stuff onto a boot-loader tape to be used to build the 24 inch distribution disk arrays.sourceThis procedure ran painfully slowly; it took about 11 hours to dump a little more than 2 MB from the tape onto the target disk, and nobody could tell me why. All they knew was that the official software had to be used to load the bootstrap routine, and then the file dumps.After killing 11 hour days with the machine for several months, I had had it; I didn't get my MS to babysit some machine. I tracked down the source to the boot loader software, learned the assembly language in which it was written and slogged through it to find the problem.The cause was that it was checking for 13 devices that could theoretically be hooked up to the system, only one of which could actually be plugged in at any given time. The remaining checks simply timed out. Compounding that was the code that copied the files from the tape to the disk. It was your basic poorly written file copy routine that we all learn not to do in CS-102:
The first rule of building your own password storage mechanisms is don’t. Like most other highly-specialized wheels, you aren’t going to do as good a job as someone who specializes in it. It’s bad enough when you write your own date mangling code, but for security-critical features, like passwords or encryption, you’re begging for trouble.Joni spotted some trouble: many of the users in the database had the same password hash. This, of course, should never happen- the password should be combined with a user-specific salt as part of the hashing, so that even if two users had the same password, they’d have different hashes.Joni investigated, and found the code used:
The company Tomasz worked for launched a new device line with more RAM than the previous generation. This was supposed to put an end to the sort of memory shortages common to embedded systems. However, it wasn't long before they began hearing from clients whose systems crashed whenever they attempted to upgrade the accompanying software package.The initial reports were met with surprise and skepticism, but the investigation soon led Tomasz and his cohorts to—you guessed it—a reproducible out-of-memory error.With RAM not an issue, they took a deeper look at the upgrade process itself. The 50MB upgrade package was supposed to be copied to a special directory so the OS could install it. In C++ on linux, this is a simple task. You could use splice() on a newer linux kernel, or sendfile() on an older one. You could also read and write one buffer at a time. Inefficient, but effective.As you may already suspect, the developers who'd written the library for handling upgrades had taken a more brillant approach, shown below. readFile() stores the entire file in a string in memory, and writeFile() places it in the desired directory. With this implementation, any device could be brought to its knees with a large-enough upgrade file.
Virginia N was trying to refactor some code, and that meant understanding where the value m_PSOC_SIG was used, and why. So, she did some searching, and found this line, which doesn’t contain our value:
One of the selling points of a language like Java is that it comes with a rich standard library of useful classes. This allows developers to completely ignore those useful features, and instead reinvent the wheel badly. Once this novel square wheel has come off the assembly line, it becomes the defacto standard for the organization.Take, for example, Caiwan’s office. They have a… special date-handling library.
Although I've retired from full time work, I still consult for lots of small mom-n-pop places. Mostly, it's little scripts to automate doing this and that. Sometimes, the boss' kid or nephew was asked to get ambitious and solve a problem. When the inevitable happens, they call me to bail them out.For the most part, it's usually something like some file got moved/renamed/deleted. Sometimes, they got ambitious and attempted to write a batch file. This time, a college freshman, who claimed to be "good with computers", had written a program to control the little scripts and jobs in an automated fashion. Apparently, it was getting too complicated for him and they asked me if I could work with it.It's a pity that Windows doesn't have some sort of way to run a task on a schedule...Anonymized, but structurally unmodified, and no, there wasn't a single comment in the file:
Our sponsor, Hired, passed us off a report they just published: “The State of Contract Workâ€. I said to myself, “Wait a second, I’m a contractor!†Well, technically, I’m more of a consultant or sometimes a trainer- one of those evil highly paid consultants who swing in, tell developers how to do their jobs, and leave behind nothing more than the smell of brimstone and invoices.The bad thing about this line of work, at least from the perspective of a TDWTF article, is that if I encounter a real WTF, it’s because someone wants me to fix it. A WTF that is getting fixed isn’t really a WTF anymore. That doesn’t mean I don’t encounter some real head-scratchers from time to time.For example, I had a client that wanted to figure out best practices around using the Cassandra database. For the unfamiliar, Cassandra is a trendy “big-data†tool, a massively distributed database with no single point of failure and limited guarantees about how consistent the data is across replicas. It’s good for blisteringly fast writes, good for reliability, and absolutely terrible for any sort of ad-hoc query and data analysis.So, I talked with them a bit about their Cassandra needs, roll into the office, and that’s when I start getting the real picture: a pointy-haired boss heard that Cassandra was cool, that FaceBook and Netflix used it a lot, and thus… they were going to use it. For everything. All of their applications, from their legacy mainframe apps, to their one-off SQL server DBs for intranet apps, to their massive supply-chain and retail business were going to run on Cassandra. They started by adopting it for the massive supply-chain and retail portion of their business, and thus were actually quite successful- it was the right tool, for the right job.Thus armed with a wrecking ball and a single success with it, every problem started to look like a building that needed to be knocked down. This lead to a lot of conversations like this:Client: So, we need to run ad-hoc reports out of Cassandra. How do we do that?
An anti-pattern that shows up from time to time here is the old “our IDE’s build output is mapped to a network drive on the web serverâ€, but “Drummer†shows us a novel new variation on that theme.It all started when a co-worker asked them, “how do I change the compiler version?†The code was built using Ant, so “Drummer†opened the build file and searched through it for a javac element- the Ant command which runs the Java compiler.They didn’t find anything, but after a more manual search, they found this:
Today's submitter writes: I wonder how many developers out there have managed, intentionally or otherwise, to have a comment Easter egg go viral within a project.It seems in the late '90's he was working on a project codenamed "Dolphin." This wasn't the GameCube; it was an ASP/VB6 N-Tier system, also known as "way less fun." One of the first phases of the project involved a few web-based forms. The architects provided them with some simple standard templates to use, such as the method header comment block. This comment block included a Purpose field, which in a moment of self-amusement our submitter changed to Porpoise throughout the VB6 classes and ASP scripts he'd written.The first phase was released, and after code review, that particular implementation was cited as the paragon that other implementations should follow. Of course, this led to rampant copy-pasta throughout the entire system. By the end of phase 2, the code comments for the Dolphin project were inextricably filled with Porpoises. Being a subtle word change, it largely went unnoticed. Every once in a while, a developer would actually notice and nearly keel over laughing.Of course, there's also a famous instance of a code comment going properly viral. Deep within the bowels of the Unix kernel, there is a method responsible for saving the CPU context when processes are switched—any time a time slice is used up, an interrupt signal is caught, a system call is made, or a page fault occurs. The code to do this in an efficient manner is horrifically complicated, so it's commented with, You are not expected to understand this. This comment can now be found on buttons, mousepads, t-shirts, hoodies, and tons of other merchandise. It's become a rallying cry of the Unix geeks, a smug way of saying, "I understand where this is from. Do you?"Have any of you ever written something that went viral, either locally within your company or across the broader Internet community? Let us know in the comments or—if you've got a good one—drop us a submission. [Advertisement] Application Release Automation for DevOps – integrating with best of breed development tools. Free for teams with up to 5 users. Download and learn more today!
Three years ago, this XKCD comic captured a lot of the problems we have with gathering requirements:Our users have no idea which kinds of problems are hard and which kinds are easy. This isn’t just for advanced machine learning classification projects- I’ve had users who assumed changing the color of an element on a page was hard (it wasn’t), to users who assumed wiring up our in-house ERP to a purchased ERP was the simplest thing ever (it wasn’t).Which brings us to Christopher Shankland’s contribution. He works for a game company, and while that often means doing game development, it often means doing tooling and platform management for the design team, like providing fancy dashboards for the designers to review how users play the game so that they can tweak the play.That lead to this conversation:
We generally don’t do news roundups when yet another major company gets hacked and leaks personally compromising data about the public. We know that “big company hacked†isn’t news, it’s a Tuesday. So the Equifax hack didn’t seem like something worth spending any time to write an article about.But then new things kept coming out. It got worse. And worse. And worse. It’s like if a dumpster caught on fire, but then the fire itself also caught on fire.If you have been living under a rock, Equifax, a company that spies on the financial behavior of Americans and sells that intelligence to banks, credit card companies, and anyone else who’s paying, was hacked, and the culprits have everything they need to steal the identities of 143 million people.That’s bad, but everything else about it is worse. First, the executives kept the breach secret for months, and then sold stock just before the news went public. That is a move so utterly brazen that they might as well be a drunk guy with no shirt shouting, “Come at me bro! Come at me!†They’re daring the Securities and Exchange Commission to do something about it, and are confident that they won’t be punished.Speaking of punishment, the CEO retired, and he’ll be crying about this over the $90M he’s collecting this year. The CIO and CSO went first, of course. They probably won’t be getting huge compensation packages, but I’m sure they’ll land cushy gigs somewhere.Said CSO, by the way, had no real qualifications to be a Chief Security Officer. Her background is in music composition.Now, I want to be really clear here: I don’t think her college degree is actually relevant. What you did in college isn’t nearly as important as your work experience, which is the real problem- she doesn’t really have that, either. She’s spent her entire career in “executive†roles, and while she was a CSO before going to Equifax, that was at First Data. Funny thing about First Data: up until 2013 (about when she left), it was in a death spiral that was fixed after some serious house-cleaning and restructuring- like clearing out dead-weight in their C-level.Don't worry about the poor shareholders, though. Remember Wells Fargo, the bank that fraudulently signed up lots of people for accounts? They list Equifax as an investment opportunity that's ready to "outperform".That’s the Peter Principle and corporate douchebaggerry in action, and it certainly starts getting me angry, but this site isn’t about class struggle- it’s about IT. And it’s on the IT side where the real WTFs come into play.Equifax spies on you and sells the results. The US government put a mild restriction on this behavior: they can spy on you, but you have the right to demand that they stop selling the results. This is a “credit freezeâ€, and every credit reporting agency- every business like Equifax- has to do this. They get to charge you money for the privilege, but they have to do it.To “secure†this transaction, when you freeze your credit, the credit reporting companies give you a “password†which you can use in the future to unfreeze it (because if you want a new credit card, you have to let Equifax share your data again). Some agencies give you a random string. Some let you choose your own password. Equifax used the timestamp on your request.The hack itself was due to an unpatched Struts installation. The flaw itself is a pretty fascinating one, where a maliciously crafted XML file gets deserialized into a ProcessBuilder object. The flaw was discovered in March, and a patch was available shortly thereafter. Apache rightfully called it “Criticalâ€, and encouraged all Struts users to apply the fix.Even if they didn’t apply the fix, Apache provided workarounds- some of which were as simple as, “Turn off the REST plugin if you’re not using it,†or “if you ARE using it, turn off the XML partâ€. It’s certainly not the easiest fix, especially if you’re on a much older version of Struts, but you could even patch just the REST plugin, cutting down on the total work.Now, if you’re paying attention, you might be saying to yourself, “Hey, Remy, didn’t you say that they were breached (initially) in March? The month the bug was discovered? Isn’t it kinda reasonable that they wouldn’t have rolled out the fix in time?†Yes, that would be reasonable: if a flaw exposed in March was exploited within a few days or even weeks of the flaw being discovered, I could understand that. But remember, the breach that actually got announced was in July- they were breached in March, and they still didn’t apply the patch. This honestly makes it worse.Even then, I’d argue that we’re giving them too much of the benefit of the doubt. I’m going to posit that they simply don’t care. Not only did they not apply the patch, they likely had no intention of applying the patch, because they assumed they’d get away with it. Remember: you are the product, not the customer. If they accidentally cut the sheep while shearing, it doesn’t matter: they’ve still got the wool.As an example of “they clearly don’t careâ€, let’s turn our attention to their Argentinian Branch, where their employee database was protected by the password admin/admin. Yes, with that super-secure password, you could log in from anywhere in the world and see the users usernames, employee IDs, and personal details. Of course, their passwords were obscured as “******â€â€¦ in the rendered DOM. A simple “View Source†would reveal the plaintext of their passwords, in true “hunter2†fashion.Don’t worry, it gets dumber. Along with the breach announcement, Equifax took to social media to direct users to a site where, upon entering their SSN, it would tell them whether or not they were compromised. That was the promise, but the reality was that it was little better than flipping a coin. Worse, the site was a thinly veiled ad for their "identity protection" service, and the agreement contained an arbitration clause which kept you from suing them.That is, at least if you went to the right site. Setting aside the wisdom of encouraging users to put confidential information into random websites, for weeks Equifax’s social media team was directing people to the wrong site! In fact, it was directing them to a site which warns about the dangers of putting confidential information into random websites.And all of that, all of that, isn’t the biggest WTF. The biggest WTF is the Social Security Number, which was never meant to be used as a private identifier, but as it’s the closest thing to unique data about every American, it substitutes for a national identification system even when it’s clearly ill-suited to the task.I’ll leave you with the CGP Grey video on the subject:[Advertisement] Release!is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!
“Throw typed exceptions,†is generically good advice in a strongly typed language, like Java. It shouldn’t be followed thoughtlessly, but it’s a good rule of thumb. Some people may need a little more on the point, though.Alexander L sends us this code:
One of the major goals of many software development teams is to take tedious, boring, simplistic manual tasks and automate them. An entire data entry team can be replaced by a single well-written application, saving the company money, greatly improving processing time, and potentially reducing errors.That is, if it’s done correctly.Peter G. worked for a state government. One of his department’s tasks involved processing carbon copies of forms for most of the state’s residents. To save costs, improve processing time, and reduce the amount of manual data entry they had to perform, the department decided to automate the process and use optical character recognition (OCR) to scan in the carbon copies and convert the handwritten data into text which was eventually entered into a database.By By Aaron Logan
Chris M works for a “solutions providerâ€. Mostly, this means taking an off-the-shelf product from Microsoft or Oracle or SAP and customizing it to fit a client’s specific needs. Since many of these clients have in-house developers, the handover usually involves training those developers up on the care and maintenance of the system. Then, a year or two later, the client comes back, complaining about the system. “It’s broken,†or “performance is terrible,†or “we need a new featureâ€. Chris then goes back out to their office, and starts taking a look at what has happened to the code in his absence.It’s things like this:
James was getting anxious to land a job that would put his newly-minted Computer Science degree to use. Six months had come to pass since he graduated and being a barista barely paid the bills. Living in a small town didn't afford him many local opportunities, so when he saw a developer job posting for an upstart telecom company, he decided to give it a shot.We do everything in-house! the posting for CallCom emphasized, piquing James' interest. He hoped that meant there would be a small in-house development team that built their systems from the ground up. Surely he could learn the ropes from them before becoming a key contributor. He filled out the online application and happily clicked Submit.Not 15 minutes later, his phone rang with a number he didn't recognize. Usually he just ignored those calls but he decided to answer. "Hi, is James available?" a nasally female voice asked, almost sounding disinterested. "This is Janine with CallCom, you applied for the developer position."Caught off guard by the suddenness of their response, James wasn't quite ready for a phone screening. "Oh, yeah, of course I did! Just now. I am very interested.""Great. Louis, the owner, would like to meet with you," Janine informed him."Ok, sure. I'm pretty open, I usually work in the evenings so I can make most days work," he replied, checking his calendar."Can you be here in an hour?" she asked. James managed to hide the fact he was freaking out about how to make it in time while assuring her he could be.He arrived at the address Janine provided after a dangerous mid-drive shave. He felt unprepared but eager to rock the interview. The front door of their suite gave way to a lobby that seemed more like a walk-in closet. Janine was sitting behind a small desk reading a trashy tabloid and barely looked up to greet him. "Louis will see you now," she motioned toward a door behind the desk and went back to reading barely plausible celebrity rumors.James stepped through the door into what could have been a walk-in closet for the first walk-in closet. A portly, sweaty man presumed to be Louis jumped up to greet him. "John! Glad you could make it on short notice. Have a seat!""Actually, it's James..." he corrected Louis, while also forgiving the mixup. "Nice to meet you. I was eager to get here to learn about this opportunity.""Well James, you were right to apply! We are a fast growing company here at CallCom and I need eager young talent like you to really drive it home!" Louis was clearly excited about his company, growing sweatier by the minute."That sounds good to me! I may not have any real-world experience yet, but I assure you that I am eager to learn from your more senior members," James replied, trying to sell his potential.Louis let out a hefty chuckle at James' mention of senior members. "Oh you mean stubborn old developers who are set in their ways? You won't be finding those around here! I believe in fresh young minds like yours, unmolded and ready to take the world by storm.""I see..." James said, growing uneasy. "I suppose then I could at least learn how your code is structured from your junior developers? The ones who do your in-house development?"Louis wiped his glistening brow with his suit coat before making the big revelation. "There are no other developers, James. It would just be you, building our fantastic new computer system from scratch! I have all the confidence in the world that you are the man for the job!"James sat for a moment and pondered what he had just heard. "I'm sorry but I don't feel comfortable with that arrangement, Louis. I thought that by saying you do everything in-house, that implied there was already a development team.""What? Oh, heavens no! In-house development means we let you work from home. Surely you can tell we don't have much office space here. So that's what it means. In. House. Got it?James quickly thanked Louis for his time and left the interconnected series of closets. In a way, James was glad for the experience. It motivated him to move out of his one horse town to a bigger city where he eventually found employment with a real in-house dev team. [Advertisement] Otter, ProGet, BuildMaster – robust, powerful, scalable, and reliable additions to your existing DevOps toolchain.
I’ve had to write a few domain-specific-languages in the past. As per Remy’s Law of Requirements Gathering, it’s been mostly because the users needed an Excel-like formula language. The danger of DSLs, of course, is that they’re often YAGNI in the extreme, or at least a sign that you don’t really understand your problem.XML, coupled with schemas, is a tool for building data-focused DSLs. If you have some complex structure, you can convert each of its features into an XML attribute. For example, if you had a grammar that looked something like this:
"So there's this developer who is the end-all, be-all try-hard of the year. We call him Shoe. He's the kind of over-engineering idiot that should never be allowed near code. And, to boot, he's super controlling."Sometimes, you'll be talking to a friend, or reading a submission, and they'll launch into a story of some crappy thing that happened to them. You expect to sympathize. You expect to agree, to tell them how much the other guy sucks. But as the tale unfolds, something starts to feel amiss.They start telling you about the guy's stand-up desk, how it makes him such a loser, such a nerd. And you laugh nervously, recalling the article you read just the other day about the health benefits of stand-up desks. But sure, they're pretty nerdy. Why not?"But then, get this. So we gave Shoe the task to minify a bunch of JavaScript files, right?"You start to feel relieved. Surely this is more fertile ground. There's a ton of bad ways to minify and concatenate files on the server-side, to save bandwidth on the way out. Is this a premature optimization story? A story of an idiot writing code that just doesn't work? An over-engineered monstrosity?"So he fires up gulp.js and gets to work."Probably over-engineered. Gulp.js lets you write arbitrary JavaScript to do your processing. It has the advantage of being the same language as the code being minified, so you don't have to switch contexts when reading it, but the disadvantage of being JavaScript and thus impossible to read."He asks how to concat JavaScript, and the room tells him the right answer: find javascripts/ -name '*.js' -exec cat {} \; > main.js"Wait, what? You blink. Surely that's not how Gulp.js is meant to work. Just piping out to shell commands? But you've never used it. Maybe that's the right answer; you don't know. So you nod along, making a sympathetic noise."Of course, this moron can't just take the advice. Shoe has to understand how it works. So he starts googling on the Internet, and when he doesn't find a better answer, he starts writing a shell script he can commit to the repo for his 'jay es minifications.'"That nagging feeling is growing stronger. But maybe the punchline is good. There's gotta be a payoff here, right?"This guy, right? Get this: he discovers that most people install gulp via npm.js. So he starts shrieking, 'This is a dependency of mah script!' and adds node.js and npm installation to the shell script!"Stronger and stronger the feeling grows, refusing to be shut out. You swallow nervously, looking for an excuse to flee the conversation."We told him, just put it in the damn readme and move on! Don't install anything on anyone else's machines! But he doesn't like this solution, either, so he finally just echoes out in the shell script, requires npm. Can you believe it? What a n00b!"That's it? That's the punchline? That's why your friend has worked himself into a lather, foaming and frothing at the mouth? Try as you might to justify it, the facts are inescapable: your friend is TRWTF.[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!
Just last week, I was teaching a group of back-end developers how to use Angular to develop front ends. One question that came up, which did suprise me a bit, was how to deal with race conditions and concurrency in JavaScript.I’m glad they asked, because it’s a good question that never occurred to me. The JavaScript runtime, of course, is single-threaded. You might use Web Workers to get multiple threads, but they use an Actor model, so there’s no shared state, and thus no need for any sort of locking.Chris R’s team did have a need for locking. Specifically, their .NET backend needed to run a long-ish bulk operation against their SqlServer. It would be triggered by an HTTP request from the client-side, AJAX-style, but only one user should be able to run it at a time.Someone, for some reason, decided that they would implement this lock in front-end JavaScript, since that’s where the AJAX calls were coming from..
The progenitor of this story prefers to be called Mr. Syntax, perhaps because of the sins his boss committed in the name of attempting to program a spreadsheet-loader so generic that it could handle any potential spreadsheet with any data arranged in any conceivable format.The boss had this idea that everything should be dynamic, even things that should be relatively straightforward to do, such as doing a web-originated bulk load of data from a spreadsheet into the database. Although only two such spreadsheet formats were in use, the boss wrote it to handle ANY spreadsheet. As you might imagine, this spawned mountains of uncommented and undocumented code to keep things generic. Sin was tasked with locating and fixing the cause of a NullPointerException that should simply never have occurred. There was no stack dump. There were no logs. It was up to Sin to seek out and destroy the problem.Just to make it interesting, this process was slow, so the web service would spawn a job that would email the user with the status of the job. Of course, if there was an error, there would inevitably be no email.It took an entire day to find and then debug through this simple sheet-loader and the mountain of unrelated embedded code, just to find that the function convertExcelSheet blindly assumed that every cell would exist in all spreadsheets, regardless of potential format differences.[OP: in the interest of brevity, I've omitted all of the methods outside the direct call-chain...]
Ah, consumer products. Regardless of what the product in question is, therre’s a certain amount of “design†that goes into the device. Not design which might make the product more user-friendly, or useful, or in any way better. No, “designâ€, which means it looks nicer on the shelf at Target, or Best Buy, or has a better image on its Amazon listing. The manufacturer wants you to buy it, but they don’t really care if you use it.This thinking extends to any software that may be on the device. This is obviously true if it’s your basic Internet of Garbage device, but it’s often true of something we depend on far more: consumer grade routers.Micha Koryak just bought a new router, and the first thing he did was peek through the code on the device. Like most routers, it has a web-based configuration tool, and thus it has a directory called “applets†which contains JavaScript.Javascript like this:
Matthew H was given a pretty basic task: save some data as a blob. This task was made more complicated by their boss’s core philosophy, though.Never. Bother. The. Customer..“Right, but if the operation fails and we can’t continue?â€Never. Bother. The. Customer.“Okay, sure, but what if they gave us bad input?â€Never. Bother. The. Customer.“Okay, sure, but what if, by hitting okay, we’re going to format their entire hard drive?â€Never. Bother. The. Customer.As such, for every method that Matthew wrote, he was compelled to write a “safe†version, like this:
Tyler G.’s “engagement managerâ€, Sheila, had a new gig for him. The Global Chemical Society, GCS, had their annual conference coming up, and their system for distributing the schedules was a set of USB thumb-drives with self-hosting web apps.“You’ll be working with two GCS representatives, Jeff and Graham,†Sheila explained. “They’ll provide you with last year’s source code, and the data for this year’s schedule. You’ll need to wire them up.â€Later that day, the four of them- Tyler, Sheila, and Jeff and Graham- joined a Skype call. Only the top of Jeff’s shiny, bald head could be seen on his webcam, and Graham had joined audio-only.Sheila managed the introductions. Tyler started the discussion by asking what format they could expect the schedule data to come in.Jeff shrugged, or at least that’s what they guessed from the way the top of his head bobbed. “Graham, do you know?â€â€œI think it might be XML,†Graham replied, his voice muffled with static and saturated with background noise. “I can’t say for sure. We’ll send a preliminary data dump first.â€The BlobThe data arrived that afternoon, as a single XML file.The first time Tyler tried to open it, Notepad++ crashed in protest. After a few attempts, he finally coaxed the editor into letting him see the file. It had no uniform format. Individual fields might be HTML-formatted strings, indecipherable base64-encoded binary blobs (with no indicator as to what data was encoded), and even their plaintext encodings switched from 8-bit to 16-bit arbitrarily.As soon as Tyler explained to Sheila what a mess the data as, she called GCS reps for another video conferece. Jeff’s shiny pate bobbed around as he listened to their complaints. Sheila finally asked, “Can you do anything to clean up the data?â€â€œNot really, no,†Jeff replied. “This is how we get the data ourselves.â€â€œAbsolutely not,†Graham concurred.“We did this last year,†Jeff replied, “and we didn’t have any trouble.â€A Lack of SupportFor weeks, Tyler worked on an importer for the XML blob. He figured out what the base64-encoded data was (PDF files), why the encoding kept changing (different language encodings), and why some text was HTML-formatted and some wasn’t (the entries were copied from email, with some as HTML and some as plaintext).Jeff and Graham had no interest in the action items assigned no them, and continued to be the largest obstacles to the project. They offered no help, they changed their minds nearly daily, and when Sheila started scheduling daily calls with them, they used those calls as an opportunity to be sarcastic and insult Tyler.Sheila, who had begun the project in a cheerful manner, started balling her fists during each call with Jeff and Graham, now nicknamed “Statler and Waldorfâ€. After one particularly grueling call, she cursed and muttered dark things about “How do they get anything done?â€After weeks of frustration, pulled hair, and cranky calls, Tyler’s importer was finished. With a few days to go before the conference, they had just enough time to hand the software off and get the USB sticks loaded.During that morning’s video conference, Jeff and Graham announced that the format had changed to CSV. Sheila, barely keeping her voice level, asked why the format had changed.“Oh, the industry standard changed,†Graham said.“And why didn’t you tell us?â€Jeff’s shiny scalp tilted as part of an offscreen shrug. “Sorry. Guess we forgot.â€The Bitter EndThe CSV-encoded data, the final and official data-dump for the conference, arrived just one day before the app was due. It came in three files, seemingly split at random, with plenty of repetition between the files. It was all the same, insanely encoded data, just wrapped as CSV rows instead of XML tags.Tyler crunched his way through an all-nighter. By morning, the importer was finished. He sent the code to GCS’s servers, went home, and collapsed.The coming Sunday, attendees would arrive at the GCS conference. They would be given a USB stick, that they could plug into their laptops. The conference app would work perfectly, taking the fractured, convoluted data, and presenting it as a scrollable, interactive calendar of panels, presentations, and convention hall hours. Some graduate student, a lab assistant to a Nobel lauerate, would open the app and wonder:“This programming thing doesn’t seem like a lot of work.â€[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!
LINQ brings functional programming and loads of syntactic sugar to .NET languages. It’s a nice feature, although as James points out, it helps if your fellow developers have even the slightest clue about what they’re doing.
The exotic and exciting life of the world-traveling contractor wasn’t exactly what Angie had been expecting. It mostly meant living in a dreary apartment on the outskirts of some city in a short drive from an industrial park where she’d go to try and keep 30-year old C code and their new ERP from fighting to the death. Six months later, she’d be off to the same apartment near the same industrial park in a different country.When the crash came, it came hard. Hard enough that Angie ditched IT and got a temp job working in a customer service call-center for a greeting card company. She wasn’t exactly the best person on the phone, and nobody was giving her stellar marks for her cheerful demeanor during her quarterly review.What her boss did notice though, is that when she did order entry, it was accurate. This was surprisingly a big deal, because the number of orders with typos coming from the other reps was remarkable. “We really appreciate your attention to detail,†he said. He offered to make the temp job permanent and start working on some career advancement within the company.Angie took it. Over months, she spent less time on the phone, and more time putting her attention to detail to work: cleaning up order entry processes. Since Angie was a developer, she wrote some scripts to streamline the process and shared them with her team. Now, her boss was praising her attention to detail and her initiative.Within a few months, the dev team offered to bring her aboard. The salary bump was nothing to sneeze at, and they didn’t care that she knew C and Java and Ruby, but not their language of choice- C#. So she moved departments, and started working for Liam.Liam was the lead architect, and back in the early days, he was their only developer. Most of the software was home-grown extensions to their ERP, or their CRM. Since engraving printing plates was itself pretty complicated, he’d whipped up a program that could generate output to control the engraving system that made printing plates.Like a lot of smaller software teams in large companies that don’t view software as a priority, the code quality was… special. For any given program, most of the code was in one gigantic do-everything class, or worse, just in the main method. Version control was naming files “Foo.cs.old†or “Foo.cs.dontuseâ€, and release management was hitting “build†and copying the output to a network share.Liam, as the lead architect, didn’t want Angie wasting her time on the “big picture†stuff. “You’ve got such a great attention to detail,†he said. This meant she ended up being the SQL and regular expression expert who also tested the programs (often in production, because that was the only way to test). The result was far fewer bugs, fewer accidents from testing in production, and happier end users.The work was messy, but it wasn’t hard, and the card company didn’t really expect a lot from their software team. Angie appreciated sleeping in the same bed every night, and actually having a social life.Late on a Friday, the head of the company’s charity efforts burst into their cube-farm. The charity team had just run a major fund drive, and now needed to send out custom “Thank You†cards. There was only problem- the template they used (which drove Liam’s program to control the engraver) needed space for one additional line of text. “We need to get these running on the presses tomorrow so we can send them out next week!â€It was a four-alarm, hair-on-fire crisis, according to the charity chief. They were more than happy to provide dinner for the team who worked late, but it needed to be done. Since Angie was “detail orientedâ€, she drew the short straw, but she needed Liam’s help to get the changes made. “I don’t understand any of this code, and I can’t follow the logic.â€â€œWell,†Liam said, “I can’t say that I do, either.â€â€œBut you wrote it!â€â€œOkay, yeah, let’s take a look.â€Over some surprisingly high-quality Thai takeout, Liam and Angie did their best to trace through the logic of the code, understanding how it consumed the template and converted it into something the engraver could understand. Because there was a lot of code-reuse by copy-and-paste, they identified three places that needed changes.“Are you sure that’s it?†Angie asked.“Yeah, absolutely.â€â€œOkay… but how do we test this?â€â€œThe only way to test it is to send it to an engraver.â€â€œOkay, well… let’s go through it again and make sure it’s right,†Angie suggested.It was rubbing up against 10PM, and Liam had enough of that. “Let’s just run it and get the plate engraved. I’ve done this sort of thing a bunch, it’ll be fine.â€It wasn’t. By the time anyone had noticed, however, the plate was already off to the presses. The resulting run cost the company $10,000 in materials, and delayed the sending of the “thank you†cards by three days, which the charity team warned could seriously hurt their charity efforts in the future. The big bosses stormed into the development team’s office, demanding: “Who’s responsible for this?â€The bus was coming, and Liam was ready to throw Angie right in front of it. “We assigned that project to Angie,†he said. She was escorted out of the office that day, and on her exit paperwork, the reason for termination was "insufficient attention to detail". [Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!
Imagine, if you will, that you have 64-bits of data. From this 64-bits of data, you need to extract a nibble, which contains the value that you care about. Now, I’m sure you’re imagining an integer with some bitmasks to extract the data, which is a perfectly sane approach. Tomasz inherited some code from his company’s German office. It took the approach of taking the 64-bits and storing the 64-bits in an eight element byte array. Then, it extracted the values from that array with code looking like this:
In the U.S., individuals are expected to file federal and state tax returns once a year by April 15. The tax forms are quite complicated, and have all sorts of sub-forms and schedules to support and detail the numbers on the main form. The tax code of the U.S. is approximately 74,000 pages of special cases.For many items, the same data needs to be entered on multiple forms, usually as the starting point for different calculations that depend upon the same information; these are duplicated again on both federal and state returns. It follows that tax preparation software needs to put the relevant numbers in all the places that they are needed.Why? Because if it doesn't, the preparer needs to manually copy numbers to each of several places, leading to all sorts of omission/accidental editing issues. Just to make it needlessly complicated interesting, in many cases, the numbers need to be transformed via some formula before copying, and the formulas vary from form to form, and from state to state. Complicating this process is the fact that tax forms stating your earnings from employers and financial institutions can be re-issued with new values if tax laws are changed too close to the end of the year. This means that new numbers need to be entered, propagated, calculated and checked on your tax forms, usually after they were initially prepared.If you're preparing your own tax returns, then they probably aren't all that complicated, and there isn't too much copy/pasting to be done. If you have multiple businesses in different states/countries, multiple properties, use multiple banks/brokerages, etc., then there are many, many forms and worksheets detailing the same information, leading to a whole lot of copy/pasting of data. When the complexity of what must be prepared gets beyond a certain point, you go to a professional preparer who will just enter your information once, and the very expensive professional tax software will put the numbers on all the relevant forms automatically. This way, it only takes a few minutes to enter the data, hit calc and a mountain of completed tax forms spews forth from the printer.Julie is a highly experienced accountant, and was partnering with a large, national chain of tax preparers that used their own in-house tax preparation software. They had a large IT team of several hundred assorted developers and support staff. Each year, they'd wait for Congress to decide on changes to the tax laws, and then begin the process of implementing those changes in their software so that they could get it to the accountants in time for tax season.Unfortunately, Congress doesn't take programming time into account when they bicker back and forth over changes to the tax law. Last year, Congress passed a whole slew of changes at the very last minute, leaving insufficient time to implement all the changes in the software (at least not without the usual magic happens here programming of experienced developers). The solution that the blockhead managers came up with was to only implement part of each change prior to distributing the software. They called this "being agile".When Julie started to prepare tax returns, she realized that the inputs to the same calculation were different on different forms. The numbers were not being propagated to the places that needed them, or worse, were being propagated incorrectly. Additionally, some calculations only performed the first 10 steps of a 12 step form. Upon raising bugs, she was told We know about these issues and have no plans to fix them!Wait, manually propagating numbers and checking every calculation defeats the purpose of using the software in the first place; the numbers can't be close, they have to be exact and consistent across forms!The debate about the importance of exact and correct calculations went back a forth and while until it was escalated sufficiently high to warrant an official response:Our software is only intended to guide you in the general direction of preparing tax returns. It only needs to be reasonably functional, not "useful".Julie, at this point, was desperate, so she contacted an insider at the company. "Is there *any* chance this is going to be working in time for tax season?""Oh, man… noooooo," her contact said. "A lot of the developers are on contract, so somebody ran the numbers comparing the hours of development time against paying penalties from audits. They figured out that they could do about… 90% of the changes. It's cheaper to just pay the penalties later."Julie is no longer partnering with this conglomerate. [Advertisement] Application Release Automation for DevOps – integrating with best of breed development tools. Free for teams with up to 5 users. Download and learn more today!
Melody got tapped to do a code review on a pull-request from a veteran team-member. It was… an interesting PR, in that very, very little changed. The code was terrible before anyone touched it- for example, the C-file started with 355 lines of variable declarations inside of the main method.It was, in fact, down around line 354 where Melody noticed the change.