The Daily WTF

Part of Python’s appeal is its rich class library. The old XKCD about import antigravity sometimes doesn’t feel that far off. Combined with a few third-party libraries, like NumPy, you can do a lot with very little code.Of course, that feels a bit like magic. As Python gurus like to say, “Explicit is better than implicit”. One of Mark’s co-workers took this perhaps a bit too far, when they started adding this import to every file:

A long time ago in my "C" programming days, I learned that when you code up anything that depends on any sort of external data, be it a file, database or socket, you should be paranoid and do it defensively. After all, you can't control those things and there's no guarantee that they will always work the way you hope. Sometimes you care about every possible error code; sometimes just success or failure. The point is to check the result of what you tried to do. Fast forward through several years of C++ and ten years into Java, and our boss calls us into the office.

Tis the season to think of trees.Our Anonymous submitter has a program with a tree in it, and it’s a relatively big one: 7 levels deep, with 200,000 leaf nodes. Oh, and it’s managed in client-side JavaScript. In other words, it’s the sort of thing you really want to make sure you’re accessing efficiently.Since it’s a tree, and since in this case, all the nodes are doubly linked (each child contains a reference to its parent), it’s easy to get from any one node to any other, with a minimal number of steps. There’s never a good reason to traverse the entire tree.Which is why Anonymous facepalmed when they found this code:

You know the story. A report is spitting out the wrong dates. Someone gets called in to investigate what’s wrong. After digging through piles of deeply nested SQL queries and views and trying to track down the problem, it turns out someone wrote their own date handling code which is wrong.Darin P found the code this time.

As programmers, we all need to fix bugs. As experienced programmers, we recognize that sometimes, the ability to fix one bug depends upon first fixing another bug. Managers, on the other hand, don't always get that simple concept.At the beginning of my career, I worked for Initrode where I wrote software to run a test-station that diagnosed assorted electronic components of jet fighters. Initrode acted as a government-supplier of the test station to another government contractor (LUserCorp) that used the station to write the test sequences to diagnose electrical faults. If the test station hardware malfunctioned, or there were bugs in the software that made the electronics tests fail to work properly, then LUserCorp could use that as an excuse for time and cost overruns. If that happened, then the government would penalize Initrode to recoup those costs.Over time and several releases of the hardware and software, a series of hardware faults and software bugs managed to creep into the system. Since LUserCorp was running behind schedule anyway, they decided that they'd use this as an excuse to hit the government for more time and money. Naturally, the brass at Initrode fought back because they didn't want to take the financial hit.After lots of political back-and-forth, an official prioritized bug list was created, and it was mandated by LUserCorp that the bugs had to be fixed in the order in which they appeared on the list.To this end, me and another junior developer were sent to LUserCorp to act as a Tiger Team. Basically, this means we are in a locked room, alone with the test station. The LUserCorp people were not allowed in the room with us. We brought the source code on our own disk pack which one of us had to be with at all times. This meant that if we went to lunch, or both had to hit the restroom at the same time, we had to power everything down and take the disk with us.The list of bugs to be addressed was provided to us only after we were on site. The first and most important bug on the list was something I had coded that had an off-by-one error in a nested loop, that only appeared at the end of the third iteration of the outer loop. Since each inner loop processed and printed the 6-line result of each of 4K tests (to a very slow thermal printer), it took 6 hours to print out 3*4K*6 => 72K lines of test results.Our software also had stop-at-test-n functionality. We noticed that bug number 2 on the list was in the stop-at-test-n functionality (it prevented that feature from being used).In the ideal case where all other functions were working, we'd do:

Brian B stumbled across a bit of code to generate UUIDs. Seeing that tag-line, I was worried that they invented their own UUID generator. The good news, is that they just use java.util.UUID. The bad news is that they don’t understand how if statements work.

Asynchronous programming is hard. Because it’s so difficult, developers are constantly trying to find ways to make it simpler, whether it’s promises or callbacks, or the async/await pattern. It gets more difficult when you need to deal with handling exceptions- when a task fails, trying to recover from that failure in a separate thread is an extra special challenge.Which brings us to Betty’s co-worker. Using C#’s Task objects, which tie into the async/await pattern, they wanted to simply ignore any exceptions thrown by one of those tasks. That’s your first WTF, of course. Their approach, however, is a larger one:

It's easy to use dictionaries/maps to solve the wrong kinds of problems, but deep down, what's more elegant than a simple hashed map structure? If you have the key, fetching the associated value back out happens in constant time, regardless of the size of the map. The same is true for inserting. In fact, hash maps only become inefficient when you start searching them.Concetta recently started a new job. Once upon a time, a developer at the office noticed that the user-facing admin pages for their product were garbage. They whipped up their own internal version, which let them accomplish tasks that were difficult, time-consuming, or downright impossible to do in the "official" front end. Time passed, someone noticed, "Hey, this is better than our actual product!", and suddenly the C# code that just lived in one folder on one developer's machine was getting refactored and cleaned up into an application they could release to the public.Concetta wasn't surprised to see that the code wasn't exactly great. She wasn't surprised that it constantly threw exceptions any time you tried to change anything. But she was surprised by this:

The line between objects and maps can sometimes get a little blurry. In languages like JavaScript, there’s really no difference between the two. In Python, the deep internals of your classes are implemented essentially as dicts, though there are ways around that behavior.In a language like C#, however, you’ve got types, you’ve got property definitions. This can offer a lot of advantages. When you layer on features like reflection, you can inspect your objects. Combine all this, and it means that if you want to serialize a data object to XML, you can usually do it in a way that’s both typesafe and generally doesn’t require much code on your part. A handful of annotations and a few method calls, and boom- any object gets serialized.Unless you work at Kara’s office, of course. When they have an object that requires serialization, they must inherit from SerializableObjectBase.

Hiring people you know is a double-edged sword. You already have an established relationship, and shared background, and an understanding of how they think and act. You’re helping a friend out, which always feels good. Then again, good friends don’t always make good co-workers, and if you limit your hiring pool to “people I know” you’re not always going to find the best people.Becky’s boss, Chaz, tends to favor his golf buddies. One of those golf buddies got hired, developed for a few months, then just gradually ghosted on the job. They never quite quit or got fired, they just started coming in less and less until they stopped coming in at all.Chaz passed the code over to Becky to fix. By “passed”, that is to say, he emailed her a zip file of the source, which was the only working copy of the code. There was no documentation, no source control, certainly no tests, and no description of what the program was actually supposed to do. “Just fix the bugs,” Chaz said.

Brian recently started a new job. That means spending some time poking around the code base, trying to get a grasp on what the software does, how it does it, and maybe a little bit of why. Since the users have some complaints about performance, that's where Brian is mostly focusing his attention.The "good" news for Brian is that the his predecessors were "geniuses", and they came up with a lot of "clever" solutions to common problems. The actually good news is that they've long since moved on to other jobs, and Brian will have a free hand in trying to revise their "cleverness".

Jay J had been helping a friend with the job hunt. As an experienced developer, with a strong network, Jay had a sense of who was hiring and what jobs were promising. One of his connections turned up a lead at Initech. Jay pointed his friend in that direction, and wished for the best."They won't let me apply," the friend explained when Jay asked how things were going. "Here, try it. These are my details. This is the link for the web application. Fill in the form and see what happens."The first thing Jay noticed when pulling up the form was that it was clearly built from a toolkit of reusable widgets. The way validations appeared, the way the page laid out- this was a bolt-together HR application built out of some enterprise solution. Nothing inherently wrong with that approach- it can save time by using reusable components.The trick, of course, is that reusable components have to be used correctly.Jay filled in the from. Name. Address. Birth date. Attach a resume. Re-enter 90% of the information that's already on the resume. Click submit.

Buford was a contract developer working at a mid-sized financial firm. He had just wrapped up a lengthy project and was looking for something new to sink his teeth into. Tanner, the manager in his area, tasked him with moving their implementation of Jenkins into "this great new thing they call The Cloud."Tanner recently returned from a conference with a bunch of swag from a company called PuffyCloud. They claimed to have the easiest cloud-based implementation of Jenkins in the business. "It's pretty much just a copy-paste job according to this whitepaper they gave me. Take a look, create some user stories, and have it done by the end of the next sprint," Tanner instructed.Buford opened up the whitepaper and soon found that PuffyCloud was certainly full of puffery. They boasted about how their approach was "production-ready for an enterprise environment" and "dozens of organizations have revolutionized their systems with PuffyCloud and the magic of our simple Docker Compose code."After getting through several pages of drivel, Buford already had a better, cheaper way in mind. He returned to Tanner's office to explain, "Look, I don't know what these PuffyCloud guys will charge, but I'm certain I can get the same result for only the cost of my time. I can make my own Docker script to install everything on a cloud-based server that we control without a costly middle-man license from them."Tanner furrowed his brow before responding, "Bah! Doing this yourself will take way too long. I guess you didn't hear me the first time I explained this PuffyCloud thing. Copy. And. Paste. Do that and you'll finish easily with time to spare in the sprint.Buford went back to reading the mind-numbing whitepaper since Tanner was clearly insistent on going with PuffyCloud. Ten minutes later, he finally got to the "production-ready for an enterprise environment" script:

In early 2002, Bert landed a job at Initech, which released its own protocol analyzer tool. Technically, they released a whole slate of protocol analyzers, data loggers, analytics tools, with overlapping features and business cases. Their product catalog had grown over the years, and was a bit of a thicket.The team Bert joined was a decent mix of talent. Some, like him, were new to the industry. There were some more experienced devs, who knew the product and the low-level internals their software needed to navigate. And then there was Herb.Herb was every stereotype of the 70s era hacker, aged up into the 00s. He had a collection of buckling spring keyboards, thought 64k of RAM was an incomprehensibly large amount, could code Assembly faster than most of the rest of the team could do C, and he knew every line of code in every product, and knew exactly what it did and why it was there.Jack, the team's project lead, was Herb's polar opposite. Young, with a shiny new MBA, a focus on the kind of networking skills that didn't involve CAT5 cable, and absolutely no useful knowledge about anything. "Our team exists to serve the business," Jack was fond of saying. "So whatever the business needs, we do. No. Matter. What."The business decided that their complicated product catalog was hurting their sales. Customers were confused about which product to purchase, some tools were receiving barely any updates, and there really wasn't enough opportunity to "upsell" additional functionality, since every product already had more functionality than most users needed. As part of this process, two older protocol analyzers and an unrelated data logger would be merged into a single product.Well, the business needed something, and Jack was confident that his team would make it happen. Since this was "just" merging software together, it couldn't possibly be all that much work. It was nothing more than some mechanical repackaging, a little tweaking, and the biggest part of the work was probably updating the help files. Jack didn't need to talk to anyone on the development team to decide how they were going to serve the business.The new product, Initech's INILYZER, would release in four months. Since it was replacing three existing products, those could be put into end-of-life. Since the INILYZER was definitely coming out in four months, they could be EOLed in four months as well.Jack's next task was to communicate this to his team at the project kickoff meeting. Jack loved to turn project kickoffs into his own personal motivational speaking exercise, because "That's how you build team morale!" So the kickoff meeting launched with dramatic intro music (provided by a tiny CD player in the corner), a rousing speech, and then the pièce de résistance: freshly made t-shirts for the whole team, emblazoned with: "The INILYZER: August 30, 2002"August 30th was Jack's predetermined release date."Well," Jack said, gleefully doing his best impression of a stadium t-shirt cannon, "does anyone have any questions before we go out and totally crush this awesome project?"Most of the team cringed, except for Herb. He held up a shirt and pointed at the date. "Yeah. Where'd this timeline come from?""It's driven by business need," Jack said."Okay, but you're never going to hit this date.""Um, Herb, we're doers at this company. We have a date, and we are going to hit that date."This meeting was on a Friday. Over the weekend, Bert and the rest of the team chucked their t-shirts into the rag pile and basically forgot about them. But not Herb. On Monday, Herb walked into the office, proudly wearing his project shirt.With a small modification, scratched in with red marker. It now read: "The INILYZER: August 30, 20023".Jack didn't notice it until about 10AM, and when he did, it prompted a meltdown. He started screaming at Herb about insubordination, respect, and team spirit. "I want that shirt off, right now! Or you're fired!""I didn't bring anything to change into," Herb said.Jack stomped off, fuming.The project continued. August 30th came and went. Summer turned to autumn, autumn into winter. Six months after officially EOLing major products without having a replacement ready to go, there was a management purge throughout the company. Jack got fired, and there was much rejoicing.Jack's replacement was Emma. It didn't take long for Emma to get a read for the new team, understand the source of the mess, and start working to get it as cleaned up as it could. When upper management started looking at Bert's team as another possible cut, it was Emma who made sure they understood that they were doing the best they could with an unrealistic timeline. Nobody else from the team got fired.August 30th, 2003, the Initech INILYZER shipped. Emma, having heard about the now infamous t-shirt story, had a small plaque made up to celebrate Herb's accuracy.Of course, with the unrealistic timeline, unrealistic goals, and management problems, the INILYZER shipped as a pile of barely usable crap. As it turns out, pretty much no customer wanted those products glued together in any way, making the whole effort pointless. It was a technical and commercial flop. The project to replace it started the very same day as its release.Emma was the PM. Unlike her predecessor, she talked with the developers, especially the most experienced team members, to figure out what the timeline and scope actually were. The project kickoff involved no t-shirts. [Advertisement] Forget logs. Next time you're struggling to replicate error, crash and performance issues in your apps - Think Raygun! Installs in minutes. Learn more.

Circa 2009, Marylou took a position at an e-commerce firm. It was a small company, which had done its startup phase right in the midst of the DotCom Crash, but somehow made it out the other end with a steady revenue stream.That itself turned out to be a problem, as once you turn a profit, investors stop investing. The company founders spent the next few years looking for investment to expand, and when they finally got it, they went on a hiring spree. That's where Marylou came in.The company had been founded by developers, but over the years, they took on more and more leadership responsibilities, and handed off the development responsibilities to Joel. Joel was Marylou's new boss."So, the first thing you'll want to do is install our CVS client and our Eclipse plugin," Joel explained, working with Marylou to get her development box set up.Marylou reached for the Eclipse plugin manager and started searching for a CVS plugin. "No, no," Joel said. "Our CVS client and plugin. I forked CVS so we can use a customized version."Marylou paused, waiting for the punchline. Joel took this silence as an opportunity to steal the mouse and start clicking around on her machine for her. "Y'see, we needed a system that let us easily stage releases before going into production. So I built one.""Um… most organizations use a separate tool that sits on top of source control for that.""Yeah, but this way, it's integrated!"Once Marylou was set up, and had the code pulled to her machine, Joel started showing off some of his "clever" solutions. "Like, I wrote this using recursion!"

Rob M caught a ticket for a bug in a C# application. Specifically, when the user picked an item off a menu, that item wouldn't get highlighted, thus defeating the purpose of the menu. Strangely, the code hadn't been touched since its first commit, back in 2015.

Setting aside cross-browser quirks, CSS is a fiendishly complicated specification. There’s a lot to it. From styling rules and how they interact with the DOM hierarchy, to the complexities of using selectors to navigate the DOM- it’s a complex tool that is also very powerful. I mean, it’s Turing complete.Shiv works with a self-proclaimed “CSS Ninja”- yes, that was actually in their resume when they got hired. They were hired on the strength of their portfolio- it looked very nice. Unfortunately, the implementation left something to be desired.For example, imagine you had twenty elements on a page which needed to be styled the same. You might choose to apply a class attribute to them, and create a single styling rule for that entire class. But that’s not what a CSS ninja does.

Today's anonymous submitter has this to say about this code: "It works fine, it's just... clever."I'm not certain about the relative cleverness of this solution, myself.

"If at first you don't succeed, try, try again."We have all encountered situations where we need to attempt an operation, with full knowledge that the operation might very well fail. And if it does, we should retry it. Usually after a delay, usually with a maximum number of retries.Today's anonymous submitter inherited a site-scraping application. Already, we're in dangerous territory- site-scraping is inherently fragile, dangerous code. It's more dangerous, of course, when it's written by the culprit behind this:

Today’s anonymous submitter is an experienced Python developer. Their team is significantly less experienced. In the interests of training, someone suggested, “Perhaps we should buy books for everyone.” Our submitter was handed a stack of possible books, and asked to pick the best one to help the team mature.One book spent some time discussing types, and the conversion between them. By way of example, it provided a handy method which would turn a string into a floating point number:

Rina works in an environment which tends to favor crunch. It's a bit of a feast or famine situation, where they'll coast for months on a pretty standard 9-5 schedule, and then bam, suddenly it's 18 hours days.No one particularly likes those periods, and code quality takes a nosedive. Co-worker Rusty, though, starts making utterly bizarre decisions when stressed, which is how Rina found this line while doing a code review:

Last week we saw an attempt to reinvent the ceil function. Tina raised a protest: "6 lines to re-implement a ceil function? We can do better."

Banks are uniquely identified via a “SWIFT Code”. It’s an ISO Standard. Having an accurate SWIFT code for a given bank is of vital importance to anyone doing financial transactions. With mergers, moves, new branches, and so on, the SWIFT codes you do business with won’t change often, but they will change.Thus, Philip wasn’t terribly surprised when he got a request to update a pile of SWIFT codes. He couldn’t make the change via the database, though, as no one had permission to do that. He couldn’t edit it through an application UI, because no one had ever built one.Instead, he had to check out a Java web service from SVN, modify the hard-coded values in the code, deploy the service to production (an entirely manual process), and then invoke the web service via cURL.

Clia was handed a pile of legacy code and told to upgrade it, but with a very important rule attached: the functionality couldn't change. Any change could break someone's workflow, and thus in the upgraded system, even the bugs had to be reproduced.Unlike most "legacy" code, this wasn't all that old- it was written in C#. Then again, C# is old enough to drive, so maybe it is old. Regardless, C# has utility methods, like, say, a ceil function. At no point in C#'s history has it lacked this basic functionality.That, of course, doesn't stop people from reimplementing it badly.

Percy's employer is an "enterprise vendor". They have a variety of products all within the "enterprise" space. Like most enterprise products, they're sold on the strength of the marketing team's ability to claim with a straight face that they're secure, robust, reliable, and performant.While the company offered a "cloud" solution for their enterprise suite, the real money was in the on premises version. With on-prem, any updates or upgrades were almost guaranteed to break the entire environment unless the customer shoveled huge piles of cash at the company to get a specialist to come out and do the upgrades.That was good for the company, but wasn't so great for the specialists. Simple patches could take weeks to perform, not because of any trickiness or difficulty, but because each upgrade step could take days to run.Percy found out about this when sitting in on a project review meeting."For our instant messaging engine," one of his peers, Elizabeth, explained, "just handling the room policy records can take upwards of ten hours at some sites. That's one SQL statement. God forbid the transaction fails for some reason.""How could it be that bad? How many records is it?" Percy asked."Only about 100,000. The problem is the way the data is structured." Elizabeth pulled up the schema.As an enterprise product, the messaging system prizes configurability and customizability over utility, performance, and common sense. It's not designed to solve a problem, but instead to allow customers to solve problems they didn't know they even had, mostly by inventing them. As a messaging/chat system, they had a concept of "rooms", and the "rooms" could have every aspect about them customized. From how usernames display in the chat, to whether files can be sent, to even whether or not copy-paste is permitted. Of course, the same interface which configured those cosmetic flags also configured role-based permissions, transport layer settings, and whether the messages were compressed before sending to the server.And with each version of the product, the specific set of fields could change.Those of you who work in RDBMSes are likely imagining a vast field of relations bound together by foreign keys and indexed to optimize likely query paths. Those of you who are more on the NoSQL side of things are likely imagining a sparse JSON/BSON document, possibly a heavily denormalized representation of the dataset.Those of you who know enterprise products, or who have read this site before, know what it actually was: gigantic XML documents stored as CLOBs in the database. To add insult to injury, they used SQL Server on the backend, which has had a native XML type with optimized query operations since the mid-2000s.Percy pulled up the schema on his own machine to explore it. At the center of the schema was a ChatRoomPolicy table. This table had three fields- a PolicyId and the RoomPolicy, which was the main gigantic XML document. In addition to that XML pile, you also had a Visibility column, which contained an XML document which contained ChatRoomVisibility elements- essentially little things, like whether attempting to send a message included a "Cancel" button, or if users could see a "Mute" button to silence the notifications.Then, of course, you had the ChatRoom table, which had a foreign key reference to the policy which controlled the chat room."So, yeah, when we do an upgrade which alters the schema of the RoomPolicy documents," Elizabeth explained, "we have to read every row, parse the XML in our upgrade script, generate the new XML, and then store it. That's if the XML changes- sometimes an upgrade only renames one key, and not every document contains that key."Percy had seen some pretty horrible things in his section of the product, but nothing quite like this. "Well, at least you can just map the XML document to a class, and mostly automate the changes, right?"Elizabeth laughed at him. "If we did that, then our customers couldn't make up their own XML keys, and then add their own procedures in our proprietary scripting language which reads them, and then I wouldn't constantly get tickets because some PM at a three-branch bank in East Pilsbury decided they wanted a new field in their chatroom and it happens to collide with one of the fields we already use."More specifically, the XML didn't have an associated schema. The upgrade script had to be able to handle any arbitrary XML document, identify the parts that the script cared about, if they existed, modify them, if necessary, hopefully without breaking anything the customer had customized, and then actually do the update.The resulting upgrades could, on a good day, get about 3 records processed every second. With some organizations, the pile of policies applied to chat rooms could extend into the hundreds of thousands (as "private chats" also counted as chat rooms and had policy documents attached). Thus, ten hours just to upgrade a single module of a single service in their enterprise software portfolio. And that's just the upgrade task."Wow, I can't imagine how long testing must take," Percy said.Elizabeth laughed, again. "That's up to the client to schedule. They usually don't." [Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

In our industry, we all know that managers cause problems when they try to, well, manage. This invariably causes us to get frustrated. Sometimes when we rebel and try to force them to do the right thing, we are the ones that pay for it with our jobs. Sometimes, they get impatient at our mortal lack of $Deity-level skills to make the magic happen fast enough for them, and we pay for that with our jobs as well.Occasionally, even though it seems as though managers never pay for their mistakes, Codethulu smiles upon us and gives us a glimpse of a Utopian world...Back in the mid 1990s, Dennis worked for privately-held Web Accessible Network Kernal, Inc. (WANKCO), a small manufacturer of large automated network test equipment. WANKCO had a couple dozen employees and had not yet developed any big-company policies. Specifically, they did not have the one where you have to use your vacation time or else lose most of it at the end of the fiscal year.Dennis was very busy as the lead engineer for a very important and very late product that was forecast to make bundles of money for WANKCO. He had been working evenings and weekends designing hardware and firmware and hadn't had a vacation for well over a year. He had lots of vacation time accrued and planned to take a nice long trip after the product was released and seen through its inevitable birthing pains.One day, WANKCO was purchased by publicly-held Initech. WANKCO quickly inherited a small herd of PHBs, and the alpha PHB informed the minions that, among other PHB things: 1) Vacation carryovers at the end of the fiscal year would be capped according to Initech policy, and 2) Dennis's project was a Critical Corporate Priority and must be finished at the earliest possible date or dire consequences would result for the non-pointy-haired among them. Development staff were to be put on a mandatory fifty-hour work week until the project was complete. A sign-in sheet was provided so they could log their arrival and quitting times each day. Several of the developers happily complied by cutting their work week down to the mandated fifty hours. Apparently, pointiness keeps some people from picking up on the obvious.The end of the fiscal year was already near when these announcements were made, and the Critical Corporate Priority was within mere weeks of completion. Dennis asked the PHBs for a one-time exception to the vacation carryover policy so that he could see the project across the finish line without sacrificing most of his accrued vacation time. They refused. Thus, with PHB-blessings, Dennis took off on a five-week vacation. With nobody else being qualified to take over his part of the work, the Critical Corporate Priority was delayed by another five weeks. The PHBs couldn't understand why the mandated fifty-hour work weeks did not advance the completion date by a single minute, although Dennis was sure that it made the PHBs appear more pointy than usual.In spite of finger pointing, insinuating emails and general PHB-isms, Initech and its WANKCO subsidiary survived the delay and the product was eventually shipped.A parade of PHBs regularly cycled through WANKCO on rotation. After ten years and several changes of ownership, WANKCO went private again. In the process the PHBs were jettisoned, illustrating that not all tales of managerial WTF have an unhappy ending.Sometimes you just have to wait 'em out. [Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

Here’s a philosophical question. Let’s say you’re searching an array. Is it clearer to use a for loop and break when you find the element, or is it better to use a while loop and break if you hit the end of the array?Most of us would likely use the for loop, but it wouldn’t be wrong to use the while- maybe just unexpected.But what if you had forgotten that while loops even exist? An anonymous submitter found this, and distilled its essence for us:

Terje worked for an IT firm that serviced the purchasing department of a global corporation. To manage purchases, the department used an enterprise shipping and warehousing system that shall be called BLA to protect the guilty. The system ran on a Citrix farm in Norway with all the most impressive resources at its command.Just after a major upgrade of BLA, sporadic performance issues began cropping up in the Brazil and Perth offices. When users tried to generate new reports, BLA would hang for several minutes. In Norway, Terje and his colleagues couldn't reproduce the problem. They went through every reasonable troubleshooting step, but failed to isolate the root cause of the issue. The vendor of BLA, also based in Norway, couldn't duplicate the issue, either. Months passed with no resolution in sight.Lest one should think that everything was sunshine and roses in Norway, the users there ran into their share of quirky issues as well. A common one was the "Printer not found" error message when a user tried to generate a report. Usually, this happened whenever a user removed or inserted their laptop into a docking station while logged into BLA. Simply rebooting Windows while the laptop was docked fixed the issue. The vendor explained that BLA had to have a default printer set within the local Windows environment so it could find out how big a piece of A4 paper was. This would ensure that BLA would always use the correct paper size for reports. This behavior seemed odd to Terje, but he kept it in mind.One day, when he was fielding yet another performance issue from Brazil, the above tidbit occurred to Terje and bestowed a spark of inspiration."Try this for me," he told the user on the phone, sitting up straighter in his chair. "Log out of Citrix, then set your default printer to a PDF creator.""What?" the user asked. "What does that have to do with anything?""I know it sounds weird, but can you just give it a try?" Terje pleaded. "If it doesn't help, you can change the setting right back.""OK ..." the user replied, skeptical.Sure enough, once the user made the change and logged back in to Citrix, he was able to generate a report almost instantly."How did that work?!" he asked."You're connecting to our Citrix server in Norway, opening BLA, and attempting to generate a report," Terje replied. "When you do that, BLA goes, 'How big is size A4 paper? I'd better ask the default printer.' In your case, the default printer was pointing to a device in your office in Brazil. So BLA was connecting from Norway to that printer halfway around the world, and was waiting for it to tell it how big A4 is. That's why everything was so slow!" Terje slapped a hand against his desk in triumph."How odd!" the user said. "I'm glad it works now, though!"Terje was even happier to have cracked that enduring mystery. He got to work assembling a knowledge base document and compiling the evidence needed to file a change request with BLA's vendor. [Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Megan K’s organization does the sane and reasonable thing: they cache all of their dependencies locally, and their CI/CD process pulls from the local cache. Since they’re in the Python world, this means pulling from PyPI using Bandersnatch.Someone set up the Bandersnatch job. Someone linked it into their CI/CD process. No one wanted to claim credit when it started failing. Every five minutes, Bandersnatch tried to contact PyPI. Every five minutes, it got a StalePage error, specifically complaining about a “stale serial”, and sent an alert email to the entire dev team. Every five minutes, the entire dev team ignored the error. For ten days.The office turned into a tense standoff. No one wanted to talk about the errors, since the first one to bring it up was going to be the one who had to fix it. Each day could be underscored by Ennio Morricone.Megan caved first. Instead of using rules to filter her inbox, she manually deleted each message. Eventually, sick of getting spammed, she did some research, and shared her findings with the developers.Megan started where anyone would start debugging HTTP requests: with curl. The curled the failing URL, first from her machine, and when that worked, from the Bandersnatch machine. That also worked. Even as a Python script, running on the same machine, sending a very similar request, failed.Megan fired up a Python REPL and imported the requests library, which is used for sending HTTP requests. She sent a request, and that failed. curl always got a fresh serial, requests always got a stale serial. Obviously, there was something different about the requests.Megan checked the headers, and found two key differences: the User-Agent and the Accept-Encoding headers. Thinking the obvious, she tried setting curl to use the same User-Agent as requests- and it kept working. Changing the User-Agent in requests didn’t fix the problem.But changing the Accept-Encoding did. If Megan sent a request with Accept-Encoding: gzip, deflate, the PyPI loadbalancer routed her to a stale cache server, every time. Accept-Encoding: '', on the other hand, always got a fresh server. By default, requests used gzip, deflate. Bandersnatch, in turn, used the default settings for requests.That of course, wasn’t enough. Eventually, the cache server that Bandersnatch was getting routed to would also become stale, so Megan had to add an arbitrary (and highly variable) parameter to her requests- specifically, a URL parameter called cache-bust which was always set to the current timestamp.An issue was raised with PyPI, which is still open at the time of this writing. Obviously, it’s some simple misconfiguration in the load balancing and caching, or perhaps an intentional configuration based on bad assumptions, but the moral of the story: Megan was glad they were caching all of their dependencies locally, just in case PyPI ever went really wrong. [Advertisement] Ensure your software is built only once and then deployed consistently across environments, by packaging your applications and components. Learn how today!

Justin Self inherited an internal build tool. Rome may not have been built in a day, but this tool was. It “simplifies” provisioning development environments, claiming machines in the environment for certain tasks, and so on.

Let's say you have an enum. You have an array of objects, where one of the fields is of that enum type. You want to filter the array of objects where the value of the enum is PAYMENTMETHOD.You also hate equals signs.Jacqueline found IntelliJ screaming out a litany of warnings, and while tracking down the reasons, she found this bit of Java:

Rodrigo was but a simple software development intern eager to prove himself. He would always volunteer for the menial tasks that nobody else wanted to do. "Nobody else" mainly consisted of Justin, Rodrigo's supervisor. Justin wasn't a big fan of doing stuff so he was glad to have an intern that was ready and willing.Justin got a request from the network administrators to create a system status application to interface with their servers. They wanted to receive alert emails every hour on the hour if anything on the servers had a conniption. If everything was ok, maintain radio silence. To do this, a simple app would need to be created to pass system health check results to Pushbullet, which would take care of sending the alerts. Rodrigo didn't even wait for Justin to finish. "I'll do it!"Rodrigo was excited to get his development environment set up with an instance of Pushbullet so he could start slinging emails. He spent the week coding and extensively testing his app and by Friday, his phone buzzed with an alert. Something about having his code send something out to his phone seemed super cool and futuristic.Rodrigo wasn't exactly sure how to simulate a system failure so his phone only got emails saying everything was hunky-dory. All he had left to do was to add a condition to not actually send the hunky-dory alerts since the network admins didn't want their phones going off every hour.Ready to proudly show off his handiwork, Rodrigo called Justin over for a demo. The demo consisted of Justin half paying attention, half texting on his phone, and saying "Yeah, ok" every couple minutes. When Rodrigo was done, Justin nonchalantly said, "Go ahead and throw it out on the production server whenever". Rodrigo wasn't expecting that but he was glad to finally have an app of his go to production.He threw the code out on the production server, fired it up, sent an email letting the admins know to expect an alert in case of trouble, then checked out for the weekend. He bragged to his friends over several beers about how his code was working all weekend while he got to have fun.Rodrigo strolled in Monday morning to find a red-faced Justin standing at his desk, arms crossed. "Rodrigo! We had a major system failure over the weekend!"Confused, Rodrigo played an optimistic card, "So when that happened, my app notified everyone... right?""No, it didn't!" Justin shouted, banging his hands down on the desk. "Your app didn't do jack squat to alert anyone, now we've lost nearly 30 hours of production data! Have a seat and think about how we're going to fix this while I review your code."Justin opened Rodrigo's source code at his computer and began scrolling through it. It didn't take long for him to find the offending line -

Initrode bought a UI widget library from Initech. Years passed, and eventually Initech went under. Initrode kept on keeping on, and kept using the library. Management shuffles happened, IT got downsized, and new development got outsourced.Merlin B worked for the company that got the contract. Somehow, someone got the source code from Initech's GUI library.

Andrew H writes “this is an interface for one of our Spring Data repositories”. If you’ve ever looked at Spring’s documentation, you know the punchline. Spring has certain naming conventions that have become a notorious.Spring Data is an ORM, and among other things, it allows you to design interfaces which are translated into a series of queries based on the naming conventions. So, for example, a method named findDistinctByTenantId would turn into a query in the database. It’s a useful convenience for simple CRUD operations, but for more complex queries, you’re still better off writing your SQL in an @Query annotation. SQL is still the best way to build complicated RDBMS queries.That doesn’t mean you have to use SQL. Andrew’s co-worker provided this method:

Hamza has some friends in the theater business. These friends had an in-house developed Java application to manage seating arrangements, and they had some problems with it. They had lots of problems with it. So Hamza cut them a deal and agreed to take a look.There were the usual litany of problems: performance was garbage, features bugged out if you didn’t precisely follow a certain path, it crashed all the time, etc. There was also an important missing feature.Seats in a theater section lay out roughly in a grid. Roughly- not every row has exactly the same number of seats, sometimes there are gaps in the middle of the row to make way for the requirements of the theater itself, and the application could handle that- for one arrangement. Seats could be removed to create standing-room sections, seats could be added to rows, seats could be reserved for various purposes, and so on. None of this was supported by the application.Hamza dug into the code which rendered the seating arrangements. Did it read from a database? Did it read from a config file? Did it have a hard-coded array?

“Hey, Kim H, can you sit in on a tech-screen for a new hire?”The Big Boss had a candidate they wanted hired, but before the hiring could actually happen, a token screening process needed to happen. Kim and a few other staffers were pulled in to screen the candidate, and the screen turned into a Blue Screen, because the candidate crashed hard. Everyone in the room gave them a thumbs down, and passed their report up the chain to the Big Boss.The Big Boss ignored their comments and hired the candidate anyway. A week later, this ended up in source control:

Anneke’s organization is the sort of company where “working” takes precedence over “working well”. Under-staffed, under-budgeted, and under unrealistic deadlines, there simply isn’t any emphasis on code quality. The result is your pretty standard pile of badness: no tests, miles of spaghetti code, fragile features and difficult to modify implementations.Recently, the powers that be discovered that they could hire half a dozen fresh-out-of-school developers on the cheap, and threw a bunch of fresh-faced kids into that mountain of garbage with no supervision. And that’s how this happened.

Art received a job offer that had some generous terms, and during the interview process, there was an ominous sense that the hiring team was absolutely desperate for someone who had done anything software related.Upon joining the team, Art found out why. Two years ago, someone had decided they needed to create a web-based storefront, and in a fit of NIH syndrome, it needed to be built from scratch. Unfortunately, they didn't have anyone working at the company with a web development background or even a software development background, so they just threw a book on JavaScript at the network admin and hoped for the best.Two years on, and they didn't have a working storefront. But they did have code like this: