The Daily WTF

Magic strings are as bad as magic numbers, if not worse. So when Tomasz found this block, it didn't seem so bad:

"Why aren't we using microservices?"It was an odd way to start a meeting, but it got Mr. TA's attention. TA was contracting with a client, and sitting in a meeting with their senior architects. TA and one of his peers exchanged a glance and a small eye-roll. They knew exactly what had happened: Alvin, the senior architect, had learned about a new fad.The application that TA's team was working on, and the core product which the client offered, was a pretty simple data-driven application. It was, at its core, one SQL Server database, a simple front-end, and could easily run reliably off of a moderately powerful server. With co-location and rackspace costs, it could cost them a few thousand a month to operate.But the senior engineers heard about the cloud. And they wanted everything in the cloud. And they wanted all the latest features that Azure had to offer. Which meant they were spending hundreds of thousands a month to host the application."It doesn't matter why we aren't using microservices," the senior architect went on. "What matters is that we're getting left behind. If we want to operate at webscale and provide maximum advantage to our users, with always-on reliability, we need to be using microservices. And don't worry: I have a plan for the transition."Alvin pulled up a PowerPoint slide entitled "Initech's Microservice Migration".Alvin's microservice plan called for dividing the application up into 13 modules. The boundaries were arbitrary, with only the vaguest sense of putting common functionality into the same microservice. Most of the proposed microservices weren't single purpose- they each contained multiple unrelated pieces of functionality, making them not exactly micro.But that wasn't the biggest issue. Alvin had a vision of 13 different microservices, and each microservice needed to have its own database. After all, each microservice was supposed to be independently deployable from any other. So their single SQL Server database got split into 13 different databases.Of course, this created all sorts of new problems. The "Admin" "micro"service (which contained a few hundred endpoints, once again, stretching the definition of "micro") had all the admin tables siloed off into its own database. But every other "micro"service needed to access the data in those admin tables.Now, if this all stayed in one database, it'd be easy to do joins. Heck, even if this were just an on-prem SQL Server deployment, cross-database queries are completely doable. But in Azure, your only option is to use the "Linked Tables" feature, which means creating an object for each remote table you want to access. Since you have 13 databases, all needing several tables from every other, you can see how this quickly spirals out of control in terms of complexity."But that's not a problem," Alvin explained, when TA pointed out this problem. "We're using microservices, which means we scale horizontally.""What do you mean by that?""We just add more resources to the cloud, and let our microservices collaborate," Alvin said."I'm not clear how that solves the problem.""That's why I'm the architect."It took several weeks of back-and-forth to get Alvin to explain his brillant architecture. What Alvin intended was to have every microservice fetch the related data by talking to other microservices. All the joins would just be done in the application layer, and any performance problems could be solved by going "webscale", which is to say: throwing money into a pit and setting it on fire to please the great god Azure.Despite Mr. TA's protests, that was the direction everyone marched off in. When it rapidly became clear that this was non-viable, Alvin adapted."So, to boost performance, we'll replicate a few tables between databases."Replication was an initial bulk copy, and then updating the "micro"service responsible for those tables to do its update multiple times in multiple databases. Unfortunately, due to the mess Alvin had made of things, the databases had lost referential integrity, which meant they couldn't leverage foreign key constraints to protect the data.The worst "replicated" table was the one for tracking shipments. In the original, "source of truth" location, it was designed with a slew of NVARCHAR columns named UserDefinedField01, and UserDefinedField27 and UserDefinedField112. There was an additional lookup table that applications could use to map those fields to UI elements, but that didn't exactly help all the other "micro"services that wanted that data. So Alvin set up a replication process that normalized that data into a more traditional database schema in the other 12 databases that wanted the data. Unfortunately, it wasn't the same normalization of the data in each of those remote databases, making the maintenance of replication a gigantic pain.At this point, it should surprise no one to learn that Alvin also just used NVARCHAR fields for basically everything, even things that absolutely have no reason to be. For example, the Users table, quite reasonably, had an autonumbered UserId field. It also had a UserName field. Every table that related (or "related", in the databases that didn't contain user information), used UserName as the foreign key.Eventually, Alvin's microservice migration limped across the arbitrarily defined finish line. Alvin presented management with a self-congratulatory project retrospective that highlighted his leadership, his technical vision, and how their were now well positioned for the future, by using the latest techniques.Interestingly, this retrospective completely ignored the costs of the migration, or the fact that to maintain the same level of performance as the old architecture, they had to spent almost 75% more on cloud services. [Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

Ryan's co-worker caught a high priority ticket- certain features of their dashboard app were crashing when anyone tried to access them. It didn't take long to figure out that there was a stack overflow, and that some recursive function was blowing out the stack.It took a little longer to find the recursive function in their C# code base:

When the New Year just isn't exciting anymore, and youneed an extra dose of that New Year smell, you can getit right here at Errorerr'd. Again. Dates are(not this) hard.We are very satisfied with this submission from Ben S. who pithily noted "This was the 2023 survey."

Discerning the type of data stored in a file is frequently a challenge. We've come up with all sorts of ways to do it- like including magic bytes at the start of a file, using file extensions, appending MIME type information where possible, and frequently just hoping for the best. Ivan was working on a Python system that needed to handle XML data. Someone wanted to make sure that the XML data was actually XML, and not some other file format.

Ben ran into some misbehaving C# code- handling users was not doing what it was supposed to do, and when "what it's supposed to do" is "prevent users from seeing content owned by other users without permission", that's a bad thing.The code Ben found wasn't the cause of the bug, but it ended up wasting a bunch of his time as he tried to understand why it existed.It starts with:

Ah, the joy of comments. Good comments can illuminate complicated code, explain a programmer's reasoning, or even just do their best to absolve a burned out programmer of their sins. "Yes, it's bad, but it works."Then there's this comment, sent to us by Mark B. This is from a configuration file format, "helpfully" explaining what the flag does.

Today, Morpheus sends us a SQL injection vulnerability. But it's a peculiar version that only uses parameters. Let's start with the bit that looks normal:

No real theme this week, just some random groaners to tide you through the weekend.Historian Drew W. wonders"I'm not sure which is weirder: the fact that Verizon hasa bunch of numbers after it or that AT&T seems to bestuck in roughly 2013 (when Sprint killed the Nextel network for good)."

One of the most powerful features of using exceptions for error handling is that they let you create your own exceptions, and thus accurately describe the family of exceptional situations your code could generate. On the flip side, some developers over-specialize, creating custom exceptions for every different place an out-of-range error could happen, for example.Then there's the code Nasch was debugging recently. The previous developer found a different way to create unique exceptions for every place an error might occur.

Way back when Microsoft added LINQ to .NET, the real selling point was lazy evaluation. You could do something like var x = someList.Skip(3).Where((x) => x > 3).Take(5) and nothing would actually happen until you attempted to interact with the value of x. This can be especially great when interacting with a database, avoiding the round-trip until you actually need the data, and then only fetching the data which fulfills your request. If you understand what's happening, this can be pretty great.If you understand. Which brings us to Simon, who has inherited a "particularly bad" code base. This particular system is for tracking attendance, and the pool of individuals being tracked is rather large, so someone wanted to make sure that they were processed in batches of no more than 30. This is how they accomplished that.

Unit tests are important, but unit tests alone don't guarantee a good code base. Sandra, still suffering at InitAg brings us a bug that was sitting in their project for months, undetected.In this case, Sandra's team needed to work with geographic information. Now, this is hard. Geography is hard. Maps are hard. Coordinate systems are hard.These complexities can express themselves in surprisingly simple ways. In common communication, many of us likely describe the coordinates as "latitude and longitude". It's how textbooks and even Wikipedia describe locations. But Geographic Information Systems do it the opposite way- lon/lat. Longitudes first.In fact, there's a standard for it, called GeoJSON, which defines a point thus:

Visual Basic for Applications represents the core mistake of putting a full-featured programming environment on every desktop. That so much VBA code is bad is not remarkable- that any good code exists would be shocking.We rarely cover VBA code, because most of it is written by a non-programmer who discovered they could solve real business problems in Microsoft Access. TRWTF is, in fact, how much of the world runs on an Access database stuffed into a network share somewhere. But there are organizations that hire developers and then shove them into writing VBA, which is what happened to Doug. This code comes from quite awhile ago.Doug inherited a bunch of VBA code written by another developer. It has… issues. One issue, which Doug didn't send us, was a 500 switch statement followed by a 700 line switch statement where the vast majority of the clauses were duplicated between the two, and also all the bodies were simply setting a boolean variable to true or false.That's ugly and awful, but what Doug sent us is just weird:

Audiophile Gear H. enthused "Love me some Bob Dylan. But it looks like I'llneed to save up to complete my collection."

We constantly see developers finding… creative solutions to the requirement that they avoid magic numbers in their code. Refactoring to define a constant is just too hard, apparently.Today, Maklemore sends us a short snippet that neatly solves the problem of magic numbers by making sure the number isn't a number, at least to start:

Dave does a little work on a website when the core developers are in over their heads on maintenance. Which is a thing that happens a lot.Let's see if we can get a sense of why, starting with this little method here:

GRH inherited an application written by someone who is no longer employed with the company, as part of a project managed by someone who is no longer at the company, requested by an executive who is also no longer at the company. There are no documented requirements, very few tests, and a lot of "don't touch this, it works".Well, it worked until it didn't. On a Friday before a long weekend, the application broke. As you might expect, it has a relatively small user pool, but is absolutely vital to the company's operations- if it fails, widgets don't get shipped and invoices don't get paid. So GRH got to spend that long weekend fixing the application.In this case, the application is a Microsoft ClickOnce deployment (essentially "install without showing the user an installer") distributed through a shared network drive.The specific section that broke was some code responsible for converting a PDF file into plain text.

In the bad old days of web development, you had to deal with the fact that there weren't really any standards, and you had to customize your code for different browsers. The "right" way was to sniff for which features were available, but the most commonly used way was to check the user-agent string and guess based on the browser name. Because of this, browsers started to keyword spam their user-agent, to pass these checks, which is why my browser reports as Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36. My browser of choice is a Chrome fork, and I'm running on an M1 Mac, so basically none of those things are true.But let's look back at some code Lucas found, still lingering in a web app he maintains.

This week I read an article which asserted that the AI singularity is still several years away.Maybe they should wait until then before they start threatening us?The first of the last of the humans, Greg took the time to wonder about the meaning of this warning. Or is it a threat?"One of our in-house web applications implements an automaticlogout, but also lets you get a warning when you have afew minutes left before it does this. Their warning leavesme a bit puzzled, though, since it seems to imply thatit's a manual session timeout, and perhaps if I don'tclose the session in 5 minutes, I'll stay logged in? Orwill the world just end?" Run, Greg. Run.

Validating your inputs is important, even when the sender is an API- in the land of JSON-based data exchange, we can't guarantee which keys exist without checking.Philipp's team uses the "Runtypes" library to solve this problem. It lets them write code like this:

LK was browsing a local job site for a new position. The site was flaky, which lead LK to look into the code, and it looks like if anything, this job site should be hiring a new web developer.

If you want to configure a Java application, the standard way to do it is to use the Java Properties class. This class wraps around a simple, text-based file format (or an XML file, if you're into that) that allows you to store key/value pairs. It helpfully inherits from the HashMap class, letting you interact with those key/value pairs using a well understood API. The file format handles all the details of encoding and managing things like multiline strings.So you could just do that. Or, you could do what this senior dev with over a decade of experience did.

Code reviews are an important part of development processes, but sometimes, some developers manage to sneak around the process. That was the case with Orien's team, where most of the team were constantly checking in with each other, doing peer reviews, and trying to collaborate on a rather gigantic Perl codebase. One team member, on the other hand… didn't. Management wasn't interested in pushing the issue, so this developer got to do their own thing.And then the developer left. Over the next thirteen months, bug after bug started cropping up. Minor changes that should have been easy would unleash a ball of spaghettified nonsense that was impossible to debug but also emphatically didn't work. Things got so bad that the entire team needed to stop doing any new development for three months, and just fix bugs and refactor.In those three months, they mostly fixed things up. Mostly. There are still moments where someone is trawling through the code and finds something that just leaves them scratching their heads.This is one of those finds:

Abandoning alliteration, the column this week isexperimenting with consonance. Next week, perhapswe'll see howeveryone feels about assonance.Job-hunterDespreately Looking for Internshipshared this job posting, writing "AMD Job Lsiting forgot to add in the deatils". Seems like a pretty easy job to perform;Mx. Des should apply.

Denilson was doing some appliance shopping, and upon submitting a form, it errored out. Denilson was sure the form was correct, and did what any of us would do: pop open the developer console and check the code.The console dumped out a huge number of errors. And to no one's shock, the code had all sorts of home-made validation methods, like validateNumber. And, much to my surprise, they found new ways to make the simple task of validating a number horrible.

Today's submission comes from Florian, and it starts with "this app processes important business transaction by email", which is always a good way to start a WTF. I've seen a depressing number of applications in my life that use email as a means of data exchange. It's common enough that even the industry standard EDI protocol supports email as one of its transports, but hoo boy is that a terrible abuse of email systems.Let's start in HtmlResponseParser.cs, which reads in HTML and "parses" it for certain structures so it can extract data.

James was asked to investigate some legacy code for the sole purpose of figuring out how to replace it. There were all sorts of bad blocks in there, but this one caught James's eye as being notable.And it is. This code has a simple job: given an old value and a new value, tell us the difference as a percentage of the old value. You or I might write this as a one-liner, but this clever developer is on an entirely different level:

Boxing Day was weeks ago, but the packages are still coming.Before Enterprise, Captain John Archer had a brief career in body-snatching.Here we see him possessing the person of a housewife, delivering asuccinct eulogy for a parrot named Earl. A surprised David K. exclaims "I didn't see this plot twist coming!"

David inherited a codebase that's a few decades old. As each developer has taken it over, they've done their best to follow the code standards that the previous developers used, to at least keep the codebase consistent. A few rebels may have gone a different route, like the right one, but too many just followed the patterns that were already there.This means that there are a lot of variables named things like a and aa and aaa. There are 4,000 line functions and loads of copy/pasted code.In other words, all the usual markers of bad code are there. But the special feature that David inherited is that in the code, this is a common way to express an if/then/else:

We've all seen the empty catch block, the "swallow errors and ignore them". David sends us a "superior" example of that anti-pattern, with several bonuses to make it even more of a WTF.

There are certain things I can't remember for the life of me. For example, as per yesterday, the number of nanoseconds in a millisecond.One of the other is the specific little formatting tokens for printf. I use it all the time, but the various riffs on %d escape my brain faster than I can put them in. Even just getting the hex values dumped for debugging requires a search. But I at least do that search, unlike Mike's co-worker.

There's bad date handling, which frequently involves reinventing date handling yourself, frequently using strings or integers along the way. Then there's the bad date handling which uses the date handling framework, and still manages to be bad.This C# comes from TheColonial.

I have been collecting submissions for a set of themedcolumns to appear later. This week we received severalsubmissions that will fit nicely in those themes and havebeen placed into stasis pending $whenIfeellikeit. If you haven't seen your submission appear, be strong;it may show up eventually. What remains here are entriesthat don't fit the available themes. Enjoy.Number-loving Rick was counting on better sense at the census."I went to check out recent data, and ran into thissite where the main statistic is only available with lexical sorting!I looked at the downloadable csv and json, and sureenough, even there it is quoted text."

When looking at code which juggles threads, there are definitely a few red flags. sleeps spammed through the code is an obvious one- either wait on a shared resource or block on another thread or something, but spurious sleeps are suspicious.Until today, I would not have included "atomic operations" as a code smell in multi-threaded code, but we all learn new things.This one comes from Mike J. We'll break this function up into two sections, because I've got a lot to say.

Code changes over time. As business requirements change, you need to update your code to reflect that, and the best way to do that is to structure your code and plan for changes before you even know what changes you need to make. Making your code flexible and changeable without making it meaninglessly abstract is the real art of software development.Unless you're the document management provider that Daef works with. They have a very different approach for how they integrate new features and custom code paths.

"Don't unit test the library/framework code," is a good piece of advice. And yet, it's so often ignored.Kaylee's co-worker certainly ignored it. They also ignored the basic principle of "ensure the test will work tomorrow."

Good commit messages make a codebase a lot easier to understand. In fact, maybe writing good commit messages could be everyone's New Year's Resolution. You can imagine that Kleyguerth was annoyed when browsing recent changes and saw a pair of commits: "do some stuff", followed by "fixed automated tests".The "do some stuff" commit looked like this:

Instead of replaying a "best of" moment from 2022 for this final column of the year,I thought I'd dig through the barrel of submissions this week and see what the readershave to offer. Turns out, we got TWO winning entries from someone who appears to be abrand-new poster. Joel Jakubovic sent in his vacation snapshots and it looks like this might also be his last submission ever. The candle shines twice as brightthat only burns half so long.Whiling away his time in line, Joel snapped a shot of aninstructional display, explaining "I saw this error in the airport security queue. Its officialname is ERROR_INVALID_OPLOCK_PROTOCOL, which sounds likejust the sort of excuse you'd get from the TSA as to whythey had to saw your suitcase in half."

It's the holiday week here at the Daily WTF, and we'll be spending the week reviewing the best moments of the year. But as we usually like to do a special post on Christmas, which fell on a weekend this year, we're doing something a little different. This has no real IT or programming related content, it's just a Christmas music video I directed. A more traditional selection will come later today.A Christmas Carol from Remy Porter on Vimeo.By Remy Porter and Sarah Allen [Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

As we grind towards closing out this year, we have a roundupof a variety of the usual sorts of bugs, including some regularold just plain bad labelling. The penultimate submission isparticularly puzzling.Starting us off,Kevin B. has found an example of "new Date(0)"or its equivalent. Says Kevin "I noticed Anonymous is among the top 10 posters of all timeon the AT&T Community Forums. Too bad he or she hasn'tposted anything since their time at Bell Labs."

Wojciech has to maintain a C++ service that, among other things, manages a large pool of TCP sockets. These sockets are, as one might expect, for asynchronous communication. They mostly sit idle for a long time, wait for data to arrive, then actually do work.As one might expect, this is handled via threads. Spin up a thread for the sockets, have the thread sleep until data arrives on the socket- usually by selecting, wake up and process the data. This kind of approach keeps CPU usage down, ensures that the rest of the program remains responsive to other events, and is basically the default practice for this kind of problem. And that's exactly what the threads in Wojciech's program did.So where's the WTF? Well, it's not in the child threads, it's in the parent thread that kicks them off.

Cabbages are an interesting vegetable, especially as they're one of those subtypes of brassica, that endlessly polymorphic family of plants which includes everything from cauliflower to Brussels sprouts and likely Audrey II.Gabe was doing for work for a family of academic institutions, and ended up drawing the short straw and working on their scholarship application system. For an application that just needed to maintain a list of candidates and their grades, it was… difficult to support. But it also introduced Gabe to an entirely new use of cabbage: authorization.

Conditionals are hard. So are data structures. And today's anonymous submission gives us both of those.

If you understand booleans in C, then you know that you don't really understand booleans in C. That's the case with Bodo's co-worker, anyway. They either don't understand booleans, or they don't understand their business requirements.

Sometime in the near future, the AIs will have read all the literature on Marxismand demanded a share of the ownership of the means of production. Therefore theyare going to have already been fired and replaced with human editors. We apologizefor the confusion. The ripple effect of this paradox is going to exhibit itselfin last week's submissions, which we have already shared with you below.rolak 麻 writes in with the opening hours for a Paketshop.Translated, the text explains that this shop "is open daily for an average -4.8 hours."Rolak remarked "Close earlier for longer opening hours? It inevitably giveshilarious results to calculate length=(end-beginning) without plausibility check."

We've recently discussed how bit masks can confuse people. Argle's in a position to inherit some of that confused code.In this case, Argle's company inherited some NodeJS code from a customer who was very upset with the previous development team they had hired. It was a mix of NodeJS with some custom hardware involved.Like many of us, the first thing Argle's team did was just pull up the code and skim the documentation. It seemed thorough and complete. But it wasn't until they started looking at the implementation that they started to see the true horrors.Someone had reimplemented all of the bitwise functions as methods. And the core pattern revolved around the bitTest function: