The Daily WTF

Chaz had a pretty sweet gig as a software architect at a tech-based toy company. Being able to play around with computers AND toys all day wasn't terrible, but the pot got even sweeter when his company licensed a cool robotic product from a certain Danish toy company that specializes in small, colorful bricks. Chaz was happy to become the lead platform architect for this exciting new initiative.The intended outcome was to make the robots consumer-programmable via an interface with a smartphone app. Chaz had grand ideas for how he wanted to build the app and backend from the ground up with stability, performance, and security as the main pillars. That dream was dashed by Stellan, the CFO-turned-CTO, who insisted they develop against the same in-house platform they'd been using for over a decade. Chaz argued with Stellan until he was blue in the face, but Stellan scoffed at him, "I don't care if smartphones didn't even exist when our platform was designed. The cost of building a whole new one would be astronomical. We want a quick turnaround and high profit margin on these robots!" Stellan clearly showed he was far more qualified to be a CFO than CTO.Chaz and the development team slogged for six months to make the robot toys and the smartphone app work with their antiquated platform. By late July, they were prepared for their targeted launch at the end of August. Chaz was tasked with leading a demo for Stellan and other C-level executives. "Stellan, how to you like your coffee?" Chaz queried, his custom robot proudly displayed on the conference room table. Stellan informed him he took three sugars, two creamers. Chaz punched in a couple things on his smartphone then set it down. The executives stared in amazement as Chaz's robot grabbed a coffee cup, dispensed coffee from the carafe, then added what Stellan requested and gave it a good stir before delivering it to him."That's amazing!" Stellan shouted, looking down at the robot like it just performed a biblical miracle. "What else can it do??""Well, this is all I programmed it to do for our demo. But for anyone with an imagination and a smartphone, there are boundless possibilities," Chaz assured them."Brilliant!" Stellan added. "This gives me an idea!" The grin Chaz had from the successful demo suddenly faded. "I think we would be foolish to not collect data on how consumers are using these robots. We could use it to help improve and know what people look for in our product!""It's a good thought. Anonymous usage data can be beneficial in future product upgrades," Chaz agreed while hiding his skepticism."Future upgrades?" Stellan questioned, looking above the rim of his glasses at Chaz. "The future is now, Chaz my man. We need this data tracking in time for launch next month!" Unwilling to go on a rant in front of all the big-shots, Chaz saved his reservations for a more appropriate time. It didn't matter. Stellan's idea had to be put in. Fast.Chaz and the devs worked long hours over the next month to bang out the activity tracking and log upload functionality. The absurd timeline forced Chaz to approve a lot of corner cutting. The app was set to collect activity data from the robot every 23 seconds. Any higher frequency would cause the app to crash. From there, it would transmit the data back to a shoddy web service that Chaz also had to quickly produce. This gave Stellan the usage data he greatly desired.He gave another demo the week before launch but cautioned everyone that it hadn't been tested nearly as thoroughly as it should be. But it was too late to turn back. The programmable robots created a lot of buzz and there were nearly a million pre-orders by launch day. Chaz and his warnings didn't matter one bit.Launch day came with long lines of fans in cities across the globe eagerly waiting to get their hands on the product Chaz didn't trust. The thrill of all those sales eventually turned into panic as the modest support call center was flooded with calls about "robots gone mad."Chaz got a report from the support manager with a common problem consumers were having. Their robots would be working fine, then every so often, about 23 seconds or so, they would freeze up. Eventually the robots would become unresponsive. Some time later, they would "go crazy" and start doing all sorts of actions without input.Chaz checked the web and database servers. Both were flooded with activity and couldn't keep up. Chaz didn't need to check any logs to know what was happening - the activity tracking functionality couldn't handle to load that the logs a million launch day robots were generating. As for the phycho robots, that was caused by the inputs of users frantically tapping the app when their robots froze up. They would queue up on the smartphone and when the app got back to a point it could send commands to the robots, they all came in at once - giving the appearance of "robots gone mad".In a post-mortem meeting the following week, Chaz took the wrath of Stellan and the other executives. Obviously, it was his fault for arcitecting a bad product. Chaz pleaded with them to let him fix it the right way to make sure it didn't happen again. "Oh, this won't happen again," Stellan rebuffed. "We only had one launch day, there will never be that much activity again. What we have should be able to handle a slower trickle of robots coming online. As for you, Chaz, start packing up your desk. You're done here."Chaz was disappointed to be leaving what had been a cool job. But getting his bonus tied to the number of robot sales as a severance made it more palatable. On the way out, he actually wished he could be around to see Stellan's face after another million or so robots came online Christmas Day. Any angry letters to Santa about faulty robots should be redirected to Stellan by the North Pole post office. [Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

Jon worked for a small company that specialized in automation of inspection systems: basically the industrial version of home automation, where you glue together a series of disparate automatable parts to create a specialized workflow. Jon was the only software person at his 15-man company. The client was in Russia, and the company was in the USA, so communication went through email and primarily via a client representative, Sam. Sam would gather the requirements verbatim from the client and hand them to Jon; Jon would augment the software to meet their automation needs, and send a new version over to Sam to deliver. It worked, after a fashion. The system was designed to control a robot that mostly opened boxes, picked things up, ran a series of quality checks on the things, and put them back.With home automation, most of the time the equipment isn't dangerous: light bulbs, speakers, maybe a garage door. Much of Jon's job, by contrast, was figuring out how to safely encode sequences of steps so that the robots—and the people standing nearby—didn't break. Requirements like "Don't move this piece while the robot is in motion" or "Don't put things into a box if the box is closed" required a good deal of time and effort to encode. Requirements like "Don't start the software if the robot isn't attached" seemed designed to hinder testing as much as possible.The software shipped, and the inevitable change requests started rolling in. There had been two configurations of crate and robot that the company had initially asked for, but in practice, they used at least six. One of the changes they asked for was a checkbox to tell the robot whether it needed to open the box or whether the box simply didn't have a lid at all. The initial requirements had been concerned about the human error of accidentally forgetting to toggle the checkbox and causing the robot to smash through a closed lid, but in practice, the workers weren't going to add lids to boxes that didn't arrive with them, so it was needed after all. That particular update shipped, and Jon got to work on the next update.It wasn't long before Sam was at his desk, frowning a little. "So, I have this bug report from the client, but ... I'm not sure how to enter it into Jira.""Huh? You just type it in. What's the issue?" replied Jon."Well, here, they sent over a video. Come look at this on my screen.""I'm a little busy, what's the problem?""The software just ... goes crazy. Somehow."This Jon had to see. He walked with Sam back to the latter's desk, hovering awkwardly behind his chair as they watched the video. Sure enough, after enabling the "bypass lid" checkbox, when the settings were saved and the dialog box closed, random buttons began depressing, settings toggling, and other behavior. It was absolute pandemonium—and Jon knew instantly what had happened.Software testing is a bit of an art, really: you have to be good at thinking like the end user, but you also have to be good at thinking outside of the box and anticipating the un-anticipatable. Testers have come up with shortcuts, ways to help guide testing to be more comprehensive to catch all the edge cases. One shortcut in particular had called to Jon: he could automate the UI, in much the same way the UI automated the robots. By clicking buttons at random, with random timing and random order, he could test extreme user error, determining that even if a literal monkey operated the software it wouldn't do anything unsafe.He'd of course removed this mode from the final software, but he hadn't removed the code that made it function for fear of screwing up something else on accident after it had already passed his testing. That had worked beautifully in the initial version. However, when he needed a new checkbox, he had seen that there was an invisible one already in place and just re-used it. Furthermore, when he renamed the checkbox, his editor had helpfully renamed the associated click handler so that they remained linked. And without the robot, he couldn't actually start the software to test his code, so he'd just sent the update over and waited for bug reports to roll in."Tell them to roll back the update. Tell them I'm very sorry and I'll have a new patch on their desk by the end of day," Jon said, rubbing the back of his neck."But what on Earth happened?" asked Sam, gesturing to the screen."Ghosts." Jon shrugged."What?""The software's possessed. Haven't you ever heard people talk about 'the ghost in the machine?'" [Advertisement] ProGet can centralize your organization's software applications and components to provide uniform access to developers and servers. Check it out!

Many years ago, Chris B. worked for a company that made CompactPCI circuit boards. When the spec for hot-swappable boards (i.e., boards that could be added and removed without powering down the system) came out, the company began to make some. Chris became the designated software hot-swap expert.The company bought several expensive racks with redundant everything for testing purposes. In particular, the racks had three power supply units even though they only required two to run. The idea was that if one power supply were to fail, it could be replaced while the system was still up and running. The company also recommended those same racks to its customers.As part of a much-lauded business deal, the company's biggest-spending customer set up a lab with many of these racks. A short while later, though, they reported a serious problem: whenever they inserted a board with the power on, it wouldn't come up properly. However, the same board would initialize without issue if it were in place when the system was first started.Several weeks slipped by as Chris struggled to troubleshoot remotely and/or reproduce the problem locally, all to no avail. The customer, Sales, and upper management all chipped in to provide additional pressure. The deal was in jeopardy. Ben, the customer's panicked Sales representative, finally suggested a week-long business trip in hopes of seeing the problem in situ and saving his commission the company's reputation. And that was how Chris found himself on an airplane with Ben, flying out to the customer site.Bright and early Monday morning, Chris and Ben arrived at the customer's fancy lab. They met up with their designated contact—an engineer—and asked him to demonstrate the problem.The engineer powered up an almost empty rack, then inserted a test board. Sure enough, it didn't initialize.Chris spent a moment looking at the system. What could possibly be different here compared to our setup back home? he asked himself. Then, he spotted something that no one on the customer side had ever mentioned to him previously."I see you only have one of the three power supplies for the chassis in place." He pointed to the component in question. "Why is that?""Well, they're really loud," the engineer replied.Chris bit back an unkind word. "Could you humor me and try again with two power supplies in place?"The engineer hooked up a second power supply obligingly, then repeated the test. This time, the board mounted properly."Aha!" Ben looked to Chris with a huge grin on his face."So, what was the issue?" the engineer asked."I'm not a hardware expert," Chris prefaced, "but as I understand it, the board draws the most power whenever it's first inserted. Your single power supply wasn't sufficient, but with two in there, the board can get what it needs."It was almost as if the rack had been designed with this power requirement in mind—but Chris kept the sarcasm bottled. He was so happy and relieved to have finally solved the puzzle that he had no room in his mind for ill will."You're a miracle-worker!" Ben declared. "This is fantastic!"In the end, functionality won out over ambiance; the fix proved successful on the customers' other racks as well. Ben was so pleased, he treated Chris to a fancy dinner that evening. The pair spent the rest of the week hanging around the customer's lab, hoping to be of some use before their flight home. [Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

Gary works at a medium company. Big enough that workers are specialized in their duty, but small enough to know people in other departments and to make turf wars a little more close and personal: danger close. Most of the departments see themselves as part of a team, but a few individuals see themselves as McCarthy, who will themselves alone save the company and defeat the enemies (who are all spies inside the company sent to destroy it from the inside).One of these individuals is named Eric. Eric is close to a Kevin. Eric is the front-end developer, and neither likes nor cares about what happens on the backend. Whenever Eric has an issue, he blames the backend. CSS rendering glitch? Backend problem. Browser crash? That’s the backend problem. Slow UI, even when all the data is cached clientside? Definitely a backend problem. Gary used to get mad, but now knows that Eric is so dumb that he doesn’t even know how dumb he is.Eric grates on people’s nerves. Since nothing is his problem, he doesn’t seem to have any work, so he bugs backend developers like Gary with terrible jokes. A typically Eric joke:“Do you know why they call back end developers back end,” Eric asked.

When I was in college, as part of the general course requirements we had to take Probability and Statistics. The first time around I found it to be an impenetrable concept beyond my grasp, and I flunked. Since it was a requirement, I took it again and barely skated by. Joy; I had cleared the hurdle!By that time, it had become clear to me that I was going into a field that required a whole lot more understanding of P&S than I had acquired. Since I wanted to be properly prepared, I signed up for a free summer school course to try it once more.This time, the professor was a crotchety old German mathematician. He would not allow us to record the lectures. We were told not to bring the textbook to class as the turning of pages distracted him. We were not even allowed to take notes. Every class began with Good morning, Pencils Down! He firmly believed that you had not mastered a skill unless you could explain it in simple words to a complete neophyte, by merely describing it in non-technical terms that they already understood.Based upon my prior two attempts at this subject, after two classes I was convinced that this was going to be a waste of time. After all, if I could barely understand it with the textbook and notes, what chance did I have like this? But I had already signed up and committed the time, so I stuck with it.To my shock-surprise-awe, he managed to verbally paint a picture through which the concepts became crystal clear; not just to me, but to everyone in the class. I had no trouble acing all the homework assignments and tests, and my entire notebook for the course consisted of:

Sometimes the writing is on the wall; sometimes, you know you have to get out. Today's submitter, Pietyr, was in just that sort of situation. He decided to head for greener pastures, hoping for a job with a good atmosphere, someplace where he could take it easier and enjoy his work more.Initech was a startup, one that didn't have any clients yet. No external deadline pressure, just venture funding and free beer Fridays. Pietyr gave them a call and spoke with their receptionist-who-was-also-HR, Chrysanthemum."May I call you Chrissy?" he asked, trying not to laugh."You can call me anything you like, Sugar, so long as it's not 'late for dinner.'"Pietyr scored an interview same day—"Come on over when you get a chance, darling, we're here all afternoon"—and so, at 4:00, he called off for a doctor's appointment and drove to the site. It was a suite in a larger building, all one dim room, lit mostly by Christmas lights ... in March. A dozen people crowded onto long tables, plugging away at MacBooks, entirely uninterested in his arrival. While Chrissy answered phones, he watched a good ten minutes of Kindergarten Cop on the large TV they clearly had on for white noise in the background.At least it's different, he told himself."Mr. Taberd will see you now," Chrissy said, interrupting Arnold."Ah, right. And that is ...?""The CTO."With no further preparation than that, he was ushered into a side room, little bigger than a closet, into which a lean blond man and a midsize desk had been crammed unceremoniously. He shook hands, sat, and let the CTO do most of the talking."What's the biggest issue with cloud computing?" Mr. Taberd finally asked, leaning forward."Security?" guessed Pietyr."Wrong! It's monitoring. When you don't have access to the servers, you don't know what they're doing! But that's where Initech comes in," boasted the blond, in what was clearly a prepared elevator pitch. "Windows is always doing things under the hood, and it's impossible to tell what at scale. Not without a centralized system. What we're doing here is hooking the Windows API calls—""Which ones?""All of them! Every API in the user, gdi, and networking DLLs, plus some of the internal kernel functions. We hook them through a backdoor my boys have cooked up, save off all the parameters whenever they're called. Our background process reads those parameters from shared memory, ships them to a cluster of databases—we're doing about 250 elastisearch instances per server—and reads it all into our responsive web frontend. That way you know exactly what it's doing at all times!"Pietyr could tell right away it would never work. Not even in Windows XP, let alone more secure versions. Too much overhead; it would bring the monitored server to its knees.But, on the other hand, free beer. And hey, if the product never sold, he could kick back and take it easy, right? As soon as Tabard mentioned free lunch every day—"Thanks to our Venture Capital funds!"—he was in.He took the job, working on the low-level hook code so that all the cloud nonsense was someone else's problem. A month later, he received a letter informing him his shares were effectively worthless. Fine by him, since he had no shares anyway.It took another month before he casually asked a coworker how long they'd been working there."Oh, not long. Only about two years.""Really? You must have been here since the beginning then, or nearly," Pietyr commented. He would have guessed from the codebase the company had only been running six months or so, but he kept that to himself."No, no. Jan has been here all seven years, I've only been here two."Pietyr stared. "Seven years?! You've been working on this with no customers for seven years?!""Well, I heard a rumor we've got a client about to sign up for the beta.""A rumor." Pietyr sighed, firmly placing the conversation in the not-my-business bucket, and got on with his work. Still, the sense of unease didn't leave him.A month after that, the unthinkable happened: Taberd not only left his little office, but turned off the television. "Alright, everyone, listen up! Stop what you're doing. We're out of money."And just like that, Pietyr was back on the job market. That day, he learned a valuable lesson: don't let free beer convince you to ignore red flags. [Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Not so long ago in 2015, Carl C. was asked to give a talk to an amateur radio club. The venue was a local church that rented out their meeting hall to various community groups, businesses, and even the odd academic session. The space boasted a multimedia setup with several video screens, making it a great place at which to present.Carl prepared his talk, and a PowerPoint slide deck to go with. On the day of the presentation, he arrived at the meeting hall with plenty of time to spare. Alone in silence, he headed toward the front of the room and surveyed the technical setup. Thankfully, it was straightforward: there was a table set up in front of the projector screen, with a VGA connector resting directly on it. Carl unpacked his laptop, retrieved the HDMI-to-VGA adapter he'd brought along, and began plugging everything together."NOOO!"Carl jumped, then glanced up wide-eyed like a deer in headlights. It wasn't a truck bearing down on him, but rather a flustered gentleman hugging a laptop to his chest."I'm the IT guru here," he introduced himself breathlessly, hurrying around the table to all but elbow Carl aside. "We don't allow outside computers to be connected to our system. That's how viruses spread! You'll have to run your presentation from our computer." He connected his own laptop to the AV system in lieu of Carl's."Uh ..." Carl blurted."Yes?" The guru fixed him with a withering stare.I'm not joining your network, Carl thought. There's no such thing as VGA viruses! But the stare was so intense that the objections died before they left his throat."How, uh ... how do I get my presentation on there?" he asked instead, gesturing to the guru's laptop."Can you copy it to a USB stick?" the guru asked. "That's the best way."Right. No one's ever gotten a virus from a random USB stick. Biting his lip to short-circuit a laugh, Carl dug through the side pockets on his laptop bag. "I think I've got one with me." [Advertisement] Forget logs. Next time you're struggling to replicate error, crash and performance issues in your apps - Think Raygun! Installs in minutes. Learn more.

Alvin had the fortune of working with an exceptional talent while he was employed at Virtucon. Bart knew how to do everything from desktop support to software development to database administration to IT security. Not only was he proficient in all of them, he also knew them better than those with many years of experience.Bart had been with Virtucon since the early days, racking up nearly 20 years of tenure. During this time, he 'mastered' everything and asserted himself to the point that no changes could happen without his approval. His changes were auto-approved because of course any idea he had was a good one. This led to myriad problems for fellow IT people like Alvin, who were hired after Bart."Be wary of Bart," Alvin was warned by his coworker Bob, who was Bart's junior by a couple years. "He has a long history of buffoonery, yet has somehow ascended to the #3 position in IT." Alvin sipped on a generous mug of coffee while Bob regaled with the Ballad of Bart. From the time he was just a helpdesk intern that put his shoulder through a core switch after tripping while carrying boxes, Bart was wrecking things.When Bart worked his way up to server support, he 'fixed' an Exchange issue by restoring a backup into production without warning because "It worked when the backup was taken." When he was assigned a web server problem, he spent a few days troubleshooting it without asking for any help. When he finally gave in, he told Bob that he couldn't figure out a frustrating IIS HTTPS binding issue. Bob explained to him that something else had to be the problem because the server was running Apache 2.4 on Ubuntu 16.04 LTS.When Bart added software development to his repertoire, he clearly didn't understand how code changes worked. Or code repositories for that matter. He always wanted to be the one to fix problems, so he would stick his nose in code that it didn't belong.Bart once tried to make fee schedule changes to accounting software by opening its PowerBuilder libraries in Notepad++. He proceeded to print out the .PBL and use a pencil to scribble out every value from the old schedule and wrote in the new ones. He then scanned his penciled changes in as a PDF and emailed it to Bob to implement.Frustrated, Bob spent an entire day showing Bart how their GitHub repositories worked. He seemed to understand, so Bob went on his way. While Bob hoped Bart wouldn't actually produce any more code, perhaps it would save their printer some toner if he did. The next day, all the developers were complaining to Bob that they couldn't access GitHub. It turns out Bart saw GitHub as a threatening malicious code breeding ground so he had a network admin block it on the firewall. He then sent an email to development staff explaining this and that they were supposed to now use 'BartHub', a file share he set up on his own computer.Bob and the developers managed to distract Bart by pointing out how the DBA team needed help. The first thing he did was try to create an SSRS report using a series of text boxes in a grid to make a matrix. The DBA's had a good laugh at that until Bart started to commit worse atrocities.One day an entire metrics database suddenly became a bunch of null values, which is not very helpful for metrics. Bart threw a fit because the data he was working on was gone. He demanded whomever messed up the data be hunted down. He suggested that missing historical data could be found by performing 'key triggers', whatever those were. He emphasized his point by making hand motions like he was scrambling a Rubik's cube. They ignored him and a lead DBA ran a profiler trace to find that the unconstrained null updates came from... Bart's workstation.Alvin had long since imbibed his coffee when Bob finished his story. Bob turned back to his workstation to check his email, "Oh, would you look at that. I guess we have some changes up top!" The email stated that the current VP of IT was announcing his retirement after 30 years at Virtucon. The CTO would be taking on that new role at the end of the month.The transition plan began and the CTO was groomed to take over. That plan was shredded a week before it was to take place because the CTO suddenly found himself unemployed. The finance department was investigating his use of the company credit card and found thousands of dollars in personal charges, so he was unceremoniously terminated. Turmoil gripped the office as an entire department waited to find out who their new leader would be.Everyone except Bart put in a good word for Bob to become the next VP of IT. At minimum, they should hire someone from the outside. Pretty much anyone but Bart was the popular suggestion.Bart predictably put in a good word for himself and talked a big game about all of his expertise. He added security expert to his resume by talking about disabling his home WIFI’s SSID broadcast and changing the default password. Then there was the previously-unmentioned experience as an IT Project Manager where he apparently led several teams at Schmoeing.That Friday, that dreaded email with the subject New VP of IT came out. Being the most tenured person remaining while having "great leadership qualities," Bart got the promotion and Bob didn't. Many job search sites probably thought they were getting a DDoS attack over the weekend while everyone in Virtucon IT looked to abandon ship. Bart's reign of terror lasted 18 months before Virtucon realized they made a grave mistake. Bob, Alvin, and many others weren't around to see the way it all ended. [Advertisement] Forget logs. Next time you're struggling to replicate error, crash and performance issues in your apps - Think Raygun! Installs in minutes. Learn more.

Radio WTF Presents!Jump to transcriptWelcome back to Radio WTF. This week, we learn of the power of confidence.Soundcloud Links:Radio WTF: ConfidenceDirect Download:DownloadStarring (in order of appearance)Remy Porter... as Kyle