by Remy Porter on (#26KKM)
Merry Holidays, and as we coast into the new year, it's time for us to reflect on some of the best/worst moments of the past one. Today, we start things off with a BANG, and a reminder that electricity is a complicated beast. --RemyMike had a different perspective on wiring and cable management- because he worked at a factory which made wires and cables. It was the early–90s, and he was in charge of babysitting a couple of VAXes and their massive, 85lb hard drives. It was an easy job: the users knew the system and needed very little support, the VAXes were practically unstoppable, and the backup battery system could keep the entire thing running for over an hour.The computers supported HR and accounting, which meant as the year ticked towards its end, Mike had to prep the system for its heaviest period of use- the year end closing processes. Through the last weeks of December, his users would be rushing to get their reports done and filed so they could take off early and enjoy the holidays.Mike had been through this rodeo before, but Reginald, the plant manager, called him up to his office. There was going to be a wrench in the works this year. Mike sat down in Reginald’s cramped and stuffy office next to Barry, the chief electrician for the plant.“Our factory draws enough power from the main grid that the power company has a substation that’s essentially dedicated to us,†Reginald explained. “But they’ve got a problem with some transformers loaded with PCBs that they want to replace, so they need to shut down that substation for a week while they do the work.â€The factory portion was easy to deal with- mid-December was a period when the assembly line was usually quiet anyway, so the company could just shift production to another facility that had some capacity. But there was no way their front-office could delay their year-end closing processes.“So, to keep the office running, we’ll be bringing in a generator truck,†Reginald said. “And that means we’re going to need to set up a cut-over from the main grid to the generator.â€From the computer-room side, the process was easy, but that didn’t stop Mike from writing up a checklist, taping it to the wall beside his desk and sharing a copy with Barry. Before the generator truck arrived, he’d already tested the process several times, ensuring that he could go from mains power to battery and back to mains power without any problem.The generator truck arrived a week before the changeover. The electricians ignored it for a few days. Just as Mike was starting to get worried about deadlines, he looked out a window and saw a trio of electricians, lead by Barry, starting to connect cables to it. Later that day, when Mike left to go to lunch, he walked past the generator truck, and noticed something odd about the cables- they were clearly single phase power cables.Typical residential power systems are single-phase alternating current- one peak, one trough. This creates “dead†moments in the cycle, where no work is being done. That’s fine for residential use- but industrial systems need three-phase power- three offset AC cycles that, when added together, guarantee current is always flowing.“Hey,†Mike said to one of the electricians loitering near the truck, “you’re planning to run some three-phase cabling, right?â€â€œNope. The factory’s shut down- this thing is just gonna run lights and heating.â€â€œAnd computers,†Mike said. “The hard drives need three-phase power.â€â€œWe’ll have to order some parts,†the electrician said.A few more days ticked by with no three-phase connections, and Mike checked in with the electricians again.“The parts are coming.â€At this point, Reginald came down from his office to the computer room. “Mike, Barry’s telling me you’re being a disruption.â€â€œWhat?â€â€œLook, there’s a chain of command,†Reginald said. “And you can’t tell the electricians how to do their job.â€â€œI’m no-â€â€œFrom now on, if you have any concerns, bring them to me.â€The day of the cut-over, the three-phase cabling finally arrived. Barry and his electricians quickly strung those cables from the generator. Mike wasn’t exactly watching them like a hawk, but he was worried- there was no way they could test the configuration while they were working so hastily. Unlike single-phase power, three-phase power could be out-of-sync, which would wreak havoc on the hard drives. He thought about bringing this up to the electricians, but recalled Reginald’s comments on the chain of command. He went to Reginald instead.“Mike,†Reginald said, shaking his head. “I know you computer guys think you know everything, but you’re not an electrician. This is a factory, Mike, and you’ve got to learn a little about what it’s like to work for a living.â€Now, the electricians and Mike needed to coordinate their efforts, but Reginald strictly enforced the idea of the chain of command. One electrician would power on the generator and announce it, “Generator’s running.†Another electrician would relay this to Barry: “Generator’s running.†Barry would relay that to Reginald. “Generator’s on.†Reginald, finally, would tell Mike.