Sunday's Site Outage
martyb writes:
Summary:
It wasn't just you; SoylentNews.org was down today (Sunday, 2021-01-03) for a few hours in the mid morning to early afternoon UTC. It seems to be back up and running, but there are some minor artifacts.
Background:
First sign was some CSS and Slashbox issues appearing on Saturday night. I was editing a story and when I tried to preview it, saw that the SlashBoxes that normally appeared on the LHS (Left-Hand Side) of the page were missing. A page refresh or two later, and things looked okay, again. A bit later I went to view a story and saw the same symptoms. This time a hard reload that ignored and cache on my system (Ctrl+F5) did the trick.
I popped onto IRC (Internet Relay Chat), reported these symptoms, and asking if anyone else was seeing the same thing. Received a couple confirmations.
Oh. Joy. And TMB (The Mighty Buzzard) still seemed to be away on vacation.
Oh well. Skipped on over to boron and ran a script to bounce the apache servers on fluorine and hydrogen. Popped back onto IRC, reported what I did, and asked if things were better. Got some affirmations. Yay!
Just in case, I hung around for another half hour or so to confirm the site was staying up and running okay. Looking good! After thanking everyone for their help, I wished everybody a good night and then headed to bed.
Sunday:
Shortly after I woke and attempted to visit the site, I was greeted by a message explaining the site was down due to DB issues. When I got back onto IRC, found that TMB was already hands-on. The site had crashed early in the morning. With the site already down, and it being Sunday morning, he decided to take advantage of the opportunity to make some backups and then do some maintenance work.
Status:
Site is back up, system loads seem back-to-normal, and things seem to be pretty much as they should be. Except... the Older Stuff slashbox that appears on the RHS (Right-Hand Side) of the main page seemed to be missing some entries. The newest entry as I write this is YouTube Class Action: Same IP Address Used To Upload 'Pirate' Movies and File DMCA Notice.
I suspect the missing entries will eventually start to stream in and repopulate the list.
tl;dr:
The DB crashed and took the site with it. TMB was soon on the scene and fixed the DB and did some other work. We're back up and running.
Thanks TMB!
Read more of this story at SoylentNews.