Discovering and recovering from PostgreSQL corruption on Matrix.org

Richard van der Hoff, a member of the team that runs the Matrix.org homeserver,has writtena detailed blog post about diagnosing and fixing a problem where Matrix roomswould simply stop working:

We know that there are plenty of users out there who will have beenaffected by the problem, and found themselves unable to communicate asa result. We very much share your frustration, and we'd like toapologise for the disruption to service.

With that said, we're glad that we were able to get to the bottomof most of the problem, and get the lost data restored within arelatively short time. If nothing else, hopefully this blog post willbe of use to future generations faced with Postgres indexcorruption!