Now in Things White People Say Aren't Racist: Swastikas. Yup.
There are certain symbols that when seen in public make you go "Oh, they racist racist." At the top of that list would probably be the swastika and the confederate flag-symbols made popular by groups whose whole M.O was killing in the name of white supremacy. Apparently though, we've had it all wrong.