Can a Requirement for Credentials Be Overcome in Science Today?

In a wide-ranging interview on Wired , Freeman Dyson notes his pride in never having been awarded a PhD, despite having achieved world-reknown for bringing the mathematics of Richard Feynman's quantum theory to life. He suggests that the PhD system is: "...good for a very small number of people who are going to spend their lives being professors. But it has become now a kind of union card that you have to have in order to have a job, whether it's being a professor or other things, and it's quite inappropriate for that. It forces people to waste years and years of their lives sort of pretending to do research for which they're not at all well-suited. In the end, they have this piece of paper which says they're qualified, but it really doesn't mean anything. The Ph.D. takes far too long and discourages women from becoming scientists, which I consider a great tragedy. So I have opposed it all my life without any success at all."

I am interested in |. answers to the following question: Why has the PhD credential become so important for careers in science and research both inside and outside academia? My best CS professor only had an undergraduate degree, for example, and I never found cause to disrespect his authority on the subjects he taught because the basis of his authority was clear in his instruction.