Iterating between theory and code

Yesterday I said on Twitter "Time to see whether practice agrees with theory, moving from LaTeX to Python. Wish me luck." I got a lot of responses to that because it describes the experience of a lot of people. Someone asked if I'd blog about this. The content is confidential, but I'll talk about the process. It's a common pattern.

I'm writing code based on a theoretical result in a journal article.

First the program gets absurd results. Theory pinpoints a bug in the code.

Then the code confirms my suspicions of an error in the paper.

The code also uncovers, not an error per se, but an important missing detail in the paper.

Then code and theory differ by about 1%. Uncertain whether this is theoretical approximation error or a subtle bug.

Then try the code for different arguments, ones which theory predicts will have less approximation error, and now code and theory differ by 0.1%.

Then I have more confidence in (my understanding of) the theory and in my code.