Dual numbers, part 1

I've recently been looking into a fascinating corner of mathematics that at first glance appears a little bit silly, but actually has far-reaching applications, from physics to numerical methods to machine learning. I thought I'd share what I've learned over the next few episodes.

I assume you recall what a complex number is, but perhaps not all of the details. A complex number is usually introduced as a pair of real numbers (a, b), where a is called the "real part" and b is called the "imaginary part".

A brief aside: it has always bugged me that these labels are unnecessarily value-laden. There is no particular "reality" that is associated with the real part; it is every bit as "imaginary" as the imaginary part. They might as well be called the "rezrov part" and the "gnusto part", but we're stuck with "real" and "imaginary". Moving on.

It would be perfectly sensible to notate these as a pair (a, b) - and indeed, when we characterize complex numbers as points on the complex plane, we often do. But it is also convenient to notate complex numbers in an algebraic form a+bi, where the i indicates the imaginary part.

Finally, we have well-known rules for adding and multiplying complex numbers. The fundamental rule is that (0+1i)² is equal to -1+0i, and the rest of the rules follow from there. For complex numbers:

(a+bi) + (c+di) = (a+c) + (b+d)i(a+bi)(c+di) = (ac-bd) + (ad+bc)i

Division is a little bit tricky and I won't write it all out here; look it up if you do not remember. We can similarly go on to derive rules for complex exponentiation, trigonometry, and so on.

Let's switch gears slightly. It is very common in mathematics, and especially calculus, to reason about a quantity "plus a little bit". That is, any old quantity, plus some positive amount that is not zero, but is much smaller than one, and really, really small compared to the original quantity. What "really, really small" means is of course dependent on context; maybe one-millionth of the original quantity is small. Maybe one-millionth is still pretty big, but one-trillionth, or one part in a googolplex, or whatever, is "really small" in our context. I'm being deliberately vague and hand-wavy here; just run with it.

Let's call our really small positive quantity I, which is the lower-case Greek letter epsilon; it is traditionally used in calculus to represent a very small quantity. Let's suppose our value is a, and we have a very small amount added to it, so that would be a+I. Hmm. What if we doubled that quantity? Plainly that would be 2a+2I. So scalar multiplication works as you'd expect. It seems reasonable that if we can have 2I then we can have 3I or for that matter 3.14I or bI for any real value of b. So once again we have an (a, b) pair of numbers representing a quantity, which we notate as a+bI. What are the algebraic rules of this thing? It certainly makes sense that addition should work as we expect, the same as complex addition:

(a+bI) + (c+dI) = (a+c) + (b+d)I

This should match our intuitions. The sum of "a plus a bit" and "c plus a bit" should be "a + c plus a bit" if there is justice in the world. What about multiplication? Let's just multiply it out using ordinary algebra:

(a+bI)(c+dI) = ac + (ad+bc)I + bdI2

We seem to have introduced an I² term there. What are we going to do with that? Remember, I is not zero but is very, very small indeed, so that thing squared must be super tiny. So tiny that we can just ignore it entirely! (Remember, I am hand-waving egregiously here.) Let's just say that I² is by definition zero, just as we said that i² is by definition -1, and see what happens. So we have:

(a+bI)(c+dI) = ac + (ad+bc)I

Again, this should match our intuitions. The product of something a bit bigger than a with something a bit bigger than c should be a bit bigger than the product of a and c.

What about division? Division by zero is illegal, and division by zero plus a tiny amount I think should also be illegal. Seems reasonable. What then is (a+bI)/(c+dI) when c is not zero? Well, if it is not zero then we can multiply top and bottom by c-dI:

a+bI (c-dI)(a+bI) ac + (bc-da)I a bc-da---- = ----------- = -------------- = - + ----- Ic+dI (c-dI)(c+dI) c2 c c2

Once again our intuition is satisfied: the quotient of "something close to a" divided by "something close to c" is close to the quotient of a and c.

This is a pretty neat number system; it is called the dual number system and it was introduced to the world in 1873 by the English mathematician William Clifford. But so far it might look like it is just a curiosity; however, it is extremely useful.

Next time on FAIC: we will implement the dual number system in C#, and discover a surprising property.