John D. Cook

For an article to be published, it has to be published somewhere. Each journal has a responsibility to select articles relevant to its readership. Articles that make new connections might be unpublishable because they don’t fit into a category. For example, I’ve seen papers rejected by theoretical journals for being too applied, and the same papers […]

One of my lightbulb moments in college was when my professor, Jim Vick, explained the Lagrange multiplier theorem. The way I’d seen it stated in a calculus text gave me no feel for why it should be true, but his explanation made sense immediately. Suppose f(x) is a function of several variables, i.e. x is a vector, and g(x) = c […]

Suppose you did a pilot study with 10 subjects and found a treatment was effective in 7 out of the 10 subjects. With no more information than this, what would you estimate the probability to be that the treatment is effective in the next subject? Easy: 0.7. Now what would you estimate the probability to be […]

I’m starting a new Twitter account for logic and formal methods: @FormalFact. Expect to see tweets about constructive logic, type theory, formal proofs, proof assistants, etc. The image for the account is a bowtie, a pun on formality. It’s also the symbol for natural join in relational algebra.

The number 3435 has the following curious property: 3435 = 33 + 44 + 33 + 55. It is called a Münchausen number, an allusion to fictional Baron Münchausen. When each digit is raised to its own power and summed, you get the original number back. The only other Münchausen number is 1. At least in […]

Beta reduction is essentially function application. If you have a function described by what it does to x and apply it to an argument t, you rewrite the xs as ts. The formal definition of β-reduction is more complicated than this in order to account for free versus bound variables, but this informal description is sufficient […]

I was catching up on Engines of our Ingenuity episodes this evening when the following line jumped out at me: If I flip a coin a million times, I’m virtually certain to get 50 percent heads and 50 percent tails. Depending on how you understand that line, it’s either imprecise or false. The more times you […]

Experience with the normal distribution makes people think all distributions have (useful) sufficient statistics [1]. If you have data from a normal distribution, then the sufficient statistics are the sample mean and sample variance. These statistics are “sufficient” in that the entire data set isn’t any more informative than those two statistics. They effectively condense […]

The other day I found myself saying that I preferred org-mode files to Jupyter notebooks because with org-mode, what you see is what you get. Then I realized I was using “what you see is what you get” (WYSISYG) in exactly the opposite of the usual sense. Jupyter notebooks are WYSIWYG in the same sense […]

Most programmers are at one extreme or another when it comes to floating point arithmetic. Some are blissfully ignorant that anything can go wrong, while others believe that danger lurks around every corner when using floating point. The limitations of floating point arithmetic are something to be aware of, and ignoring these limitations can cause problems, like crashing […]

Here’s an interesting quote omparing writing proofs and writing programs: Building proofs and programs are very similar activities, but there is one important difference: when looking for a proof it is often enough to find one, however complex it is. On the other hand, not all programs satisfying a specification are alike: even if the […]

Statistics can be useful, even if it’s idealizations fall apart on close inspection. For example, take English letter frequencies. These frequencies are fairly well known. E is the most common letter, followed by T, then A, etc. The string of letters “ETAOIN SHRDLU” comes from the days of Linotype when letters were arranged in that order, […]

When people ask me what calculus is, my usual answer is “the mathematics of change,” studying things that change continually. Algebra is essentially static, studying things frozen in time and space. Calculus studies things that move, shapes that bend, etc. Algebra deals with things that are exact and consequently can be fragile. Calculus deals with […]

As systems get larger and more complex, we need new tools to test whether these systems are correctly specified and implemented. These tools may not be new per se, but they may be applied with new urgency. Dimensional analysis is a well-established method of error detection. Simply checking that you’re not doing something like adding […]

This evening I ran across an unexpected reference to spherical trigonometry: Thomas Hales’ lecture on lessons learned from the formal proof of the Kepler conjecture. He mentions at one point a lemma that was awkward to prove in its original form, but that became trivial when he looked at its spherical dual. The sides of […]

The set of primitive recursive (PR) functions is the smallest set of functions of several integer arguments satisfying five axioms: Constant functions are PR. The function that picks the ith element of a list of n arguments is PR. The successor function S(n) = n+1 is PR. PR functions are closed under composition. PR functions are closed under primitive […]

There’s no notion of continuity in linear algebra per se. It’s not part of the definition of a vector space. But a finite dimensional vector space over the reals is isomorphic to a Euclidean space of the same dimension, and so we usually think of such spaces as Euclidean. (We’ll only going to consider real vector spaces […]

The applied math featured here tends to be fairly sophisticated, but there’s a lot you can do with the basics as we’ll see in the following interview with Trevor Dawson of Borrego Solar, a company specializing in grid-connected solar PV systems. JC: Can you say a little about yourself? TD: I’m Trevor Dawson, I’m 25, born in the […]

Let S be the area of triangle T in three-dimensional space. Let A, B, and C be area of the projections of T to the xy, yz, and xz planes respectively. Then S2 = A2 + B2 + C2. There’s an elegant proof of this theorem here using differential forms. Below I’ll sketch a less elegant but more elementary proof. You could prove the identity above by using the fact that the […]

Neal Stephenson coins a useful word Amistics in his novel Seveneves: … it was a question of Amistics, which was a term that had been coined ages ago by a Moiran anthropologist to talk about the choices that different cultures made as to which technologies they would, and would not, make part of their lives. […]

Suppose you have an n by m chessboard. How many ways can you cover the chessboard with dominoes? It turns out there’s a remarkable closed-form solution: Here are some questions you may have. But what if n and m are both odd? You can’t tile such a board with dominoes. Yes, in that case the formula evaluates to […]

Amplitude modulated signals sound rough to the human ear. The perceived roughness increases with modulation frequency, then decreases, and eventually disappears. The point where roughness reaches is maximum depends on the the carrier signal, but for a 1 kHz tone roughness reaches a maximum for modulation at 70 Hz. Roughness also increases as a function […]

There are two uses of the word scalar, one from linear algebra and another from tensor calculus. In linear algebra, vector spaces have a field of scalars. This is where the coefficients in linear combinations come from. For real vector spaces, the scalars are real numbers. For complex vector spaces, the scalars are complex numbers. […]

In the first post in this series I mentioned several apparently unrelated things that are all called tensors, one of these being objects that behave a certain way under changes of coordinates. That’s what we’ll look at this time. In the second post we said that a tensor is a multilinear functional. A k-tensor takes k vectors and […]

In the previous post, we defined the tensor product of two tensors, but you’ll often see tensor products of spaces. How are these tensor products defined? Tensor product splines For example, you may have seen tensor product splines. Suppose you have a function over a rectangle that you’d like to approximate by patching together polynomials so that […]

The simplest definition of a tensor is that it is a multilinear functional, i.e. a function that takes several vectors, returns a number, and is linear in each argument. Tensors over real vector spaces return real numbers, tensors over complex vector spaces return complex numbers, and you could work over other fields if you’d like. A dot product is […]

The word “tensor” is shrouded in mystery. The same term is applied to many different things that don’t appear to have much in common with each other. You might have heared that a tensor is a box of numbers. Just as a matrix is a rectangular collection of numbers, a tensor could be a cube of […]

“Mathematics compares the most diverse phenomena and discovers the secret analogies that unite them.” — Joseph Fourier The above quote makes me think of a connection Fourier made between triangles and thermodynamics. Trigonometric functions were first studied because they relate angles in a right triangle to ratios of the lengths of the triangle’s sides. For […]

“Research is what I’m doing when I don’t know what I’m doing.” — Wernher von Braun I find Shinichi Mochizuki’s proof of the abc conjecture fascinating. Not the content of the proof—which I do not understand in the least—but the circumstances of the proof. Most mathematics, no matter how arcane it appears to outsiders, is […]

I had a discussion recently about whether things are really continuous in the real world. Strictly speaking, maybe not, but practically yes. The same is true of all mathematical properties. There are no circles in the real world, not in the Platonic sense of a mathematical circle. But a circle is a very useful abstraction, […]

Fractional integrals are easier to define than fractional derivatives. And for sufficiently smooth functions, you can use the former to define the latter. The Riemann-Liouville fractional integral starts from the observation that for positive integer n, This motivates a definition of fractional integrals which is valid for any complex α with positive real part. Derivatives and integrals are […]

Eberhard Zwicker proposed a model for combining several psychoacoustic metrics into one metric to quantify annoyance. It is a function of three things: N5, the 95th percentile of loudness, measured in sone (which is confusingly called the 5th percentile) ωS, a function of sharpness in asper and of loudness ωFR, fluctuation strength (in vacil), roughness (in […]

I’ve posted an online calculator to convert between two commonly used units of loudness, phon and sone. The page describes the purpose of both units and explains how to convert between them.

The most recent episode of 99% Invisible tells the story of the Corp of Engineers’ enormous physical model of the Mississippi basin, nearly half of the area of the continental US. Spanning over 200 acres, the model was built during WWII and was shut down in 1993. Here are some of my favorite lines from the show: […]

There are many ways to define fractional derivatives, and in general they coincide on nice classes of functions. A long time ago I wrote about one way to define fractional derivatives using Fourier transforms. From that post: Here’s one way fractional derivatives could be defined. Suppose the Fourier transform of f(x) is g(ξ). Then for […]

Maybe from the headline you were expecting a blank post? No, that’s not where I’m going. Yesterday I was on Amazon.com and noticed that nearly all the books they recommended for me were either about Lisp or mountain climbing. I thought this was odd, and mentioned it on Twitter. Carl Vogel had a witty reply: […]

There are four basic types of integral equations. There are many other integral equations, but if you are familiar with these four, you have a good overview of the classical theory. All four involve the unknown function φ(x) in an integral with a kernel K(x, y) and all have an input function f(x). In all […]

Fluctuation strength is similar to roughness, though at much lower modulation frequencies. Fluctuation strength is measured in vacils (from vacilare in Latin or vacillate in English). Police sirens are a good example of sounds with high fluctuation strength. Fluctuation strength reaches its maximum at a modulation frequency of around 4 Hz. For much higher modulation frequencies, one […]

Acoustic roughness is measured in aspers (from the Latin word for rough). An asper is the roughness of a 1 kHz tone, at 60 dB, 100% modulated at 70 Hz. That is, the signal (1 + sin(140πt)) sin(2000πt) where t is time in seconds. Here’s what that sounds like (if you play this at 60 […]

The Mittag-Leffler function is a generalization of the exponential function. Since k!= Γ(k + 1), we can write the exponential function’s power series as and we can generalize this to the Mittag=Leffler function which reduces to the exponential function when α = β = 1. There are a few other values of α and β for […]

A few weeks ago I started a series of posts on various things you could do with a functional fold. In the first post I mentioned that the idea came from a paper by Brian Beckman on Kalman filters and folds: This post was inspired by a paper by Brian Beckman (in progress) that shows how […]

I heard someone say the other day that the advantage of formal software validation methods is that they let you explore the corners, cases where intuition doesn’t naturally take you. This made me think of corners in the geometric sense. If you have a sphere in a box in high dimensions, nearly all the volume […]

The best way to develop software doesn’t exist. There’s only the best way that you know of, for a particular problem, with particular people, given their skills, experience, and constraints.

Presentation order People best understand computer programs in a different order than compilers do. This is a key idea of literate programming, and one that distinguishes literate programs from heavily commented programs. Traditional source code, no matter how heavily commented, is presented in the order dictated by the compiler. The computer is the primary audience. Literate programming is more humanistic in […]

This post explores a sentence from the book Single Digits: Any number in Pascal’s triangle that is not in the outer two layers will appear at least three times, usually four. Pascal’s triangle contains the binomial coefficients C(n, r) where n ranges over non-negative numbers and r ranges from 0 to n. The outer layers are the elements with r equal to […]

One of the things I learned from my tenure as a software project manager was that a project is more likely to succeed if there’s a way to get where you want to go continuously. You want to move a project from A to B gradually, keeping a working code base all along the way. At […]

The best teams have people with complementary skills, but similar work ethic. Academic assignments are the opposite. There’s not much variation in skills, in part because students haven’t yet developed specialized skills, and in part because students are in the same class because they have similar interests. The biggest variation is likely to be work ethic. It’s […]

Suppose two captains, A and B, are choosing people for their teams. To make things fair, the two captains alternate choices: A, B, A, B, etc. This is much better than simply letting A choose his team first and leaving B the dregs, but it still gives A a substantial advantage. If each captain picks the best remaining […]

If a real polynomial in one variable is a sum of squares, it obviously cannot be negative. For example, the polynomial p(x) = (x2 – 3)2 + (x + 7)2 is obviously never negative for real values of x. What about the converse: If a real polynomial is never negative, is it a sum of […]

The dormitory I lived in as an undergraduate had its own five-digit zip code at one time. It was rumored to be the largest dorm in the US, or maybe the largest west of the Mississippi, or something like that. There were about 3,000 of us living there. Although the dorm had enough people to justify […]