John D. Cook

Six blog posts on mathematics and privacy: Randomized response and Bayes’ theorem Big aggregate queries can still violate privacy Quantifying privacy loss Database anonymization for testing Toxic pairs and re-identification Adding Laplacian or Gaussian noise to a database

Ten years ago I started writing this blog. Since then I’ve written about 2700 posts. Thank you all for reading, commenting, and sharing. Update: For highlights of my posts over the years, see Tim Hopper’s post John Cook’s Ten Year Blogging Endeavour.

A new record for the largest known prime was announced yesterday: This number has 23,249,425 digits when written in base 10. In base 2, 2p – 1 is a sequence of p ones. For example, 31 = 25 -1 which is 11111 in binary. So in binary, the new record prime is a string of 77,232,917 […]

The Nyquist sampling theorem says that a band-limited signal can be recovered from evenly-spaced samples. If the highest frequency component of the signal is fc then the function needs to be sampled at a frequency of at least the Nyquist frequency 2fc. Or to put it another way, the spacing between samples needs to be […]

Someone wrote to me the other day asking if I could explain a probability example from the Wall Street Journal. (“Proving Investment Success Takes Time,” Spencer Jakab, November 25, 2017.) Victor Haghani … and two colleagues told several hundred acquaintances who worked in finance that they would flip two coins, one that was normal and […]

Exponential sums can make intricate patterns. Last year I made a page that displays a different page each day, using the month, day, and year as parameters in the expression below. The images plot the partial sums of this sum. This was yesterday’s image. Today’s image is surprisingly plain if we use y = 18. This […]

Retiring professor Leonard Fabiano contacted me looking to give away a set of technical books, mostly chemical engineering books. If you’re interested please email him at lenfab@live.com. Here are the books: Click on the image to see a larger version. Two titles are not possible to read in the photo. These are Conduction of heat […]

Robert Banks’s book Towing Icebergs, Falling Dominoes, and Other Adventures in Applied Mathematics describes the Eiffel Tower’s shape as approximately the logarithmic curve where y* and x0 are chosen to match the tower’s dimensions. Here’s a plot of the curve: And here’s the code that produced the plot: from numpy import log, exp, linspace, vectorize import matplotlib.pyplot […]

These have been the most popular math-related posts here this year. Golden powers are nearly integers How efficient is Morse code? Finding numbers in pi Common words used as technical terms Sierpinski triangle strikes again See also a list of the top five computing-related posts.

In the previous post, I alluded to using Hermite polynomials in conjunction with higher-order Laplace approximation. In this post I’ll expand on what that means. Hermite polynomials are orthogonal polynomials over the real line with respect to the weight given by the standard normal distribution. (There are two conventions for defining Hermite polynomials, what Wikipedia […]

Yesterday’s post presented the most common form of Laplace approximation, the second order version, and mentioned in passing that there are higher order versions. The extension to higher order is not trivial, so post gives a high level overview of how you’d do it. The previous post looked at integrating exp( log( g(x) ) ) by […]

Define and This integral comes up in Bayesian logisitic regression with a uniform (improper) prior. We will use this integral to illustrate a simple case of Laplace approximation. The idea of Laplace approximation is to approximate the integral of a Gaussian-like function by the integral of a (scaled) Gaussian with the same mode and same […]

These have been the most popular computing-related posts here this year. Programming language life expectancy SHA1 no longer recommended, but hardly a failure The most disliked programming language Improving on the Unix shell One practical application of functional programming I plan to post a list of the top file math-related posts soon.

One of the things about academia that most surprised and disappointed me was the low regard for scholarship. Exploration is tolerated as long as it results in a profusion of journal articles, and of course grant money, but is otherwise frowned upon. For example, I know someone who ruined his academic career by writing a massive […]

Tyler Cowen’s latest blog post gives advice for learning about modern China. He says that “books about sequences of dynasties are mind-numbing and not readily absorbed” and recommends finding other entry points before reading about dynasties. Find an “entry point” into China of independent intrinsic interest to you, be it basketball, artificial intelligence, Chinese opera, […]

Here’s a fact that has been rediscovered many times in many different contexts: The way you parenthesize matrix products can greatly change the time it takes to compute the product. This is important, for example, for the back propagation algorithm in deep learning. Let A, B, and C be matrices that are compatible for multiplication. Then (AB)C = A(BC). […]

Most sources that present Taylor’s theorem for functions of several variables stop at second order terms. One reason is that one or two terms are good enough for many applications. But the bigger reason is that things get more complicated when you include higher order terms. The kth order term in Taylor’s theorem is a rank k […]

I’ll be presenting at a webinar on Wednesday, December 13 at 1:00 PM Eastern. The title of the presentation is “Seven questions a statistician and answer for an attorney.” I will discuss, among other things, when common sense applies and when correct analysis can be counter-intuitive. There will be ample time at the end of […]

This post relates moment generating functions to the Laplace transform and to exponential generating functions. It also brings in connections to the z-transform and the Fourier transform. Thanks to Brian Borchers who suggested the subject of this post in a comment on a previous post on transforms and convolutions. Moment generating functions The moment generating function (MGF) of […]

The Shannon wavelet has an interesting plot: Given the complexity of the plot, the function definition is surprisingly simple: The Fourier transform is even simpler: it’s the indicator function of [-2π, -π] ∪ [π, 2π], i.e. the function that is 1 on the intervals [-2π, -π] and [π, 2π] but zero everywhere else. The Shannon […]

There are many theorems of the form where f and g are functions, T is an integral transform, and * is a kind of convolution. In words, the transform of a convolution is the product of transforms. When the transformation changes, the notion of convolution changes. Here are three examples. Fourier transform and convolution With the Fourier transform […]

Last week I wrote about Jentzsch’s theorem. It says that if the power series of function has a finite radius of convergence, the set of zeros of the partial sums of the series will cluster around and fill in the boundary of convergence. This post will look at the power series for the gamma function […]

From Orthogonal Polynomials and Special Functions by Richard Askey: At first the results we needed were in the literature but after a while we ran out of known results and had to learn something about special functions. This was a very unsettling experience for there were very few places to go to really learn about […]

The last digits of Fibonacci numbers repeat with period 60. This is something I’ve written about before. The 61st Fibonacci number is 2504730781961. The 62nd is 4052739537881. Since these end in 1 and 1, the 63rd Fibonacci number must end in 2, etc. and so the pattern starts over. It’s not obvious that the cycle should […]

Take a function that has a power series with a finite radius of convergence. Then the zeros of the partial sums will be dense around the boundary of convergence. That is Jentzsch’s theorem. Here are a couple plots to visualize Jentzsch’s theorem using the plotting scheme described in this post. First, we take the function f(z) […]

This post shows a connection between three families of orthogonal polynomials—Legendre, Chebyshev, and Jacobi—and the beta distribution. Legendre, Chebyshev, and Jacobi polynomials A family of polynomials Pk is orthogonal over the interval [-1, 1] with respect to a weight w(x) if whenever m ≠ n. If w(x) = 1, we get the Legendre polynomials. If w(x) = (1 […]

I’ve mentioned the Runge phenomenon in a couple posts before. Here I’m going to go into a little more detail. First of all, the “Runge” here is Carl David Tolmé Runge, better known for the Runge-Kutta algorithm for numerically solving differential equations. His name rhymes with cowabunga, not with sponge. Runge showed that polynomial interpolation […]

The previous post compared bits of information to answers in a game of Twenty Questions. The optimal strategy for playing Twenty Questions is for each question to split the remaining possibilities in half. There are a couple ways to justify this strategy: mixmax and average. The minmax approach is to minimize the worse thing that […]

I’ve had data privacy on my mind a lot lately because I’ve been doing some consulting projects in that arena. When I saw a tweet from Tim Hopper a little while ago, my first thought was “How many bits of PII is that?”. [1] π Things Only Left Handed Introverts Over 6′ 5″ with O+ […]

The Pareto probability distribution has density for x ≥ 1 where a > 0 is a shape parameter. The Pareto distribution and the Pareto principle (i.e. “80-20” rule) are named after the same person, the Italian economist Vilfredo Pareto. Samples from a Pareto distribution obey Benford’s law in the limit as the parameter a goes to […]

Random number generation is typically a two step process: first generate a uniformly distributed value, then transform that value to have the desired distribution. The former is the hard part, but also the part more likely to have been done for you in a library. The latter is relatively easy in principle, though some distributions […]

The beta-binomial model is the “hello world” example of Bayesian statistics. I would call it a toy model, except it is actually useful. It’s not nearly as complicated as most models used in application, but it illustrates the basics of Bayesian inference. Because it’s a conjugate model, the calculations work out trivially. For more on […]

Suppose you want to prevent your data science team from being able to find out information on individual customers, but you do want them to be able to get overall statistics. So you implement two policies. Data scientists can only query aggregate statistics, such as counts and averages. These aggregate statistics must be based on […]

It’s easy to visualize function from two real variables to one real variable: Use the function value as the height of a surface over its input value. But what if you have one more dimension in the output? A complex function of a complex variable is equivalent to a function from two real variables to two […]

The Kullback-Leibler divergence between two probability distributions is a measure of how different the two distributions are. It is sometimes called a distance, but it’s not a distance in the usual sense because it’s not symmetric. At first this asymmetry may seem like a bug, but it’s a feature. We’ll explain why it’s useful to measure […]

As companies get into data analysis for the first time, many of them are going to start by making the same mistakes that were common a century ago, then gradually recapitulate the development of modern statistics.

Fitting a polynomial to a function at more points might not produce a better approximation. This is Faber’s theorem, something I wrote about the other day. If the function you’re interpolating is smooth, then interpolating at more points may or may not improve the fit of the interpolation, depending on where you put the points. […]

Fourier-Bessel series are analogous to Fourier series. And like Fourier series, they converge pointwise near a discontinuity with the same kind of overshoot and undershoot known as the Gibbs phenomenon. Fourier-Bessel series Bessel functions come up naturally when working in polar coordinates, just as sines and cosines come up naturally when working in rectangular coordinates. […]

I’m experimenting with making animated versions of the kinds of images I wrote about in my previous post. Here’s an animated version of the exponential sum of the day for 12/4/17. Why that date? I wanted to start with something with a fairly small period, and that one looked interesting. I’ll have to do something […]

How do you create a database for testing that is like your production database? It depends on in what way you want the test database to be “like” the production one. Replacing sensitive data Companies often use an old version of their production database for testing. But what if the production database has sensitive information […]

The exponential sum of the day draws a line between consecutive partial sums of where m, d, and y are the current month, day, and two-digit year. The four most recent images show how different these plots can be. These images are from 10/30/17, 10/31/17, 11/1/17, and 11/2/17. Consecutive dates often produce very different images for a couple […]

I just got an evaluation copy of The Best Writing on Mathematics 2017. My favorite chapter was Inverse Yogiisms by Lloyd N. Trefethen. Trefethen gives several famous Yogi Berra quotes and concludes that Yogiisms are statements that, if taken literally, are meaningless or contradictory or nonsensical or tautological—yet nevertheless convey something true. An inverse yogiism is […]

According to this post from Stack Overflow, Perl is the most disliked programming language. I have fond memories of writing Perl, though it’s been a long time since I used it. I mostly wrote scripts for file munging, the task it does best, and never had to maintain someone else’s Perl code. Under different circumstances […]

Let ω(n) be the number of distinct prime factors of x. A theorem of Landau says that for N large, then for randomly selected positive integers less than N, ω-1 has a Poisson(log log N) distribution. This statement holds in the limit as N goes to infinity. Apparently N has to be extremely large before […]

Yesterday I heard that the county I live in, Harris County, is the 3rd largest is the United States. (In population. It’s nowhere near the largest in area.) Somehow I’ve lived here a couple decades without knowing that. Houston is the 4th largest city in the US, so it’s no shock that Harris County the […]

I was playing around with something in Mathematica and one of the images that came out of it surprised me. It’s a contour plot for the system function of a low pass filter. H[z_] := 0.05634*(1 + 1/z)*(1 - 1.0166/z + 1/z^2) / ((1 - 0.683/z)*(1 - 1.4461/z + 0.7957/z^2)) ContourPlot[ Arg[H[Exp[I (x + I […]

Series solutions to differential equations can be grubby or elegant, depending on your perspective. Power series solutions At one level, there’s nothing profound going on. To find a series solution to a differential equation, assume the solution has a power series, stick the series into the equation, and solve for the coefficients. The mechanics are […]

You can find any integer you want as a substring of the digits in π. (Probably. See footnote for details.) So you could encode a number by reporting where it appears. If you want to encode a single digit, the best you can do is break even: it takes at least one digit to specify […]

Terry Tao’s most recent blog post looks at the Chowla conjecture theoretically. This post looks at the same conjecture empirically using Python. (Which is much easier!) The Liouville function λ(n) is (-1)Ω(n) where Ω(n) is the number of prime factors of n counted with multiplicity. So, for example, Ω(9) = 2 because even though 9 […]

Time series analysis and digital signal processing are closely related. Unfortunately, the two fields use different terms to refer to the same things. Suppose you have a sequence of inputs x[n] and a sequence of outputs y[n] for integers n. Moving average / FIR If each output depends on a linear combination of a finite number of previous […]