John D. Cook

Our adventure starts with the following ordinary differential equation: Analytic solution We can solve this equation in closed-form, depending on your definition of closed-form, by multiplying by an integrating factor. The left side factors to and so The indefinite integral above cannot be evaluated in elementary terms, though it can be evaluated in terms of […]

Ratios of gamma functions come up often in applications. If the two gamma function arguments differ by an integer, then it’s easy to calculate their ratio exactly by using (repeatedly if necessary) the fact at Γ(x + 1) = x Γ(x). If the arguments differ by 1/2, there is no closed formula, but the there […]

Differential privacy adds Laplace-distributed random noise to data to protect individual privacy. (More on that here.) Although it’s simple to generate Laplacian random values, the Laplace distribution is not always one of the built-in options for random number generation libraries. The Laplace distribution with scale β has density The Laplace distribution is also called the double […]

I heard somewhere that Pluto receives more sunlight than you might think, enough to read by, and that sunlight on Pluto is much brighter than moonlight on Earth. I forget where I heard that, but I’ve done a back-of-the-envelope calculation to confirm that it’s true. Pluto is about 40 AU from the sun, i.e. forty times […]

Suppose you know a number is between 30 and 42. You want to guess the number while minimizing how wrong you could be in the worst case. Then you’d guess the midpoint of the two ends, which gives you 36. But suppose you want to minimize the worst case relative error? If you chose 36, […]

The previous post looked at how much information is contained in zip codes. This post will look at how much information is contained in someone’s age, birthday, and birth date. Combining zip code with birthdate will demonstrate the plausibility of Latanya Sweeney’s famous result [1] that 87% of the US population can be identified based […]

If you know someone’s US zip code, how much do you know about them? We can use entropy to measure the amount of information in bits. To quantify the amount of information in a zip code, we need to know how many zip codes there are, and how evenly people are divided into zip codes. […]

Dark debt is a form of technical debt that is invisible until it causes failures. The term was coined in the STELLA conference and codified in the conference report. Dark debt is found in complex systems and the anomalies it generates are complex system failures. Dark debt is not recognizable at the time of creation. … It […]

Given two functions f and g, the product rule tells you how to take the first derivative of their product, and the chain rule tells you how to take the first derivative of their composition. What if you want to take higher order derivatives? You could repeatedly apply basic calculus rules, but there are formulas […]

Pick random numbers uniformly between 0 and 1, adding them as you go, and stop when you get a result bigger than 1. How many numbers would you expect to add together on average? You need at least two samples, and often two are enough, but you might get any number, and those larger numbers […]

Inverting a power series As a student, one of the things that seemed curious to me about power series was that a function might have a simple power series, but its inverse could be much more complicated. For example, the coefficients in the series for arctangent have a very simple pattern but the coefficients in […]

Here’s an identity relating Fibonacci numbers and binomial coefficients. You can think of it as a finite sum or an infinite sum: binomial coefficients are zero when the numerator is an integer and the denominator is a larger integer. See the general definition of binomial coefficients. Source: Sam E. Ganis. Notes on the Fibonacci Sequence. The […]

This evening, after I got off a phone call discussing a project with a colleague, I thought “Huh, I guess you could call that project management.” I worked as a project manager earlier in my career, but what I’m doing now feels completely different and much more pleasant. Strip away the bureaucracy and politics, and […]

There’s now an “Animate” link on the exponential sum pages that lets you watch the curves being drawn. Sometimes these are surprising. The plot of the partial sums might bounce all over in the process of filling in a very symmetric plot. Here’s an example of that.

Probability is full of theorems that say that probability density approximates another as some parameter becomes large. All the dashed lines in the diagram below indicate a relationship like this. You can find details of what everything in the diagram means here. How can you quantify these approximations? One way is to use Kullback-Leibler […]

I’ve run into potential polynomials a lot lately. Most sources I’ve seen are either unclear on how they are defined or use a notation that doesn’t sit well in my brain. Also, they’ll either give an implicit or explicit definition but not both. Both formulations are important. The implicit formulation suggests how potential polynomials arise […]

Bell polynomials come up naturally in a variety of contexts: combinatorics, analysis, statistics, etc. Unfortunately, the variations on Bell polynomials are confusingly named. Analogy with differential equations There are Bell polynomials of one variable and Bell polynomials of several variables. The latter are sometimes called “partial” Bell polynomials. In differential equations, “ordinary” means univariate (ODEs) […]

If you visit this blog once in a while, here are a few ways to hear from me more regularly. Subscription You can subscribe to the blog via RSS or email. I often use SVG images because they look great on a variety of devices, but most email clients won’t display that format. If you […]

A “squircle” is a sort of compromise between a square and circle, but one that differs from a square with rounded corners. It’s a shape you’ll see, for example, in some of Apple’s designs. A straight line has zero curvature, and a circle with radius r has curvature 1/r. So in a rounded square the […]

Suppose you have a differential equation of the form If the function f(x) is constant, the differential equation is easy to solve in closed form. But when it is not constant, it is very unlikely that closed form solutions exist. But there may be useful closed form approximate solutions. There is no linear term in […]

Yesterday I said on Twitter “Time to see whether practice agrees with theory, moving from LaTeX to Python. Wish me luck.” I got a lot of responses to that because it describes the experience of a lot of people. Someone asked if I’d blog about this. The content is confidential, but I’ll talk about the […]

Sir Michael Atiyah recommends Hermann Weyl’s book The Classical Groups for its clarity and beautiful prose. From my interview with Atiyah: Hermann Weyl is my great model. He used to write beautiful literature. Reading it was a joy because he put a lot of thought into it. Hermann Weyl wrote a book called The Classical Groups, […]

Consider the following Taylor series for sin(θ/7) and the following two functions based on the series, one takes only the first non-zero term def short_series(x): return 0.14285714*x and a second that three non-zero terms. def long_series(x): return 0.1425714*x - 4.85908649e-04*x**3 + 4.9582515e-07*x**5 Which is more accurate? Let’s make a couple plots plot to see. First […]

Here’s an apparent paradox. You’ll hear that Monte Carlo methods are independent of dimension, and that they scale poorly with dimension. How can both statements be true? The most obvious way to compute multiple integrals is to use product methods, analogous to the way you learn to compute multiple integrals by hand. Unfortunately the amount […]

Spheres and balls are examples of common words that take on a technical meaning in math, as I wrote about here. Recall the the unit sphere in n dimensions is the set of points with distance 1 from the origin. The unit ball is the set of points of distance less than or equal to 1 from the […]

For a random variable X, the kth moment of X is the expected value of Xk. For any random variable X with 0 mean, or negative mean, there’s an inequality that bounds the 3rd moment, m3 in terms of the 4th moment, m4: The following example shows that this bound is the best possible. Define u […]

Here’s a useful LaTeX command that I learned about recently: \underbrace. It does what it sounds like it does. It puts a brace under its argument. I used this a few days ago in the post on the new prime record when I wanted to show that the record prime is written in hexadecimal as […]

The syntax of regular expressions in Emacs is a little disappointing, but the ways you can use regular expressions in Emacs is impressive. I’ve written before about the syntax of Emacs regular expressions. It’s a pretty conservative subset of the features you may be used to from other environments as summarized in the diagram below. But […]

An Easter egg is a hidden feature, a kind of joke. The term was first used in video games but the idea is broader and older than that. For example, Alfred Hitchcock made a brief appearance in all his movies. And I recently heard that there’s a pineapple or reference to a pineapple in every […]

Suppose you have a function of n variables f. The kth derivative of f is a kth order tensor [1] with nk components. Not all those tensor components are unique. Assuming our function f is smooth, the order in which partial derivatives are taken doesn’t matter. It only matters which variables you differentiate with respect […]

One study will say that coffee is good for you and then another will say it’s bad for you. Ditto with wine and many other things. So which is it: are these things good for you or bad for you? Probably neither. That is, these things that are endlessly studied with contradictory conclusions must not […]

Take two desks of cards and shuffle them. They can be standard 52-card decks, though the number of cards in the decks doesn’t matter as long as they’re the same and the decks are fairly large. Now count the number of times the two desks match, i.e. how many times the same card is in […]

If you view a 3 × 3 magic square as a matrix and raise it to the third power, the result is also a magic square. More generally, if you multiply an odd number of 3 × 3 magic squares together, the result is a magic square. For example, here are three magic squares that […]

General Data Protection Regulation The European GDPR (General Data Protection Regulation) was adopted in 2016 and becomes enforceable in May of this year. Article 17 mandates a right to erasure, more commonly called the right to be forgotten. A right to be forgotten is tricky. It’s not immediately clear what this means or to what […]

A few years ago someone asked me what was my most useful undergraduate math class. My first thought was topology. I have never directly applied topology for a client. Nobody has ever approached me wanting to know, for example, whether two objects were in the same homotopy class. But I believe topology was one of […]

I’ve often reviewed books on this site and may review other products some day. I wanted to let readers and potential vendors know what my policies are regarding product reviews. I don’t get paid for reviews. I review things that I find interesting and think that readers would find interesting. I don’t do reviews with […]

Pick a large number n. Divide n by each of the positive integers up to n and round the results up to the nearest integer. On average, how far do you round up? Or in terms of probability, what is the expected distance between a fraction n/r, where n is large and fixed and r is chosen randomly […]

Six blog posts on mathematics and privacy: Randomized response and Bayes’ theorem Big aggregate queries can still violate privacy Quantifying privacy loss Database anonymization for testing Toxic pairs and re-identification Adding Laplacian or Gaussian noise to a database

Ten years ago I started writing this blog. Since then I’ve written about 2700 posts. Thank you all for reading, commenting, and sharing. Update: For highlights of my posts over the years, see Tim Hopper’s post John Cook’s Ten Year Blogging Endeavour.

A new record for the largest known prime was announced yesterday: This number has 23,249,425 digits when written in base 10. In base 2, 2p – 1 is a sequence of p ones. For example, 31 = 25 -1 which is 11111 in binary. So in binary, the new record prime is a string of 77,232,917 […]

The Nyquist sampling theorem says that a band-limited signal can be recovered from evenly-spaced samples. If the highest frequency component of the signal is fc then the function needs to be sampled at a frequency of at least the Nyquist frequency 2fc. Or to put it another way, the spacing between samples needs to be […]

Someone wrote to me the other day asking if I could explain a probability example from the Wall Street Journal. (“Proving Investment Success Takes Time,” Spencer Jakab, November 25, 2017.) Victor Haghani … and two colleagues told several hundred acquaintances who worked in finance that they would flip two coins, one that was normal and […]

Exponential sums can make intricate patterns. Last year I made a page that displays a different page each day, using the month, day, and year as parameters in the expression below. The images plot the partial sums of this sum. This was yesterday’s image. Today’s image is surprisingly plain if we use y = 18. This […]

Retiring professor Leonard Fabiano contacted me looking to give away a set of technical books, mostly chemical engineering books. If you’re interested please email him at lenfab@live.com. Here are the books: Click on the image to see a larger version. Two titles are not possible to read in the photo. These are Conduction of heat […]

Robert Banks’s book Towing Icebergs, Falling Dominoes, and Other Adventures in Applied Mathematics describes the Eiffel Tower’s shape as approximately the logarithmic curve where y* and x0 are chosen to match the tower’s dimensions. Here’s a plot of the curve: And here’s the code that produced the plot: from numpy import log, exp, linspace, vectorize import matplotlib.pyplot […]

These have been the most popular math-related posts here this year. Golden powers are nearly integers How efficient is Morse code? Finding numbers in pi Common words used as technical terms Sierpinski triangle strikes again See also a list of the top five computing-related posts.

In the previous post, I alluded to using Hermite polynomials in conjunction with higher-order Laplace approximation. In this post I’ll expand on what that means. Hermite polynomials are orthogonal polynomials over the real line with respect to the weight given by the standard normal distribution. (There are two conventions for defining Hermite polynomials, what Wikipedia […]

Yesterday’s post presented the most common form of Laplace approximation, the second order version, and mentioned in passing that there are higher order versions. The extension to higher order is not trivial, so post gives a high level overview of how you’d do it. The previous post looked at integrating exp( log( g(x) ) ) by […]

Define and This integral comes up in Bayesian logisitic regression with a uniform (improper) prior. We will use this integral to illustrate a simple case of Laplace approximation. The idea of Laplace approximation is to approximate the integral of a Gaussian-like function by the integral of a (scaled) Gaussian with the same mode and same […]

These have been the most popular computing-related posts here this year. Programming language life expectancy SHA1 no longer recommended, but hardly a failure The most disliked programming language Improving on the Unix shell One practical application of functional programming I plan to post a list of the top file math-related posts soon.