John D. Cook

In an undergraduate course on differential equations, you see an equation, you find a solution. Wax on, wax off. It would be reasonable to assume that the differential equation is simply an obstacle to overcome and that once you have the solution, the differential equation has done its job and can be discarded. It would [...]The post What good is a DE once you have the solution? first appeared on John D. Cook.

Newton's method for solving f(x) = 0 converges quickly once it starts to converge. Specifically, it converges quadratically: the error is roughly squared at each step. This means that the number of correct decimal places doubles at each iteration of Newton's method. Doubling the number of decimal places is nice, but what if we could [...]The post Halley's variation on Newton's method first appeared on John D. Cook.

It's hard to imagine doing anything like Midjourney's image generation without neural networks. The same is true of ChatGPT's text generation. But a lot of business tasks do not require AI, and in fact would be better off not using AI. Three reasons why: Statistical models are easier to interpret. When a model has a [...]The post Three advantages of non-AI models first appeared on John D. Cook.

Suppose you have a bivariate normal distribution with correlation where Then the level sets of the density function are ellipses, and the eccentricity e of the ellipses is related to the correlation by Plots For example, suppose = 0.8. Here's a plot of the density function f(x, y). And here is a [...]The post Eccentricity of bivariate normal level sets first appeared on John D. Cook.

The incomplete beta function is defined by It is called incomplete because the limit of integration doesn't necessarily run all the way up to 1. When z = 1 the incomplete beta function is simply the beta function discussed in the previous post [1]. The incomplete beta function is proportional to the CDF of a [...]The post Bounds on the incomplete beta function (beta CDF) first appeared on John D. Cook.

The beta function B(x, y) is defined by and is the normalizing constant for the beta probability distribution. It is related to the gamma function via The beta function comes up often in applications. It can be challenging to work with, however, and so estimates for the function are welcome. The function 1/xy gives simple [...]The post Upper and lower bounds on the beta function first appeared on John D. Cook.

Last week the function came up in a blog post as the solution to the equation n!=bn. After writing that post I stumbled upon an article [1] that begins by saying, in my notation, The function b(x) ... has many applications in pure and applied mathematics. The article mentions a couple applications but mostly focuses [...]The post nth root of n! first appeared on John D. Cook.

Neural network activation functions transform the output of one layer of the neural net into the input for another layer. These functions are nonlinear because the universal approximation theorem, the theorem that basically says a two-layer neural net can approximate any function, requires these functions to be nonlinear. Activation functions often have two-part definitions, defined [...]The post Activation functions and Iverson brackets first appeared on John D. Cook.

The previous post mentioned the group S4, the group of all permutations of a set with four elements. This post will show a way to visualize this group. The Mathematica command CayleyGraph[ SymmetricGroup[4], VertexLabels -> Placed["Name", Center], VertexSize -> 0.4] generates the graph below. This is an interesting image, but what does it mean? The [...]The post Cayley graphs in Mathematica first appeared on John D. Cook.

I had a fleeting thought about how little I use group theory when I realize I used it just this week. A couple days ago I needed to know which permutations of 4 elements commute with reversal. Ifr takes a sequence and reverses it, I need to find all permutations p such that p [...]The post Permutations and centralizers in Mathematica first appeared on John D. Cook.

I recently stumbled across the identity while reading [1]. Here x is the floor of x, the largest integer not larger than x. My first thought was that this was hard to believe. Although the floor function is very simple, its interactions with other functions tends to be complicated. I was surprised that such a [...]The post Floors and roots first appeared on John D. Cook.

Here's a graphical way to multiply two positive numbers a and b using the parabola y = x^2. Start at the origin, move a units to the left, then go up vertically to the parabola, and draw a point. Go back to the origin, move b units to the right, go up vertically to the [...]The post Multiplication via parabola first appeared on John D. Cook.

When I worked at MD Anderson Cancer Center, we spent a lot of compute cycles evaluating the function g(a, b, c, d), defined as the probability that a sample from a beta(a, b) random variable is larger than a sample from a beta(c, d) random variable. This function was often in the inner loop of [...]The post Beta inequalities and cross ratios first appeared on John D. Cook.

The nth Dedekind number M(n) is the number of monotone Boolean functions of n variables. The 9th Dedekind number was recently computed to be M(9) = 286386577668298411128469151667598498812366. The previous post defines monotone Boolean functions and explicitly enumerates the functions for one, two, or three variables. As that post demonstrates, M(1) = 3, M(2) = 6, [...]The post The 10th Dedekind number first appeared on John D. Cook.

The 9th Dedekind number was recently computed. What is a Dedekind number and why was it a big deal to compute just the 9th one of them? We need to define a couple terms before we can define Dedekind numbers. A Boolean function is a function whose inputs are 0's and 1's and whose output [...]The post Enumerating monotone Boolean functions first appeared on John D. Cook.

The motion of a falling body of mass m is given by where the term -kvr accounts for drag due to air resistance. One can derive r = 2 under simple physical assumptions, but if I remember correctly other values of r may be more realistic in certain circumstances. I don't know much about the [...]The post Drag equation exponent variation first appeared on John D. Cook.

Suppose you select a 100-digit number at random. How many distinct prime factors do you think it would have? The answer is smaller than you might think: most likely between 5 and 6. The function (n) returns the number of distinct prime factors of n [1]. A theorem of Hardy and Ramanujan says that as [...]The post Numbers don't typically have many prime factors first appeared on John D. Cook.

The previous post mentioned that 24! 1024 and 25! 1025. For every n, there is some base b such that n! = bn. For example, 30! 1230. It's easy to find b [1]: What's interesting is that b is very nearly a linear function of n. In hindsight it's clear that this [...]The post Every factorial is a power first appeared on John D. Cook.

Suppose you'd like to have a very rough idea how large n! is for, say, n less than 100. If you're familiar with such things, your Pavlovian response to factorial" and approximation" is Stirling's approximation. Although Stirling's approximation is extremely useful-I probably use it every few weeks-it is not conducive to mental calculation. The cut [...]The post Mentally approximating factorials first appeared on John D. Cook.

How are the first digits of primes distributed? Do some digits appear as first digits of primes more often that others? How should we even frame the problem? There are an infinite number of primes that begin with each digit, so the cardinalities of the sets of primes beginning with each digit are the same. [...]The post Leading digits of primes first appeared on John D. Cook.

When I first started programming I'd occasionally get an error because a delimiter wasn't matched. Maybe I had a { without a matching }, for example. Or if I were writing Pascal, which I did back in the day, I might have had a BEGIN statement without a corresponding END statement. At some point I [...]The post Matching delimiters and chiastic patterns first appeared on John D. Cook.

Yesterday I wrote about the equation of a circle through three points. This post will be similar, looking at the equation of an ellipse or hyperbola through five points. As with the earlier post, we can write down an elegant equation right away. Making the equation practical will take more work. The equation of a [...]The post Equation of an ellipse or hyperbola through five points first appeared on John D. Cook.

Let P be a parabola. Draw tangents to P at three different points. These three lines intersect to form a triangle. Theorem: The circumcircle of this triangle passes through the focus of P. [1] In the image above, the dashed lines are tangents and the black dot is the focus of the parabola. (See this [...]The post A parabola, a triangle, and a circle first appeared on John D. Cook.

A few months ago I wrote about several equations that come up in geometric calculations that all have the form of a determinant equal to 0, where one column of the determinant is all 1s. The equation of a circle through three points (x1, y1), (x2, y2), and (x3, y3) has this form: This is [...]The post Equation of a circle through three points first appeared on John D. Cook.

Suppose a planet is in an elliptical orbit around the sun with semimajor axis a and semiminor axis b. Then the average distance of the planet to the sun over time equals a(1 + e^2/2) where the eccentricity e satisfies e^2 = 1 - b^2/a^2. You can find a proof of this statement in [1]. [...]The post Set of orbits with the same average distance to sun first appeared on John D. Cook.

The previous post discussed the functions as test cases for plotting. This post will look at using the same functions as test cases for integration. As you can see from the plot of f30(x) below, the function is continuous, but the derivative of the function has a lot of discontinuities. The integrals of Steinerberger's [...]The post Pushing numerical integration software to its limits first appeared on John D. Cook.

Stefan Steinerberger defines an amusing sequence of functions" in [1] by Here's a plot of f30(x): As you can see, fn(x) has a lot of local minima, and the number of local minima increases rapidly with n. Here's a naive attempt to produce the plot above using Python. from numpy import sin, pi, linspace import [...]The post Plotting a function with a lot of local minima first appeared on John D. Cook.

This post is an addendum to the recent post Reviewing a thousand things. We're going to look again at the coupon collector problem, randomly sampling a set of N things with replacement until we've collected one of everything. As noted before, for largeNthe expected number of draws before you've seen everything at least once is [...]The post Collecting a large number of coupons first appeared on John D. Cook.

Here's a curious fact. The graphs of cotangent and secant cross at the same height as the graphs of tangent and cosecant, and this common height is the square root of the golden ratio . It's also the case that the graphs of hyperbolic cosecant and hyperbolic cosine, and the graphs of hyperbolic sine and [...]The post Trig crossings and root of gold first appeared on John D. Cook.

The previous post looked at an application of the beta-binomial distribution. The probability mass function for a beta-binomial with parameters n, a, and b is given by The mean μ and the variance σ² are given by Solving for a and b to meet a specified mean and variance appears at first to require solving […]The post Beta-binomial with given mean and variance first appeared on John D. Cook.

About half of children are boys and half are girls, but that doesn’t mean that every couple is equally likely to have a boy or a girl each time they conceive a child. And evidence suggests that indeed the probability of conceiving a girl varies per couple. I will simplify things for this post and […]The post Babies and the beta-binomial distribution first appeared on John D. Cook.

Suppose the vertices of two triangles are given by complex numbers a, b, c and x, y, z. The two triangles are similar if This can be found in G. H. Hardy’s classic A Course of Pure Mathematics. It’s on page 93 in the 10th edition. Corollary The theorem above generalizes a result from an […]The post Similar triangles and complex numbers first appeared on John D. Cook.

Here’s something I found surprising: the powers of a 2×2 matrix have a fairly simple closed form. Also, the derivation is only one page [1]. LetA be a 2×2 matrix with eigenvalues α and β. (3Blue1Brown made a nice jingle for finding the eigenvalues of a 2×2 matrix.) If α = β then the nth […]The post Powers of a 2×2 matrix in closed form first appeared on John D. Cook.

“Instant classic” is, of course, an oxymoron. A classic is something that has passed the test of time, and by definition that cannot happen instantly. But how long should the test of time last? In his book Love What Lasts, Joshua Gibbs argues that 100 years after the death of the artist is about the […]The post Instant classic first appeared on John D. Cook.

Suppose you have a random number generator that returns numbers between 1 and N. The birthday problem asks how many random numbers would you have to output before there’s a 50-50 chance that you’ll repeat a number. The coupon collector problem asks how many numbers you expect to generate before you’ve seen all N numbers […]The post Occupancy problem distribution first appeared on John D. Cook.

One of these days I’d like to read Feller’s probability book slowly. He often says clever things in passing that are easy to miss. Here’s an example from Feller [1] that I overlooked until I saw it cited elsewhere. Suppose an urn contains n marbles, n1 red and n2 black. When r marbles are drawn […]The post Hypergeometric distribution symmetry first appeared on John D. Cook.

Suppose you take the arithmetic mean and the geometric mean of the first n integers. The ratio of these two means converges to e/2 as n grows [1]. In symbols, Now suppose we wanted to visualize the convergence by plotting the expression on the left side for a sequence of ns. First let’s let n […]The post AM over GM first appeared on John D. Cook.

I was bewildered by my first exposure to category theory. My first semester in graduate school I had a textbook with definitions like “A gadget is an object G such that whenever you have this unfamiliar constellation of dots and arrows, you’re allowed to draw another arrow from here to there.” What? Why?! I revisited […]The post Category theory without categories first appeared on John D. Cook.

I’ve mentioned the harmonic mean multiple times here, most recently last week. The harmonic mean pops up in many contexts. The contraharmonic mean is a variation on the harmonic mean that comes up occasionally, though not as often as its better known sibling. Definition The contraharmonic mean of two positive numbers a and b is […]The post Contraharmonic mean first appeared on John D. Cook.

I recently noticed something in a book I’ve had for five years: the bibliography section ends with a histogram of publication dates for references. I’ve used the book over the last few years, but maybe I haven’t needed to look at the bibliography before. This is taken from Bernstein’s Matrix Mathematics. I wrote a review […]The post Bibliography histogram first appeared on John D. Cook.

Let A be an n × n matrix over a field F. The cofactor of an element Aij is the matrix formed by removing the ith row and jth column, denoted A[i, j]. This terminology is less than ideal. The matrix just described is called the cofactor of Aij, but it would more accurately be […]The post Cofactors, determinants, and adjugates first appeared on John D. Cook.

I’ve written several times about the arithmetic-geometric mean and variations. Take the arithmetic and geometric mean of two positive numbers a and b. Then take the arithmetic and geometric of the means from the previous step. Repeat ad infinitum and the result converges to a limit. This limit is called the arthmetic-geometric mean or AGM. […]The post Arithmetic-harmonic mean first appeared on John D. Cook.

A few days ago I wrote that circulant matrices all have the same eigenvectors. This post will show that it follows that circulant matrices commute with each other. Recall that a circulant matrix is a square matrix in which the rows are cyclic permutations of each other. If we number the rows from 0, then […]The post Circulant matrices commute first appeared on John D. Cook.

The previous post discussed an unusual algebraic structure on the real interval (-1, 1) inspired by (and applied to) special relativity. We defined an addition operator ⊕ by How might we extend this from the interval (-1, 1) to the unit disk in the complex plane? The definition won’t transfer over unmodified because it does […]The post Relativity, complex numbers, and gyrovectors first appeared on John D. Cook.

I usually think of an instructor as someone who unpacks things, such as unpacking the meaning of an obscure word or explaining a difficult concept. Last night I was trying to read some unbearably dry medical/legal material and thought about how an instructor might also pack things, wrapping dry material in some sort of story […]The post Packing versus unpacking first appeared on John D. Cook.

It’s well known that you cannot map a sphere onto the plane without distortion. You can’t map the entire sphere to the plane at all because a sphere and a plane are not topologically equivalent. But even if you want to map a relatively small portion of globe to paper, say France, with about 0.1% […]The post How faithful can a map be? first appeared on John D. Cook.

If a three-digit number looks like it might be prime, there’s about a 2 in 3 chance that it is. To be more precise about what it means for a number to “look like a prime,” let’s say that a number is obviously composite if it is divisible by 2, 3, 5, or 11. Then […]The post Recognizing three-digit primes first appeared on John D. Cook.

Suppose you randomly sample points from a unit cube. How far apart are pairs of points on average? My intuition would say that the expected distance either has a simple closed form or no closed form at all. To my surprise, the result is somewhere in between: a complicated closed form. Computing the expected value […]The post Expected distance between points in a cube first appeared on John D. Cook.

I was looking back at a post about the Soviet license plate game and was reminded of the amusing identity sin (n!)° = 0 for n ≥ 6. Would it be possible to find sin (n!)° in closed form for all positive integers n? For this post I’ll make an exception to my usual rule […]The post Sine of factorial degrees first appeared on John D. Cook.

Here’s a simple but surprising theorem from digital signal processing: linear, time-invariant (LTI) operators commute. The order in which you apply LTI operators does not matter. Linear in DSP means just you’d expect from seeing linear defined anywhere else: An operator L is linear if given any two signals x1 and x2, and any two […]The post LTI operators commute first appeared on John D. Cook.