John D. Cook

Suppose you want to fit a line to three data points. If no line passes exactly through your points, what's the best compromise you could make? Chebyshev suggested the best thing to do is find the minmax line, the line that minimizes the maximum error. That is, for each candidate line, find the vertical distance [...]The post Best line to fit three points first appeared on John D. Cook.

Let A, B, C, and D be positive numbers. Then Lehman's inequality says Proof by circuit This inequality can be proved analytically, but Lehman's proof is interesting because he uses electrical circuits [1]. Let A, B, C, and D be the resistances of resistors arranges as in the circuit on the left. Resistors R1 and [...]The post Lehman's inequality, circuits, and LaTeX first appeared on John D. Cook.

A few weeks ago I wrote about the contraharmonic mean. Given two positive numbers a and b, their contraharmonic mean is This mean has the unusual property that increasing one of the two inputs could decrease the mean. If you take the partial derivative with respect to a you can see that it is zero [...]The post Contraharmonic mean quirk first appeared on John D. Cook.

I had a tweet Twitter go viral yesterday, at least relatively viral. Elon Musk could tweet a punctuation mark and get an orders of magnitude more traffic, but this was viral by my standards [1]. That schadenfreude-like feeling when you realize something you felt you should learn but didn't is now obsolete." Apparently this resonated [...]The post Technological schadenfreude first appeared on John D. Cook.

Differential equations in the complex plane are different from differential equations on the real line. Suppose you have an nth order linear differential equation as follows. The real case If the as are continuous, real-valued functions on an open interval of the real line, then there are n linearly independent solutions over that interval. The [...]The post Complex differential equations first appeared on John D. Cook.

I create nearly all my documents in LaTeX, even documents that might be easier to create in Word. The reason is that even if a particular document would be easier to write in Word, my workflow is more efficient if everything is in LaTeX. LaTeX makes small, plain text files that work well with version [...]The post Convert LaTeX to Microsoft Word first appeared on John D. Cook.

Years ago I wrote about how you can make a decent approximation to the Gaussian density by shifting and scaling the cosine function. This observation dates back at least as far as 1961. Turns out you can do even better by raising the cosine term to a power. Not only can you get a good [...]The post Cosine power approximation to Gaussian density first appeared on John D. Cook.

The two latest blog post have elaborated on the fact that when a function satisfies a second order differential equation, once you've evaluated the function and its derivative you get the second derivative for free, or at least for cheap. Sometimes functions are so closely related that software libraries bundle them together. For example, in [...]The post Buy one, get one free first appeared on John D. Cook.

In an undergraduate course on differential equations, you see an equation, you find a solution. Wax on, wax off. It would be reasonable to assume that the differential equation is simply an obstacle to overcome and that once you have the solution, the differential equation has done its job and can be discarded. It would [...]The post What good is a DE once you have the solution? first appeared on John D. Cook.

Newton's method for solving f(x) = 0 converges quickly once it starts to converge. Specifically, it converges quadratically: the error is roughly squared at each step. This means that the number of correct decimal places doubles at each iteration of Newton's method. Doubling the number of decimal places is nice, but what if we could [...]The post Halley's variation on Newton's method first appeared on John D. Cook.

It's hard to imagine doing anything like Midjourney's image generation without neural networks. The same is true of ChatGPT's text generation. But a lot of business tasks do not require AI, and in fact would be better off not using AI. Three reasons why: Statistical models are easier to interpret. When a model has a [...]The post Three advantages of non-AI models first appeared on John D. Cook.

Suppose you have a bivariate normal distribution with correlation where Then the level sets of the density function are ellipses, and the eccentricity e of the ellipses is related to the correlation by Plots For example, suppose = 0.8. Here's a plot of the density function f(x, y). And here is a [...]The post Eccentricity of bivariate normal level sets first appeared on John D. Cook.

The incomplete beta function is defined by It is called incomplete because the limit of integration doesn't necessarily run all the way up to 1. When z = 1 the incomplete beta function is simply the beta function discussed in the previous post [1]. The incomplete beta function is proportional to the CDF of a [...]The post Bounds on the incomplete beta function (beta CDF) first appeared on John D. Cook.

The beta function B(x, y) is defined by and is the normalizing constant for the beta probability distribution. It is related to the gamma function via The beta function comes up often in applications. It can be challenging to work with, however, and so estimates for the function are welcome. The function 1/xy gives simple [...]The post Upper and lower bounds on the beta function first appeared on John D. Cook.

Last week the function came up in a blog post as the solution to the equation n!=bn. After writing that post I stumbled upon an article [1] that begins by saying, in my notation, The function b(x) ... has many applications in pure and applied mathematics. The article mentions a couple applications but mostly focuses [...]The post nth root of n! first appeared on John D. Cook.

Neural network activation functions transform the output of one layer of the neural net into the input for another layer. These functions are nonlinear because the universal approximation theorem, the theorem that basically says a two-layer neural net can approximate any function, requires these functions to be nonlinear. Activation functions often have two-part definitions, defined [...]The post Activation functions and Iverson brackets first appeared on John D. Cook.

The previous post mentioned the group S4, the group of all permutations of a set with four elements. This post will show a way to visualize this group. The Mathematica command CayleyGraph[ SymmetricGroup[4], VertexLabels -> Placed["Name", Center], VertexSize -> 0.4] generates the graph below. This is an interesting image, but what does it mean? The [...]The post Cayley graphs in Mathematica first appeared on John D. Cook.

I had a fleeting thought about how little I use group theory when I realize I used it just this week. A couple days ago I needed to know which permutations of 4 elements commute with reversal. Ifr takes a sequence and reverses it, I need to find all permutations p such that p [...]The post Permutations and centralizers in Mathematica first appeared on John D. Cook.

I recently stumbled across the identity while reading [1]. Here x is the floor of x, the largest integer not larger than x. My first thought was that this was hard to believe. Although the floor function is very simple, its interactions with other functions tends to be complicated. I was surprised that such a [...]The post Floors and roots first appeared on John D. Cook.

Here's a graphical way to multiply two positive numbers a and b using the parabola y = x^2. Start at the origin, move a units to the left, then go up vertically to the parabola, and draw a point. Go back to the origin, move b units to the right, go up vertically to the [...]The post Multiplication via parabola first appeared on John D. Cook.

When I worked at MD Anderson Cancer Center, we spent a lot of compute cycles evaluating the function g(a, b, c, d), defined as the probability that a sample from a beta(a, b) random variable is larger than a sample from a beta(c, d) random variable. This function was often in the inner loop of [...]The post Beta inequalities and cross ratios first appeared on John D. Cook.

The nth Dedekind number M(n) is the number of monotone Boolean functions of n variables. The 9th Dedekind number was recently computed to be M(9) = 286386577668298411128469151667598498812366. The previous post defines monotone Boolean functions and explicitly enumerates the functions for one, two, or three variables. As that post demonstrates, M(1) = 3, M(2) = 6, [...]The post The 10th Dedekind number first appeared on John D. Cook.

The 9th Dedekind number was recently computed. What is a Dedekind number and why was it a big deal to compute just the 9th one of them? We need to define a couple terms before we can define Dedekind numbers. A Boolean function is a function whose inputs are 0's and 1's and whose output [...]The post Enumerating monotone Boolean functions first appeared on John D. Cook.

The motion of a falling body of mass m is given by where the term -kvr accounts for drag due to air resistance. One can derive r = 2 under simple physical assumptions, but if I remember correctly other values of r may be more realistic in certain circumstances. I don't know much about the [...]The post Drag equation exponent variation first appeared on John D. Cook.

Suppose you select a 100-digit number at random. How many distinct prime factors do you think it would have? The answer is smaller than you might think: most likely between 5 and 6. The function (n) returns the number of distinct prime factors of n [1]. A theorem of Hardy and Ramanujan says that as [...]The post Numbers don't typically have many prime factors first appeared on John D. Cook.

The previous post mentioned that 24! 1024 and 25! 1025. For every n, there is some base b such that n! = bn. For example, 30! 1230. It's easy to find b [1]: What's interesting is that b is very nearly a linear function of n. In hindsight it's clear that this [...]The post Every factorial is a power first appeared on John D. Cook.

Suppose you'd like to have a very rough idea how large n! is for, say, n less than 100. If you're familiar with such things, your Pavlovian response to factorial" and approximation" is Stirling's approximation. Although Stirling's approximation is extremely useful-I probably use it every few weeks-it is not conducive to mental calculation. The cut [...]The post Mentally approximating factorials first appeared on John D. Cook.

How are the first digits of primes distributed? Do some digits appear as first digits of primes more often that others? How should we even frame the problem? There are an infinite number of primes that begin with each digit, so the cardinalities of the sets of primes beginning with each digit are the same. [...]The post Leading digits of primes first appeared on John D. Cook.

When I first started programming I'd occasionally get an error because a delimiter wasn't matched. Maybe I had a { without a matching }, for example. Or if I were writing Pascal, which I did back in the day, I might have had a BEGIN statement without a corresponding END statement. At some point I [...]The post Matching delimiters and chiastic patterns first appeared on John D. Cook.

Yesterday I wrote about the equation of a circle through three points. This post will be similar, looking at the equation of an ellipse or hyperbola through five points. As with the earlier post, we can write down an elegant equation right away. Making the equation practical will take more work. The equation of a [...]The post Equation of an ellipse or hyperbola through five points first appeared on John D. Cook.

Let P be a parabola. Draw tangents to P at three different points. These three lines intersect to form a triangle. Theorem: The circumcircle of this triangle passes through the focus of P. [1] In the image above, the dashed lines are tangents and the black dot is the focus of the parabola. (See this [...]The post A parabola, a triangle, and a circle first appeared on John D. Cook.

A few months ago I wrote about several equations that come up in geometric calculations that all have the form of a determinant equal to 0, where one column of the determinant is all 1s. The equation of a circle through three points (x1, y1), (x2, y2), and (x3, y3) has this form: This is [...]The post Equation of a circle through three points first appeared on John D. Cook.

Suppose a planet is in an elliptical orbit around the sun with semimajor axis a and semiminor axis b. Then the average distance of the planet to the sun over time equals a(1 + e^2/2) where the eccentricity e satisfies e^2 = 1 - b^2/a^2. You can find a proof of this statement in [1]. [...]The post Set of orbits with the same average distance to sun first appeared on John D. Cook.

The previous post discussed the functions as test cases for plotting. This post will look at using the same functions as test cases for integration. As you can see from the plot of f30(x) below, the function is continuous, but the derivative of the function has a lot of discontinuities. The integrals of Steinerberger's [...]The post Pushing numerical integration software to its limits first appeared on John D. Cook.

Stefan Steinerberger defines an amusing sequence of functions" in [1] by Here's a plot of f30(x): As you can see, fn(x) has a lot of local minima, and the number of local minima increases rapidly with n. Here's a naive attempt to produce the plot above using Python. from numpy import sin, pi, linspace import [...]The post Plotting a function with a lot of local minima first appeared on John D. Cook.

This post is an addendum to the recent post Reviewing a thousand things. We're going to look again at the coupon collector problem, randomly sampling a set of N things with replacement until we've collected one of everything. As noted before, for largeNthe expected number of draws before you've seen everything at least once is [...]The post Collecting a large number of coupons first appeared on John D. Cook.

Here's a curious fact. The graphs of cotangent and secant cross at the same height as the graphs of tangent and cosecant, and this common height is the square root of the golden ratio . It's also the case that the graphs of hyperbolic cosecant and hyperbolic cosine, and the graphs of hyperbolic sine and [...]The post Trig crossings and root of gold first appeared on John D. Cook.

The previous post looked at an application of the beta-binomial distribution. The probability mass function for a beta-binomial with parameters n, a, and b is given by The mean μ and the variance σ² are given by Solving for a and b to meet a specified mean and variance appears at first to require solving […]The post Beta-binomial with given mean and variance first appeared on John D. Cook.

About half of children are boys and half are girls, but that doesn’t mean that every couple is equally likely to have a boy or a girl each time they conceive a child. And evidence suggests that indeed the probability of conceiving a girl varies per couple. I will simplify things for this post and […]The post Babies and the beta-binomial distribution first appeared on John D. Cook.

Suppose the vertices of two triangles are given by complex numbers a, b, c and x, y, z. The two triangles are similar if This can be found in G. H. Hardy’s classic A Course of Pure Mathematics. It’s on page 93 in the 10th edition. Corollary The theorem above generalizes a result from an […]The post Similar triangles and complex numbers first appeared on John D. Cook.

Here’s something I found surprising: the powers of a 2×2 matrix have a fairly simple closed form. Also, the derivation is only one page [1]. LetA be a 2×2 matrix with eigenvalues α and β. (3Blue1Brown made a nice jingle for finding the eigenvalues of a 2×2 matrix.) If α = β then the nth […]The post Powers of a 2×2 matrix in closed form first appeared on John D. Cook.

“Instant classic” is, of course, an oxymoron. A classic is something that has passed the test of time, and by definition that cannot happen instantly. But how long should the test of time last? In his book Love What Lasts, Joshua Gibbs argues that 100 years after the death of the artist is about the […]The post Instant classic first appeared on John D. Cook.

Suppose you have a random number generator that returns numbers between 1 and N. The birthday problem asks how many random numbers would you have to output before there’s a 50-50 chance that you’ll repeat a number. The coupon collector problem asks how many numbers you expect to generate before you’ve seen all N numbers […]The post Occupancy problem distribution first appeared on John D. Cook.

One of these days I’d like to read Feller’s probability book slowly. He often says clever things in passing that are easy to miss. Here’s an example from Feller [1] that I overlooked until I saw it cited elsewhere. Suppose an urn contains n marbles, n1 red and n2 black. When r marbles are drawn […]The post Hypergeometric distribution symmetry first appeared on John D. Cook.

Suppose you take the arithmetic mean and the geometric mean of the first n integers. The ratio of these two means converges to e/2 as n grows [1]. In symbols, Now suppose we wanted to visualize the convergence by plotting the expression on the left side for a sequence of ns. First let’s let n […]The post AM over GM first appeared on John D. Cook.

I was bewildered by my first exposure to category theory. My first semester in graduate school I had a textbook with definitions like “A gadget is an object G such that whenever you have this unfamiliar constellation of dots and arrows, you’re allowed to draw another arrow from here to there.” What? Why?! I revisited […]The post Category theory without categories first appeared on John D. Cook.

I’ve mentioned the harmonic mean multiple times here, most recently last week. The harmonic mean pops up in many contexts. The contraharmonic mean is a variation on the harmonic mean that comes up occasionally, though not as often as its better known sibling. Definition The contraharmonic mean of two positive numbers a and b is […]The post Contraharmonic mean first appeared on John D. Cook.

I recently noticed something in a book I’ve had for five years: the bibliography section ends with a histogram of publication dates for references. I’ve used the book over the last few years, but maybe I haven’t needed to look at the bibliography before. This is taken from Bernstein’s Matrix Mathematics. I wrote a review […]The post Bibliography histogram first appeared on John D. Cook.

Let A be an n × n matrix over a field F. The cofactor of an element Aij is the matrix formed by removing the ith row and jth column, denoted A[i, j]. This terminology is less than ideal. The matrix just described is called the cofactor of Aij, but it would more accurately be […]The post Cofactors, determinants, and adjugates first appeared on John D. Cook.

I’ve written several times about the arithmetic-geometric mean and variations. Take the arithmetic and geometric mean of two positive numbers a and b. Then take the arithmetic and geometric of the means from the previous step. Repeat ad infinitum and the result converges to a limit. This limit is called the arthmetic-geometric mean or AGM. […]The post Arithmetic-harmonic mean first appeared on John D. Cook.