John D. Cook

It’s well known that a binomial random variable can be approximated by a Poisson random variable, and under what circumstances the approximation is particularly good. See, for example, this post. A binomial random variable is the sum of iid (independent, identically distributed) Bernoulli random variables. But what if the Bernoulli random variables don’t have the […]The post Sum of independent but differently distributed variables first appeared on John D. Cook.

HAKMEM is a collection of tricks and trivia from the MIT AI lab written in 1972. I’ve mentioned HAKMEM here before. The image below came from HAKMEM item 123, explained here. I ran across HAKMEM item 143 while writing my previous two blog posts about the arithmetic-geometric mean (AGM). This entry, due to Gene Salamin, […]The post HAKMEM, AGM, and Pi first appeared on John D. Cook.

In the previous post I said that Jonathan and Peter Borwein figured out how to use the rapid convergence of the AGM to compute various functions, including logarithms. This post will show how to compute logarithms using the AGM. First I need to define the Jacobi theta functions Note that if q is very small, […]The post Computing logs with the AGM first appeared on John D. Cook.

Suppose you are visited by aliens from halfway across the galaxy. After asking you a lot of questions, they give you a parting gift, little black boxes can compute x – x²/2 + x³/3 – … with unbelievable speed and accuracy. You say thank you and your visitors vanish. You get back home and wonder […]The post The magic AGM box first appeared on John D. Cook.

This post will state a couple forms of the Cauchy-Schwarz inequality and then present the lesser-known reverse of the Cauchy-Schwarz inequality due to Pólya and Szegö. Cauchy-Schwarz inequality The summation form of the Cauchy-Schwarz inequality says that for sequences of real numbers xn and yn. The integral form of the Cauchy-Schwarz inequality says that for […]The post Reversed Cauchy-Schwarz inequality first appeared on John D. Cook.

I recently got a copy of Methods of Theoretical Physics by Morse and Feshbach. It’s a dense book, literally and metaphorically. I wondered whether it might be the densest book I own, so I weighed some of my weightier books. Morse and Feshbach has density 1.005 g/cm³, denser than water. Gravitation by Misner, Thorne, and […]The post My densest books first appeared on John D. Cook.

Pick a quaternion p = p0 + p1i + p2j + p3k and consider the function that acts on quaternions by multiplying them on the left by p. If we think of q as a vector in R4 then this is a linear function of q, and so it can be represented by multiplication by […]The post Quaternion product as a matrix product first appeared on John D. Cook.

If a vector space has an inner product, it has a norm: you can define the norm of a vector to be the square root of the inner product of the vector with itself. You can use the defining properties of an inner product to show that This is a form of the so-called polarization […]The post Inner product from norm first appeared on John D. Cook.

I’ve used Feedburner to allow people to subscribe to this blog via email. That service is going away, and so I just moved everyone over to MailerLite. I turned off Feedburner email, so nobody should get duplicate email. Feedburner’s RSS service is still going, for now, but most RSS subscribers use my RSS feed without […]The post Email subscription switchover first appeared on John D. Cook.

A couple days ago I wrote about Gauss’ trick to multiply two complex numbers using only 3 real multiplications. This post will do something similar with quaternions. Just as the most direct approach to computing complex products requires 4 real multiplications, the most direct approach to quaternion products requires 16 real multiplications. (See the function […]The post Quaternion products with fewer real products first appeared on John D. Cook.

You can use quaternions to describe rotations and quaternion products to carry out these rotations. These products have the form qpq* where q represents a rotation, q* is its conjugate, and p is the the vector being rotated. (That’s leaving out some details that we’ll get to shortly.) The primary advantage of using quaternions to […]The post Faster quaternion product rotations first appeared on John D. Cook.

I’ve written before about how the word “tensor” is applied to several different things, and that the connection between them isn’t immediately obvious. I thought about writing more about that some day, but I recently became aware of a paper that does this more thoroughly than I ever could. The paper looks at three notions […]The post Three kinds of tensor first appeared on John D. Cook.

The recent post on the wave equation on a disk showed that the Laplace operator has a different form in polar coordinates than it does in Cartesian coordinates. In general, the Laplacian is not simply the sum of the second derivatives with respect to each variable. Mathematica has a function, unsurprisingly called Laplacian, that will […]The post Laplacian in various coordinate systems first appeared on John D. Cook.

The most straight-forward way of multiplying two complex numbers requires four multiplications of real numbers: def mult(a, b, c, d): re = a*c - b*d im = b*c + a*d return (re, im) Gauss [1] discovered a way to do this with only three multiplications: def gauss(a, b, c, d): t1 = a*c t2 = […]The post Gauss algorithm for complex multiplication first appeared on John D. Cook.

This post will tie together many things I’ve blogged about before. The previous post justified separation of variables. This post will illustrate separation of variables. Also, this post will show why you might care about Bessel functions and their zeros. I’ve written about Bessel functions before, and said that Bessel functions are to polar coordinates […]The post Vibrating circular membranes first appeared on John D. Cook.

The separation of variables technique for solving partial differential equations looks like a magic trick the first time you see it. The lecturer, or author if you’re more self-taught, makes an audacious assumption, like pulling a rabbit out of a hat, and it works. For example, you might first see the heat equation ut = […]The post Justifying separation of variables first appeared on John D. Cook.

If you’re puzzled by the title of this post, allow me to explain. A natural reaction would be “Isn’t geometry intrinsically visual?” Indeed, geometry is motivated by things we can visualize. But modern developments of geometry have become heavy with formal machinery, so much so that one could reasonably ask “What happened to the geometry?” […]The post Visual geometry first appeared on John D. Cook.

I picked up a used copy of Quaternions and Rotation Sequences by Jack B. Kuipers for a project I’m starting to work on. The feedback I’ve seen on the book says it has good content but also has lots of typos. My copy has a fair number of corrections that someone penciled in. Someone on […]The post Volunteer-generated errata pages first appeared on John D. Cook.

Suppose in a company of N employees, m are chosen randomly for drug screening. In two independent screenings, what is the probability that someone will be picked both times? It may be unlikely that any given individual will be picked twice, while being very likely that someone will be picked twice. Imagine m employees being […]The post Random drug screening first appeared on John D. Cook.

As I mentioned a couple weeks ago, Feedburner, the service I’ve been using for blog email subscriptions, is shutting down. I’m switching over to MailerLite. The new email subscription is up and running. You can sign up here if you’d like. If you’re already subscribed via Feedburner, there’s no need to sign up again with […]The post Blog email subscription first appeared on John D. Cook.

A friend and I were discussing how to analyze his data one time and at the end of the conversation he said “So, basically you’re recommending division.” And indeed I was. The conclusion was to divide one thing by another. I’ve also recommended to clients that they use an extended Kalman filter or homomorphic encryption; […]The post Recommending division first appeared on John D. Cook.

Several weeks ago I wrote about the Fourier uncertainty principle which gives a lower bound on the product of the variance of a function f and the variance of its Fourier transform. This post expands on the earlier post by quoting some results from a recent paper [1]. Gaussian density The earlier post said that the […]The post Fourier, Gauss, and Heisenberg first appeared on John D. Cook.

Lambda calculus is simple. The definitions and rules of lambda calculus would easily fit on an index card. But you can’t really “read” lambda calculus so much as you can mentally execute it. Not many people can look at more than a few characters of lambda calculus and have any idea what it represents, though […]The post More readable lambda calculus first appeared on John D. Cook.

The RIPEMD-160 secure hash function may be best known these days for its role as part of the implementation of Bitcoin. I’ve wondered what “RIPEMD” stands for, and today I stumbled on an explanation [1]: “RIPEMD” stands for “RIPE Message Digest,” where “RIPE” stands for “RACE Integrity Primitives Evaluation” and where “RACE” stands for “Research […]The post What does RIPEMD stand for? first appeared on John D. Cook.

I’m reading Voyage to Alpha Centauri and ran into a question about relativity. The book says in one place that their ship is moving a 56.7% of the speed of light, and in another place it says that time moves about 20% slower for them relative to folks on Earth. Are those two statements consistent? […]The post Time dilation in SF and GPS first appeared on John D. Cook.

The previous post looked at the sequence kn mod 1 where k = log2(3). This value of k arose out of exploring perfect fifths and octaves. We saw that the sequence above fills in the unit interval fairly well, though not exactly regularly. The first 12 elements of the sequence nearly divide the unit interval […]The post A gentle introduction to QMC and discrepancy first appeared on John D. Cook.

In music, a perfect fifth is the interval between two notes whose frequencies are in 3:2 ratio. For example, the interval from an A at 440 Hz and an E at 660 Hz is a perfect fifth. Going up by 12 perfect fifths is very nearly the same as going up 7 octaves. That is, […]The post Perfect fifths, octaves, and an ergodic map first appeared on John D. Cook.

Occasionally I will use a row of three asterisks to indicate the separation between the body of a blog post and the footnotes, as I did in my previous post. *** This page quotes a typesetter who said she calls this a zareba. The OED defines a zareba as a kind of cattle pen but […]The post Zareba, asterism, and dinkus first appeared on John D. Cook.

The following trig identity looks like a mistake but is correct: sin(x + y) sin(x – y) = (sin(x) + sin(y)) (sin(x) – sin(y)) It looks as if someone fallaciously expanded sin(x + y) “=” sin(x) + sin(y) and sin(x – y) “=” sin(x) – sin(y). Although both expansions are wrong, their product is correct. […]The post A unique property of sine first appeared on John D. Cook.

The previous post explains how the Maidenhead location system works. The first character in a location code specifies the longitude in 20 degree increments and the second character specifies the latitude in 10 degree increments. Both are a letter from A through R that breaks the possible range down into 18 parts. (The longitude range […]The post Information theory and coordinates first appeared on John D. Cook.

The Maidenhead Locator System encodes a pair of longitude and latitude coordinates in a slightly complicated but ingenious way. Amateur radio operators using this geocoding system to describe locations. The Wikipedia article on the subject describes the what of the system, but I’d like to say more about the why of the system. I’ll also […]The post Maidenhead geocode system first appeared on John D. Cook.

The plastic ratio is given by The Dutch architect Dom Hans van der Laan gave the number this name in 1928. He used “plastic” as an allusion to a 3D construction of the number, analogous to the 2D construction of the golden ratio. Here’s a plastic rectangle, a rectangle whose sides have the proportions of […]The post Plastic number feels plastic first appeared on John D. Cook.

What is the distribution of coins you’d expect to see in a coin jar? This post will make a guess based on the following simplifying assumptions. Change is in US currency, with coins in denominations of 1, 5, 10, and 25 cents. The change due in a transaction is uniformly distributed from 0 to 99 […]The post Coin distribution first appeared on John D. Cook.

The exponential sum page on this site draws lines between the consecutive partial sums of where m is the month, d is the day, and y is the last two digits of the year. I get mixed feedback on my exponential sum page. Some people find it even more interesting than I do and have […]The post Why exponential sums are interesting first appeared on John D. Cook.

Lawyers write Word documents and mathematicians write LaTeX documents. Of course makes collaboration awkward, but there are ways to make it better. One solution is to simply use Word. People who use LaTeX probably know how to use Word, even if they’d rather not, and asking someone else to learn LaTeX is a non-starter. So […]The post LaTeX and Lawyers first appeared on John D. Cook.

I first heard of the book A Pattern Language sometime in the 1990s. I had left academia, for the first time [1], and was working as a software developer. Although the book is about architecture, software developers were exited about the book because of its analogs to software development patterns. The “Gang of Four” book […]The post A Pattern Language first appeared on John D. Cook.

Email subscription for this web site has been provided by Google Feedburner. This service is going away in July. I don’t know yet what I will replace Feedburner with. Do any of you know of an alternative that automatically sends out email when there’s a new post? I use Mail Chip to distribute by monthly […]The post Email subscription changing first appeared on John D. Cook.

This post is an expansion on a comment by Nathan Hannon: My favorite definition of e is “the base whose version of the decibel is the largest”. I hadn’t seen that before. Sounds interesting. What does it mean? First attempt My first thought was that the statement meant that the function b logb(x) is largest […]The post The base with the largest decibel first appeared on John D. Cook.

I’ve written before about Rényi entropy Hq and most recently q-log entropy Sq, two generalizations of Shannon entropy. There are simple equations relating Rényi entropy and q-log entropy if we measure both in nats: I mentioned in the post on q-log entropy that there were two possible ways it could be defined. The equation above […]The post Relating Rényi entropy and q-log entropy first appeared on John D. Cook.

The most common way to quantify entropy is Shannon entropy. That’s what people usually mean when they say “entropy” without further qualifications. A while back I wrote about Renyi entropy as a generalization of Shannon entropy. This time I’ll explore a different generalization called q-log entropy, a.k.a. Tsallis entropy. The definition of Shannon entropy includes […]The post Generalizing Shannon entropy with q-logs first appeared on John D. Cook.

Last week my family and I made a tour of the five national parks in Utah. In Canyonlands National Park, my son-in-law noticed some grates at the bottom of a hill just a few feet off the road we were walking on. The area was not restricted. We walked over to investigate and found that […]The post Stumbling on an abandoned uranium mine first appeared on John D. Cook.

Logarithms in all bases are proportional. Specifically loga(x) = logb(x) / logb(a) for any bases a and b. One way to read this is to say that if you already know about logarithms base b, logarithms base a aren’t anything fundamentally new [1]. They’re proportional to logarithms base b, and in fact the proportionality constant […]The post Decibel to log in new base first appeared on John D. Cook.

Here are the five most popular posts so far this year. Self-reproducing cellular automata Simple trig approximations How to mentally calculate logs Trig functions across programming languages Gell-Mann amnesia The posts above had the most page views. But just counting page views doesn’t measure what regular readers necessarily most enjoy reading. It’s heavily influenced by […]The post Most popular posts this year first appeared on John D. Cook.

The trace of a square matrix is the sum of the elements on its main diagonal. The order in which you multiply matrices matters: in general, matrix multiplication is not commutative. But the trace of a product of matrices may or may not depend on the order of multiplication. Specifically, trace doesn’t change if you […]The post Cyclic permutations and trace first appeared on John D. Cook.

3Blue1Brown has a nice new video on how to calculate the eigenvalues of 2×2 matrices. The most common way to find the eigenvalues of a 2×2 matrix A is working straight from the definition, solving det(A – λI) = 0. This is fine when you’re learning what eigenvalues are. But if you’ve already learned all […]The post Trick for 2×2 eigenvalues first appeared on John D. Cook.

Here’s a way to find a 95% confidence interval for any parameter θ. With probability 0.95, return the real line. With probability 0.05, return the empty set. Clearly 95% of the time this procedure will return an interval that contains θ. This example shows the difference between a confidence interval and a credible interval. A […]The post Universal confidence interval first appeared on John D. Cook.

When you leave the comfort of the real numbers, you might be mistaken about what a polynomial is. Suppose you’re looking at polynomials over some finite field. Why would you do that? Numerous practical applications, but that’s a topic for another post. You look in some reference that’s talking about polynomials and you see things […]The post What is a polynomial? first appeared on John D. Cook.

Suppose you do N trials of something that can succeed or fail. After your experiment you want to present a point estimate and a confidence interval. Or if you’re a Bayesian, you want to present a posterior mean and a credible interval. The numerical results hardly differ, though the two interpretations differ. If you got […]The post Confidence interval widths first appeared on John D. Cook.

The previous post looked at a cellular automaton introduced by Edward Fredkin. It has only two states: a cell is on or off. At each step, each cell is set to the sum of the states of the neighboring cells mod 2. So a cell is on if it had an odd number neighbors turned […]The post Multicolor reproducing cellular automata first appeared on John D. Cook.

Edward Fredkin is best known these days for the Fredkin gate, a universal reversible circuit. I recently found out that Fredkin was one of the pioneers in cellular automata. His student Edwin Banks describes a cellular automaton introduced by Fredkin [1]. Edward Fredkin of MIT has described the following interesting cellular space. If the states […]The post Self-reproducing cellular automata first appeared on John D. Cook.