John D. Cook

I was watching one of Brian Douglas’ videos on control theory (Discrete Control #5) and ran into a simple derivation of an approximation I presented earlier. Back in April I wrote several post on simple approximations for log, exp, etc. In this post I gave an approximation for the exponential function: The control theory video […]The post Simple derivation of exponential approximation first appeared on John D. Cook.

Converting text to all upper case or all lower case is a fairly common task. One way to convert text to upper case would be to use the tr utility to replace the letters a through z with the letters A through Z. For example, $ echo Now is the time | tr '[a-z]' '[A-Z]' […]The post Upper case, lower case, title case first appeared on John D. Cook.

NIST publication 800-90B, Recommendations for the Entropy Sources Used for Random Bit Generation, contains an interesting continued fraction. Define a function where is the incomplete gamma function. NIST 800-90B gives [1] a continued fraction implement F, but the continued fraction includes a parameter k for no apparent reason. NIST cites a paper that in turn […]The post Continued fraction from entropy source testing first appeared on John D. Cook.

Several people responded to my previous post asserting that screen readers would not be able to read text formatted via Unicode variants. Maybe some screen readers can’t handle this, but there’s no reason they couldn’t. Before I go any further, I’d like to repeat my disclaimer from the previous post: It’s a dirty hack, and […]The post Removing Unicode formatting first appeared on John D. Cook.

Twitter does not directly provide support for formatting text in bold, italic, etc. But it does support Unicode characters [1], and so a hack to get around the formatting limitation is to replace letters with Unicode variants. For example, you could tweet How to include bold or italic text in a tweet. I cheated in […]The post How to format text in Twitter first appeared on John D. Cook.

The Lotka-Volterra equations are a system of nonlinear differential equations for modeling a predator-prey ecosystem. After a suitable change of units the equations can be written in the form where ab = 1. Here x(t) is the population of prey at time t and y(t) is the population of predators. For example, maybe x represents rabbits […]The post Predator-Prey period first appeared on John D. Cook.

Yesterday I wrote about rates of convergence for natural cubic splines. This brief post reports similar results for more boundary conditions. As explained in the earlier post, a cubic spline that interpolates a function f at n+1 points satisfies 4n -2 equations in 4n variables. Two more equations are necessary to uniquely determine the interpolating […]The post Error estimates for splines with more boundary conditions first appeared on John D. Cook.

A theorem by Paley and Wiener says that a Fourier series with random coefficients produces Brownian motion on [0, 2π]. Specifically, produces Brownian motion on [0, 2π]. Here the Zs are standard normal (Gaussian) random variables. Here is a plot of 10 instances of this process. Here’s the Python code that produced the plots. import […]The post Random Fourier series first appeared on John D. Cook.

Suppose you want to approximate a function with a polynomial by interpolating it at evenly spaced points. You might reasonably expect that the more points you use, the better the approximation will be. That might be true, but it might not. As explained here, for some functions the maximum approximation error actually increases as the […]The post Rate of natural cubic spline convergence first appeared on John D. Cook.

Yesterday I wrote about how to interpolate a Brownian path. If you’ve created a discrete Brownian path with a certain step size, you can go back and fill in at smaller steps as if you’d generated the values from the beginning. This means that instead of generating samples in order as a random walk, you […]The post The fractal nature of Brownian motion first appeared on John D. Cook.

Accuracy isn’t everything. Sometimes when you’re approximating a function you care more about matching a function’s qualitative behavior than matching its numerical values. One example is interpolating monotonic data. Say your data show that increasing input always increases output. It’s likely you want a function that interpolates your data to have the same property and […]The post Monotonic interpolation first appeared on John D. Cook.

Let W(t) be a standard Wiener process, a.k.a. a one-dimensional Brownian motion. We can produce a discrete realization of W by first setting W(0) = 0. Then let W(1) be a sample from a N(0, 1) random variable. Then let W(2) be W(1) plus another N(0, 1) sample. At each integer n > 0, W(n) […]The post Interpolating Brownian motion first appeared on John D. Cook.

Ran across a fun quote this afternoon: There are in this world optimists who feel that any symbol that starts off with an integral sign must necessarily denote something that will have every property that they should like an integral to possess. This of course is quite annoying to us rigorous mathematicians; what is even […]The post Anything that starts with an integral sign first appeared on John D. Cook.

The top sides of your ceiling fan blades are dusty because of a boundary layer effect. When the blades spin, a thin layer of air above the blades moves with the blades. That’s why the fan doesn’t throw off the dust. A mathematical model may have very different behavior in two large regions, with a […]The post Technological boundary layer first appeared on John D. Cook.

Ronald Doerfler presents a simple approximation for the cosine of an angle in degrees in his book Dead Reckoning. cos d° ≈ 1 – d(d + 1)/7000 The approximation is good to about three digits for angles between 0° and 40°. Although cosine is an even function, the approximation above is not, and the error […]The post Mentally compute cosine first appeared on John D. Cook.

The solution to a differential equation is called oscillatory if its set of zeros is unbounded. This does not necessarily mean that the solution is periodic, but that it crosses the horizontal axis infinitely often. Fowler [1] studied the following differential equation by demonstrates both oscillatory and nonoscillatory behavior. x” + tσ |x|γ sgn x […]The post Oscillatory differential equations first appeared on John D. Cook.

The AGM (arithmetic-geometric mean) is the limiting value you get when you take the arithmetic and geometric means of two numbers, then take the arithmetic and geometric means of the results, over and over. am = lambda x, y: (x + y)*0.5 gm = lambda x, y: (x * y)**0.5 def agm(x, y): while x […]The post Extending AGM to three variables first appeared on John D. Cook.

The previous post looked at the average ℓp distance between points in the ℓp disk. This post looks at a related question, the average distance to the center. Unlike the previous post, we will look at dimension n greater than 2. The paper [1] cited earlier ends with this statement: It should be pointed out […]The post Average distance to the middle first appeared on John D. Cook.

This post will define ℓp distance and look at the average ℓp distance between points in the ℓp unit disk. We will compute this distance by simulation. ℓp distance The ℓp distance between two points (x1, y1) and (x2, y2) is The ℓp unit disk is the points whose ℓp distance from the origin is […]The post Average lp distance in a disk first appeared on John D. Cook.

The other day I ran across a Rick Beato video entitled “The most complex pop song of all time.” I thought the song would be something by a cerebral group like Rush, but instead it’s “Never Gonna Let You Go” by Sérgio Mendes. The song made it to #4 on the weekly pop charts in […]The post Complexity below the surface first appeared on John D. Cook.

Math and physics use Greek letters constantly, but seldom do they use letters from any other alphabet. The only Cyrillic letter I recall seeing in math is sha (Ш, U+0428) for the so-called Dirc comb distribution. One Hebrew letter is commonly used in math, and that’s aleph (א, U+05D0). Aleph is used fairly often, but […]The post Hebrew letters spotted in applied math first appeared on John D. Cook.

It’s well known that a binomial random variable can be approximated by a Poisson random variable, and under what circumstances the approximation is particularly good. See, for example, this post. A binomial random variable is the sum of iid (independent, identically distributed) Bernoulli random variables. But what if the Bernoulli random variables don’t have the […]The post Sum of independent but differently distributed variables first appeared on John D. Cook.

HAKMEM is a collection of tricks and trivia from the MIT AI lab written in 1972. I’ve mentioned HAKMEM here before. The image below came from HAKMEM item 123, explained here. I ran across HAKMEM item 143 while writing my previous two blog posts about the arithmetic-geometric mean (AGM). This entry, due to Gene Salamin, […]The post HAKMEM, AGM, and Pi first appeared on John D. Cook.

In the previous post I said that Jonathan and Peter Borwein figured out how to use the rapid convergence of the AGM to compute various functions, including logarithms. This post will show how to compute logarithms using the AGM. First I need to define the Jacobi theta functions Note that if q is very small, […]The post Computing logs with the AGM first appeared on John D. Cook.

Suppose you are visited by aliens from halfway across the galaxy. After asking you a lot of questions, they give you a parting gift, little black boxes can compute x – x²/2 + x³/3 – … with unbelievable speed and accuracy. You say thank you and your visitors vanish. You get back home and wonder […]The post The magic AGM box first appeared on John D. Cook.

This post will state a couple forms of the Cauchy-Schwarz inequality and then present the lesser-known reverse of the Cauchy-Schwarz inequality due to Pólya and Szegö. Cauchy-Schwarz inequality The summation form of the Cauchy-Schwarz inequality says that for sequences of real numbers xn and yn. The integral form of the Cauchy-Schwarz inequality says that for […]The post Reversed Cauchy-Schwarz inequality first appeared on John D. Cook.

I recently got a copy of Methods of Theoretical Physics by Morse and Feshbach. It’s a dense book, literally and metaphorically. I wondered whether it might be the densest book I own, so I weighed some of my weightier books. Morse and Feshbach has density 1.005 g/cm³, denser than water. Gravitation by Misner, Thorne, and […]The post My densest books first appeared on John D. Cook.

Pick a quaternion p = p0 + p1i + p2j + p3k and consider the function that acts on quaternions by multiplying them on the left by p. If we think of q as a vector in R4 then this is a linear function of q, and so it can be represented by multiplication by […]The post Quaternion product as a matrix product first appeared on John D. Cook.

If a vector space has an inner product, it has a norm: you can define the norm of a vector to be the square root of the inner product of the vector with itself. You can use the defining properties of an inner product to show that This is a form of the so-called polarization […]The post Inner product from norm first appeared on John D. Cook.

I’ve used Feedburner to allow people to subscribe to this blog via email. That service is going away, and so I just moved everyone over to MailerLite. I turned off Feedburner email, so nobody should get duplicate email. Feedburner’s RSS service is still going, for now, but most RSS subscribers use my RSS feed without […]The post Email subscription switchover first appeared on John D. Cook.

A couple days ago I wrote about Gauss’ trick to multiply two complex numbers using only 3 real multiplications. This post will do something similar with quaternions. Just as the most direct approach to computing complex products requires 4 real multiplications, the most direct approach to quaternion products requires 16 real multiplications. (See the function […]The post Quaternion products with fewer real products first appeared on John D. Cook.

You can use quaternions to describe rotations and quaternion products to carry out these rotations. These products have the form qpq* where q represents a rotation, q* is its conjugate, and p is the the vector being rotated. (That’s leaving out some details that we’ll get to shortly.) The primary advantage of using quaternions to […]The post Faster quaternion product rotations first appeared on John D. Cook.

I’ve written before about how the word “tensor” is applied to several different things, and that the connection between them isn’t immediately obvious. I thought about writing more about that some day, but I recently became aware of a paper that does this more thoroughly than I ever could. The paper looks at three notions […]The post Three kinds of tensor first appeared on John D. Cook.

The recent post on the wave equation on a disk showed that the Laplace operator has a different form in polar coordinates than it does in Cartesian coordinates. In general, the Laplacian is not simply the sum of the second derivatives with respect to each variable. Mathematica has a function, unsurprisingly called Laplacian, that will […]The post Laplacian in various coordinate systems first appeared on John D. Cook.

The most straight-forward way of multiplying two complex numbers requires four multiplications of real numbers: def mult(a, b, c, d): re = a*c - b*d im = b*c + a*d return (re, im) Gauss [1] discovered a way to do this with only three multiplications: def gauss(a, b, c, d): t1 = a*c t2 = […]The post Gauss algorithm for complex multiplication first appeared on John D. Cook.

This post will tie together many things I’ve blogged about before. The previous post justified separation of variables. This post will illustrate separation of variables. Also, this post will show why you might care about Bessel functions and their zeros. I’ve written about Bessel functions before, and said that Bessel functions are to polar coordinates […]The post Vibrating circular membranes first appeared on John D. Cook.

The separation of variables technique for solving partial differential equations looks like a magic trick the first time you see it. The lecturer, or author if you’re more self-taught, makes an audacious assumption, like pulling a rabbit out of a hat, and it works. For example, you might first see the heat equation ut = […]The post Justifying separation of variables first appeared on John D. Cook.

If you’re puzzled by the title of this post, allow me to explain. A natural reaction would be “Isn’t geometry intrinsically visual?” Indeed, geometry is motivated by things we can visualize. But modern developments of geometry have become heavy with formal machinery, so much so that one could reasonably ask “What happened to the geometry?” […]The post Visual geometry first appeared on John D. Cook.

I picked up a used copy of Quaternions and Rotation Sequences by Jack B. Kuipers for a project I’m starting to work on. The feedback I’ve seen on the book says it has good content but also has lots of typos. My copy has a fair number of corrections that someone penciled in. Someone on […]The post Volunteer-generated errata pages first appeared on John D. Cook.

Suppose in a company of N employees, m are chosen randomly for drug screening. In two independent screenings, what is the probability that someone will be picked both times? It may be unlikely that any given individual will be picked twice, while being very likely that someone will be picked twice. Imagine m employees being […]The post Random drug screening first appeared on John D. Cook.

As I mentioned a couple weeks ago, Feedburner, the service I’ve been using for blog email subscriptions, is shutting down. I’m switching over to MailerLite. The new email subscription is up and running. You can sign up here if you’d like. If you’re already subscribed via Feedburner, there’s no need to sign up again with […]The post Blog email subscription first appeared on John D. Cook.

A friend and I were discussing how to analyze his data one time and at the end of the conversation he said “So, basically you’re recommending division.” And indeed I was. The conclusion was to divide one thing by another. I’ve also recommended to clients that they use an extended Kalman filter or homomorphic encryption; […]The post Recommending division first appeared on John D. Cook.

Several weeks ago I wrote about the Fourier uncertainty principle which gives a lower bound on the product of the variance of a function f and the variance of its Fourier transform. This post expands on the earlier post by quoting some results from a recent paper [1]. Gaussian density The earlier post said that the […]The post Fourier, Gauss, and Heisenberg first appeared on John D. Cook.

Lambda calculus is simple. The definitions and rules of lambda calculus would easily fit on an index card. But you can’t really “read” lambda calculus so much as you can mentally execute it. Not many people can look at more than a few characters of lambda calculus and have any idea what it represents, though […]The post More readable lambda calculus first appeared on John D. Cook.

The RIPEMD-160 secure hash function may be best known these days for its role as part of the implementation of Bitcoin. I’ve wondered what “RIPEMD” stands for, and today I stumbled on an explanation [1]: “RIPEMD” stands for “RIPE Message Digest,” where “RIPE” stands for “RACE Integrity Primitives Evaluation” and where “RACE” stands for “Research […]The post What does RIPEMD stand for? first appeared on John D. Cook.

I’m reading Voyage to Alpha Centauri and ran into a question about relativity. The book says in one place that their ship is moving a 56.7% of the speed of light, and in another place it says that time moves about 20% slower for them relative to folks on Earth. Are those two statements consistent? […]The post Time dilation in SF and GPS first appeared on John D. Cook.

The previous post looked at the sequence kn mod 1 where k = log2(3). This value of k arose out of exploring perfect fifths and octaves. We saw that the sequence above fills in the unit interval fairly well, though not exactly regularly. The first 12 elements of the sequence nearly divide the unit interval […]The post A gentle introduction to QMC and discrepancy first appeared on John D. Cook.

In music, a perfect fifth is the interval between two notes whose frequencies are in 3:2 ratio. For example, the interval from an A at 440 Hz and an E at 660 Hz is a perfect fifth. Going up by 12 perfect fifths is very nearly the same as going up 7 octaves. That is, […]The post Perfect fifths, octaves, and an ergodic map first appeared on John D. Cook.

Occasionally I will use a row of three asterisks to indicate the separation between the body of a blog post and the footnotes, as I did in my previous post. *** This page quotes a typesetter who said she calls this a zareba. The OED defines a zareba as a kind of cattle pen but […]The post Zareba, asterism, and dinkus first appeared on John D. Cook.

The following trig identity looks like a mistake but is correct: sin(x + y) sin(x – y) = (sin(x) + sin(y)) (sin(x) – sin(y)) It looks as if someone fallaciously expanded sin(x + y) “=” sin(x) + sin(y) and sin(x – y) “=” sin(x) – sin(y). Although both expansions are wrong, their product is correct. […]The post A unique property of sine first appeared on John D. Cook.