John D. Cook

There are two uses of the word scalar, one from linear algebra and another from tensor calculus. In linear algebra, vector spaces have a field of scalars. This is where the coefficients in linear combinations come from. For real vector spaces, the scalars are real numbers. For complex vector spaces, the scalars are complex numbers. […]

In the first post in this series I mentioned several apparently unrelated things that are all called tensors, one of these being objects that behave a certain way under changes of coordinates. That’s what we’ll look at this time. In the second post we said that a tensor is a multilinear functional. A k-tensor takes k vectors and […]

In the previous post, we defined the tensor product of two tensors, but you’ll often see tensor products of spaces. How are these tensor products defined? Tensor product splines For example, you may have seen tensor product splines. Suppose you have a function over a rectangle that you’d like to approximate by patching together polynomials so that […]

The simplest definition of a tensor is that it is a multilinear functional, i.e. a function that takes several vectors, returns a number, and is linear in each argument. Tensors over real vector spaces return real numbers, tensors over complex vector spaces return complex numbers, and you could work over other fields if you’d like. A dot product is […]

The word “tensor” is shrouded in mystery. The same term is applied to many different things that don’t appear to have much in common with each other. You might have heared that a tensor is a box of numbers. Just as a matrix is a rectangular collection of numbers, a tensor could be a cube of […]

“Mathematics compares the most diverse phenomena and discovers the secret analogies that unite them.” — Joseph Fourier The above quote makes me think of a connection Fourier made between triangles and thermodynamics. Trigonometric functions were first studied because they relate angles in a right triangle to ratios of the lengths of the triangle’s sides. For […]

“Research is what I’m doing when I don’t know what I’m doing.” — Wernher von Braun I find Shinichi Mochizuki’s proof of the abc conjecture fascinating. Not the content of the proof—which I do not understand in the least—but the circumstances of the proof. Most mathematics, no matter how arcane it appears to outsiders, is […]

I had a discussion recently about whether things are really continuous in the real world. Strictly speaking, maybe not, but practically yes. The same is true of all mathematical properties. There are no circles in the real world, not in the Platonic sense of a mathematical circle. But a circle is a very useful abstraction, […]

Fractional integrals are easier to define than fractional derivatives. And for sufficiently smooth functions, you can use the former to define the latter. The Riemann-Liouville fractional integral starts from the observation that for positive integer n, This motivates a definition of fractional integrals which is valid for any complex α with positive real part. Derivatives and integrals are […]

Eberhard Zwicker proposed a model for combining several psychoacoustic metrics into one metric to quantify annoyance. It is a function of three things: N5, the 95th percentile of loudness, measured in sone (which is confusingly called the 5th percentile) ωS, a function of sharpness in asper and of loudness ωFR, fluctuation strength (in vacil), roughness (in […]

I’ve posted an online calculator to convert between two commonly used units of loudness, phon and sone. The page describes the purpose of both units and explains how to convert between them.

The most recent episode of 99% Invisible tells the story of the Corp of Engineers’ enormous physical model of the Mississippi basin, nearly half of the area of the continental US. Spanning over 200 acres, the model was built during WWII and was shut down in 1993. Here are some of my favorite lines from the show: […]

There are many ways to define fractional derivatives, and in general they coincide on nice classes of functions. A long time ago I wrote about one way to define fractional derivatives using Fourier transforms. From that post: Here’s one way fractional derivatives could be defined. Suppose the Fourier transform of f(x) is g(ξ). Then for […]

Maybe from the headline you were expecting a blank post? No, that’s not where I’m going. Yesterday I was on Amazon.com and noticed that nearly all the books they recommended for me were either about Lisp or mountain climbing. I thought this was odd, and mentioned it on Twitter. Carl Vogel had a witty reply: […]

There are four basic types of integral equations. There are many other integral equations, but if you are familiar with these four, you have a good overview of the classical theory. All four involve the unknown function φ(x) in an integral with a kernel K(x, y) and all have an input function f(x). In all […]

Fluctuation strength is similar to roughness, though at much lower modulation frequencies. Fluctuation strength is measured in vacils (from vacilare in Latin or vacillate in English). Police sirens are a good example of sounds with high fluctuation strength. Fluctuation strength reaches its maximum at a modulation frequency of around 4 Hz. For much higher modulation frequencies, one […]

Acoustic roughness is measured in aspers (from the Latin word for rough). An asper is the roughness of a 1 kHz tone, at 60 dB, 100% modulated at 70 Hz. That is, the signal (1 + sin(140πt)) sin(2000πt) where t is time in seconds. Here’s what that sounds like (if you play this at 60 […]

The Mittag-Leffler function is a generalization of the exponential function. Since k!= Γ(k + 1), we can write the exponential function’s power series as and we can generalize this to the Mittag=Leffler function which reduces to the exponential function when α = β = 1. There are a few other values of α and β for […]

A few weeks ago I started a series of posts on various things you could do with a functional fold. In the first post I mentioned that the idea came from a paper by Brian Beckman on Kalman filters and folds: This post was inspired by a paper by Brian Beckman (in progress) that shows how […]

I heard someone say the other day that the advantage of formal software validation methods is that they let you explore the corners, cases where intuition doesn’t naturally take you. This made me think of corners in the geometric sense. If you have a sphere in a box in high dimensions, nearly all the volume […]

The best way to develop software doesn’t exist. There’s only the best way that you know of, for a particular problem, with particular people, given their skills, experience, and constraints.

Presentation order People best understand computer programs in a different order than compilers do. This is a key idea of literate programming, and one that distinguishes literate programs from heavily commented programs. Traditional source code, no matter how heavily commented, is presented in the order dictated by the compiler. The computer is the primary audience. Literate programming is more humanistic in […]

This post explores a sentence from the book Single Digits: Any number in Pascal’s triangle that is not in the outer two layers will appear at least three times, usually four. Pascal’s triangle contains the binomial coefficients C(n, r) where n ranges over non-negative numbers and r ranges from 0 to n. The outer layers are the elements with r equal to […]

One of the things I learned from my tenure as a software project manager was that a project is more likely to succeed if there’s a way to get where you want to go continuously. You want to move a project from A to B gradually, keeping a working code base all along the way. At […]

The best teams have people with complementary skills, but similar work ethic. Academic assignments are the opposite. There’s not much variation in skills, in part because students haven’t yet developed specialized skills, and in part because students are in the same class because they have similar interests. The biggest variation is likely to be work ethic. It’s […]

Suppose two captains, A and B, are choosing people for their teams. To make things fair, the two captains alternate choices: A, B, A, B, etc. This is much better than simply letting A choose his team first and leaving B the dregs, but it still gives A a substantial advantage. If each captain picks the best remaining […]

If a real polynomial in one variable is a sum of squares, it obviously cannot be negative. For example, the polynomial p(x) = (x2 – 3)2 + (x + 7)2 is obviously never negative for real values of x. What about the converse: If a real polynomial is never negative, is it a sum of […]

The dormitory I lived in as an undergraduate had its own five-digit zip code at one time. It was rumored to be the largest dorm in the US, or maybe the largest west of the Mississippi, or something like that. There were about 3,000 of us living there. Although the dorm had enough people to justify […]

When I was in college, I sat in on a communication workshop for Latin American preachers. This was unusual since I’m neither Latin American nor a preacher, but I’m glad I was there. I learned several things in that workshop that I’ve used ever since. For example, when you’re gesturing about something moving forward in time, move your […]

I’m starting a new Twitter account @FunctorFact for functional programming and category theory. These two subjects have a lot of overlap, and some tweets will combine both, but many will be strictly about one or the other. So some content will be strictly about programming, some strictly about math, and some combining ideas from both.

Telephone numbers typically have around three distinct prime factors. The length of a phone number varies by country, but US a phone number is a 10 digit number, and 10-digit numbers are often divisible by three different prime numbers, give or take a couple. Assuming phone numbers are scattered among possible 10-digit numbers in a way that […]

A few days ago I wrote about creating ASCII art in Emacs using ditaa. Out of curiosity, I wanted to try making the Five Lemma diagram. [1] The examples in the ditaa site all have arrows between boxes, but you don’t have to have boxes. Here’s the ditaa source: A₀ ---> A₁ ---> A₂ ---> […]

A while back I wrote about how the leading digits of factorials follow Benford’s law. That is, if you look at just the first digit of a sequence of factorials, they are not evenly distributed. Instead, 1’s are most popular, then 2’s, etc. Specifically, the proportion of factorials starting with n is roughly log10(1 + 1/n). […]

This post uses a hypothesis test for proportions to look at a couple conjectures in number theory. It is similar to my earlier post on the chi-square test and prime remainders. You could read this as a post on statistics or a post on number theory, depending on which you’re less familiar with. Using statistical […]

Four posts about musical instrument acoustics: Electric guitars Kettledrums Saxophones Leaf blowers

Once in a while something big suddenly makes sense. Far more often, things make sense a little at a time. I was talking with someone a few days ago, and we both said that we never had an epiphany when category theory suddenly made sense. Instead, we both said we had a sequence of micro-epiphanies. […]

Yesterday I wrote about ASCII art diagrams and gave four reasons you might want to use this ancient approach to creating simple diagrams: It could be quicker than creating a graphical image . You can paste them into plain text documents like source code files. They can be produced programmatically. There is software to turn ASCII […]

“Technology is additive.” — Kevin Kelly Old technologies never die. Instead, their range of application shrinks. Or maybe it grows when conditions change. ASCII art, drawing pictures with fixed-width plain text characters, is no longer how many people want to produce diagrams. Just fire up Adobe Illustrator and you get incomparably more resolution of expression. […]

The other day I spoke to Chris Toomey from thoughtbot. Chris runs Upcase, thoughtbot’s online platform for learning about Rails, test-driven development, clean code, and more. I was curious about his work with Ruby on Rails since I know little about that world. And at a little deeper level, I wanted to get his thoughts […]

I’ve written several posts lately about various algorithms that can be expressed as functional folds: updating conjugate Bayesian models solving differential equations computing sample statistics These have all been numerical algorithms. Insertion sort is an example of a non-numerical algorithm that could be implemented as a fold. Insertion sort is not the fastest sorting algorithm. […]

First Brian Hayes wrote an excellent post about the remainders when primes are divided by other primes. Then I wrote a follow-on just focusing on the first part of his post. He mostly looked at pairs of primes, but I wanted to look in more detail at the first part of his post, simulating dice […]

Folds in functional programming are often introduced as a way to find the sum or product of items in a list. In this case the fold state has the same type as the list items. But more generally the fold state could have a different type, and this allows more interesting applications of folds. Previous […]

Yesterday Brian Hayes wrote a post about the distribution of primes. He showed how you could take the remainder when primes are divided by 7 and produce something that looks like rolls of six-sided dice. Here we apply the chi-square goodness of fit test to show that the rolls are too evenly distributed to mimic […]

I’ve been thinking lately about different things I’ve tried that didn’t work out and how grateful I am that they did not. The first one that comes to mind is my academic career. If I’d been more successful with grants and publications as a postdoc, it would have been harder to decide to leave academia. […]

Here are 10 blog posts I wrote earlier this year about spectral graph theory, studying graphs via the eigenvalues of matrices associated with the graphs. Measuring graph connectivity with eigenvalues Graph regularity and Laplacian eigenvalues Adding an edge increases eigenvalues Spectral coordinates in Python Bipartite graphs and the signless Lapacian Spectra of random graphs Can […]

One of the most widely used numerical algorithms for solving differential equation is the 4th order Runge-Kutta method. This post shows how the Runge-Kutta method can be written naturally as a fold over the set of points where the solution is needed. These points do not need to be evenly spaced. Given a differential equation […]

Bayesian calculations are intrinsically recursive: The posterior distribution from one step becomes the prior for the next step. With conjugate models, the prior and posterior belong to the same family of distributions. If a distribution from this family is determined by a fixed set of parameters, we only need to update these parameters at each step. This updating process is […]

A retronym is a new name created for an old thing, often made necessary by technological changes. For example, we have terms like “postal mail” or “snail mail” for what used to simply be “mail” because email has become the default. What was once called a “transmission” is now called a “manual transmission” since most […]

The park in the photo above flooded. And that’s a good thing. It’s designed to flood so that homes don’t. It’s not really a park that flooded. It’s a flood control project that most of the time doubles as a park. Ordinarily the park has a lake, but a few days a year the park is […]

Kalman filtering is a mixture of differential equations and statistics. Kalman filters are commonly used in tracking applications, such as tracking the location of a space probe or tracking the amount of charge left in a cell phone battery. Kalman filters provide a way to synthesize theoretical predictions and actual measurements, accounting for error in […]