John D. Cook

I've written three posts on cosine similarity lately. The first looked at interpreting cosine similarity. The second looked at an approximation related to the first. The third looked at how ranking according to cosine similarity works better than cosine similarity itself. Normalized word vectors are points on a high dimensional sphere, and geometry in high [...]The post Converting between cosine similarity and concentration ratio first appeared on John D. Cook.

Yesterday I wrote about the canonical example of how vector embeddings of words add: king" - man" + woman" queen" This should be interpreted as saying that the word vector forking, minus the word vector forman, plus the word vector forwoman, is in some sense close to the word vector forqueen. This post will [...]The post Coffee + milk latte first appeared on John D. Cook.

The product of four consecutive Fibonacci numbers equals the product of two consecutive integers. For example, 3 * 5 * 8 * 13 = 39 * 40. I ran across this theorem in a note [1] that says The product of any four consecutive Fibonacci numbers is twice a triangular number." Since triangular numbers have [...]The post Fibonacci product first appeared on John D. Cook.

The previous post looked at how to interpret cosine similarity, or equivalently angles between word vectors. In a high-dimensional space, randomly chosen vectors are likely nearly perpendicular, and so relatively large angles, such as 50, indicate very closely related words. Another way to look at this, as explained in the previous post, is that in [...]The post Simple approximation for spherical cap area first appeared on John D. Cook.

Machine learning represents words as vectors and measures the similarity of words by the angles between the vectors. For vectorsxandy, where is the angle between the vectors, and so This is the cosine similarity between the words represented by x and y. Small angles have large cosines, and so words with larger cosine similarities [...]The post What counts as a large cosine similarity? first appeared on John D. Cook.

Suppose I give you ann-digit number and ask you what it represents. This seems impossible, and in theory itis impossible. But in practice it's often possible. Apps on a phone may automatically interpret a 10-digit number as a phone number or a 16-digit number as a package tracking number. And very often these interpretations are [...]The post Guessing the meaning of a number first appeared on John D. Cook.

The Greek letter (beta) and the German letter (eszett) look similar, especially in some fonts. Now suppose an OCR program sees some character that could be a beta or could be an eszett. It could calculate some kind of distance between between the pixel pattern of the character and the pixel patterns of [...]The post Bayesian OCR first appeared on John D. Cook.

The idea of using computers to assist with proofs is not new. The first major computer-assisted proof was published in 1976, the proof of the four color theorem by Kenneth Appel and Wolfgang Haken. The authors reduced the proof of the four color theorem to verifying calculations on 1,834 configurations, each checked by a computer [...]The post A 50-year-old computer-assisted proof first appeared on John D. Cook.

A rising tide may lift all boats, but the AI tide lifts some boats much more than others. By all accounts, the best programmers have had the biggest productivity boost from AI. And top tier mathematicians are using AI to settle long-standing mathematical conjectures. AI is a powerful tool, but tools don't come to life [...]The post AI is an intelligence multiplier first appeared on John D. Cook.

Yesterday OpenAI announced a proof that settled a long-standing question about the Navier-Stokes equations from fluid dynamics. The announcement has created a lot of buzz, as one would expect. But there's an aspect of OpenAI's work that I haven't seen anyone talk about: they posted a Lean 4 formal proof at the same time as [...]The post The part of Navier-Stokes no one is talking about first appeared on John D. Cook.

There are rumors that a long-standing math problem, one of the Millennium Prize problems, has been solved. The problem concerns technical properties of solutions to the Navier-Stokes equations [1], a set of equations that describe the dynamics of fluid flow. Popular accounts of the problem are often oversimplified and misleading. Some reports will speak of [...]The post Navier-Stokes in the news first appeared on John D. Cook.

The Online Etymological Dictionary gives the following etymology forgrok: grok (v.) understand empathically," 1961, an arbitrary formation by U.S. science fiction writer Robert A. Heinlein (1907-1988) in his book Stranger in a Strange Land." In the book it is a transliteration of a Martian word and is said to mean etymologically to drink." It attained [...]The post Ngram error rate first appeared on John D. Cook.

The previous post discussed the motivation for and application of the rank-trace theorem. This post will give a proof. SupposeA is a real symmetric matrix. The rank-trace inequality says where tr is the trace operator, the sum of the elements along the diagonal of the matrix. Terse proof Here's the proof in a nutshell: diagonalizeA [...]The post Proof of the rank-trace theorem first appeared on John D. Cook.

Suppose you want to know the rank of an n * n matrix A, the number of linearly independent rows ofA, or equivalently the number of linearly independent columns. There are at least three difficulties. Difficulties in computing rank First of all, rank is not a continuous function of a matrix. Since rank is an [...]The post Computing a lower bound on matrix rank first appeared on John D. Cook.

NVIDIA has offered to buy Hugging Face for $12,930,300,000. 129303 is the Unicode code point for the Hugging Face emoj (U+1F917), which you can verify with the following Python code. >>> import unicodedata >>> 129303 == 0x1F917 True >>> unicodedata.name(chr(0x1F917)) 'HUGGING FACE' Related posts Prevent characters from displaying as emoji Unicode, Tolkien, and Privacy Unicode [...]The post Hugging Face Easter Egg first appeared on John D. Cook.

Eric Lu announced on X today that he has factored RSA-260, a number N with 260 digits (862 bits) that is the product of two large primes [1]. RSA numbers are challenge problems posed to gauge the security of RSA encryption, which rests on the difficulty of factoring large numbers [2]. The naming scheme is [...]The post New RSA number factored first appeared on John D. Cook.

I just found out Brian Beckman and I got a patent on work we did for GSI Technology [1]. Nearly all the work I do is under an NDA, so I don't often get a chance to talk about my projects. This work is public now that it's in a patent; I suppose it has [...]The post Patented application of linear algebra first appeared on John D. Cook.

I watched a few videos this morning, looking for ideas of what I could use AI to do. In one video, someone had an agent monitor tech news sites every 30 minutes to notify him of a variety of developments. No doubt that's less effort than visiting a bunch of sites every half hour, but [...]The post Making the unnecessary easier first appeared on John D. Cook.

This post provides a couple examples to go along with two earlier posts. The pattern we're illustrating is families of polynomialspn(x) that each satisfy a differential equation and a three-term recurrence. The differential equations have a second solutionqn(x) that is the larger solution with respect tox but the smaller solution with respect ton. In both [...]The post Second solutions first appeared on John D. Cook.

AI-powered coding agents increase productivity for many developers. But do these agents produce good-quality code? Some say this doesn't matter, we are heading toward dark software factories with source code never inspected, and maybe we should eliminate source code altogether-source code is the new assembly code". Others have a different view. Humans sometimes need to [...]The post What is the quality of software that AI writes? first appeared on John D. Cook.

When you're interested in studying a family of functions, it can be useful to look at a differential equation that the functions solve. This is a theme I've written about several times, most recently hereand here, but also three years ago here. Orthogonal polynomials are mathematically elegant as well as very useful in applications [1]. [...]The post Junk solutions first appeared on John D. Cook.

When I hear the term ultrasphericalI think of something extremely spherical. For example, a baseball is spherical, but a billiard ball is more spherical. Maybe a highly polished billiard ball is ultraspherical. Using this line of thought, the term ultraspherical polynomialis inexplicable. This is an example of the arcane terminology I wrote about recently. In [...]The post Ultraspherical first appeared on John D. Cook.

The previous post gave several examples of three-term recurrence relations for special functions. These relations can be computationally useful, but they have to be applied carefully. Several years ago I wrote a post on stable and unstable recurrences. In that post I show that the stability of the recurrence relation for Bessel functions produces depends [...]The post Numerical (in)stability of recurrence relations first appeared on John D. Cook.

There many examples of families of functions where each function can be computed as a linear combination of the two previous terms wherea andb are functions ofx and possibly n. This is called a three-term recurrence formula. It's amazing how often you can run into three-term recurrence formulas. There are theorems that give conditions for [...]The post Three-term recurrences first appeared on John D. Cook.

Probability density function must integrate to 1, and so if you know a density function up to a constant, the constant is determined. When you're looking at a probability densityf(x) for the first time, it helps to ignore the normalizing constant. Concentrate on the part of the function involving x and know that the normalizing [...]The post The von Mises-Fisher distribution first appeared on John D. Cook.

Special functions often have arcane names that not very helpful without some context. The previous post goes into some reasons for this. This post will expand on a point at the end of the post about modified" functions. Things are given their names for reasons. Discovering those reasons may help you understand their motivation and [...]The post What exactly is modified about a modified Bessel function? first appeared on John D. Cook.

Special functions are special because they're useful. They can also be shrouded in arcane terminology. These two facts are related. The more widely useful a function is, the more likely it is that the function will be discovered independently multiple times. Independent discoveries lead to varying definitions and notations. For example, there are two widely [...]The post Why special function terminology is arcane first appeared on John D. Cook.

Suppose you wanted to find the distance between Earth and Mars over time. To first approximation, both planets orbit the sun in elliptic orbits in the same plane. If you wanted to be more accurate, you'd need to take into account the fact that the orbit of Mars is tilted about 1.85 relative to the [...]The post The difference orbit inclination makes first appeared on John D. Cook.

There's a pizza shop near my home with a sign out front that says Coming Soon." When I drove by it this morning I thought about how you would model the time until an event happens that is coming soon." Suppose I look at the sign one day and guess how many days until the [...]The post Coming soon first appeared on John D. Cook.

A few weeks ago I wrote about the number system used in labeling spreadsheet columns. Labels run from A through Z, then AA through AZ, etc. This looks a lot like base 26, but it's not quite the same. It has no analog of zero. If Z were like zero, Y would be followed by [...]The post How would you know whether an ancient culture had zero? first appeared on John D. Cook.

I like AI-generated ASCII diagrams. Because nobody would ask AI to generate ASCII diagrams, and so, it's congruous. I like incongruity [1]. Aside from the incongruity of using a gazillion-parameter neural network to make 1970's style ASCII art, ASCII diagrams have some uses. They're absolutely tiny compared to image files. But more importantly they can [...]The post AI-generated ASCII diagrams first appeared on John D. Cook.

A new paper just came out, The Maximum-Area Small Polygon Problem. The paper solves the problem of finding, for each n, the n-gon with diameter 1 and maximum area. For odd n, the solution is what you might expect: a regular n-gon. I would expect this to be the solution for even n as well, [...]The post Big little hexagon first appeared on John D. Cook.

The imbalance conjecture is now a theorem. James Alexander Schreib and Yousof Yavari posted a proof last week. What does the conjecture theorem say? Start with a graphG with no edge between two nodes of the same degree. Then for every edge, calculate the absolute value of the difference of the degree of each end. [...]The post The imbalance theorem first appeared on John D. Cook.

Suppose you have a planet in an elliptical orbit around a star. The math is identical for any light object orbiting a heavy object, such as a moon or satellite orbiting a planet, but we'll call the heavy object a star and the light object a planet. The center of the star is not quite [...]The post Mean distance to the sun first appeared on John D. Cook.

The first post in the recent series of posts on Hadamard matrices describes a way of constructing new Hadamard matrices from two other Hadamard matrices by taking their Kronecker product. Starting with a Hadamard matrixH0 and a Hadamard matrix G, you can construct a sequence of Hadamard matrices by Hn+1 = G Hn for [...]The post Proportion of 1s in a Hadamard matrix first appeared on John D. Cook.

Error correcting codes are most simply described in terms of the errors they can certainly correct. For example, the Hadamard code used for the Mariner 9 probe to Mars encoded each 6-bit pixel to a 32-bit codeword in such a way that the original pixel could be recovered if no more than 7 bits were [...]The post Probability of correcting errors first appeared on John D. Cook.

Hadamard matrices are in the news following the recent announcement of a newly discovered Hadamard matrix. I've written three posts on Hadamard matrices recently, one as a sort of introduction and two on applications: the error correcting code used in the Mariner 9 probe and constructing sphere packings. A Hadamard matrix is an orthogonal matrix [...]The post Compressing a Hadamard matrix first appeared on John D. Cook.

Yesterday Levent Alpoge announced that he and his colleagues had discovered a new Hadamard matrix using Claude AI. That motivated a post I wrote this morning on how to construct Hadamard matrices. I mentioned in that post that these matrices arise in applications. This evening I gave an example, describing how NASA used a Hadamard [...]The post Hadamard Codes and Sphere Packing first appeared on John D. Cook.

NASA set Mariner 9 to photograph Mars in 1971. The images had to be encoded for transmission using an error-correcting code, otherwise they would be significantly corrupted when they were received on Earth. The images were encoded for transmission using a code based on Hadamard matrices, specifically a (32, 6, 16) Hadamard code. This means [...]The post How NASA's Mariner 9 probe encoded images first appeared on John D. Cook.

A Hadamard matrix is an orthogonal matrix whose entries are all either 1 or - 1. For example is a Hadamard matrix of order 2. True to Stigler's law of eponymy, James Joseph Sylvester investigated Hadamard matrices before Jacques Hadamard. Sylvester saw how to bootstrap the example above into more examples. If H is a [...]The post Constructing Hadamard matrices first appeared on John D. Cook.

Suppose you've never worked at the command line and you're reading a book about the bash shell. You read that !$ is a shortcut to refer to the last word of the previous command. That little fact will almost certainly not stick in your head for a couple reasons. First, you probably see no need [...]The post Cryptic but consistent first appeared on John D. Cook.

I was reading a blog post on boat names because it was on Hacker News this morning. The post contained a link to a data set on dog names in NYC and I poked around the data a little. The top names were not at all what I expected, but then again this is limited [...]The post Dogs and fat tails first appeared on John D. Cook.

Suppose you were able to go back in time, to an era before computers, and give someone contemporary cryptography. Encryption methods that are essentially unbreakable now would certainly be unbreakable then. But there's a catch: not only do attackers not have computers, neither do users. Manual cryptography If you told someone about RSA encryption, for [...]The post Manually unbreakable cryptography first appeared on John D. Cook.

The following extraordinary paragraph comes from Knuth's TAOCP Volume 4A, right before the last set of exercises. Many of the exercises below ask a modern reader to find and/or to correct errors in the literature of bygone days. The point is not to gloat over how smart we are in the 21st century; the point [...]The post Learning from historical mistakes first appeared on John D. Cook.

In science and engineering classes, you might describe a system using Newton's laws and end up with a differential equation. You then solve the differential equation, analytically or numerically, to see how the solutions behave. You might also do the opposite, especially in a mathematics class: look at what differential equation a set of functions [...]The post Inverse differential equations first appeared on John D. Cook.

I was reading a book on the history of the discovery of the structure of DNA [1] and was surprised by a few passing references to Bessel functions. According to Claude, When X-rays are diffracted by a helical structure, the resulting diffraction pattern breaks into a series of horizontal layer lines." Cochran, Crick, and Vand [...]The post DNA and Bessel functions first appeared on John D. Cook.

At the end of my post on how not to calculate cosine I said that the first step in calculating cosine, particularly cosine of a large number, would be to do range reduction. This post will present a simple range reduction method by Cody and Waite that is adequate for moderately large arguments. If you [...]The post A simple range reduction method first appeared on John D. Cook.

I have a program that shares files between my laptop and my phone. It works well, except for apostrophes. When I type an apostrophe ' on my laptop, it becomes aTM on my phone. And when I type 's on my phone, it becomes on my laptop. Apparently the phone turns the apostrophe (U+0027) [...]The post Corrupted apostrophes first appeared on John D. Cook.

Calculus professors with no experience in numerical computing will tell students that computers calculate trig functions with power series. They don't. I worked on the implementation of trig functions in hardware, and I can assure you we didn't just use power series. Power series are an excellent way to calculate functions near the center of [...]The post How not to calculate cosine first appeared on John D. Cook.

In a footnote to the previous post, I said that Python's math library can calculate the logarithm of extremely large numbers but not the cosine. This post will expand on that comment. In this post I'll use n = 200! as my example rather than 1000! because this value of N is larger than the [...]The post cos(200!) first appeared on John D. Cook.