John D. Cook

The fundamental solution" to a PDE solves the equation with the right-hand side set to . Intuitively, you can think of the delta function as striking something with a hammer in order to see how it rings. An aside on rigor A novice might be OK with the explanation above. A sophomore might rightly object [...]The post Fundamental solution first appeared on John D. Cook.

The following quote stuck with me when I read it years ago. Looking back I appreciate it even more. Now, when solving differential equations, or indeed solving any problem, it is permissible to use any methods at all, no matter how dubious, provided that once the solution has been found it can be proved to [...]The post No matter how dubious first appeared on John D. Cook.

What to do first after scoping out and starting a data science project? I've started an unsupervised learning project based on textual data. The first thing I like to do is actually look at the data. Is it noisy? What are the features-complex feature engineering needed? How heterogeneous? What generalization and overfitting challenges? Analysis can [...]The post Looking at Your Data first appeared on John D. Cook.

An ellipse has equation and a hyperbola has equation Similarly the superellipse has equation and the superhyperbola When p = 2, the absolute value signs are unnecessary and the superellipse and superhyperbola reduce to the ellipse and hyperbola respectively. Increasingp makes the superellipse more like a rectangle. But unlike a rectangle with rounded corners, the [...]The post Superhyperbola first appeared on John D. Cook.

The glass disk game is played on a grid. You have translucent colored glass disks you can either place on an edge or a vertex. There are two kinds of disks that can be placed on an edge: blue or yellow. A vertex with a blue and yellow disk looks green. There are two kinds [...]The post The glass disk game first appeared on John D. Cook.

This post started out as a thread on X. Here I've edited it into a blog post. The image below and the fact cited can be found in JPL Publication 89-24. In 1960 it didn't seem that it would be possible to explore the solar system beyond Jupiter without greatly improved propulsion. Then the gravitational [...]The post Voyager's slingshot maneuvers first appeared on John D. Cook.

When Voyager 1 and 2 left Earth, their internal computers were programmed to use Golay error correction codes. Images transmitted from Jupiter and Saturn were encoded using Golay codes. After leaving Saturn, the software was upgraded to use Reed-Solomon error correction codes. I didn't realize how much difference the change of encoding made until I [...]The post Advantages of Reed-Solomon codes over Golay codes first appeared on John D. Cook.

The previous post mentioned that Martin Gardner announced that Ramanujan's conjecture that exp(163) in an integer had been proven. This was an April Fool's joke in 1975. Gardner said Working by hand, he [Ramanujan] found the value to be 262537412640768743.999999999999... The calculations were tedious, and he was unable to verify the next decimal digits. Calculating [...]The post Hand calculating exp(x) first appeared on John D. Cook.

Yesterday I ran across this approximation for and posted it on X. 3 log(640320) / 163 This approximation is good to 15 decimal places, and so the approximation is exact to within the limits of floating point arithmetic. I said in a follow up comment that the 163 term looked familiar from [...]The post Very accurate pi approximation first appeared on John D. Cook.

What is the interior of a circle? Obvious. What is the interior of a parabola? Not quite as obvious. What is the interior of a hyperbola? Not at all obvious. Is it possible to define interior in a way that applies to all conic sections? Circles If you remove a circle from the plane, there [...]The post Interior of a conic first appeared on John D. Cook.

Community best practices are useful for helping use a software product more effectively. I've just completed a small project using the Z3 solver. Here are some things I've learned: My project involves an optimization problem: for a subset of Boolean variables, maximize the count of how many are true. My specific problem is solved much [...]The post Lessons Learned With the Z3 SAT/SMT Solver first appeared on John D. Cook.

The last chord of the James Bond theme is known as the James Bond chord. If you've ever seen a Bond movie, you'll recognize it. Here's a quick video showing how to play it on guitar. I tried asking Grok 3 to generate some LilyPond code for sheet music a few days ago. LilyPond is [...]The post Grokking the James Bond chord first appeared on John D. Cook.

The last several posts have looked at counting two kinds of permutations: those that leave no consecutive integers and those that leave no integer fixed. As n grows large, the proportion of permutations ofn elements that fall into both classes approaches 1/e. This post will look a little closer and as how fast each proportion [...]The post Rates of convergence first appeared on John D. Cook.

Yesterday I wrote a couple of posts about a combinatorics question that lead to OEIS sequence A000255. That page has this intriguing line: This comment derives from a family of recurrences found by Malin Sjodahl for a combinatorial problem for certain quark and gluon diagrams (Feb 27 2010) I love how pulling on a thread [...]The post Gluons, quarks, letters, and envelopes first appeared on John D. Cook.

The previous post looked at how to compute Q(n), the number of permutations of1, 2, 3, ..., n + 1 that contain no consecutive integers. We found a way to numerically compute Q(n) but no analytic expression that would let us compute asymptotics. The sequence Q(n) is sequence A000255 in OEIS, and OEIS gives the [...]The post Example using a generating function to find asymptotic behavior first appeared on John D. Cook.

I was working on something this week that made me wonder how many permutations break up all consecutive elements. For example, if we're looking at permutations of abcde thenacebd counts, but acdbe does not. I'd like to count the number of such permutations, and estimate for large N the number of permutations ofN elements with [...]The post Permutations with no consecutive elements first appeared on John D. Cook.

The following equation, known as the BBP formula [1], will let you compute the nth digit of directly without having to compute the previous digits. I've seen this claim many times, but I've never seen it explained in much detail how this formula lets you extract digits of . First of all, this formula [...]The post Computing the nth digit of directly first appeared on John D. Cook.

Colossus The xAI Colossus supercomputer contains 100,000 NVIDIA H100 GPUs. Upgrades are planned, ultimately up to as much as a million GPUs. The H100 has theoretical peak speed of at least 60 teraFLOPs (FP64 tensor core), though the actual number depends on the power and frequency cap settings on the GPUs. Admittedly FP64 is overkill [...]The post Colossus versus El Capitan: A Tale of Two Supercomputers first appeared on John D. Cook.

When I'm in the mood to write, you can often follow a chain of though in my posts. Recently, a post on LLM tokenization lead to a post on how Unicode characters are tokenized, which led to a post on Unicode surrogates. The latter ended by touching on Unicode's PUA (Private Use Area), which of [...]The post Unicode, Tolkien, and Privacy first appeared on John D. Cook.

At the highest level, Unicode is a simply a list of symbols. But when you look closer you find that isn't entirely true. Some of the symbols are sorta meta symbols. And while a list of symbols is not complicated, this list is adjacent to a lot of complexity. I've explored various rabbit holes of [...]The post Unicode surrogates first appeared on John D. Cook.

I recently ran across the article Something weird is happening with LLMs and chess. One of the things it mentions is how the a minor variation in a prompt can have a large impact on the ability of an LLM to play chess. One extremely strange thing I noticed was that if I gave a [...]The post Practical consequences of tokenization details first appeared on John D. Cook.

I mentioned in the previous post that not every Unicode character corresponds to a token in ChatGPT. Specifically I'm looking at gpt-3.5-turbo in tiktoken. There are 100,256 possible tokens and 155,063 Unicode characters, so the pigeon hole principle says not every character corresponds to a token. I was curious about the relationship between tokens and [...]The post ChatGPT tokens and Unicode first appeared on John D. Cook.

Large language models operate on tokens, not words, though tokens roughly correspond to words. A list of words would not be practical. There is no definitive list of all English words, much less all words in all languages. Still, tokens correspond roughly to words, while being more flexible. Words are typically turned into tokens using [...]The post The difference between tokens and words first appeared on John D. Cook.

The GELU (Gaussian Error Linear Units) activation function was proposed in [1]. This function is x (x) where is the CDF of a standard normal random variable. As you might guess, the motivation for the function involves probability. See [1] for details. The GELU function is not too far from the more familiar ReLU, [...]The post A simpler GELU activation function approximation first appeared on John D. Cook.

A common technique for memorizing numbers is to associate numbers with words. The Major mnemonic system does this by associating consonant sounds with each digit. You form words by inserting vowels as you please. There are many possible encodings of numbers, but sometimes you want to pick a canonical word for each number, what's commonly [...]The post Spreading out words in space first appeared on John D. Cook.

Databasetechnology is a mature field, and techniques for optimizing databases are well understood. However, surprises can still happen. Certain performance optimizations you might expect to be automatic are not really. I'm working with a legacy code developed some time ago, before modern notions of separation of concerns between code business logic and data storage. The [...]The post On Making Databases Run Faster first appeared on John D. Cook.

Like the previous post, this post also seeks to approximately reproduce a hand-drawn plot. This time the goal is reproduce figure 7.3 from A&S page 298. This plot is a visualizing of the function of a complex variable w(z) = exp(-z^2) erfc(- iz) where erfc is the complementary error function. A&S calls the graph above [...]The post Duplicating a hand-drawn contour plot first appeared on John D. Cook.

The plots in old (i.e. pre-computer) math books are impressive. These plots took a lot of effort to produce, and so they weren't created lightly. Consequently they tend to be aesthetically and mathematically interesting. A few weeks ago I recreated a plot from A&S using Mathematica, and today I'd like to do the same for [...]The post Reproducing a hand-drawn plot first appeared on John D. Cook.

Gaussian elimination is pretty simple if all goes well, and in introductory courses all does go well. You have an n * n matrix A, an n * 1 column vector b, and you want to find an n * 1 column vector x such that Ax = b. You subtract multiples of the first [...]The post What is partial pivoting? first appeared on John D. Cook.

An earlier post needed to calculate how much the speed of a planet varies in orbit. The planet moves fastest as perihelion, the point in its orbit closes to the sun, and it moves slowest at aphelion, when it is furthest from the sun. The ratio of the maximum to minimum speed turns out to [...]The post Max and min orbital speed first appeared on John D. Cook.

The previous post looked at one challenge with designing a calendar for Mars, namely how to vary the number of days per month so that each month corresponds to a change of 30 degrees with respect to the sun. This is a bigger problem on Mars than here on Earth. That post assumed that a [...]The post Martian Leap Years first appeared on John D. Cook.

I recently started reading The Case for Mars by Robert Zubrin. This post will unpack one line from that book regarding creating a calendar for Mars: Equipartitioned months don't work for Mars, because the planet's orbit is elliptical, which causes the seasons to be of unequal length. This sentence doesn't sit well at first for [...]The post A calendar for Mars first appeared on John D. Cook.

Making your code to run faster starts with understanding where in the code the runtime is actually spent. But suppose, for whatever reason, the code profiling tools won't work? I recently used MS Visual Studio on a legacy C++ code. The code crashed shortly after startup when attempting to profile, though otherwise the code ran [...]The post Code Profiling Without a Profiler first appeared on John D. Cook.

Is the shape of a triangle determined by its perimeter and area? In other words, if two triangles have the same area and the same perimeter, are the triangles similar? [1] It's plausible. A triangle has three degrees of freedom: the lengths of the three sides. Specifying the area and perimeter removes two degrees of [...]The post Do perimeter and area determine a triangle? first appeared on John D. Cook.

When my children were little, I read the Little House on the Prairie books aloud to them and I naturally saw the books through the eyes of a child. Last night I started reading the books by myself for the first time and saw them very differently. Laura Ingalls Wilder wrote the Little House books [...]The post Settlers versus Hipsters first appeared on John D. Cook.

The previous post looked at how iterations of converge to the golden ratio . That post said we could start at any positive x. We could even start at any x > -3/4 because that would be enough for the derivative of (1 + x) to be less than 1, which means the iteration will [...]The post Complex golden convergence first appeared on John D. Cook.

The golden ratio satisfies the following equation. The proof most commonly given is to let x equal the right-hand side of the equation, then observe that x^2 = 1 + x, the quadratic equation for the golden ratio. The quadratic has two roots: and -1/. Since x > 1, x = . This [...]The post Golden convergence first appeared on John D. Cook.

The Major mnemonic system makes numbers easier to memorize by encoding them as words. Each digit corresponds to one or more consonant sounds, and you can fill in vowels as you wish. In August 2022 I tried creating a few images using DALL-E 2. The results were disappointing and sometimes disturbing. To illustrate the use [...]The post Mnemonic images with Grok 3 first appeared on John D. Cook.

Is it possible to come up with truly innovative ideas when you're not part of the institutions where the expertise resides? According to one study, the answer would seem to be No." The book, The Sociology of Philosophies by Randall Collins, makes a case for how great ideas through history have always developed, almost [...]The post Standing with Intellectual Giants first appeared on John D. Cook.

This morning I read Alberto Romeo's article Grok 3: Another Win For The Bitter Lesson. I wasn't clear on what exactly the bitter lesson was, so I followed the link to Romeo's previous article GPT-4: The Bitterer Lesson, which lead to Rich Sutton's original article The Bitter Lesson. Sutton opens his article by saying The [...]The post Rich Sutton's bitter lesson of AI first appeared on John D. Cook.

The multiplication table of a finite group forms a Latin square. You form the multiplication table of a finite group just as you would the multiplication tables from your childhood: list the elements along the top and side of a grid and fill in each square with the products. In the context of group theory [...]The post Multiplication tables and Latin squares first appeared on John D. Cook.

The Buenos Aires constant is 2.92005097731613... What's so special about this number? Let's see when we use it to initialize the following Python script. s = 2.920050977316134 for _ in range(10): i = int(s) print(i) s = i*(1 + s - i) What does this print? 2, 3, 5, 7, 11, 13, 17, 19, 23, [...]The post The Buenos Aires constant first appeared on John D. Cook.

The other day MathMatize posted roses are red books go on a shelf 1+2+3+4+ ... with a photo of Ramanujan on X. This was an allusion to the bizarre equation 1 + 2 + 3 + ... = - 1/12. This comes up often enough that I wanted to write a post that I could [...]The post 1 + 2 + 3 + ... = -1/12 first appeared on John D. Cook.

The cosine of a multiple of can be written as a polynomial in cos . For example, cos 3 = 4 cos3 - 3 cos and cos 4 = 8 cos4 - 8 cos2 + 1. But it may or may not be possible to write the sine of a [...]The post Multiple angle asymmetry first appeared on John D. Cook.

I read something once about an American telegraph operator who had to switch over to using Russian Morse code during WWII. I wondered how hard that would be, but let it go. The idea came back to me and I decided to settle it. It would be hard to switch from being able to recognize [...]The post Russian Morse Code first appeared on John D. Cook.

Two families of orthogonal polynomials are named after Chebyshev because he explored their properties. These are prosaically named Chebyshev polynomials of the first and second kind. I recently learned there are Chebyshev polynomials of the third and fourth kind as well. You might call these posthumous Chebyshev polynomials. They were not developed by Mr. Chebyshev, [...]The post Posthumous Chebyshev Polynomials first appeared on John D. Cook.

I recently ran across an interesting family of Pythagorean triples [1]. You can verify that a^2 + b^2 = c^2 for all n. Sparseness When written in binary, a has only two bits set, and c has only four bits set. It's not as immediately obvious, but b has only two bits that are not [...]The post Sparse binary Pythagorean triples first appeared on John D. Cook.

The reactions to the new DeepSeek-R1 AI model in recent days seem limitless. Some say it runs so much faster than existing models that we will no longer need the billions of dollars in compute hardware that big tech is preparing to buy. Is that plausible? To get an answer, we need only look [...]The post DeepSeek-R1: Do we need less compute now? first appeared on John D. Cook.

The previous post discussed complex numbers, dual numbers, and double numbers. All three systems are constructed by adding some element to the real numbers that has some special algebraic property. The complex numbers are constructed by adding an element i such that i^2 = -1. The dual numbers add an element 0 with [...]The post Matrix representations of number systems first appeared on John D. Cook.

The complex numbers are formed by adding an element i to the real numbers such that i^2 = - 1. We can create other number systems by adding other elements to the reals. One example is dual numbers. Here we add a number 0 with the property ^2 = 0. Dual numbers have [...]The post Euler's formula for dual numbers and double numbers first appeared on John D. Cook.