John D. Cook

When a function is not differentiable in the classical sense there are multiple ways to compute a generalized derivative. This post will look at three generalizations of the classical derivative, each applied to the ReLU (rectified linear unit) function. The ReLU function is a commonly used activation function for neural networks. It's also called the [...]The post Three ways to differentiate ReLU first appeared on John D. Cook.

Someone said a technique is a trick that works twice. I wanted to see if I could get anything interesting by turning the trick in the previous post into a technique. The trick created a high-order approximation by subtracting a multiple one even function from another. Even functions only have even-order terms, and by using [...]The post Turning a trick into a technique first appeared on John D. Cook.

Suppose you have an arc a, a portion of a circle of radius r, and you know two things: the length c of the chord of the arc, and the lengthbof the chord of half the arc, illustrated below. Here is the central angle of the arc. Then the length of the arc, r, [...]The post Circular arc approximation first appeared on John D. Cook.

The previous post looks at the nonlinear pendulum equation and what difference it makes to the solutions if you linearize the equation. If the initial displacement is small enough, you can simply replace sin with . If the initial displacement is larger, you can improve the accuracy quite a bit by solving the linearized [...]The post Closed-form solution to the nonlinear pendulum equation first appeared on John D. Cook.

There's a nice formula for thenth derivative of a product. It looks a lot like the binomial theorem. There is also a formula for thenth derivative of a quotient, but it's more complicated and less known. We start by writing the quotient rule in an unusual way. Applying the quotient rule twice gives the following. [...]The post nth derivative of a quotient first appeared on John D. Cook.

The equation of motion for a pendulum is the differential equation whereg is the acceleration due to gravity and is the length of the pendulum. When this is presented in an introductory physics class, the instructor will immediately say something like we're only interested in the case where is small, so we can [...]The post How nonlinearity affects a pendulum first appeared on John D. Cook.

The previous post gave a simple and accurate approximation for the smaller angle of a right triangle. Given aright triangle with sides a,b, andc, wherea is the shortest side andc is the hypotenuse, the angle opposite side a is approximately in radians. The previous post worked in degrees, but here we'll use radians. If the [...]The post Approximation to solve an oblique triangle first appeared on John D. Cook.

Suppose you have a right triangle with sidesa,b, andc, wherea is the shortest side andc is the hypotenuse. Then the following approximation from [1] for the angle Aopposite side a seems too simple and too accurate to be true. In degrees, A a 172 / (b + 2c). The approximation above only involves simple [...]The post Simple approximation for solving a right triangle first appeared on John D. Cook.

AI coding agents improved greatly last summer, and again last December-January. Here are my experiences since my last post on the subject. The models feel subjectively much smarter. They can accomplish a much broader range of tasks. They seem to have a larger, more comprehensive in-depth view of the code base and what you are [...]The post An AI Odyssey, Part 4: Astounding Coding Agents first appeared on John D. Cook.

A few days ago I wrote a post on Newton's diameter theorem. The theorem says to plot the curve formed by the solutions to f(x, y) = 0 where f is a polynomial in x and y of degree n. Next plot several parallel lines that cross the curve at n points and find the [...]The post More on Newton's diameter theorem first appeared on John D. Cook.

The previous post looked at the FP4 4-bit floating point format. This post will look at another 4-bit floating point format, NF4, and higher precision analogs. NF4 and FP4 are common bitsandbytes 4-bit data types. If you download LLM weights from Hugging Face quantized to four bits, the weights might be in NF4 or FP4 [...]The post Gaussian distributed weights for LLMs first appeared on John D. Cook.

In ancient times, floating point numbers were stored in 32 bits. Then somewhere along the way 64 bits became standard. The C programming language retains the ancient lore, using float to refer to a 32-bit floating point number and double to refer to a floating point number with double the number of bits. Python simply [...]The post 4-bit floating point FP4 first appeared on John D. Cook.

Letf(x,y) be annth degree polynomial inx andy. In general, a straight line will cross the zero set off inn locations [1]. Newton defined a diameter to be any line that crosses the zero set off exactlyn times. If f(x,y) =x^2 +y^2 - 1 then the zero set off is a circle and diameters of the [...]The post Newton diameters first appeared on John D. Cook.

The post AI and reasoning: Are we in danger? first appeared on John D. Cook.

We know a lot about Robert Oppenheimer and Los Alamos national laboratory from the movie.Many AI researchers today read avidly the classic work by Richard Rhodes, the making of the atomic bomb. But there's a stronger connection between Oppenheimers work and current AI research that's sometimes not noticed. By the end of World War [...]The post Oppenheimer's legacy: How the national labs forged the age of hyperscale AI first appeared on John D. Cook.

If you know the distance d to a satellite, you can compute a circle of points that passes through your location. That's because you're at the intersection of two spheres-the earth's surface and a sphere of radius d centered on the satellite-and the intersection of two spheres is a circle. Said another way, one observation [...]The post Intersecting spheres and GPS first appeared on John D. Cook.

The Wikipedia article on modern triangle geometry has an image labeled Artzt parabolas" with no explanation. A quick search didn't turn up anything about Artzt parabolas [1], but apparently the parabolas go through pairs of vertices with tangents parallel to the sides. The general form of a conic section is ax^2 + bxy + cy^2 [...]The post Finding a parabola through two points with given slopes first appeared on John D. Cook.

Andrzej Odrzywolek recently posted an article on arXiv showing that you can obtain all the elementary functions from just the function and the constant 1. The following equations, taken from the paper's supplement, show how to bootstrap addition, subtraction, multiplication, and division from the eml function. See the paper and supplement for how to obtain [...]The post Mathematical minimalism first appeared on John D. Cook.

The date of Easter The church fixed Easter to be the first Sunday after the first full moon after the Spring equinox. They were choosing a date in the Roman (Julian) calendar to commemorate an event whose date was known according to the Jewish lunisolar calendar, hence the reference to equinoxes and full moons. The [...]The post Lunar period approximations first appeared on John D. Cook.

Today is Orthodox Easter. Western churches celebrated Easter last week. Why are the Eastern and Western dates of Easter different? Is Eastern Easter always later than Western Easter? How far apart can the two dates be? Why the dates differ Easter is on the first Sunday after the first full moon in Spring [1]. East [...]The post The gap between Eastern and Western Easter first appeared on John D. Cook.

There's a lot of mathematics just off the beaten path. You can spend a career in math and yet not know all there is to know about even the most basic areas of math. For example, this post will demonstrate something you may not have seen about decimal forms of fractions. Let p > 5 [...]The post Distribution of digits in fractions first appeared on John D. Cook.

Saw a post on X saying that the latitude of the Pyramid of Giza is the same as the speed of light. I looked into this, expecting it to be approximately true. It's exactly true in the sense that the speed of light in vacuum is 299,792,458 m/s and the line of latitude 29.9792458 N [...]The post The Great Pyramid of Giza and the Speed of Light first appeared on John D. Cook.

I recently ran across a post on X describing a process for creating a random fractal. First, pick a random point c inside a hexagon. Then at each subsequent step, pick a random side of the hexagon and create the triangle formed by that side andc. Updatec to be the center of the new triangle [...]The post Random hexagon fractal first appeared on John D. Cook.

I recently found out about Andrica's conjecture: the square roots of consecutive primes are less than 1 apart. In symbols, Andrica's conjecture says that if pn and pn+1 are consecutive prime numbers, then pn+1 - pn < 1. This has been empirically verified for primes up to 2 * 1019. If the conjecture is true, [...]The post Root prime gap first appeared on John D. Cook.

Last week I wrote about the orbit of Artemis II. The orbit of Artemis I was much more interesting. Because Artemis I was unmanned, it could spend a lot more time in orbit. The Artemis I mission took 25 days while Artemis II will take 10 days. Artemis I took an unusual path, orbiting the [...]The post A Three- and a Four- Body Problem first appeared on John D. Cook.

Landauer's principle gives a lower bound on the amount of energy it takes to erase one bit of information: E >= log(2) kB T where kB is the Boltzmann constant and T is the ambient temperature in Kelvin. The lower bound applies no matter how the bit is physically stored. There is no theoretical lower [...]The post Toffoli gates are all you need first appeared on John D. Cook.

The best way to run AI and remain HIPAA compliant is to run it locally on your own hardware, instead of transferring protected health information (PHI) to a remote server by using a cloud-hosted service like ChatGPT or Claude. [1]. There are HIPAA-compliant cloud options, but they're both restrictive and expensive. Even enterprise options are [...]The post HIPAA compliant AI first appeared on John D. Cook.

This post will look at the problem of updating an average grade as a very simple special case of Bayesian statistics and of Kalman filtering. Suppose you're keeping up with your average grade in a class, and you know your average after n tests, all weighted equally. m = (x1 + x2 + x3 + [...]The post Kalman and Bayes average grades first appeared on John D. Cook.

I used the term perilune in yesterday's post about the flight path of Artemis II. When Artemis isclosest to the moon it will be furthest from earth because its closest approach to the moon, its perilune, is on the side of the moon opposite earth. Perilune is sometimes called periselene. The two terms come from [...]The post Roman moon, Greek moon first appeared on John D. Cook.

I was looking through an old geometry book [1] and saw a hyperbolic analog of Napier's mnemonic for spherical trigonometry. In hindsight of course there's a hyperbolic analog: there's a hyperbolic analog of everything. But I was surprised because I'd never thought of this before. I suppose the spherical version is famous because of its [...]The post Hyperbolic version of Napier's mnemonic first appeared on John D. Cook.

The Artemis II mission launched yesterday. Much like the Apollo 8 mission in 1968, the goal is to go around the moon in preparation for a future mission that will land on the moon. And like Apollo 13, the mission will swing around the moon rather than entering lunar orbit. Artemis II will deliberately follow [...]The post Artemis II, Apollo 8, and Apollo 13 first appeared on John D. Cook.

Pentagonal numbers are truncated triangular numbers. You can take the diagram that illustrates thenth pentagonal number and warp it into the base of the image that illustrates the (2n - 1)st triangular number. If you added a diagram for the (n - 1)st triangular number to the bottom of the image on the right, you'd [...]The post Pentagonal numbers are truncated triangular numbers first appeared on John D. Cook.

I'm skeptical that quantum computing will become practical. However, if it does become practical before we're prepared, the world's financial system could collapse. Everyone agrees we should prepare for quantum computing, even those of us who doubt it will be practical any time soon. Quantum computers exist now, but the question is when and if [...]The post Quantum Y2K first appeared on John D. Cook.

Peter Vogel posted the following image on X yesterday. The receive side of the coin is a decision tree for decoding Morse code. The shape is what makes this one interesting. Decision trees are typically not very compact. Each branch is usually on its own horizontal level, with diagonal lines going down from each node [...]The post Morse code tree first appeared on John D. Cook.

While shopping on a major e-commerce site, I wanted to get an answer to an obscure question about a certain product. Not finding the answer immediately on the product page, I thought I'd try clicking the AI shopping assistant helper tool to ask this question. I waited with anticipation for an answer I would expect [...]The post An AI Odyssey, Part 3: Lost Needle in the Haystack first appeared on John D. Cook.

Suppose you have a calculator or math library that only handles real arguments but you need to evaluate sin(3 + 4i). What do you do? If you're using Python, for example, and you don't have NumPy installed, you can use the built-in math library, but it will not accept complex inputs. >>> import math >>> [...]The post Computing sine and cosine of complex arguments with only real functions first appeared on John D. Cook.

I alluded to Lebesgue constants in the previous post without giving them a name. There I said that the bound on order ninterpolation error has the form whereh is the spacing between interpolation points and is the error in the tabulated values. The constantc depends on the function f being interpolated, and to a [...]The post Lebesgue constants first appeared on John D. Cook.

Richard Feynman said that almost everything becomes interesting if you look into it deeply enough. Looking up numbers in a table is certainly not interesting, but it becomes more interesting when you dig into how well you can fill in the gaps. If you want to know the value of a tabulated function between values [...]The post How much precision can you squeeze out of a table? first appeared on John D. Cook.

The previous post looked at an inequality discovered by Dmitri Mendeleev and generalized by Andrey Markov: Theorem (Markov): If P(x) is a real polynomial of degreen, and |P(x)| 1 on [-1, 1] then |P'(x)| n^2 on [-1, 1]. IfP(x) is a trigonometric polynomial then Bernstein proved that the bound decreases from n^2 ton. Theorem [...]The post From Mendeleev to Fourier first appeared on John D. Cook.

Dmitri Mendeleev is best known for creating the first periodic table of chemical elements. He also discovered an interesting mathematical theorem. Empirical research led him to a question about interpolation, which in turn led him to a theorem about polynomials and their derivatives. I ran across Mendeleev's theorem via a paper by Boas [1]. The [...]The post Mendeleev's inequality first appeared on John D. Cook.

A few years ago I wrote about comm, a utility that lets you do set theory at the command line. It's a really useful little program, but it has two drawbacks: the syntax is hard to remember, and the input files must be sorted. If A and B are two sorted lists, comm A B [...]The post Set intersection and difference at the command line first appeared on John D. Cook.

The hardest part of using regular expressions is not crafting regular expressions per se. In my opinion, the two hardest parts are minor syntax variations between implementations, and all the environmental stuff outside of regular expressionsper se. Embedded regular expression modifiers address one of the environmental complications by putting the modifier in the regular expression [...]The post Embedded regex flags first appeared on John D. Cook.

The gamma function (z) extends the factorial function from integers to complex numbers. (Technically, (z + 1) extends factorial.) There are other ways to extend the factorial function, so what makes the gamma function the right choice? The most common answer is the Bohr-Mollerup theorem. This theorem says that if f: (0, ) (0, [...]The post A lesser-known characterization of the gamma function first appeared on John D. Cook.

The alternating series test is part of the standard calculus curriculum. It says that if you truncate an alternating series, the remainder is bounded by the first term that was left out. This fact goes by in a blur for most students, but it becomes useful later if you need to do numerical computing. To [...]The post Tighter bounds on alternating series remainder first appeared on John D. Cook.

If a number has a finite but nonzero fractional part, so do all its powers. I recently ran across a proof in [1] that is shorter than I expected. Theorem: Supposer is a real number thatis not an integer, and the decimal part ofr terminates. Then rk is not an integer for any positive integer [...]The post Powers don't clear fractions first appeared on John D. Cook.

The previous post introduced the idea of a twelve-tone row, a permutation of the twelve pitch classes of a chromatic scale. The earlier post also introduced a group of operations on a tone row with elements P, R, I, and RI. Here P, which stands for prime", is the identity operator: it leaves the tone [...]The post Tone row operations first appeared on John D. Cook.

Atonal music is difficult to compose because it defies human instincts. It takes discipline to write something so unpleasant to listen to. One technique that composers use to keep their music from falling into tonal patterns is the twelve-tone row. The composer creates some permutation of the 12 notes in a chromatic scale and then [...]The post Twelve-tone composition first appeared on John D. Cook.

Notice anything special about the following sequence? 8 6 10 3 1 11 1 3 6 8 12 9 7 10 4 2 5 11 2 4 7 9 5 12 Each of the numbers 1 through 12 appear twice. Between the two 1s there is one number. Between the two 2s there are two [...]The post Langford series first appeared on John D. Cook.

Lilypond is a TeX-like typesetting language for sheet music. I've had good results asking AI to generate Lilypond code, which is surprising given the obscurity of the language. There can't be that much publicly available Lilypond code to train on. I've mostly generated Lilypond code for posts related to music theory, such as the post [...]The post Typesetting sheet music with AI first appeared on John D. Cook.

In the previous two posts, we looked at why Mathematica and SymPy did not simplify sinh(arccosh(x)) to (x^2 - 1) as one might expect. After understanding why sinh(arccosh(x)) doesn't simplify nicely, it's natural to ask why sin(arccos(x)) does simplify nicely. In this post I sketched a proof of several identities including sin(arccos(x)) = (1 - [...]The post Inverse cosine first appeared on John D. Cook.