John D. Cook

Differential privacy can be rigid and overly conservative in practice, and so finding ways to relax pure differential privacy while retaining its benefits is an active area of research. Two approaches to doing this are concentrated differential privacy [1] and Renyi differential privacy [3]. Concentrated differential privacy was used in reporting results from the 2020 [...]The post Zero-Concentrated Differential Privacy first appeared on John D. Cook.

Let's work our way up to differentially private stochastic gradient descent (DP-SGD) a little at a time. We'll first look at gradient descent, then stochastic gradient descent, then finally differentially private stochastic gradient descent. Gradient descent We'll start with gradient descent. Suppose you have a function of several variables f(x) where x is a vector. [...]The post Differentially private stochastic gradient descent first appeared on John D. Cook.

Probability density functions are independent of physical units. The normal distribution, for example, works just as well when describing weights or times. But sticking in units anyway is useful. Normal distribution example Suppose you're trying to remember the probability density function for the normal distribution. Is the correct form or or or maybe some other [...]The post Using dimensional analysis to check probability calculations first appeared on John D. Cook.

Differential privacy protects user privacy by adding randomness as necessary to the results of queries to a database containing private data. Local differential privacy protects user privacy by adding randomness before the data is inserted to the database. Using the visualization from this post, differential privacy takes the left and bottom (blue) path through the [...]The post Randomized response and local differential privacy first appeared on John D. Cook.

Machine learning models can memorize fragments of their training data and return these fragments verbatim. I've seen instances, for example, where I believe an LLM returned phrases verbatim from this site. It's easy to imagine how medical data might leak this way. How might you prevent this? And how might you do it in a [...]The post PATE framework for differentially private machine learning first appeared on John D. Cook.

There are many ways to describe the distance between two probability distributions. The previous two posts looked at using the p-norm to measure the difference between the PDFs and using Kullbach-Leibler divergence. Earth mover's distance (EMD) is yet another approach. Imagine a probability distribution on ^2 as a pile of dirt. Earth mover's distance measures [...]The post Earth mover's distance first appeared on John D. Cook.

The previous post looked at the best approximation to a normal density by normal density with a different mean. Dan Piponi suggested in the comments that it would be good to look at the Kullback-Leibler (KL) divergence. The previous post looked at the difference from between two densities from an analytic perspective, solving the problem [...]The post KL divergence from normal to normal first appeared on John D. Cook.

In my previous post on approximating a logistic distribution with a normal distribution I accidentally said something about approximating a normal with a normal. Obviously the best approximation to a probability distribution is itself. As Norbert Wiener said The best material model of a cat is another, or preferably the same, cat." But this made [...]The post Normal approximation to normal first appeared on John D. Cook.

In a recent post I pointed out that a soliton, a solution to the KdV equation, looks a lot like a normal density for fixed x. As someone pointed out in the comments, one way to look at this is that the soliton is exactly proportional to the density of a logistic distribution, and it's [...]The post Logistic / Normal approximation first appeared on John D. Cook.

Tens of millions of fax machines still exist. My business line gets calls from modems and fax machines fairly often. Maybe my number is close to that of a fax machine. Fax machines and health care Fax machines are especially common in health care. I remember when I was working at MD Anderson Cancer Center [...]The post Fax machines in the 21st century first appeared on John D. Cook.

I got an email from someone saying the RSS feed for this site stopped working. Anyone else having this problem? I subscribe to my RSS feed and it's working fine for me. It may be that there are variations on the RSS feed, and the version I'm using works while the variation some others use [...]The post Blog RSS feed first appeared on John D. Cook.

Rarely does a nonlinear differential equation, especially a nonlinear partial differential equation, have a closed-form solution. But that is the case for the Korteweg-De Vries equation. (Technically I should say it's rare for a naturally-occurring nonlinear differential equation to have a closed-form solution. You can always start with a solution and cook up a contrived [...]The post Solitons and the KdV equation first appeared on John D. Cook.

The other day I saw an image of a large disk centered on Paris subjected to the Mercator projection. I was playing around in Mathematica and made similar images for different projections. Each image below is a disk of radius 4200 km centered on Paris (latitude 49, longitude 2). All images were produced with the [...]The post A disk around Paris first appeared on John D. Cook.

The HIPAA Safe Harbor provision says that data can be considered deidentified if 18 kinds of data are removed or reported at low resolution. At the end of the list of 18 items, there is an extra category, sometimes informally called the 19th rule: The covered entity does not have actual knowledge that the information [...]The post The 19th rule of HIPAA Safe Harbor first appeared on John D. Cook.

I saw a comment from Christos Argyropoulos on Twitter implying that there's a good scientific community on Bluesky, so I went there and looked around a little bit. I have account, but I haven't done much with it. I was surprised that a fair number of people had followed me on Bluesky even though I [...]The post Bluesky first appeared on John D. Cook.

The -i flag to ask sed to edit a file in place works differently on Linux and MacOS. If you want to create a backup of your file before you edit it, say with the extension .bak, then on Linux you would run sed -i.bak myfile but for the version of sed that ships with [...]The post Portable sed -i across MacOS and Linux first appeared on John D. Cook.

From Love What Lasts, Joshua Gibbs: ... there are too many things in the world to care equally about them all. The sheer volume of things ... demands that we have hierarchical standards by which to judge their value, or else we are condemned to give our lives over entirely to what is nearest, easiest, [...]The post Nearest, easiest, and most accessible first appeared on John D. Cook.

Draw three circles of radius r that intersect at a single point. Then draw a triangle connecting the remaining three points of intersection. (Each pair of circles intersects in two points, one of which is the point where all three circles intersect, so there are three other intersection points.) Then the circumcircle of the triangle, [...]The post Johnson circle theorem first appeared on John D. Cook.

Let Q be a convex quadrilateral with at most two parallel sides. Draw the two diagonals then draw a line through their midpoints. This line is called the Newton line. (The requirement that at most two sides are parallel insures that the midpoints are distinct and so there is a unique line joining them.) In [...]The post Newton line first appeared on John D. Cook.

This post is a follow-on to a discussion that started on Twitter yesterday. This tweet must have resonated with a lot of people because it's had over 250,000 views so far. You almost have to study advanced math to solve basic math problems. Sometimes a high school student can solve a real world problem that [...]The post Homework problems are rigged first appeared on John D. Cook.

The last couple article have looked at various kinds of mean. The Python code for four of these means is trivial: gm = lambda a, b: (a*b)**0.5 am = lambda a, b: (a + b)/2 hm = lambda a, b: 2*a*b/(a+b) chm = lambda a, b: (a**2 + b**2)/(a + b) But the arithmetic-geometric mean [...]The post Python code for means first appeared on John D. Cook.

in an earlier post I said that the arithmetic mean of two frequencies an octave apart is an interval of a perfect fifth, and the geometric mean gives a tritone. This post will look at a few other means. Intervals The harmonic mean (HM) gives a perfect fourth. The arithmetic-geometric mean (AGM) gives a pitch [...]The post More ways of splitting the octave first appeared on John D. Cook.

This afternoon I wrote a brief post about Terence Tao's new paper A Maclaurin type inequality. That paper builds on two classical inequalities: Newton's inequality and Maclaurin's inequality. The previous post expanded a bit on Newton's inequality. This post will do the same for Maclaurin's inequality. As before, let x be a list of real [...]The post Maclaurin's inequality first appeared on John D. Cook.

The previous post mentioned Newton's inequality. This post will explore this inequality. Let x be a list of real numbers and define Sn(x) to be the average over all products of n elements from x. Newton's inequality says that Sn-1 Sn+1 S^2n In more terminology more recent than Newton, we say that the sequence [...]The post Newton's inequality and log concave sequences first appeared on John D. Cook.

Terence Tao has a new paper out that relates to a couple things I've written about recently. Elementary symmetric polynomials came up when developing the general equations for tangent sum andhyperbolic tangent sum. The latter post goes into more detail. Before that, means of symmetric functions, not necessarily elementary polynomials or even polynomials, came up [...]The post U statistics and a new paper by Terence Tao first appeared on John D. Cook.

The latest episode of Erik Seligman's podcast is entitled The Grim State of Modern Pizza. Although you might not realize it from the title, the post is about fraud detection. GRIM stands for Granularity-Related Inconsistency of Means. In a nutshell, the test looks for means (averages) that are not possible on number theoretic grounds. If [...]The post Detecting fraud with the GRIM test first appeared on John D. Cook.

A few weeks ago I wrote about how the dissonance of a musical interval is related to the complexity of the frequency ratio as a fraction, where complexity is measured by the sum of the numerator and denominator. Consonant intervals have simple frequency ratios and dissonant intervals have complex frequency ratios. By this measure, the [...]The post Tritone first appeared on John D. Cook.

The relation between a function and its power series is subtle. In a calculus class you'll see equations of the form series = function" which may need some footnotes. Maybe the series only represents the function over part of its domain: the function extends further than the power series representation. Starting with the power series, [...]The post When a function cannot be extended first appeared on John D. Cook.

Yesterday I wrote a post that looked at the hyperbolic tangent sum for x and y strictly between -1 and 1. This sum arises when adding velocities in special relativity. The post ended with a description of the expression for in terms of elementary symmetric polynomials but did not offer a proof. This post will [...]The post Tanh and elementary symmetric polynomials first appeared on John D. Cook.

Earlier this week I wrote about several ways to generalize trig functions. Since trig functions have addition theorems like a natural question is whether generalized trig functions also have addition theorems. Hyperbolic functions have well-known addition theorems analogous to the addition theorems above. This isn't too surprising since circular and hyperbolic functions are fundamentally two [...]The post Addition theorems first appeared on John D. Cook.

In the previous post I said I was trying remember where I'd seen the tangent sum applied. I mentioned a couple near misses, and it turns out that what I was trying to remember was another near miss. What I'd seen before was not the tangent sum but the hyperbolic tangent sum. Several people suggested [...]The post Hyperbolic tangent sum first appeared on John D. Cook.

When I was writing my post on lemniscate functions yesterday, a line from the Wikipedia article seemed familiar for reasons I cannot place. Defining a tangent-sum operator as a b := tan(arctan a + arctan b) gives cl^2z sl^2 z = 1. I feel like I've seen this tangent-sum used before, but [...]The post Tangent sum first appeared on John D. Cook.

We begin with a couple examples. First, the set of linear transformations from one vector space to another is itself a vector space. Second, the set of continuous linear operators from one Banach space to another is itself a Banach space. Or maybe better, this set can be made into a Banach space. In the [...]The post Enriched categories first appeared on John D. Cook.

This is the fourth post in a series on generalizations of sine and cosine. The first post looked at defining sine as the inverse of the inverse sine. The reason for this unusual approach is that the inverse sine is given in terms of an arc length and an integral. We can generalize sine by [...]The post p-norm trig functions and squigonometry" first appeared on John D. Cook.

This is the third post in a series on generalizing sine and cosine. The previous post looked at a generalization of the sine and cosine functions that come from replacing a circle with a lemniscate, a curve that looks like a figure eight. This post looks at replacing the circle with a hyperbola. On the [...]The post Geometric derivation of hyperbolic trig functions first appeared on John D. Cook.

In the previous post I said that you could define the inverse sine as the function that gives the arc length along a circle, then define sine to be the inverse of the inverse sine. The purpose of such a backward definition is that it generalizes to other curves besides the circle. For example, it [...]The post Lemniscate functions first appeared on John D. Cook.

In a recent post I mentioned in passing that trigonometry can be generalized from functions associated with a circle to functions associated with other curves. This post will go into that a little further. The equation of the unit circle is and so in the first quadrant The length of an arc from (1, 0) [...]The post Generalized trigonometry first appeared on John D. Cook.

Let G be a directed graph whose nodes are the positive integers and whose edges represent relations between two integers. In our first example we'll draw an edge from x to y if x is a multiple of y. In our second example we'll draw an edge from x to y if x >= y. [...]The post From graph theory to category theory first appeared on John D. Cook.

Test functions are how you can make sense of functions that aren't really functions. The canonical example is the Dirac delta function" that is infinite at the origin, zero everywhere else, and integrates to 1. That description is contradictory: a function that is 0 almost everywhere integrates to 0, even if you work in extended [...]The post Test functions first appeared on John D. Cook.

This article will probably only be of interest to a small number of readers. Those unfamiliar with category theory may find it bewildering, and those well versed in category theory may find it trivial. My hope is that someone in between, someone just starting to get a handle on category theory, will find it helpful. [...]The post Groups vs Abelian groups: Pedantic or profound? first appeared on John D. Cook.

The Depths of Wikipedia twitter account posted a screenshot about supereggs that's popular at the moment. It says there's no way this is real. they must be making these words up above a screenshot from the Wikipedia article on supereggs saying The definition can be changed to have an equality rather than an inequality; this [...]The post Supereggs, squigonometry, and squircles first appeared on John D. Cook.

Meredith Whittaker posted on Twitter that In addition to being the best in privacy, Signal is also the best in not subjecting you to corny AI' features no one asked for or wants. I love the phrase corny AI." That's exactly what a lot of AI features are. Would you like help composing that tweet?" [...]The post Corny AI first appeared on John D. Cook.

The star-like image above is today's exponential sum. The exponential sum page on my site generates a new image each day by putting the numbers of the day's month, day, and year into the equation and connecting the partial sums in the complex plane. Here m is the month, d is the day, and y [...]The post Today's star first appeared on John D. Cook.

Coupon collector problem Suppose you have a bag of balls labeled 1 through 1,000. You draw draw balls one at a time and put them back after each draw. How many draws would you have to make before you've seen every ball at least once? This is the coupon collector problem with N = 1000, [...]The post Consecutive coupon collector problem first appeared on John D. Cook.

Monte Carlo integration is not as simple in practice as it is often introduced. A homework problem might as you to integrate a function of two variables by selecting random points from a cube and counting how many of the points fall below the graph of the function. This would indeed give you an estimate [...]The post Regular solids and Monte Carlo integration first appeared on John D. Cook.

The previous post describes the hoops I jumped through to enter Unicode characters on a Mac. Here's a script to run from the command line that will copy Unicode characters to the system clipboard. It runs anywhere the Python module pyperclip runs. #!/usr/bin/env python3 import sys import pyperclip cp = sys.argv[1] ch = eval(f"chr(0x{cp})") print(ch) [...]The post Cross-platform way to enter Unicode characters first appeared on John D. Cook.

Setting up Unicode on my MacBook took some research, so I'm leaving myself a note here if I need to do it again. Maybe it'll help someone else too. Update: I've gotten some feedback on this article that suggest people imagine that I want to use this approach to enter large quantities of text, such [...]The post Using Unicode on MacOS first appeared on John D. Cook.

About three years ago I ran across a strange coordinate system in which familiar functions lead to interesting plots. The system is called circular coordinates" but it is not polar coordinates. This morning I was playing around with this again. Here's a plot of f(x) = x. And here's a plot of f(x) = cos(8x). [...]The post Circular coordinate art first appeared on John D. Cook.

Today's date, US style, is 9/26/2023, and there is only one group, up to isomorphism, of size 9262023. You could verify this in Mathematica with the command FiniteGroupCount[9262023] which returns 1. For a given n, when is there only one group of size n? There are two requirements. First, n has to be the product [...]The post When there is only one group of a given size first appeared on John D. Cook.

Simple groups are the building blocks of groups similar to the way prime numbers are the building blocks of integers. This post will unpack this analogy in two ways: How do simple groups compare to prime numbers? How does the composition of simple groups compare to the composition of prime numbers? The former analogy is [...]The post Analogy between prime numbers and simple groups first appeared on John D. Cook.