Feed john-d-cook John D. Cook

Favorite IconJohn D. Cook

Link https://www.johndcook.com/blog
Feed http://feeds.feedburner.com/TheEndeavour?format=xml
Updated 2025-03-07 01:01
Elliptic curve P-384
The various elliptic curves used in ellitpic curve cryptography (ECC) have different properties, and we’ve looked at several of them before. For example, Curve25519 is implemented very efficiently, and the parameters were transparently chosen. Curve1174 is interesting because it’s an Edwards curve and has a special addition formula. This post looks at curve P-384. What’s […]
Bessel function crossings
The previous looked at the angles that graphs make when they cross. For example, sin(x) and cos(x) always cross with the same angle. The same holds for sin(kx) and cos(kx) since the k simply rescales the x-axis. The post ended with wondering about functions analogous to sine and cosine, such as Bessel functions. This post […]
Orthogonal graphs
Colin Wright posted a tweet yesterday that said that the plots of cosine and tangent are orthogonal. Here’s a plot so you can see for yourself. Jim Simons replied with a proof so short it fits in a tweet: The product of the derivatives is -sin(x)sec²(x) = -tan(x)/cos(x), which is -1 if cos(x)=tan(x). This made […]
Fascination burnout
Here a little dialog from Anathem by Neal Stephenson that I can relate to: “… I don’t care …” Asribalt was horrified. “But how can you not be fascinated by—” “I am fascinated,” I insisted. “That’s the problem. I’m suffering from fascination burnout. Of all the things that are fascinating, I have to choose just […]
Area and volume of Menger sponge
The Menger sponge is the fractal you get by starting with a cube, dividing each face into a 3 by 3 grid (like a Rubik’s cube) and removing the middle square of each face and everything behind it. That’s M1, the Menger sponge at the 1st stage of its construction. The next stage repeats this […]
Regular expression for ICD-9 and ICD-10 codes
Suppose you’re searching for medical diagnosis codes in the middle of free text. One way to go about this would be to search for each of the roughly 14,000 ICD-9 codes and each of the roughly 70,000 ICD-10 codes. A simpler approach would be to use regular expressions, though that may not be as precise. […]
A misunderstanding of complexity
Iterating simple rules can lead to complex behavior. Many examples of this are photogenic, and so they’re good for popular articles. It’s fun to look at fractals and such. I’ve written several articles like that here, such as the post that included the image below. But there’s something in popular articles on complexity that bothers […]
Improving on the sieve of Eratosthenes
Ancient algorithm Eratosthenes had a good idea for finding all primes less than an upper bound N over 22 centuries ago. Make a list of the numbers 2 to N. Circle 2, then scratch out all the larger multiples of 2 up to N. Then move on to 3. Circle it, and scratch out all […]
How category theory is applied
Instead of asking whether an area of mathematics can be applied, it’s more useful to as how it can be applied. Differential equations are directly and commonly applied. Ask yourself what laws govern the motion of some system, write down these laws as differential equations, then solve them. Statistical models are similarly direct: propose a […]
Rare and strange ICD-10 codes
ICD-10 is a set of around 70,000 diagnosis codes. ICD stands for International Statistical Classification of Diseases and Related Health Problems. The verbosity of the name is foreshadowing. Some of the ICD-10 codes are awfully specific, and bizarre. For example, V95.4: Unspecified spacecraft accident injuring occupant V97.33XA: Sucked into jet engine, initial encounter V97.33XD: Sucked […]
State privacy laws to watch
A Massachusetts court ruled this week that obtaining real-time cell phone location data requires a warrant. Utah has passed a law that goes into effect next month that goes further. Police in Utah will need a warrant to obtain location data or to search someone’s electronic files. (Surely electronic files are the contemporary equivalent of […]
Quantum leaps
A literal quantum leap is a discrete change, typically extremely small [1]. A metaphorical quantum leap is a sudden, large change. I can’t think of a good metaphor for a small but discrete change. I was reaching for such a metaphor recently and my first thought was “quantum leap,” though that would imply something much […]
Professional, amateur, and something else
I opened a blog posts a while back by saying One of the differences between amateur and professional software development is whether you’re writing software for yourself or for someone else. It’s like the difference between keeping a journal and being a journalist. This morning I saw where someone pulled that quote and I thought […]
Easter and exponential sums
For the last couple years, the exponential sum of the day for Easter Sunday has been a cross. This was not planned, since the image each day is determined by the numbers that make up the date, as explained here. This was the exponential sum for last Easter last year, April 1, 2018: and this […]
Groups in categories
The first time I saw a reference to a “group in a category” I misread it as something in the category of groups. But that’s not what it means. Due to an unfortunately choice of terminology, “in” is more subtle than just membership in a class. This is related to another potentially misleading term, algebraic […]
What is an isogeny?
The previous post said that isogenies between elliptic curves are the basis for a quantum-resistant encryption method, but we didn’t say what an isogeny is. It’s difficult to look up what an isogeny is. You’ll find several definitions, and they seem incomplete or incompatible. If you go to Wikipedia, you’ll read “an isogeny is a […]
Isogeny-based encryption
If and when large quantum computers become practical, all currently widely deployed method for public key cryptography will break. Even the most optimistic proponents of quantum computing believe such computers are years away, maybe decades. But it also takes years, maybe decades, to develop, test, and deploy new encryption methods, and so researchers are working […]
Calling Python from Mathematica
The Mathematica function ExternalEvalute lets you call Python from Mathematica. However, there are a few wrinkles. I first pasted in an example from the Mathematica documentation and it failed. ExternalEvaluate[ "Python", {"def f(x): return x**2", "f(3)"} ] It turns out you (may) have to tell Mathematica where to find Python. I ran the following, tried […]
Random projection
Last night after dinner, the conversation turned to high-dimensional geometry. (I realize how odd that sentence sounds; I was with some unusual company.) Someone brought up the fact that two randomly chosen vectors in a high-dimensional space are very likely to be nearly orthogonal. This is a surprising but well known fact. Next the conversation […]
A truly horrible random number generator
I needed a bad random number generator for an illustration, and chose RANDU, possibly the worst random number generator that was ever widely deployed. Donald Knuth comments on RANDU in the second volume of his magnum opus. When this chapter was first written in the late 1960’s, a truly horrible random number generator called RANDU […]
Maybe you should’t script it after all
Programmers have an easier time scaling up than scaling down. You could call this foresight or over-engineering, depending on how things work out. Scaling is a matter of placing bets. Experienced programmers are rightfully suspicious of claims that something only needs to be done once, or that quick-and-dirty will be OK [*]. They’ve been burned […]
Squircle perimeter and the isoparametric problem
If you have a fixed length of rope and you want to enclose the most area inside the rope, make it into a circle. This is the solution to the so-called isoparametric problem. Dido’s problem is similar. If one side of your bounded area is given by a straight line, make your rope into a […]
Taking the derivative of a muscle car
I’ve been getting a lot of spam lately saying my web site does not rank well on “certain keywords.” This is of course true: no web site ranks well for every keyword. I was joking about this on Twitter, saying that my site does not rank well for women’s shoes, muscle cars, or snails because […]
Safe Harbor and the calendar rollover problem
Data privacy is subtle and difficult to regulate. The lawmakers who wrote the HIPAA privacy regulations took a stab at what would protect privacy when they crafted the “Safe Harbor” list. The list is neither necessary or sufficient, depending on context, but it’s a start. Extreme values of any measurement are more likely to lead […]
Data privacy Twitter account
My newest Twitter account is Data Privacy (@data_tip). There I post tweets about ways to protect your privacy, statistical disclosure limitation, etc. I had a clever idea for the icon, or so I thought. I started with the default Twitter icon, a sort of stylized anonymous person, and colored it with the same blue and […]
Ratio of Lebesgue norm ball volumes
As dimension increases, the ratio of volume between a unit ball and a unit cube goes to zero. Said another way, if you have a high-dimensional ball inside a high-dimensional box, nearly all the volume is in the corners. This is a surprising result when you first see it, but it’s well known among people […]
Higher dimensional squircles
The previous post looked at what exponent makes the area of a squircle midway between the area of a square and circle of the same radius. We could ask the analogous question in three dimensions, or in any dimension. (What do you call a shape between a cube and a sphere? A cuere? A sphube?) […]
History of the “Squircle”
Architect Peter Panholzer coined the term “squircle” in the summer of 1966 while working for Gerald Robinson. Robinson had seen a Scientific American article on the superellipse shape popularized by Piet Hein and suggested Panholzer use the shape in a project. Piet Hein used the term superellipse for a compromise between an ellipse and a […]
Covered entities: TMPRA extends HIPAA
The US HIPAA law only protects the privacy of health data held by “covered entities,” which essentially means health care providers and insurance companies. If you give your heart monitoring data or DNA to your doctor, it comes under HIPAA. If you give it to Fitbit or 23andMe, it does not. Government entities are not […]
Inferring religion from fitness data
Fitness monitors reveal more information than most people realize. For example, it may be possible to infer someone’s religious beliefs from their heart rate data. If you have location data, it’s trivial to tell whether someone is attending religious services. But you could make a reasonable guess from cardio monitoring data alone. Muslim prayers occur […]
Putting topological data analysis in context
I got a review copy of The Mathematics of Data recently. Five of the six chapters are relatively conventional, a mixture of topics in numerical linear algebra, optimization, and probability. The final chapter, written by Robert Ghrist, is entitled Homological Algebra and Data. Those who grew up with Sesame Street may recall the song “Which […]
Assumed technologies
I just had a client ship me a laptop. We never discussed what OS the computer would run. I haven’t opened the box yet, but I imagine it’s running Windows 10. I’ve had clients assume I run Windows, but also others who assume I run Linux or Mac. I don’t recall anyone asking me whether […]
Elementary solutions to differential equations
Differential equations rarely have closed-form solutions. Some do, and these are emphasized in textbooks. For this post we want to look specifically at homogeneous second order linear equations: y ” + a(x) y‘ + b(x) y = 0. If the coefficient functions a and b are constant, then the solution can be written down in terms […]
Finite rings
It occurred to me recently that I rarely hear about finite rings. I did a Google Ngram search to make sure this isn’t just my experience. Source Why are finite groups and finite fields common while finite rings are not? Finite groups have relatively weak algebraic structure, and demonstrate a lot of variety. Finite fields […]
Monads and generalized elements
Paolo Perrone gives a nice, succinct motivation for monads in the introduction to his article on probability and monads. … a monad is like a consistent way of extending spaces to include generalized elements of a specific kind. He develops this idea briefly, and links to his dissertation where he gives a longer exposition (pages […]
Mixing error-correcting codes and cryptography
Secret codes and error-correcting codes have nothing to do with each other. Except when they do! Error-correcting codes Error correcting code make digital communication possible. Without some way to detect and correct errors, the corruption of a single bit could wreak havoc. A simple example of an error-detection code is check sums. A more sophisticated […]
US Army applying new areas of math
Many times on this blog I’ve argued that the difference between pure and applied math is motivation. As my graduate advisor used to say, “Applied mathematics is not a subject classification. It’s an attitude.” Traditionally there was general agreement regarding what is pure math and what is applied. Number theory and topology, for example, are […]
Riffing on mistakes
I mentioned on Twitter yesterday that one way to relieve the boredom of grading math papers is to explore mistakes. If a statement is wrong, what would it take to make it right? Is it approximately correct? Is there some different context where it is correct? Several people said they’d like to see examples, so […]
A genius can admit finding things difficult
Karen Uhlenbeck has just received the Abel Prize. Many say that the Fields Medal is the analog of the Nobel Prize for mathematics, but others say that the Abel Prize is a better analog. The Abel prize is a recognition of achievement over a career whereas the Fields Medal is only awarded for work done […]
Thermocouple polynomials and other sundries
I was looking up something on the NIST (National Institute of Standards and Technology) web site the other day and ran across thermocouple polynomials. I wondered what that could be, assuming “thermocouple” was a metaphor for some algebraic property. No, it refers to physical thermocouples. The polynomials are functions for computing voltage as a function […]
Digital signatures with oil and vinegar
“Unbalanced oil and vinegar” is a colorful name for a cryptographic signature method. This post will give a high-level description of the method and explain where the name comes from. The RSA encryption algorithm depends on the fact that computers can easily multiply enormous numbers, but they cannot efficiently factor the product of two enormous […]
Counting irreducible polynomials over finite fields
You can construct a finite field of order pn for any prime p and positive integer n. The elements are polynomials modulo an irreducible polynomial of degree n, with coefficients in the integers mod p. The choice of irreducible polynomial matters, though the fields you get from any two choices will be isomorphic. For example, […]
Scaling up differential privacy: lessons from the US Census
The paper Issues Encountered Deploying Differential Privacy describes some of the difficulties the US Census Bureau has run into while deploying differential privacy for the 2020 census. It’s not surprising that they would have difficulties. It’s surprising that they would even consider applying differential privacy on such an enormous scale. If your data project is […]
Average distance between planets
What is the closest planet to Earth? The planet whose orbit is closest to the orbit of Earth is clearly Venus. But what planet is closest? That changes over time. If Venus is between the Earth and the sun, Venus is the closest planet to Earth. But if Mercury is between the Earth and the […]
All elliptic curves over fields of order 2 and 3
Introductions to elliptic curves often start by saying that elliptic curves have the form y² = x³ + ax + b. where 4a³ + 27b² ≠ 0. Then later they say “except over fields of characteristic 2 or 3.” What does characteristic 2 or 3 mean? The order of a finite field is the number of […]
US Census Bureau embraces differential privacy
The US Census Bureau is convinced that traditional methods of statistical disclosure limitation have not done enough to protect privacy. These methods may have been adequate in the past, but it no longer makes sense to implicitly assume that those who would like to violate privacy have limited resources or limited motivation. The Bureau has […]
Efficient modular arithmetic technique for Curve25519
Daniel Bernstein’s Curve25519 is the elliptic curve y² = x³ + 486662x² + x over the prime field with order p = 2255 – 19. The curve is a popular choice in elliptic curve cryptography because its design choices are transparently justified [1] and because cryptography over the curve can be implemented very efficiently. This […]
Why isn’t CPU time more valuable?
Here’s something I find puzzling: why isn’t CPU time more valuable? I first thought about this when I was working for MD Anderson Cancer Center, maybe around 2002. Our research in adaptive clinical trial methods required bursts of CPU time. We might need hundreds of hours of CPU time for a simulation, then nothing while […]
Chaos + Chaos = Order
If you take these chaotic-looking values for your x-coordinates and these chaotic-looking values for your y coordinates you get this image that looks more ordered. The image above is today’s exponential sum.
An attack on RSA with exponent 3
As I noted in this post, RSA encryption is often carried out reusing exponents. Sometimes the exponent is exponent 3, which is subject to an attack we’ll describe below [1]. (The most common exponent is 65537.) Suppose the same message m is sent to three recipients and all three use exponent e = 3. Each […]
...34353637383940414243...