Using PractRand to test an RNG

Yesterday I wrote about my experience using NIST STS to test an entropy extractor, a filtering procedure that produces unbiased bits from biased sources. This post will look at testing the same entropy extractor using the Practically Random (PractRand) test suite. The results were much worse this time, which speaks to the limitations of both the entropy extractor and the STS test suite.

Difficulties with PractRand

I built PractRand on Linux with no difficulty. There were multiple warnings but no errors. But when I piped a file of bits into RNG_test I got a message saying "error reading standard input".

A couple years ago I ran PractRand using my Mac with no problems (results here) so I switched over to the machine I'd used before. The latest version of PractRand (pre95) produces compile errors, but the version I'd used before (93) compiled successfully.

Piping bits to RNG_test from the command line worked when I used the program RNG_output included with PractRand, but piping my own file of bits to the test suite failed with the same error I saw on Linux. I applied a patch posted by Daniel Lemire and was able to pipe my file to the test program. I went back to Linux to see if I could get version 93 to work there, but no luck.

Testing bits from a file is unusual; it's faster to generate random numbers and send them directly to the test without going to disk and back, and so naturally that's what people will do more often. But I wanted to use bits from a file because I need to do that on another project.

Test results

The entropy extractor failed 47 out of 151 tests using 128 MB of data. Five other tests reported anomalies ranging from "mildly suspicious" to "very suspicious." The tests that failed had p-values of 10^-16 and smaller. Most of the p-values were less than can be represented by a 64-bit floating point number. PractRand must use extended precision to report just how infinitesimally small some of the p-values are.

To make sure the test failures weren't due to some anomaly with piping the file into RNG_test I repeated my procedure using output from the Melissa O'Neill's PCG generator. All tests passed with no anomalies.

First do no harm

What if you run the entropy extractor on a high-quality source of random bits? When I reran the experiment on three streams from PCG, the result passed all PractRand tests without any anomalies. This suggests that applying the I1/4RNG entropy extractor improves random streams, though perhaps not as much as we'd like, and at a minimum it makes streams no worse.

More on random number generation

• Manipulating an RNG
• Adiantum and Cha Cha
• Random number generator testing