TestU01 small crush test suite
In recent posts I've written about using RNG test suites on the output of the I1/4RNG entropy extractor. This is probably the last post in the series. I've looked at NIST STS, PractRand, and DIEHARDER before. In this post I'll be looking at TestU01.
TestU01 includes three batteries of tests: Small Crush, Crush, and Big Crush. The entropy extractor failed the smallest of the three, so I didn't go on to the larger suites. Small Crush isn't small; it used over 22 billion 32-bit samples as input, about 0.84 GB of data. Crush uses two orders of magnitude more data, and Big Crush uses another order of magnitude more data than Crush.
SmallCrush consists of 10 tests:
- smarsa_BirthdaySpacings
- sknuth_Collision
- sknuth_Gap
- sknuth_SimpPoker
- sknuth_CouponCollector
- sknuth_MaxOft
- svaria_WeightDistrib
- smarsa_MatrixRank
- sstring_HammingIndep
- swalk_RandomWalk1
The test names begin with s, followed by a prefix indicating the origin of the test. For example, knuth refers to Donald Knuth's tests in volume 2 of TAOCP and marsa refers to George Marsaglia. The remainder of the name is more descriptive, such as SimpPoker for Knuth's simple poker test.
The output of the entropy extractor failed four of the tests, failure being defined as producing a p-value less than 10-300. The other tests passed without issue, meaning they returned p-values in the range [0.001, 0.999].
Recall from earlier posts that I1/4RNG entropy extractor takes three possibly biased bit streams and produces an unbiased bit stream, provided each of the input streams has min-entropy of at least 1/3. I produced biased streams by taking the bitwise OR of two consecutive values, producing a stream with probability 0.75 of being a 1 and probability 0.25 of being a 0. The result passed all STS and DIEHARDER tests, but failed some PractRand and Test01 SmallCrush tests. This is consistent with the generally held opinion that STS and DIEHARDER are relatively weak tests and PractRand and TestU01 are more rigorous tests.
I applied the entropy extractor to PCG without creating a biased stream, and the result passed PractRand and TestIU01 SmallCrush. Presumably it would have passed STS and DIEHARDER as well. This confirms that the extractor does no harm to a high-quality stream of pseudorandom bits. It largely removes the bias from biased streams, enough to pass the easier two test suites but not enough to pass the two more demanding test suites.
Previous posts in this series