NIST SP800-22 Test Protocol
How Eormen applies the 15 statistical tests documented in NIST SP800-22 Rev. 1a to 1 GiB entropy-block inputs, with the recorded implementation details and result limitations.
Overview
This document explains the recorded results from the NIST SP800-22 Rev. 1a statistical test suite as applied to 1 GiB entropy-block inputs generated by Eormen systems. The results show whether the implemented tests rejected their null hypotheses at the configured significance level. They do not, by themselves, prove randomness or establish that a generator is suitable for a particular cryptographic application.
What is NIST SP800-22 Testing?
NIST Special Publication 800-22 Revision 1a documents 15 statistical tests for outputs from random and pseudorandom number generators used in cryptographic applications. Each test examines a defined statistical property and reports a p-value against its null hypothesis.
Passing these tests records that the tested data did not produce statistical evidence sufficient to reject those hypotheses at the selected threshold. NIST describes statistical testing as a first step: it does not prove that the output is random and does not replace analysis of the generator design or cryptanalysis.
Testing 1 GiB Blocks: The Challenge and Solution
Processing a 1 GiB source payload (over 8.5 billion bits) required memory and computational engineering beyond a simple in-memory implementation. The signed report records the implementation parameters and results; independent conformance review would be required to establish equivalence with another implementation.
Key Enhancements Implemented
- Memory-efficient processing: All tests were enhanced to process data in streaming chunks (typically 1 MiB at a time), enabling analysis of the full 1 GiB blocks without requiring excessive RAM.
- Computational optimisation: Advanced vectorised operations and optimised algorithms reduced processing time from potentially days to manageable timeframes whilst maintaining mathematical rigour.
- Recorded input coverage: The signed report records a 1 GiB test input. Each test then applies its documented sequence length, sampling and parameter requirements; the individual result record identifies what was analysed.
- Recorded methodology: The implementation is intended to follow the formulas, thresholds and statistical methods documented by NIST. The signed result artefact records the specification, threshold and per-test outputs so that the implementation and results can be reviewed.
File Structure and Metadata
EORM entropy blocks contain:
- 1,073,741,824 bytes (1 GiB) of entropy data
- 64 bytes of metadata appended after the data
| Metadata field | Byte offset | Size | Description |
|---|---|---|---|
| Nonce | 0–15 | 16 bytes | Unique identifier for this generation session |
| Timestamp | 16–23 | 8 bytes (little-endian) | Generation time |
| Filename | 24–55 | 32 bytes (UTF-8) | Original filename |
| File size | 56–63 | 8 bytes (little-endian) | Total file size |
Understanding the Test Results
The JSON results file contains four main sections:
1. Test Identification
- nonce: 16-byte unique identifier extracted from the EORM block metadata.
- data_only_sha256: SHA-256 hash of the 1 GiB entropy data (excludes metadata).
- complete_file_sha256: SHA-256 hash of the entire file (data + metadata).
- metadata_only_sha256: SHA-256 hash of the 64-byte metadata section.
- generation_timestamp_utc: When the entropy block was created.
- data_size_bytes: Confirms exactly 1,073,741,824 bytes (1 GiB) were tested.
2. Test Configuration
- specification: Confirms NIST SP800-22 Rev. 1a compliance.
- significance_level: 0.01 (the hypothesis-test threshold, denoted α).
- pass_criteria: Tests pass when p_value ≥ 0.01.
3. Individual Test Results
Each of the 15 tests provides the result (PASSED or FAILED), a p-value evaluated against α = 0.01, a plain-English assessment of what the test found, and the specific metrics measured. A larger p-value is not a measure of “more randomness”.
4. Test Suite Summary
The overall result shows PASSED only if all 15 implemented tests meet their recorded acceptance criteria. It summarises those test outcomes and does not certify generator security.
The 15 NIST Tests
All 15 tests use inputs derived from the certified 1 GiB payload according to their recorded sequence and parameter requirements. Select a test to see what it measures and what a passing result means.
- What it measures
- Whether the data contains approximately equal numbers of 0s and 1s.
- How Eormen tested 1 GiB
- Processed the entire file in chunks, maintaining running counts without loading all 8.5 billion bits into memory simultaneously.
- What PASSED means
- The entropy has the expected 50/50 bit balance of random data.
- What it measures
- Whether the proportion of 1s remains consistent across different sections of the data.
- How Eormen tested 1 GiB
- Divided the data into optimal-sized blocks (10,000 bits each for 1 GiB files) and verified frequency consistency across all blocks.
- What PASSED means
- No sections of the data show unusual concentrations of 0s or 1s.
- What it measures
- Whether sequences of consecutive identical bits (runs) occur as frequently as expected in random data.
- How Eormen tested 1 GiB
- Tracked runs across chunk boundaries to ensure accurate counting throughout the entire file.
- What PASSED means
- The data transitions between 0s and 1s at the expected rate for random sequences.
- What it measures
- Whether the longest sequences of consecutive 1s match expectations for random data.
- How Eormen tested 1 GiB
- Analysed blocks of 10,000 bits each, categorising longest runs without storing block contents.
- What PASSED means
- No unexpectedly long sequences of 1s that might indicate patterns.
- What it measures
- Whether binary matrices formed from the data have the expected mathematical properties of random matrices.
- How Eormen tested 1 GiB
- Constructed and analysed thousands of 32×32 bit matrices using streaming techniques.
- What PASSED means
- The data’s mathematical structure matches that of random sequences.
- What it measures
- Whether the data contains periodic patterns when analysed in the frequency domain.
- How Eormen tested 1 GiB
- Applied spectral analysis to the first million bits (sufficient for detecting periodic features per NIST guidelines).
- What PASSED means
- No hidden repeating patterns detected in the entropy.
- What it measures
- Whether specific bit patterns occur with the expected frequency.
- How Eormen tested 1 GiB
- Searched for a randomly selected 9-bit pattern across million-bit blocks using efficient pattern matching.
- What PASSED means
- Pattern occurrences match random distribution expectations.
- What it measures
- How often a specific pattern (nine consecutive 1s) appears when searching with overlaps.
- How Eormen tested 1 GiB
- Scanned 968 blocks of 1,032 bits each for the test pattern using memory-efficient algorithms.
- What PASSED means
- The test pattern appears with appropriate frequency for random data.
- What it measures
- Whether pattern recurrence indicates statistically detectable compressibility under this test.
- How Eormen tested 1 GiB
- Analysed pattern recurrence distances using sophisticated streaming algorithms that handle millions of patterns efficiently.
- What PASSED means
- The test did not detect significant compressibility at the configured threshold.
- What it measures
- Whether bit sequences have appropriate algorithmic complexity.
- How Eormen tested 1 GiB
- Applied the Berlekamp-Massey algorithm to 2,000 blocks of 500 bits each, computing linear complexity distributions.
- What PASSED means
- The data exhibits the complex structure expected from random sequences.
- What it measures
- Whether all possible bit patterns of given lengths occur with equal frequency.
- How Eormen tested 1 GiB
- Tracked frequencies of all 16-bit patterns (65,536 possibilities) across 100 million bits using sparse storage techniques.
- What PASSED means
- No bit patterns appear significantly more or less than expected.
- What it measures
- The rate of entropy generation by comparing pattern frequencies at different lengths.
- How Eormen tested 1 GiB
- Compared 10-bit and 11-bit pattern frequencies across 10 million bits using unified tracking systems.
- What PASSED means
- The measured pattern-frequency relationship was consistent with this test’s null hypothesis.
- What it measures
- Whether the running sum of the data (treating 0 as −1 and 1 as +1) stays within expected bounds.
- How Eormen tested 1 GiB
- Performed both forward and backward cumulative sum calculations, with the backward test using the last 100 million bits.
- What PASSED means
- No systematic bias causing drift in cumulative values.
- What it measures
- How often the cumulative sum visits specific values during “excursions” from zero.
- How Eormen tested 1 GiB
- Identified and analysed all complete cycles in the cumulative sum using streaming detection.
- What PASSED means
- State visits within cycles match random walk theory.
- What it measures
- The total number of times specific cumulative sum values are reached.
- How Eormen tested 1 GiB
- Tracked visits to 18 different states throughout the entire file using memory-efficient counting.
- What PASSED means
- Overall state visit frequencies align with theoretical predictions.
Interpreting the Results
Understanding P-Values
- P-value ≥ 0.01: The test does not reject its null hypothesis at α = 0.01.
- P-value < 0.01: The test rejects its null hypothesis at α = 0.01 and the result requires investigation.
- A p-value is evaluated against the chosen threshold; a higher value does not mean that the data are “more random”.
Overall Assessment
- All Tests PASSED: None of the 15 implemented tests rejected its null hypothesis at the configured threshold. This does not prove randomness or establish cryptographic suitability.
- Any Test FAILED: Indicates potential weaknesses that should be investigated.
Common Questions
Why test the full 1 GiB when smaller samples might suffice?
A larger source payload permits broader test coverage and more observations. It cannot prove that no pattern exists at every scale, and individual tests still use their specified sequence lengths and parameters.
Can these results be compared to standard NIST test results?
The report records the NIST revision, threshold, input identity and per-test outputs needed for comparison. A defensible equivalence claim also requires review of the implementation, parameters, sequence construction and acceptance logic used by each system.
File Authenticity
The results file includes cryptographic verification using a three-tier hashing system:
- Data-only SHA-256: Verifies the entropy data independently. Remains constant for reproductions.
- Complete file SHA-256: Verifies the entire file has not been modified.
- Metadata-only SHA-256: Verifies the metadata section.
- Nonce: Links results to the specific EORM generation session.
- Timestamp: Unix timestamp confirming when the entropy was generated.
This three-tier approach ensures both data integrity and proper attribution to the original generation session. The data-only hash remains constant for reproductions, whilst the complete file hash verifies the entire file has not been modified.