Reference suite¶
The tests take their ground truth from an independent generator in reference/. It computes
every value with Arb ball arithmetic through
python-flint, and never imports scatrs. The values are
committed as JSON Lines files in reference/data/. The Rust tests read those files, so
running the tests does not need Arb.
Record format¶
Each line of a data file is one record, a JSON object with these keys:
Key |
Contents |
|---|---|
|
the generator that wrote it (also the file name) |
|
a name, unique within the file |
|
|
|
the arguments, as exact doubles, integers or strings |
|
named sets of values, each as the parallel arrays below |
The manifest calls this layout format version 2. Every stored value \(v\) is represented in three ways:
hi,lo,lo2: three doubles. Their exact sum approximates \(v\) to about 160 bits.hiis the double nearest to the midpoint of the Arb ball, andhi == fl(hi + lo), so(hi, lo)is a normalised double-double.lo2is the double nearest to the rest.dec: a decimal string. \(v\) as Arb prints it, with up to 25 significant digits, or fewer when the ball is too wide to justify 25. It is for people to read; the tests use the doubles.err: an error bound. A rigorous upper bound on \(\lvert v - (\text{hi} + \text{lo} + \text{lo2})\rvert\), which includes the radius of the Arb ball.
The third word lets the tests measure the error of a double-double result almost exactly.
RefValues::dd_error in scatrs/tests/common/mod.rs returns
\(\lvert \text{got} - (\text{hi} + \text{lo} + \text{lo2})\rvert + \text{err}\). It forms the
difference with error-free transformations and compensated summation, so the result is within
about \(2^{-150}\lvert v\rvert\) of a rigorous bound on \(\lvert \text{got} - v\rvert\). That is
negligible against double-double errors of order \(2^{-106}\).
Labels¶
A record is certified when every source of error is rigorously bounded, including any truncation of a series. It is high-precision when some error, such as a series tail, is not yet rigorously bounded. Labels are never overstated.
The 358 records of the arithmetic, the constants and the special functions are all certified:
the arithmetic, constants and decimal values are exact rationals or Arb balls;
the elementary and complex functions are Arb’s rigorous implementations;
the Gauss–Legendre roots and weights come from Arb’s certified Legendre root finder;
the Wigner values come from the finite explicit sum, or, at very large degrees, from Arb’s Legendre functions.
The Mie records are certified where a proven bound covers the series tail, and high-precision otherwise: 297 of the 306 series records of values (the other 9 would need more than 60 000 terms), and none of the 80 series records of derivatives, which have no tail bound yet. Their coefficient records are all certified. Homogeneous spheres (Lorenz–Mie) derives the bound.
Files¶
reference/data/manifest.json lists every data file. Each entry gives:
the python-flint and FLINT versions that produced it (0.9.0 and 3.6.0 for every file now);
the generator’s working precision in bits;
the number of records;
the SHA-256 of the file.
File |
Records |
Contents |
|---|---|---|
|
62 |
the |
|
9 |
true results of double-double |
|
4 |
|
|
38 |
19 functions ( |
|
2 |
exact values of 300 random decimal literals and of hand-picked edge cases |
|
14 |
|
|
17 |
see below |
|
740 |
see below |
|
8 |
see below |
|
612 |
see below |
|
160 |
see below |
|
36 |
see below |
|
20 |
see below |
|
212 |
see below |
The edge cases sit at the edges of each function’s domain and of its algorithm’s branches:
overflow and underflow thresholds, and subnormal inputs;
arguments next to 1;
the switch points between series, reductions and fast and wide-range paths;
for
sinandcos, the double closest to a multiple of \(\pi/2\) in every binade below \(2^{50}\) (fortan, below \(2^{21}\)).
gauss_legendre.jsonl holds 17 rules, nodes and weights:
full rules for \(n\) = 1, 2, 3, 4, 5, 8, 16, 33, 64, 100, 128, 257, 500 and 1000;
partial rules for \(n\) = 5000 and 20 000, holding the 8 nodes nearest each of \(\pm1\) and the 4 nearest 0;
the same 20 nodes of the largest rule scatrs accepts, \(n = 2^{17}\).
wigner_d.jsonl holds 212 records:
Records |
Contents |
|---|---|
170 |
every degree \(l \le 100\) of 17 pairs \((m, n)\) at 10 values of \(x\), including \(\pm1\) and doubles a few ulps from them |
1 |
\((0, 2)\) at \(x = 0.3\), every degree up to 300 |
25 |
sampled degrees up to \(l = 2000\), for \((0, 0)\), \((2, \pm2)\), \((0, 2)\), \((50, -37)\), \((-37, -50)\) and \((7, 150)\) |
2 |
sampled degrees up to \(l = 4000\), next to the poles |
7 |
five degrees from \(10^5\) to \(2.5\cdot10^5\), for \((0, 0)\) and \((0, 2)\), from Arb’s Legendre functions |
7 |
start values below the normal double range, with later degrees that are normal doubles |
mie.jsonl holds two records per sphere, for 306 spheres: the 19 size parameters
\(10^{-6}, 10^{-4}, 10^{-2}, 0.1, 0.5, 1, 2, 5, 10, 20, 50, 100, 200, 500, 10^3,
2\cdot10^3, 5\cdot10^3, 10^4, 2\cdot10^4\) against 14 refractive indices (near 1, water and
aerosol, high index, strong absorption and metal-like), and 40 random spheres in the validated
domain. A ,coefficients record holds \(a_n\) and \(b_n\) at up to 11 orders; a ,series
record holds the efficiencies, \(g\), and \(S_1\), \(S_2\) at 9 angles from 0° to 180°.
mie_derivatives.jsonl holds the same for 80 spheres, with the derivatives of each
output instead of the values.
mie_greek.jsonl holds two records per case, for 18 cases from \(x = 0.1\) to 200 (17 spheres, one at two orders), of
the \(N\)-term series with the default f64 truncation \(N\). A ,phase record holds the
normalized phase matrix \(F_{11}\), \(F_{12}\), \(F_{33}\), \(F_{34}\) at 7 angles from 0° to 180°; a
,greek record holds \(\alpha_1, \ldots, \alpha_4, \beta_1, \beta_2\) up to all orders \(2N\)
(13 spheres) or a lower order \(L\) (5 spheres), normalized so that \(\alpha_1^0 = 1\). The
coefficients come from Gauss–Legendre quadrature on Arb’s own Legendre roots, exact for these
polynomial integrands, and the Wigner d-functions from their three-term recurrence in ball
arithmetic, so every value is certified.
spheroid.jsonl holds two records per spheroid, for 10 spheroids with aspect ratios from
1/5 to 5 (a sphere among them), equal-volume size parameters from 0.5 to 6, and indices from
water to 3 + 0.1i, computed without scatrs’s algorithm: the generator evaluates the EBCM from
its definition, the surface integrals of the cross products of the vector spherical wave
functions with no simplification of the integrands, on Arb’s own Gauss–Legendre rule over the
whole range of \(\cos\theta\), and solves for \(T^m\) in ball arithmetic. A ,N=…,n_theta=… record
holds \(Q_{\mathrm{ext}}\) and \(Q_{\mathrm{sca}}\) at a given truncation and a sample of \(T^m\) for
three modes (the diagonals of \(T^{11}\) and \(T^{22}\) and the first column of each block),
certified for that truncation; a ,converged record holds \(Q_{\mathrm{ext}}\),
\(Q_{\mathrm{sca}}\) and \(Q_{\mathrm{abs}}\) with \(N\) and the quadrature grown until they settle
to \(10^{-17}\), high-precision.
layered.jsonl holds two records per layered sphere, for 370 particles, computed without
scatrs’s algorithm: the generator evaluates \(\psi_n\) and \(\xi_n\) of every layer’s arguments
and matches the fields at each interface directly, all in ball arithmetic. The particles are
two-layer spheres of 9 pairs of indices (black carbon in sulfate, water on ice, a silica core
in a gold-like shell, absorbing cores in a clear shell, high contrast, near unity and an air
shell) with core radius fractions \(10^{-3}\), 0.3, 0.9 and 0.999, at outer size parameters from
\(10^{-6}\) to \(10^4\); three- and ten-layer particles; and 40 random particles of 2 to 8
layers. A ,coefficients record holds \(a_n\) and \(b_n\) at up to 11 orders, certified; a
,series record the efficiencies, \(g\), and \(S_1\), \(S_2\) at 9 angles, summed to
\(\lceil x + 18x^{1/3} + 32 \rceil\) terms without a tail bound, so high-precision.
layered_derivatives.jsonl holds, for 8 particles of up to three layers, the derivatives
of the coefficients and the efficiencies with respect to every layer’s \(x_l\), \(\mathrm{Re}\,m_l\)
and \(\mathrm{Im}\,m_l\), from five-point central differences in ball arithmetic
(high-precision).
The committed data take about 18 MB, within the 20 MB budget, which a test enforces.
How the tests use it¶
Tolerances and output. Each accuracy test compares scatrs with every value of the relevant records and asserts the documented bound. It also prints its worst measured error (run
cargo test -- --nocaptureto see it).Loading. The loader rejects malformed records, duplicate case names and output arrays of unequal length.
Slow cases. Records that take too long to recompute are checked by ignored tests, which CI runs weekly, and which run on demand with
cargo test -p scatrs --release -- --ignored. Today that is the 20 000-point Gauss–Legendre rule.The largest rule. The rule with \(2^{17}\) points is checked root by root, by a unit test.
Tolerance policy. A test tolerance is never loosened without a written justification in the test.
Regenerating the data¶
uv run --directory reference python -m scatrs_reference generate # every file
uv run --directory reference python -m scatrs_reference generate --only wigner_d
uv run --directory reference python -m pytest
python-flint is pinned exactly (0.9.0), and the generators round every input they draw with
Arb, never with the platform’s maths library, so regeneration gives the same bytes on every
platform. The generator’s own tests regenerate constants.jsonl, check every committed file
against its SHA-256 in the manifest, and reject generator code that would call the maths
library. CI regenerates every file on Linux for every change, and on Linux and macOS weekly,
and fails on any difference.
The manifest’s precision for wigner_d.jsonl, 800 bits, is only the starting precision. The
generator doubles it as needed, to at most 51 200 bits, until each ball is narrow enough: a
radius of at most \(2^{-200}\) relative or \(2^{-1300}\) absolute. The absolute criterion is meant
for values that are zero.