Experimental: temporal assessment (opt-in)

Warning

Experimental and opt-in. Nothing here runs unless you call it. import atom_hifi, atom-hifi init and atom-hifi run are unchanged, and the API may change. The ratio B/E is a temporal separation diagnostic. It is not Atom-HiFi’s selection fidelity L/R, and nothing in this mode fits a model, picks a K or a cutoff, or names a winner.

atom_hifi.experimental.temporal assesses saved group labels over time. Given labels (frames × entities, -1 = rejected) from one or more candidate labelings of the same trajectory, it measures:

• how well the groups separate in their visit durations (B);

• how much each group’s durations change between an early and a late window (E).

It then reports exact Pareto fronts over (K, B/E, native acceptance). It needs numpy and the standard library only. It does not need a trajectory, a fitted model, a descriptor or a molecular renderer.

Quickstart

python -m atom_hifi.experimental.temporal --write-example demo     # synthetic example (installed)
python -m atom_hifi.experimental.temporal --config demo/config.yaml

The run writes demo/results/. Read it back from Python, independently of where it was written:

from atom_hifi.experimental.temporal import assessment
res = assessment.read_results('demo/results')        # checks schema_version and file sha256
res['fronts'][:3], res['summary']['status_explanations']

The same computation from Python, on labels you already have:

import numpy as np
from atom_hifi.experimental.temporal import assessment, record

src = record.SourceSelection('run7/traj.xyz#sha256=<file hash>')   # WHAT was observed
pairs = []
for K in (8, 12, 16):
    labels = np.load(f'labels_K{K}.npy')                             # frames x entities, -1 = rejected
    rec = record.LabelRecord.from_labels(labels, entity_ids=ids, n_groups=K, source=src,
                                         record_id=f'K{K}')
    a = assessment.assess_record(rec, split=labels.shape[0] // 2, candidate_id=f'K{K}', K=K,
                                 namespace='my_system',
                                 descriptor_policy={'cutoff': 5.0, 'image_policy': 'wrapped',
                                                    'reference_cell_policy': 'stored'},
                                 pca_policy='fixed_k:8', preprocessing=('raw',),
                                 exact_budget={'max_seconds': 60})
    pairs += a.pairs
front_rows, relation_rows, summaries = assessment.fronts(pairs)

assessment.run(mapping_or_path) and the CLI use this same code path. They give byte-identical tables for the same inputs (verified by the test suite).

Configuration (YAML or JSON)

Unknown keys are refused. Relative paths resolve against the config file’s directory.

Key	Meaning
output_dir	Result directory. Existing outputs are refused unless you pass --overwrite.
windows.split	Explicit early/late split frame s: the windows are [0, s) and [s, n). There is no default.
windows.segments	Optional [[start, stop], ...] uniformly sampled row ranges. Rows outside them are unobserved, and every segment boundary censors runs.
preprocessing	Any of raw (primary) and bridge1 (sensitivity: a one-pass simultaneous bridge of single-frame interruptions inside each segment ∩ window).
candidates, weightings	tau (mean duration) and/or P (duration distribution, exact W1); visit and/or equal.
frame_interval, time_unit	Metadata only. Durations are in frames.
exact_budget	Optional {max_seconds, max_work} (finite, positive). It enables exact Level-2b escalation (see below).
exports	Optional {descriptions: bool, runpacks: bool} per-input exports (see Outputs).
inputs[i]	path (2-D integer .npy), candidate_id, K, namespace, descriptor_policy {cutoff, image_policy, reference_cell_policy}, pca_policy ({kind: fixed_k, k}, {kind: native_auto} or an identity string). Optional: n_groups (default K), entity_ids (list or .npy), record_id, source {id, frame_start, frame_step}.

Errors print error: ... and exit with status 2. They name the input (inputs[i] (id, path)) and the problem: invalid values, unsupported formats, ambiguous sampling, undeclared or different source populations, and output collisions.

Identity: which candidates may share a front

A front compares candidates on one observed population. The mode keeps four identities separate:

1. Source population. What was observed. The caller declares it: an opaque source_id for the physical record (for example a trajectory name plus its file hash), and the affine frame map source frame = frame_start + frame_step × row, together with the entity ids.

2. Sampling/layout. Frame count, entity order, segments and sampling metadata.

3. Analysis context. The early/late windows, preprocessing, candidate and weighting.

4. Candidate/model provenance. namespace, descriptor policy, PCA policy, K and the label hash.

The front rules:

• In a context with two or more members, every member must declare a source, and the sources, populations, layouts and windows must be equal. Otherwise the run stops with IdentityError.

• Shapes, labels, record ids, candidate ids and file paths are never used to infer a population. Two trajectories, or two slices of one trajectory, can match in every array dimension.

• A single-member context is allowed. Its identity is recorded as declared or undeclared.

• Different labels of one declared population (different K, different models) are exactly what a front compares.

Pooling run tables (runs.pool_runs):

• Pooling refuses two record ids that declare the same source_id when their observed frames and entities overlap. These would be the same observations counted twice under two aliases.

• Identical label content in distinct records is not duplication. It is kept, and Runs.notes records it when provenance is undeclared.

Limitation. Only affine frame selections are expressible. Any other selection needs its own source_id, and overlap between differently declared sources is not detected.

What B and E are

For each preprocessing, and for the complete visits of each window:

• A visit is a maximal run of one accepted group by one entity inside one observed segment ∩ window, with an exclusive exit, so duration = exit − entry frames.

• A run touching a window edge or a segment boundary is censored. Only complete visits enter B and E.

• A rejection (-1) ends a visit. An unobserved gap censors it instead.

• Full-window complete visits are never built by concatenating half-window visits.

The quantities:

• Joint set: G_joint = {g : n_g^E ≥ 1 and n_g^L ≥ 1}.

• Weights:

  • visit: w_g = (n_g^E + n_g^L) / Σ_joint (n^E + n^L);

  • equal: w_g = 1/|G_joint|.

  The same weights apply to both windows, B and E.

• B_tau = ½ Σ_{g,h} w_g w_h (|μ_g^E − μ_h^E| + |μ_g^L − μ_h^L|) over all ordered pairs.

• E_tau = Σ_g w_g |μ_g^E − μ_g^L|.

• B_P and E_P are the same with the exact empirical W1 distance of the integer duration distributions.

Statuses (status_explanations in run_summary.json):

Status	Meaning
finite	E > 0 exactly
undefined_no_between_separation_and_no_window_difference	B = E = 0
zero_denominator_positive_separation	E = 0 < B
unavailable_empty_joint_set	—
numerical_indeterminate	Production floats contradict the exact identities

Only finite points enter a front. The others are listed in a register and never ranked.

What B/E does not prove:

• It is a descriptive contrast of complete-visit durations between groups, relative to their early/late drift.

• It does not show that the groups are kinetically distinct states.

• It does not measure model quality, and it is not a substitute for L/R.

• Sampling uncertainty is not quantified. The certification below is arithmetic only.

• A higher ratio is not a recommendation: any K/cutoff choice stays with the analyst.

Coverage. Every candidate row carries the exact coverage_* integers:

• occupied, complete and joint groups;

• visits and occupancy per window;

• left-only, right-only and both-censored counts;

• rejected occupancy;

• bridge changes (unknown → empty, never 0).

Native acceptance (the front’s third objective) is accepted raw entity-frames / observed entity-frames over the whole record, the same for every preprocessing.

Exact arithmetic and the optional budget

• Production values. B and E are computed in canonical float order, from exact integer numerators.

• Level-1 certification. Every finite row carries binary64 enclosures of the exact B and E.

• Front comparisons use, in order:

  1. exact zero identities;

  2. separated Level-1 intervals of B_a·E_b versus B_b·E_a;

  3. only when exact_budget is set, exact Fraction arithmetic (Level 2b) within that budget.

  A pair that none of these orders stays unresolved. That is a valid result, never a float ranking.

• Omitting exact_budget is valid: no exact work runs at all.

• Caching. Exact components are computed at most once per (input, preprocessing). Success and exhaustion are both cached for that run only.

• production_inside_enclosure = False is not an error. The canonical production float can round just outside the enclosure of the exact value; fronts use the enclosure.

Fronts

• One front per context: namespace, descriptor policy, PCA policy, preprocessing, window, sample scope, candidate and weighting. Contexts are never pooled.

• Objectives: minimise K, maximise the finite B/E, maximise native acceptance.

• Rows in fronts.csv:

  • each row has a front_status: nondominated, dominated (with dominated_by), unresolved (with unresolved_with) or register;

  • ratio maximisers are listed with ties.

  front_relations.csv lists every pairwise ratio relation with its certification level and evidence.

• Not included: an elbow, a winner, or a joint temporal/structural score.

Outputs

File	Content
candidates.csv / .json	One row per (input, preprocessing, candidate, weighting): B, E, ratio, status, enclosures, coverage, acceptance (numerator/denominator), record fingerprint, label path as written in the config
fronts.csv, front_relations.csv	Fronts and certified relations
group_contributions.csv	Per joint group: exact integers (n, s, ntau, nw1, den, visit mass), μ, Δμ, W1, weights and the four E contributions
run_summary.json	schema_version, resolved config (no absolute paths), input provenance with identities, per-context front summaries and identity records, status explanations, exact-escalation record, environment, sha256 manifest of the other files
descriptions/NN_<id>/ (opt-in)	Per input and per (preprocessing × full/early/late window): accounting.json, aggregates.json, groups_*.csv, frequencies_*.csv, runpacks.npz (lossless run tables; see runs.unpack_runs) and DESCRIPTION.json

Formats:

• Floats are written round-trip exact: repr in CSV, JSON numbers. NaN is nan in CSV and null in JSON.

• Descriptions are off by default. Measured cost for one four-input configuration at 14,955 frames × 736 entities: about 34 s and 36 MB of output, versus about 3 s without them.

• runs.unpack_runs treats a runpack as untrusted input and refuses a malformed one with ValueError:

  • non-integer, non-positive or oversized durations;

  • inconsistent offsets or observed pieces;

  • unknown or incomplete layouts;

  • impossible bridge1 change counts;

  • values whose exact arithmetic would leave int64;

  • two touching runs of one entity with the same label inside one observed piece. Runs must be maximal; equal labels across a gap, segment, entity or record boundary are fine.

  It never merges, rounds or drops runs. A Runs table constructed directly in Python is trusted as given.

Reusing existing labels or a saved model (no campaign rerun)

Precomputed labels. Point inputs[i].path at the saved .npy arrays and declare their common source.

A saved KMeans judge. Embedded rows can be labelled with the judge’s own deployed rule:

from atom_hifi.experimental import judges
judge = judges.load_judge('judge_final.pkl', expected_sha256=FILE_SHA, element='O',
                          expected_embed_op_sha256=OP_SHA)           # verified before unpickling
labels = judges.label_from_judge(judge, z, 'O')                     # z: rows embedded with that operator

Notes on label_from_judge:

• It reproduces the nearest-centroid rule (q > 1 strictly → -1), then the judge’s refined group order and element offset. Labels are therefore global group ids:

  • use n_groups = offset + number of groups for that element;

  • or subtract the offset before building the record.

• It refuses, rather than relabels:

  • other backends;

  • an explicit classifier override;

  • an abstain_radius fit;

  • a judge without refined ids.

• Nothing is refitted, and λ is never altered.

Other opt-in adapters.

• atom_hifi.experimental.periodic (wrap_atoms, corrected_center_fn): integer-lattice wrapping for descriptor evaluation on unwrapped periodic coordinates. Fully periodic cells only. ASE constraints do not block the translation.

• atom_hifi.experimental.descriptor: ED-SOAP with independently set channel cutoffs.

• judges.PCAPolicy: fixed_k:k vs native_auto:broken_stick identities.

These never replace the production descriptor or the PCA defaults.

The PIT-WATER example is an example

The mode was developed on a PIT-WATER interface trajectory (O atoms, 14,955 frames). That study’s outcomes are results of that system, its fresh reference cell (46.8533 Å, with an unverified exact-run provenance) and its labels:

• a 4.8 Å cutoff;

• the O70 discussion;

• its fixed-21 and native-auto fronts.

They are not constants, defaults or recommendations of this mode. The generic package contains no PIT thresholds, indices, frame counts or cells.

Limitations

• Uniform sampling inside each declared segment only. Irregular timestamps are refused, never interpolated.

• Durations are in frames. frame_interval is metadata.

• Only affine source-frame selections are declarable (see Identity).

• B/E is descriptive. Its statistical uncertainty is not estimated, and fronts certify arithmetic only.

• Exact Level-2b escalation can be expensive for many groups with long duration supports. Bound it with exact_budget, or leave it off and keep unresolved pairs.