Experimental: temporal assessment

Opt-in, experimental modules (see Experimental: temporal assessment (opt-in)). Nothing in the default package imports them.

Canonical labels-first temporal assessment API (experimental, opt-in), shared by Python and CLI.

Flow per input: integer labels (frames x entities, -1 = rejected) -> LabelRecord -> early/late runs at an explicit split, per preprocessing -> assess() (B, E, ratio, status, Level-1 enclosures, coverage) -> one CandidateRow per (preprocessing, candidate, weighting) -> fronts(), one front per context (namespace, descriptor policy, PCA policy, preprocessing, window, sample scope, candidate, weighting). The ratio B/E is a temporal separation diagnostic, NOT HiFi’s fidelity L/R, and nothing here selects a K, cutoff or winner.

Python:

from atom_hifi.experimental.temporal import assessment, record
rec = record.LabelRecord.from_labels(labels, entity_ids=ids, n_groups=K,
                                     source=record.SourceSelection('traj.xyz#sha256=...'))
a = assessment.assess_record(rec, split=s, candidate_id='K12', K=12, namespace='mine',
                             descriptor_policy={...}, pca_policy='fixed_k:8')
front_rows, relation_rows, summaries = assessment.fronts(a.pairs + b.pairs)

or assessment.run(config) with the same mapping the CLI reads from YAML/JSON (see resolve_config()), then assessment.read_results(output_dir).

Identity contract. A front compares candidates of ONE observed population at one sampling and one pair of windows. check_front_identity() (called by fronts(), hence by run() and the CLI) requires every member of a context with two or more members to declare the same source population (SourceSelection), the same layout and the same windows. Shapes, labels, record ids, candidate ids and paths never establish a population.

Exact escalation is optional: without exact_budget every pair that the exact-zero identities and Level-1 enclosures cannot order stays 'unresolved' (a valid result). With a budget each (input, preprocessing) computes exact components at most once; success or exhaustion is cached for that assessment only.

exception atom_hifi.experimental.temporal.assessment.ConfigError[source]

Bases: ValueError

Invalid configuration or input (actionable message; CLI exit status 2).

exception atom_hifi.experimental.temporal.assessment.IdentityError[source]

Bases: ValueError

Candidates that may not share a front: undeclared or different source population, layout or windows.

atom_hifi.experimental.temporal.assessment.resolve_config(config, *, base_dir=None)[source]

Validated, fully resolved configuration from a YAML/JSON path or a mapping.

Relative paths resolve against base_dir (default: the config file’s directory, or the current directory for a mapping). Unknown keys are refused. Keys:

output_dir: out                      # required
windows: {split: S}                  # required explicit early/late split; optional segments
preprocessing: [raw, bridge1]        # required, from raw / bridge1
candidates: [tau, P]                 # required
weightings: [visit, equal]           # required
frame_interval: 1                    # optional metadata; durations are in frames
time_unit: frame                     # optional
n_bins: 8                            # optional entry bins (descriptions only)
exact_budget: {max_seconds: 60}      # optional; enables Level-2b escalation
exports: {descriptions: false, runpacks: false}   # optional per-input exports
inputs:                              # required, one per candidate labeling
  - path: labels.npy                 # 2-D integer .npy, -1 = rejected
    candidate_id: K12
    K: 12
    namespace: mine
    descriptor_policy: {cutoff: ..., image_policy: ..., reference_cell_policy: ...}
    pca_policy: {kind: fixed_k, k: 8}   # or {kind: native_auto}, or an identity string
    source: {id: 'traj.xyz#sha256=...', frame_start: 0, frame_step: 1}   # needed to share a front
    n_groups: 12                     # optional (default K)
    entity_ids: [..] or ids.npy      # optional
    record_id: K12                   # optional (default candidate_id)
class atom_hifi.experimental.temporal.assessment.InputAssessment(pairs, provenance, contributions, results, record, providers=<factory>)[source]

Bases: object

Assessment of one labeling: candidate pairs (CandidateRow, extras), the input provenance, per-group contributions and the raw per-preprocessing results. run() releases record and results (sets them to None) once the rows exist, so at most one input’s label array is held in memory at a time.

Parameters:
pairs: list
provenance: dict
contributions: list
results: dict
record: object
providers: dict
atom_hifi.experimental.temporal.assessment.assess_record(record, *, split, candidate_id, K, namespace, descriptor_policy, pca_policy, preprocessing=('raw',), candidates=('tau', 'P'), weightings=('visit', 'equal'), n_groups=None, exact_budget=None, label_path=None, input_index=0)[source]

Assess one LabelRecord at the explicit early/late split.

descriptor_policy is a DescriptorPolicy or a {cutoff, image_policy, reference_cell_policy} mapping (metadata only); pca_policy an identity string, a {kind, k} mapping or a PCAPolicy. exact_budget (optional {max_seconds, max_work}) enables Level-2b escalation; it is validated as in the configuration. Returns an InputAssessment.

atom_hifi.experimental.temporal.assessment.assess_input(item, cfg, *, input_index=0)[source]

Assessment of one resolved config input (see resolve_config()).

atom_hifi.experimental.temporal.assessment.describe_record(record, *, split, preprocessing=('raw',), resolution=None, n_bins=8, runpacks=True)[source]

Per-window descriptions (full, early, late) x preprocessing of one record.

Returns {'accounting': [...], 'aggregates': [...], 'tables': {(prep, window): {'groups', 'frequencies'}}, 'runpacks': {'<prep>_<window>': pack}} using accounting(), group_tables(), aggregate() and pack_runs(). Descriptive only: no front, threshold or filter.

atom_hifi.experimental.temporal.assessment.check_front_identity(members)[source]

Validate that the (CandidateRow, extras) members of ONE context may share a front.

With two or more members, every member must carry a declared source population and all must share population_key, layout_key and the early/late windows; otherwise IdentityError names the difference. A single member is always allowed. Returns the identity record of the context (population_status ‘declared’ or ‘undeclared’).

atom_hifi.experimental.temporal.assessment.fronts(pairs)[source]

Fronts per context of (CandidateRow, extras) pairs (first-seen order; rows in input order), after check_front_identity(). Returns (front rows, relation rows, summaries).

atom_hifi.experimental.temporal.assessment.candidate_record(row, extras)[source]

One candidates.csv row (plus coverage_* columns) of a (CandidateRow, extras) pair.

atom_hifi.experimental.temporal.assessment.run(config, *, overwrite=False, base_dir=None)[source]

Run one configuration (path or mapping, see resolve_config()) and write its outputs.

Existing outputs are refused unless overwrite; collisions are checked before anything is written. Returns the summary written to run_summary.json.

atom_hifi.experimental.temporal.assessment.read_results(output_dir, *, verify=True)[source]

Read a finished run’s outputs (independent of where they were written).

Returns {'summary', 'candidates', 'fronts', 'relations', 'contributions'}; candidates come from candidates.json (typed, NaN -> None), the CSV tables with integer/boolean columns typed. verify checks schema_version and every file’s sha256 against the manifest in run_summary.json. Fronts are read as persisted, never rebuilt.

Label records: validated integer labels, explicit identities and uniform sampling segments.

A LabelRecord is one tracked trajectory of per-frame group labels for a fixed set of entities (for example atoms). labels[t, i] is the label of entity i at frame t; -1 means rejected/abstained, every other value is an accepted group id >= 0.

Sampling contract

  • Frames are the time unit of every duration. frame_interval/time_unit are metadata for conversion and for refusing to pool records sampled differently.

  • segments are explicit, sorted, non-overlapping [start, stop) row ranges of uniformly sampled consecutive frames. Rows outside every segment are unobserved. A segment boundary (a gap, a restart, dropped frames) censors both sides: runs never cross it.

  • Optional timestamps are validated, never used to invent durations: within a segment the step must equal frame_interval (exactly, or within an explicit timestamp_rtol) and timestamps must increase strictly across the whole record. Anything else is rejected.

  • The early/late split is always an explicit argument; there is no default split.

  • Two records are independent: there is no concatenation API, and the same entity id in two records is never treated as the same tracked identity (see runs.pool_runs()).

Identities (kept separate)

  • Source population – WHAT was observed: an optional caller-declared SourceSelection (an opaque source_id of the physical record, e.g. a trajectory name plus its file hash, and the affine frame map source frame = frame_start + frame_step * row) together with the entity ids. It is never inferred from shapes, labels, record ids or file paths; undeclared records have population_key is None.

  • Sampling/layout – LabelRecord.layout_key: frame count, entity order, segments, sampling metadata. Two different trajectories (or two slices of one) can share a layout.

  • Content – LabelRecord.fingerprint (labels included): integrity of early/late halves only. Equal content never proves the same population, and distinct candidates of one population have different labels.

exception atom_hifi.experimental.temporal.record.SamplingError[source]

Bases: ValueError

Irregular, ambiguous or unsupported sampling metadata.

class atom_hifi.experimental.temporal.record.SourceSelection(source_id, frame_start=0, frame_step=1)[source]

Bases: object

Declared source population of a record: which physical record, which of its frames.

source_id is an opaque, caller-declared identity of the physical record (for example 'run7/traj.xyz#sha256=...'); it is never a candidate id or a path chosen by this package. Row r of the label array is source frame frame_start + frame_step * r. Only affine selections are expressible: a non-affine selection needs its own source_id, and overlap between differently declared sources cannot be detected.

Parameters:

  • source_id (str)

  • frame_start (int)

  • frame_step (int)

source_id: str
frame_start: int = 0
frame_step: int = 1
frames(rows)[source]

Source frame numbers of label rows rows (int64).

as_dict()[source]
class atom_hifi.experimental.temporal.record.LabelRecord(labels, entity_ids, segments, frame_interval, time_unit, record_id, timestamps=None, fingerprint=None, source=None)[source]

Bases: object

One validated label record; construct with from_labels().

from_labels() stores read-only COPIES of labels, identities and timestamps, so changing the caller’s arrays afterwards cannot bypass validation. Direct LabelRecord(...) construction is internal: it neither validates nor copies.

Parameters:
entity_ids: ndarray
segments: tuple
frame_interval: float
time_unit: str
record_id: str
timestamps: ndarray | None = None
fingerprint: str | None = None
source: SourceSelection | None = None
classmethod from_labels(labels, *, entity_ids=None, segments=None, frame_interval=1, time_unit='frame', timestamps=None, timestamp_rtol=0.0, record_id='record', n_groups=None, source=None)[source]

Validate and wrap labels (frames x entities, integer, -1 = rejected).

entity_ids are the original identities (default 0..N-1), integer or unicode strings, and must be unique. segments defaults to one segment spanning every row. timestamps (optional) are checked against frame_interval; timestamp_rtol is an explicit tolerance for floating-point timestamps (default exact). n_groups (optional) bounds the labels. source (optional SourceSelection) declares the observed source population; it does not change fingerprint.

property n_frames
property n_entities
property population_key

Hash of the declared source population (source id, frame map, frame count, entity ids), or None when no SourceSelection was declared. Labels are excluded.

property layout_key

Hash of the sampling/layout (frame count, entity order, segments, frame interval, time unit, timestamps). Equal layouts do NOT imply the same population.

full_window()[source]

The whole row range (0, n_frames).

check_window(window)[source]

Validated (start, stop) row window inside the record.

half_windows(split)[source]

((0, split), (split, n_frames)) for an explicit split frame (no default).

observed_intervals(window)[source]

Non-empty segment ∩ window pieces, shape (k, 2), in time order.

Every piece boundary censors: a run touching one is left/right-censored.

observed_frames(window)[source]

Number of observed frames per entity inside window.

Literal runs/visits, exclusive censoring, one-pass bridge1, accounting and a lossless runpack.

A run is a maximal contiguous sequence of one label for one entity inside one observed piece (segment ∩ window), rejected runs included. Coordinates are absolute row indices with an EXCLUSIVE exit, so duration = exit - entry. A run touching a piece boundary (window edge or segment boundary) is left- and/or right-censored; only accepted runs with neither flag are complete visits. Ported from the accepted v2 extract_runs/bridge1/accounting/ pack_runs and the PF stats_from_runs (same values and run order; vectorised over entities).

class atom_hifi.experimental.temporal.runs.RunSource(record_id, window, intervals, entity_offset, n_entities, frame_interval, time_unit, record_frames=None, record_fingerprint=None, source=None)[source]

Bases: object

Where a block of runs came from: one record and one window.

record_frames is the record’s frame count (its full window is (0, record_frames)); None when unknown, e.g. for a run table rebuilt from an archived v2 runpack. source is the record’s declared SourceSelection (None when undeclared).

Parameters:
record_id: str
window: tuple
intervals: ndarray
entity_offset: int
n_entities: int
frame_interval: float
time_unit: str
record_frames: int | None = None
record_fingerprint: str | None = None
source: SourceSelection | None = None
property observed_frames
class atom_hifi.experimental.temporal.runs.BridgeChanges(frames_changed, accepted_to_accepted_changes, rejected_to_accepted_changes, overlapping_candidate_pairs)[source]

Bases: object

Change categories of one bridge1 application (counts over all entities).

Parameters:

  • frames_changed (int)

  • accepted_to_accepted_changes (int)

  • rejected_to_accepted_changes (int)

  • overlapping_candidate_pairs (int)

frames_changed: int
accepted_to_accepted_changes: int
rejected_to_accepted_changes: int
overlapping_candidate_pairs: int
class atom_hifi.experimental.temporal.runs.Runs(group, duration, entity, entry, exit, left_censored, right_censored, entry_bin, source, sources, entity_ids, n_bins, preprocessing, bridge=None, bridge_mask=None, qualified_ids=False, notes=())[source]

Bases: object

Run table (v2 order: entity-major, then entry) plus provenance.

entity indexes entity_ids; source indexes sources (one per record/window). bridge/bridge_mask exist for preprocessing='bridge1': the mask marks changed labels over the observed rows of the window stacked in time order (frames x entities). bridge is None for a bridge1 table means the change counts are UNKNOWN (never zero). qualified_ids is True when entity_ids are already (record_id, entity_id) pairs (output of pool_runs()). notes carries pooling remarks (never a silent change). Tables from extract_runs() are maximal by construction and unpack_runs() validates encoded (untrusted) run data; a Runs constructed directly is trusted as given.

Parameters:
group: ndarray
duration: ndarray
entity: ndarray
entry: ndarray
exit: ndarray
left_censored: ndarray
right_censored: ndarray
entry_bin: ndarray
source: ndarray
sources: tuple
entity_ids: ndarray
n_bins: int
preprocessing: str
bridge: BridgeChanges | None = None
bridge_mask: ndarray | None = None
qualified_ids: bool = False
notes: tuple = ()
property n_entities
complete()[source]

Accepted runs with observed entry and exit inside their piece.

censoring_class()[source]

Mutually exclusive class per run: -1 rejected, else COMPLETE/LEFT_ONLY/RIGHT_ONLY/BOTH_CENSORED.

atom_hifi.experimental.temporal.runs.bridge1(block)[source]

v2 one-pass simultaneous bridge on a contiguous block (frames x entities).

For interior t, the label becomes A iff the ORIGINAL neighbours at t-1 and t+1 both equal an accepted A and the original middle differs. Returns (bridged, change) with change of shape (len(block) - 2, entities) (empty if the block has fewer than three frames).

atom_hifi.experimental.temporal.runs.extract_runs(record, window, *, preprocessing='raw', n_bins=8)[source]

Maximal runs of every entity inside window (absolute rows, exclusive exit).

preprocessing='bridge1' applies the one-pass bridge inside each segment ∩ window piece only (never across a gap or the window edge). Calling this with the full window gives the separate full-window overview; with a half window, the within-half sensitivity. entry_bin = floor(n_bins * (entry - start) / (stop - start)) is window-relative.

atom_hifi.experimental.temporal.runs.pool_runs(tables)[source]

Explicitly pool run tables of DIFFERENT records without linking identities.

Entities are re-indexed per record ((record_id, entity_id) identities, qualified once: re-pooling a pooled table keeps its pairs), no run is joined across records, and records must share frame_interval/time_unit/preprocessing/n_bins; duplicate record ids are refused as ambiguous. Pooled bridge1 change counts are unknown if any input’s are.

Duplicates are defined by declared provenance, never by content: two records declaring the same source_id whose observed source frames and entities intersect are refused (the same observations under two aliases). Equal label content is NOT duplication – distinct records may legitimately coincide – so such records are kept, and a remark is added to notes when at least one of them has no declared source.

atom_hifi.experimental.temporal.runs.native_acceptance(record)[source]

(accepted, observed) raw entity-frame counts over the record’s full window.

accepted counts labels >= 0 on observed rows (every segment), observed all observed entity-frames; identical to the accepted_occupancy_frames and total_entity_frames of accounting() on the raw full-window runs. Not reduced, so the exact acceptance is Fraction(accepted, observed). Preprocessing never changes it.

atom_hifi.experimental.temporal.runs.accounting(runs, *, resolution, window_name, sample_scope=None)[source]

Exhaustive run/occupancy accounting (v2 definitions, generic names).

Verifies that complete, left-only, right-only and both-censored visits partition the accepted visits and that runs cover every observed entity-frame exactly once. window_name is a label only; sample_scope defaults to the scope derived from the windows ('full_primary' for the record’s full window, 'secondary_half' for one half) and must be given explicitly for any other window.

atom_hifi.experimental.temporal.runs.half_stats(runs, n_groups)[source]

Exact per-group integers of one window (PF stats_from_runs definitions).

Returns n (complete visits), s (their summed duration), occ, one, left_only/right_only/both, ogrp (entity x group complete-visit indicator) and lossless frequency rows fg, fd, fc sorted by (group, duration), plus rejected and bridge-change counts. These are the inputs of separation.separation(). Bridge-change counts are 0 for raw tables and None (unknown, reason in bridge_changes_reason) for a bridge1 table whose counts were not kept, e.g. one rebuilt from an archived v2 runpack.

atom_hifi.experimental.temporal.runs.pack_runs(runs)[source]

Lossless array pack of one record/window run table (v2 layout, generalised).

Keys group, duration, offsets, start, stop follow v2 (smallest of int16/32/64, uint16/32/64, int32/64 that holds the values; PIT data give v2’s dtypes) plus intervals (observed pieces), n_bins, entity_ids, record_id, preprocessing, frame_interval, time_unit and, when known, record_frames, record_fingerprint, the four bridge1 change counts bridge_changes (in BRIDGE_FIELDS order) and the declared source (source_id, source_frame_start, source_frame_step). The per-frame bridge_mask is not packed. Save with np.savez_compressed.

atom_hifi.experimental.temporal.runs.unpack_runs(pack, *, preprocessing=None)[source]

Rebuild the run table from pack_runs() arrays (or an accepted v2 runpack).

v2 packs have exactly the keys group, duration, offsets, start, stop: one piece [start, stop) and 8 bins are used; their record length and bridge1 change counts are unknown (record_frames=None, bridge=None), never assumed. preprocessing labels a pack that does not store it and must agree with one that does.

A pack is untrusted input. It is validated before any cast or calculation, and a malformed pack is refused with ValueError naming the field, entity, piece or run; it is never repaired (no merging, rounding, dropping or relabelling) and never modified:

  • the layout is the complete pack_runs() layout or the v2 layout (no unknown keys);

  • window, intervals (the observed pieces: non-empty, inside the window, in time order, touching allowed), offsets, labels (integers >= -1) and durations have integer dtypes and consistent shapes; every duration is a positive integer that fits its piece;

  • the runs of every entity tile its observed pieces exactly, in time order;

  • runs are maximal: two consecutive runs of one entity inside one piece never carry the same label (rejected -1 included). Equal labels across a piece boundary (a gap or a segment restart, as stored in intervals), an entity or a record boundary are legitimate;

  • bridge1 change counts are consistent with the pieces, and every sum and product (run coordinates, entry bins) is checked against the int64 range before it is computed; a pack whose exact values fit is never refused for size.

Between-group separation B and within-group early/late discrepancy E (two candidates, two weightings).

This ratio B/E is a temporal separation diagnostic. It is NOT HiFi’s fidelity F = L/R, and the names here never say “fidelity”; archived_names maps them to the archived temporal-fidelity-Pareto (PF) column names for regression checks only.

Definitions (per paired early/late dataset; frames are the unit)

  • Joint set G_joint = {g : n_g^E >= 1 and n_g^L >= 1} (complete visits). Every candidate and weighting uses this set; the same weights serve both halves, numerator and denominator.

  • w_visit = (n_E + n_L) / T with T the joint total, w_equal = 1/|G_joint|.

  • B_tau = 1/2 sum_{g,h} w_g w_h (|mu_g^E - mu_h^E| + |mu_g^L - mu_h^L|) over ordered pairs with the zero diagonal; E_tau = sum_g w_g |mu_g^E - mu_g^L|.

  • B_P/E_P: the same with the exact empirical W1 distance of complete-visit durations.

Production arithmetic (a port of PF’s canonical float algorithm, same operation order)

S_w(v) = 2 sum_b (m_b/T) A_b with A_1 = 0, A_b = A_{b-1} + (M_{<b}/T)(v_b - v_{b-1}) over exact-tie blocks of the stably sorted values; every sum is a sequential ascending sum (np.cumsum, never pairwise np.sum). B_tau uses float means s/n; B_P integrates S_w of float CDF ordinates C/n over the union support in increasing duration. E is the sequential sum of the ascending-sorted per-group terms w_g * Delta_g with Delta_g = N_g / (n_E n_L) and EXACT integer numerators N_tau = |s_E n_L - s_L n_E| and N_W1 = sum_i |C_E n_L - C_L n_E| (u_{i+1}-u_i). A guard from observed maxima runs BEFORE any product; if a bound could exceed int64 the same algorithm runs on Python integers (path='python_int') with exact, unclamped numerators. The guard bound GUARD_LIMIT = 2**62 may only be lowered (guard_limit), never raised. Statuses come from the exact integer identities (empty joint set, B = 0, E = 0), never from a float magnitude threshold; the one float-derived status, numerical_indeterminate, marks production floats that are non-finite or contradict those identities. Such points go to the non-finite register; the Level-1 enclosure does not repair them.

Arithmetic certification (labelled ARITHMETIC, not statistical confidence)

Level 1 (level1()): binary64 interval enclosures of exact B and E from exact rational blocks with one-ulp outward rounding (port of PF exact_verify.level1); orderings and ties come from exact integer cross-multiplication, so a float-sort disagreement is repaired instead of aborting. Level 2b (exact_components()): exact Fraction values under a budget.

exception atom_hifi.experimental.temporal.separation.BudgetExhausted[source]

Bases: RuntimeError

Exact (Level 2b) evaluation stopped by its recorded budget; the relation stays unresolved.

atom_hifi.experimental.temporal.separation.joint_set(early, late)[source]

Ascending ids of groups with at least one complete visit in both halves.

atom_hifi.experimental.temporal.separation.status_code(G, b_zero, e_zero, b, e)[source]

PF status: unavailable (empty joint set), indeterminate (floats contradict the exact identities or are non-finite), finite (E > 0), undefined (B = E = 0) or divergent (B > 0, E = 0).

atom_hifi.experimental.temporal.separation.separation(n_groups, early, late, *, guard_limit=4611686018427387904, chunk_elements=1048576)[source]

Production B/E for both candidates and weightings of one paired early/late dataset.

early/late are mappings with n, s (length n_groups) and frequency rows fg, fd, fc sorted by (group, duration), e.g. from runs.half_stats(). Returns a dict with path (‘int64’ or ‘python_int’), joint_ids, PF-ordered raw_float/raw_int, exact integer ntau/nw1/den and visit_mass (int64 arrays, or object arrays of unclamped Python ints on the Python-integer path), per-group float dmu/w1, the weights and per-group E_contributions that the canonical sums add, and per-combination combos[(candidate, weighting)] with B, E, ratio, half components and status. guard_limit may only lower GUARD_LIMIT (ValueError otherwise).

atom_hifi.experimental.temporal.separation.level1(n_groups, early, late, result, *, chunk_elements=1048576)[source]

Level-1 enclosures of the exact B and E of every combination (port of PF exact_verify.level1).

Uses the production result for the joint set, the integer path and the exact numerators. Inside the int64 guard all integer work is int64; on result['path'] == 'python_int' it uses Python integers with the exact, unclamped ntau/nw1/den. Returns None for an empty joint set, else {'path', 'enclosures': {combo: (B_lo, B_hi, E_lo, E_hi)}, 'halves', 'reordered_rows'} where reordered_rows counts float-sort orders repaired exactly.

atom_hifi.experimental.temporal.separation.exact_components(n_groups, early, late, *, max_seconds=None, max_work=None)[source]

Exact Fraction B and E of every combination (port of PF exact_components('fraction')).

max_work bounds the predicted number of exact block updates (joint groups x support points); max_seconds bounds the wall time of the whole evaluation, checked throughout and once more before returning. Exceeding either raises BudgetExhausted. Returns None for an empty joint set, else {combo: (B, E)}. With neither bound the evaluation is unbounded: a low-level compatibility default. The public assessment route (assessment) only calls it under an explicit finite budget.

atom_hifi.experimental.temporal.separation.reliability(result, enclosures=None, exact=None)[source]

ARITHMETIC reliability fields per combination (rounding error of the exact value only).

enclosures: level1() output (or None); exact: exact_components() output (or None). Relative errors are NaN with reason 'exact_zero' when the exact value is zero (absolute error and enclosure widths remain), or 'exact_not_computed'.

atom_hifi.experimental.temporal.separation.coverage(early, late, joint_ids)[source]

Exact integer coverage/censoring/bridge accounting of one paired dataset (PF coverage).

Bridge-change fields are None when a half’s counts are unknown (see runs.half_stats(), whose bridge_changes_reason says why); they are never reported as 0.

atom_hifi.experimental.temporal.separation.check_halves(early_runs, late_runs)[source]

Refuse (ValueError) an early/late pair that is not the two halves of the same record(s).

Both tables must share preprocessing, and source by source (in order) record identity, the declared source population (both undeclared, or equal), entities, frame_interval and time_unit; the early window must end at or before the late start (not swapped/overlapping), and the two windows must be (0, s) and (s, n_frames) of that record, i.e. record.LabelRecord.half_windows(). Duration distributions from different sampling or different records are never paired.

atom_hifi.experimental.temporal.separation.assess(early_runs, late_runs, n_groups, *, guard_limit=4611686018427387904, with_level1=True)[source]

Convenience: half statistics, production B/E, Level-1 enclosures, coverage and reliability.

The pair is validated first by check_halves(), and guard_limit may only lower GUARD_LIMIT; both refuse with ValueError.

Per-group complete-visit descriptions (v2 definitions), returned as lists of row dicts.

Definitions are inherited unchanged from the accepted v2 code: mean, median, linear-interpolated quantiles, sample SD (ddof=1), CV = SD/mean, unscaled MAD, D_MAD = 1.4826*MAD/median, one-frame fraction, entity support/concentration, the mean-matched geometric comparator q = 1/mu, CV_geom = sqrt(1 - 1/mu), and Q_raw/Q_log on all complete visits. Means/frequencies need one complete visit; dispersion needs two (otherwise NaN with a reason). Every occupied accepted group keeps a row. The 20 visits / 5 entities / 4 bins support view is a supplementary flag set, never a filter. The numpy call order of v2 is kept (complete visits in run order per group) so values are bitwise comparable with archived v2 tables. Rows use generic entity names (archived oxygen names: archived_names, regression only).

atom_hifi.experimental.temporal.describe.q_stat(duration, group, log=False)[source]

In-sample mean-explained variation 1 - SS_W/SS_T over all complete visits (v2).

y = log1p(d) when log. NaN when fewer than two visits or zero total variance.

atom_hifi.experimental.temporal.describe.group_tables(runs, *, resolution, window_name, sample_scope=None)[source]

All occupied accepted groups: per-group rows, exact duration frequencies/ECDF, entity and bin counts.

Returns (groups, frequencies, entity_counts, bin_counts) as lists of dicts. sample_scope follows runs.accounting() (derived from the windows, never the name).

atom_hifi.experimental.temporal.describe.aggregate(runs, groups, account)[source]

Window summary over all complete visits plus group-equal/episode-weighted D_MAD and CV (v2).

atom_hifi.experimental.temporal.describe.support_diagnostic(groups, summary, *, min_visits=20, min_entities=5, min_bins=4)[source]

Supplementary support flags per group (never a filter) and the qualified-subset summary.

Defaults are the inherited v2 thresholds (20 complete visits, 5 distinct entities, entries in 4 of the run table’s n_bins bins); every group keeps its row with its condition flags.

Exact three-objective fronts: minimise K, maximise the finite ratio B/E, maximise acceptance.

Port of the PF pareto.py front logic and exact_verify.compare_f comparator.

  • a dominates b iff K_a <= K_b, ratio_a >= ratio_b and A_a >= A_b with one strict;

  • K and the acceptance A are exact (int, or fractions.Fraction / an integer numerator over a common denominator); floats are refused;

  • a ratio order is certified only by exact zeros (integer identities), separated Level-1 intervals of B_a*E_b vs B_b*E_a, or exact Fraction cross-products computed on demand under a recorded budget; otherwise the relation is 'unresolved' (no relative-gap rule);

  • non-finite statuses go to a register and never enter a front; extrema keep ties and are taken over the non-dominated members (unresolved members are reported through front_status and the ratio-maximizer unresolved list, never ranked);

  • build_front() refuses rows that mix namespace, descriptor policy (cutoff, image policy, reference-cell policy), PCA policy, preprocessing, window, sample scope, candidate or weighting. It does NOT verify that the rows describe one source population: callers pass already validated rows (the public route assessment.fronts() checks the declared population, layout and window identities first).

class atom_hifi.experimental.temporal.pareto.VerifiedPoint(label, b_zero, enclosure, exact=None, budget=None)[source]

Bases: object

A finite-status point: exact-zero flag, Level-1 enclosure and an optional exact provider.

exact is a zero-argument callable returning exact (B, E) Fractions; it may raise BudgetExhausted. budget is recorded in the comparison evidence. Both outcomes are cached per point: an exhausted budget is not paid again by later comparisons.

exact_values()[source]
atom_hifi.experimental.temporal.pareto.synthetic_point(label, B, E, budget=None)[source]

VerifiedPoint from exact Fractions (tests/synthetic cases). float(Fraction) is correctly rounded, so one ulp either side encloses the exact value.

atom_hifi.experimental.temporal.pareto.compare_ratio(a, b)[source]

(relation, level, evidence) for ratio_a vs ratio_b of two finite points (E > 0 exactly).

relation in {‘gt’, ‘lt’, ‘eq’, ‘unresolved’}; level in {‘exact_zero’, ‘level1_binary64_interval’, ‘level2b_exact_fraction’, ‘unresolved’}.

atom_hifi.experimental.temporal.pareto.relations(points, compare)[source]

Pairwise ratio relations among finite points: rel[(i, j)] for ratio_i vs ratio_j.

atom_hifi.experimental.temporal.pareto.dominance(points, rel)[source]

dominated_by and unresolved maps for finite points with exact keys K and A.

atom_hifi.experimental.temporal.pareto.front(points, compare)[source]

points: dicts with ‘id’, exact ‘K’ and ‘A’, ‘status’ (STATUS string) and comparator payload.

Returns (rows, witnesses, relation_table, register, rel); rows carry front_status in {‘nondominated’, ‘dominated’, ‘unresolved’} for finite points.

atom_hifi.experimental.temporal.pareto.ratio_maximizers(points, compare=None, rel=None)[source]

Finite-ratio maximizers with ties kept; returns (maximizers, unresolved_candidates).

atom_hifi.experimental.temporal.pareto.extrema(points, rows, maximizers, unresolved)[source]

Observed property extrema of one front (ties kept): ratio maximum (+unresolved), and the minimum K / maximum A among non-dominated members. Descriptive, never an operating point.

class atom_hifi.experimental.temporal.pareto.DescriptorPolicy(cutoff, image_policy, reference_cell_policy)[source]

Bases: object

Descriptor identity of a candidate: cutoff, periodic-image policy, reference-cell policy.

Parameters:

  • cutoff (Any)

  • image_policy (str)

  • reference_cell_policy (Any)

cutoff: Any
image_policy: str
reference_cell_policy: Any
class atom_hifi.experimental.temporal.pareto.CandidateRow(candidate_id, namespace, descriptor_policy, pca_policy, K, preprocessing, window, sample_scope, candidate, weighting, B, E, ratio, status_code, enclosures, coverage=<factory>, acceptance=None, B_exact_zero=None, exact=None, exact_budget=None)[source]

Bases: object

One candidate-table row (one K, one candidate, one weighting, one context).

enclosures are the Level-1 ARITHMETIC enclosures (B_lo, B_hi, E_lo, E_hi); acceptance is exact (Fraction(accepted, all entity-frames) or an int numerator over a denominator shared by every row of the front); exact optionally returns exact (B, E).

Parameters:
candidate_id: str
namespace: str
descriptor_policy: DescriptorPolicy
pca_policy: str
K: int
preprocessing: str
window: str
sample_scope: str
candidate: str
weighting: str
B: float
E: float
ratio: float
status_code: int
enclosures: tuple | None
coverage: Mapping
acceptance: Any = None
B_exact_zero: bool | None = None
exact: Callable | None = None
exact_budget: Mapping | None = None
property status
context()[source]
verified_point()[source]
atom_hifi.experimental.temporal.pareto.check_single_context(rows)[source]

Raise ValueError naming every context field that differs between rows.

atom_hifi.experimental.temporal.pareto.build_front(rows)[source]

Front, register, witnesses, relations and extrema of candidate rows sharing one context.

Small deterministic synthetic example of the EXPERIMENTAL temporal assessment (installed).

One synthetic record (synthetic:latent_record_1, 48 frames x 6 entities) is labelled at three resolutions K2/K3/K4 – deterministic partitions of the same latent fine states, with candidate-specific abstention (-1) – so all three describe ONE observed population and may share a front. K5_padded reuses K3’s labels at a larger nominal K (dominated by K3 through an exact Level-2b equality), K3 also appears under a second PCA-policy identity (its own fronts), and a separate record (synthetic:repeat_record, own namespace) has identical early/late halves (E = 0 < B: the non-finite register). The latent chain changes its stay probabilities at the split, so the early/late comparison is informative (E > 0).

The arrays are generated by examples/experimental_temporal/make_labels.py in the repository (seed and partitions documented there) and embedded here as text rows: one string per frame, one character per entity, - = rejected. Descriptor/PCA metadata are placeholders for the context identity only; nothing here is a PIT setting or a recommendation.

python -m atom_hifi.experimental.temporal –write-example DIR python -m atom_hifi.experimental.temporal –config DIR/config.yaml

atom_hifi.experimental.temporal.example.arrays()[source]

The example label arrays (int8, frames x entities, -1 = rejected).

atom_hifi.experimental.temporal.example.config()[source]

The example configuration as a mapping (equal to CONFIG_YAML); paths are relative.

atom_hifi.experimental.temporal.example.write_example(directory, *, overwrite=False)[source]

Write config.yaml and the label arrays into directory (created if needed).

Existing files are refused unless overwrite. Returns {'config': path, 'arrays': [paths]}.

Opt-in adapters

labels/centers dataclass fields are described in their class docstrings (excluded here so that they do not shadow the cross-references of atom_hifi.training.cluster_base).

Explicit corrected periodic-image adapter for descriptor evaluation (experimental, opt-in).

Why

DScribe’s periodic SOAP (used by ED-SOAP through decaf) builds periodic copies of the raw positions m lattice vectors away, with m_a in [-n_a, n_a] and n_a = ceil(R / h_a) (R = cutoff + Gaussian padding, h_a = perpendicular cell height). It never wraps positions. When coordinates are unwrapped (e.g. an MD trajectory that lets atoms drift several cells), a neighbour whose required lattice image lies outside that copy range is silently dropped and contributes exactly zero.

What

wrap_atoms() returns a copy whose atoms (and optional coordinate centres) are translated by integer lattice vectors into the unit cell, r' = r + n @ C (row vectors). Atoms already inside the cell are left bitwise unchanged. With every atom and centre inside one cell, the required image of any neighbour satisfies |k_a| <= ceil(R/h_a) whenever R/h_a is not an integer, so DScribe’s own copy range is complete – including multi-image cells with h_a < R (proof sketch: for an image vector v = (s_j - s_i + k) @ C the projection onto the unit normal of the face spanned by the other two cell vectors is (s_j - s_i + k)_a h_a, so |v| <= R implies |s_j - s_i + k_a| <= R/h_a; with s in [0, 1] this bounds |k_a| by floor(R/h_a) + 1 = ceil(R/h_a)).

Image convention (used by every integer comparison): row vectors, cell C with lattice vectors as rows, displacement_ij(k) = r_j + k @ C - r_i. Under translations r'_j = r_j + n_j @ C and r'_i = r_i + n_i @ C the same physical neighbour has k' = k + n_i - n_j; canonical_image() maps observed offsets back.

Scope: only fully periodic cells (pbc == (True, True, True)) are supported in this stage; anything else, non-finite positions, a degenerate cell or a radius whose R/h is (numerically) an integer raises. Nothing here changes any Atom-HiFi default.

class atom_hifi.experimental.periodic.WrapResult(atoms, shifts, centers, center_shifts)[source]

Bases: object

A wrapped copy and the integer translations that produced it (r' = r + shifts @ cell).

Parameters:
atoms: object
shifts: ndarray
center_shifts: ndarray | None
atom_hifi.experimental.periodic.cell_heights(cell)[source]

Perpendicular heights h_a = V / |a_b x a_c| of a row-vector cell.

Return type:

ndarray

atom_hifi.experimental.periodic.legacy_copy_counts(cell, radius)[source]

Copies per side DScribe generates on each periodic axis: ceil(radius / h_a).

Parameters:

radius (float)

Return type:

ndarray

atom_hifi.experimental.periodic.check_supported(atoms, radii=(), tol=1e-06)[source]

Raise on configurations this adapter does not certify.

Parameters:
Return type:

None

atom_hifi.experimental.periodic.wrap_atoms(atoms, radii=(), centers=None, tol=1e-06, frac_guard=1e-09)[source]

Return a copy translated into the unit cell by integer lattice vectors.

centers may be an (m, 3) array of coordinate centres; they are translated by the same rule (their own integer shifts) and returned. Atom-index centres need no handling: an index into the wrapped copy is the wrapped centre. ASE constraints (e.g. FixAtoms) are kept on the copy but do not block the lattice translation: a periodic image is the same physical configuration, not a move.

Parameters:
Return type:

WrapResult

atom_hifi.experimental.periodic.canonical_image(k_observed, n_center, n_neighbor)[source]

Map an image offset observed on translated coordinates back to the source convention.

k = k' - n_i + n_j for centre translation n_i and neighbour translation n_j.

Return type:

ndarray

atom_hifi.experimental.periodic.enumerate_images(positions, cell, center, radius, guard=1e-09)[source]

Independent complete enumeration of all (atom, image) pairs within radius of center.

Works on arbitrary (unwrapped) coordinates: for each atom j the fractional relative displacement delta = (r_j - c) @ inv(C) bounds the admissible images per axis by |delta_a + k_a| <= R / h_a; candidates are widened by one on each side and filtered by the Cartesian distance. Pairs with |d - R| < guard are flagged rather than decided. Returns (j, k, d, in_guard_band) with k in the convention of the given positions.

Parameters:
atom_hifi.experimental.periodic.corrected_center_fn(center_fn, radii)[source]

Wrap an Atom-HiFi center_descriptor_fn(structures, atom_ids_per_structure).

Each structure is replaced by its integer-wrapped copy before the unchanged descriptor function runs; atom-index centres keep their identity and row order. radii (the descriptor’s cutoff-plus-padding radii) are required: the copy-range check needs them.

Parameters:
Return type:

Callable

Decoupled ED-SOAP centre descriptor with absolute radii (experimental, opt-in).

Why

Production ED-SOAP (atom_hifi.soap_ML.soap.generate_ED_SOAP(), called by runner._build_descriptor_fn) ties both channels to one lengthscale: decaf forms r = lengthscale * cut and sigma = lengthscale * (cut / 8) for the long and the short channel. A cutoff study needs the four values set independently.

What

EDSoapSpec holds the ABSOLUTE long/short cutoffs and Gaussian widths (Angstrom), the radial/angular maxima and the species, all explicit (nothing is defaulted, nothing is system-specific). center_fn() returns a center_descriptor_fn(structures, atom_ids_per_structure) that calls decaf.get_SOAP(structure, 1.0, SOAP_L_cut=r_long, SOAP_L_sigma=sigma_long, SOAP_S_cut=r_short, SOAP_S_sigma=sigma_short, ...). With NNDistance = 1.0 decaf’s products 1.0 * r and 1.0 * sigma are exact, so the values handed to DScribe are exactly the given ones. Output layout and normalisation are decaf’s: [long | short] per row, each channel L2-normalised per row.

Identity with production: EDSoapSpec.from_production() forms the absolute values as the production float products (lengthscale * cut and lengthscale * (cut / 8)), in which case the rows are bitwise the production rows. Anything else is a different descriptor, by construction.

The function composes with atom_hifi.experimental.periodic.corrected_center_fn() (pass support_radii() as its radii). Importing this module does no IO and does not import decaf or DScribe; both are imported when a descriptor is evaluated.

class atom_hifi.experimental.descriptor.EDSoapSpec(r_long, sigma_long, r_short, sigma_short, n_long, l_long, n_short, l_short, species, periodic=True)[source]

Bases: object

Absolute ED-SOAP parameters (Angstrom). Every field is explicit.

species are atomic numbers in the order handed to DScribe (as production passes them); periodic is the master switch, honoured per structure only where pbc is set (the production rule).

Parameters:
r_long: float
sigma_long: float
r_short: float
sigma_short: float
n_long: int
l_long: int
n_short: int
l_short: int
species: tuple
periodic: bool = True
classmethod from_production(*, lengthscale, l_cut, s_cut, l_nmax, l_lmax, s_nmax, s_lmax, species, periodic=True)[source]

The production descriptor as absolute values: r = lengthscale * cut and sigma = lengthscale * (cut / 8) per channel (the exact float products decaf forms).

identity()[source]

Stable, exact identity record (floats as repr, i.e. round-trip exact).

atom_hifi.experimental.descriptor.center_fn(spec)[source]

An Atom-HiFi center_descriptor_fn(structures, atom_ids_per_structure) for spec.

Parameters:

spec (EDSoapSpec)

Return type:

Callable

atom_hifi.experimental.descriptor.ed_soap_rows(structures, atom_ids_per_structure, spec)[source]

ED-SOAP rows at the requested centres of every structure, stacked in structure order.

atom_ids_per_structure is None (every atom of every structure) or one index array per structure (an empty array means every atom, as in production).

Parameters:
Return type:

ndarray

atom_hifi.experimental.descriptor.support_radii(spec)[source]

DScribe’s own neighbour-search radius per channel, (long, short): the cutoff plus DScribe’s Gaussian padding, read from DScribe objects built with the spec’s values (the radii atom_hifi.experimental.periodic.corrected_center_fn() certifies).

Parameters:

spec (EDSoapSpec)

Return type:

tuple

Saved-judge labelling and PCA-policy identities (experimental, opt-in).

Saved-judge labeller

label_rows() is the nearest-centroid rule of a saved KMeans judge on already-embedded rows z of any embedding dimension: Atom-HiFi’s own NearestCentroidClassifier (squared distances, first minimum on exact ties, q = d1**2 / lambda_open, OOD iff q > 1 STRICTLY -> label -1), then the judge’s refined group order local_of_old and element offset. Diagnostics are the decision margins q - 1 (OOD iff > 0) and the nearest-centroid gap d2 - d1 (an exact tie gives 0). label_from_judge() reads those inputs from a loaded judge. Nothing is fitted or re-embedded.

load_judge() verifies the file sha256 BEFORE unpickling and the element’s stored embedding operator against an identity the CALLER supplies (no system constants live here). The operator is read from the stored field only: a judge without one is refused, never re-embedded (recomputing it would be a PCA on the reference data).

PCA policies

PCAPolicy names how an embedding operator is chosen: fixed_k (top k eigenvectors) or native_auto (the broken-stick dimension). Identities are distinct strings ('fixed_k:21' vs 'native_auto:broken_stick') even when the broken-stick dimension equals k. eigendecomposition() / broken_stick_dim() / PCAPolicy.operator() implement decaf’s get_PCA rule with decaf’s own arithmetic: covariance of the centred rows (X - mean)^T (X - mean) / N, full eigh, eigenpairs in descending order (argsort()[::-1]), d_auto = #{k : bs_k <= lambda_k / sum(lambda)} with bs_k = (1/D) sum_{j=k..D} 1/j, operator = leading eigenvectors, projection UNCENTRED (rows @ operator), no whitening, no randomness. Calling it is the caller’s decision: importing this module computes nothing.

Importing this module does no IO and imports neither decaf, DScribe nor the Atom-HiFi training stack; those load when a judge is loaded or rows are labelled.

atom_hifi.experimental.judges.array_sha256(a)[source]

sha256(dtype.str + json(list(shape)) + C-order bytes) of an array (0-d kept 0-d).

Return type:

str

atom_hifi.experimental.judges.file_sha256(path, block=16777216)[source]
Return type:

str

atom_hifi.experimental.judges.load_judge(path, *, expected_sha256, element, expected_n_groups=None, expected_embed_op_sha256=None, expected_embed_op=None)[source]

Load a saved atom_hifi.Judge after verifying its identity.

expected_sha256 (file) is checked before unpickling. The element’s stored embedding operator must match expected_embed_op_sha256 (array_sha256()) and/or expected_embed_op (dtype, shape and bytes); at least one is required. expected_n_groups optionally pins the fitted group count. Raises ValueError on any mismatch.

atom_hifi.experimental.judges.label_rows(z, centers, lambda_open, local_of_old, offset=0, *, chunk=32, diagnostics=False)[source]

Nearest-centroid labels of embedded rows z (shape (..., d)), -1 = OOD.

centers (n_groups, d) and lambda_open (squared-distance units) are the saved fit; local_of_old maps a fitted centre index to its refined local id and offset is the element’s group-id offset. chunk leading-axis entries are classified per call (the rule is row-wise, so chunking never changes a value). Labels use the smallest of int16/int32/int64 holding every id. With diagnostics returns (labels, q_margin, gap) where q_margin = q - 1 and gap = d2 - d1.

atom_hifi.experimental.judges.label_from_judge(judge, z, element, *, chunk=32, diagnostics=False)[source]

label_rows() with the centres, lambda_open, refined order and offset of judge’s element fit.

Only the case where this rule IS the saved judge’s own deployed classifier is accepted: a KMeans fit with abstain_lambda and no abstain_radius, no explicit classifier override (which production ClusterFit.classify would use first) and applied refined group ids. Anything else raises ValueError instead of labelling with a different rule.

class atom_hifi.experimental.judges.PCAPolicy(kind, k=None)[source]

Bases: object

How an embedding operator is chosen: fixed_k (with k) or native_auto.

Parameters:
kind: str
k: int | None = None
classmethod fixed(k)[source]
classmethod native_auto()[source]
property identity: str

Stable identity string; never equal across kinds, whatever the realized dimension.

classmethod from_identity(text)[source]
operator(rows)[source]

The operator this policy selects on rows (see pca_operator()).

class atom_hifi.experimental.judges.PCAOperator(policy, embed_op, dim, eigenvalues, broken_stick_dim, n_rows)[source]

Bases: object

An embedding operator and the eigen-spectrum it was cut from.

embed_op is (D, dim); eigenvalues are all D covariance eigenvalues in descending order; broken_stick_dim is reported for both policies (for fixed_k it is information only).

Parameters:
policy: PCAPolicy
embed_op: ndarray
dim: int
eigenvalues: ndarray
broken_stick_dim: int
n_rows: int
property identity
project(rows)[source]

Uncentred projection rows @ embed_op (decaf embed_PCA).

atom_hifi.experimental.judges.broken_stick(segments)[source]

bs_k = (1/D) sum_{j=k..D} 1/j for k = 1..D (decaf get_broken_stick arithmetic: each suffix is its own np.sum).

Return type:

ndarray

atom_hifi.experimental.judges.broken_stick_dim(eigenvalues)[source]

#{k : bs_k <= lambda_k / sum(lambda)} over all D eigenvalues (descending order).

Return type:

int

atom_hifi.experimental.judges.eigendecomposition(rows)[source]

Descending eigenvalues and eigenvectors of the covariance of the centred rows.

atom_hifi.experimental.judges.pca_operator(rows, policy)[source]

Operator chosen by policy on rows: top-k (fixed_k) or top-d_auto (native_auto) eigenvectors of one eigendecomposition.

Return type:

PCAOperator

atom_hifi.experimental.judges.ARRAY_SHA256_DEFINITION = 'sha256(dtype.str + json(list(shape)) + C-order bytes)'

hash definition used for array identities (matches the stage’s historical array hash)