# Experimental: temporal assessment (opt-in)

```{warning}
**Experimental and opt-in.** Nothing here runs unless you call it. `import atom_hifi`,
`atom-hifi init` and `atom-hifi run` are unchanged, and the API may change.
The ratio **B/E is a temporal separation diagnostic. It is not Atom-HiFi's selection
fidelity L/R**, and nothing in this mode fits a model, picks a K or a cutoff, or names a winner.
```

`atom_hifi.experimental.temporal` assesses **saved group labels over time**. Given
labels (frames × entities, `-1` = rejected) from one or more candidate labelings of the *same*
trajectory, it measures:
- how well the groups separate in their visit durations (**B**);
- how much each group's durations change between an early and a late window (**E**).

It then reports exact Pareto fronts over (K, B/E, native acceptance). It needs numpy and the
standard library only. It does not need a trajectory, a fitted model, a descriptor or a
molecular renderer.

## Quickstart

```bash
python -m atom_hifi.experimental.temporal --write-example demo     # synthetic example (installed)
python -m atom_hifi.experimental.temporal --config demo/config.yaml
```

The run writes `demo/results/`. Read it back from Python, independently of where it was written:

```python
from atom_hifi.experimental.temporal import assessment
res = assessment.read_results('demo/results')        # checks schema_version and file sha256
res['fronts'][:3], res['summary']['status_explanations']
```

The same computation from Python, on labels you already have:

```python
import numpy as np
from atom_hifi.experimental.temporal import assessment, record

src = record.SourceSelection('run7/traj.xyz#sha256=<file hash>')   # WHAT was observed
pairs = []
for K in (8, 12, 16):
    labels = np.load(f'labels_K{K}.npy')                             # frames x entities, -1 = rejected
    rec = record.LabelRecord.from_labels(labels, entity_ids=ids, n_groups=K, source=src,
                                         record_id=f'K{K}')
    a = assessment.assess_record(rec, split=labels.shape[0] // 2, candidate_id=f'K{K}', K=K,
                                 namespace='my_system',
                                 descriptor_policy={'cutoff': 5.0, 'image_policy': 'wrapped',
                                                    'reference_cell_policy': 'stored'},
                                 pca_policy='fixed_k:8', preprocessing=('raw',),
                                 exact_budget={'max_seconds': 60})
    pairs += a.pairs
front_rows, relation_rows, summaries = assessment.fronts(pairs)
```

`assessment.run(mapping_or_path)` and the CLI use this same code path. They give
byte-identical tables for the same inputs (verified by the test suite).

## Configuration (YAML or JSON)

Unknown keys are refused. Relative paths resolve against the config file's directory.

| Key | Meaning |
|---|---|
| `output_dir` | Result directory. Existing outputs are refused unless you pass `--overwrite`. |
| `windows.split` | **Explicit** early/late split frame `s`: the windows are `[0, s)` and `[s, n)`. There is no default. |
| `windows.segments` | Optional `[[start, stop], ...]` uniformly sampled row ranges. Rows outside them are unobserved, and every segment boundary censors runs. |
| `preprocessing` | Any of `raw` (primary) and `bridge1` (sensitivity: a one-pass simultaneous bridge of single-frame interruptions inside each segment ∩ window). |
| `candidates`, `weightings` | `tau` (mean duration) and/or `P` (duration distribution, exact W1); `visit` and/or `equal`. |
| `frame_interval`, `time_unit` | Metadata only. Durations are in frames. |
| `exact_budget` | Optional `{max_seconds, max_work}` (finite, positive). It enables exact Level-2b escalation (see below). |
| `exports` | Optional `{descriptions: bool, runpacks: bool}` per-input exports (see Outputs). |
| `inputs[i]` | `path` (2-D integer `.npy`), `candidate_id`, `K`, `namespace`, `descriptor_policy` `{cutoff, image_policy, reference_cell_policy}`, `pca_policy` (`{kind: fixed_k, k}`, `{kind: native_auto}` or an identity string). Optional: `n_groups` (default K), `entity_ids` (list or `.npy`), `record_id`, **`source`** `{id, frame_start, frame_step}`. |

Errors print `error: ...` and exit with status 2. They name the input (`inputs[i] (id, path)`)
and the problem: invalid values, unsupported formats, ambiguous sampling, undeclared or
different source populations, and output collisions.

## Identity: which candidates may share a front

A front compares candidates on **one observed population**. The mode keeps four identities
separate:
1. **Source population.** What was observed. The caller *declares* it: an opaque `source_id`
   for the physical record (for example a trajectory name plus its file hash), and the affine
   frame map `source frame = frame_start + frame_step × row`, together with the entity ids.
2. **Sampling/layout.** Frame count, entity order, segments and sampling metadata.
3. **Analysis context.** The early/late windows, preprocessing, candidate and weighting.
4. **Candidate/model provenance.** namespace, descriptor policy, PCA policy, K and the
   label hash.

The front rules:
- In a context with **two or more** members, every member must declare a source, and the
  sources, populations, layouts and windows must be equal. Otherwise the run stops with
  `IdentityError`.
- Shapes, labels, record ids, candidate ids and file paths are **never** used to infer a
  population. Two trajectories, or two slices of one trajectory, can match in every array
  dimension.
- A single-member context is allowed. Its identity is recorded as `declared` or `undeclared`.
- Different labels of one declared population (different K, different models) are exactly what
  a front compares.

Pooling run tables (`runs.pool_runs`):
- Pooling refuses two record ids that declare the same `source_id` when their observed frames
  and entities overlap. These would be the same observations counted twice under two aliases.
- Identical label content in distinct records is **not** duplication. It is kept, and
  `Runs.notes` records it when provenance is undeclared.

**Limitation.** Only affine frame selections are expressible. Any other selection needs its
own `source_id`, and overlap between differently declared sources is not detected.

## What B and E are

For each preprocessing, and for the complete visits of each window:
- A **visit** is a maximal run of one accepted group by one entity inside one observed
  segment ∩ window, with an exclusive exit, so duration = exit − entry frames.
- A run touching a window edge or a segment boundary is censored. Only complete visits enter
  B and E.
- A rejection (`-1`) ends a visit. An unobserved gap censors it instead.
- Full-window complete visits are never built by concatenating half-window visits.

The quantities:
- **Joint set:** `G_joint = {g : n_g^E ≥ 1 and n_g^L ≥ 1}`.
- **Weights:**
  - `visit`: `w_g = (n_g^E + n_g^L) / Σ_joint (n^E + n^L)`;
  - `equal`: `w_g = 1/|G_joint|`.

  The same weights apply to both windows, B and E.
- `B_tau = ½ Σ_{g,h} w_g w_h (|μ_g^E − μ_h^E| + |μ_g^L − μ_h^L|)` over all ordered pairs.
- `E_tau = Σ_g w_g |μ_g^E − μ_g^L|`.
- `B_P` and `E_P` are the same with the exact empirical W1 distance of the integer duration
  distributions.

**Statuses** (`status_explanations` in `run_summary.json`):

| Status | Meaning |
|---|---|
| `finite` | E > 0 exactly |
| `undefined_no_between_separation_and_no_window_difference` | B = E = 0 |
| `zero_denominator_positive_separation` | E = 0 < B |
| `unavailable_empty_joint_set` | — |
| `numerical_indeterminate` | Production floats contradict the exact identities |

Only `finite` points enter a front. The others are listed in a register and never ranked.

**What B/E does not prove:**
- It is a descriptive contrast of complete-visit durations between groups, relative to their
  early/late drift.
- It does not show that the groups are kinetically distinct states.
- It does not measure model quality, and it is not a substitute for L/R.
- Sampling uncertainty is not quantified. The certification below is arithmetic only.
- A higher ratio is not a recommendation: any K/cutoff choice stays with the analyst.

**Coverage.** Every candidate row carries the exact `coverage_*` integers:
- occupied, complete and joint groups;
- visits and occupancy per window;
- left-only, right-only and both-censored counts;
- rejected occupancy;
- bridge changes (unknown → empty, never 0).

**Native acceptance** (the front's third objective) is accepted raw entity-frames / observed
entity-frames over the whole record, the same for every preprocessing.

## Exact arithmetic and the optional budget

- **Production values.** B and E are computed in canonical float order, from exact integer
  numerators.
- **Level-1 certification.** Every finite row carries binary64 enclosures of the exact B and E.
- **Front comparisons** use, in order:
  1. exact zero identities;
  2. separated Level-1 intervals of `B_a·E_b` versus `B_b·E_a`;
  3. only when `exact_budget` is set, exact `Fraction` arithmetic (Level 2b) within that
     budget.

  A pair that none of these orders stays **`unresolved`**. That is a valid result, never a
  float ranking.
- **Omitting `exact_budget`** is valid: no exact work runs at all.
- **Caching.** Exact components are computed at most once per (input, preprocessing).
  Success and exhaustion are both cached for that run only.
- **`production_inside_enclosure = False`** is not an error. The canonical production float can
  round just outside the enclosure of the exact value; fronts use the enclosure.

## Fronts

- **One front per context:** namespace, descriptor policy, PCA policy, preprocessing, window,
  sample scope, candidate and weighting. Contexts are never pooled.
- **Objectives:** minimise K, maximise the finite B/E, maximise native acceptance.
- **Rows in `fronts.csv`:**
  - each row has a `front_status`: `nondominated`, `dominated` (with `dominated_by`),
    `unresolved` (with `unresolved_with`) or `register`;
  - ratio maximisers are listed with ties.

  `front_relations.csv` lists every pairwise ratio relation with its certification level and
  evidence.
- **Not included:** an elbow, a winner, or a joint temporal/structural score.

## Outputs

| File | Content |
|---|---|
| `candidates.csv` / `.json` | One row per (input, preprocessing, candidate, weighting): B, E, ratio, status, enclosures, coverage, acceptance (numerator/denominator), record fingerprint, label path *as written in the config* |
| `fronts.csv`, `front_relations.csv` | Fronts and certified relations |
| `group_contributions.csv` | Per joint group: exact integers (n, s, ntau, nw1, den, visit mass), μ, Δμ, W1, weights and the four E contributions |
| `run_summary.json` | `schema_version`, resolved config (no absolute paths), input provenance with identities, per-context front summaries and identity records, status explanations, exact-escalation record, environment, sha256 manifest of the other files |
| `descriptions/NN_<id>/` (opt-in) | Per input and per (preprocessing × full/early/late window): `accounting.json`, `aggregates.json`, `groups_*.csv`, `frequencies_*.csv`, `runpacks.npz` (lossless run tables; see `runs.unpack_runs`) and `DESCRIPTION.json` |

Formats:
- Floats are written round-trip exact: `repr` in CSV, JSON numbers. NaN is `nan` in CSV and
  `null` in JSON.
- Descriptions are off by default. Measured cost for one four-input configuration at 14,955
  frames × 736 entities: about 34 s and 36 MB of output, versus about 3 s without them.
- `runs.unpack_runs` treats a runpack as untrusted input and refuses a malformed one with
  `ValueError`:
  - non-integer, non-positive or oversized durations;
  - inconsistent offsets or observed pieces;
  - unknown or incomplete layouts;
  - impossible bridge1 change counts;
  - values whose exact arithmetic would leave int64;
  - two touching runs of one entity with the same label inside one observed piece. Runs must be
    maximal; equal labels across a gap, segment, entity or record boundary are fine.

  It never merges, rounds or drops runs. A `Runs` table constructed directly in Python is trusted
  as given.

## Reusing existing labels or a saved model (no campaign rerun)

**Precomputed labels.** Point `inputs[i].path` at the saved `.npy` arrays and declare their
common `source`.

**A saved KMeans judge.** Embedded rows can be labelled with the judge's own deployed rule:

```python
from atom_hifi.experimental import judges
judge = judges.load_judge('judge_final.pkl', expected_sha256=FILE_SHA, element='O',
                          expected_embed_op_sha256=OP_SHA)           # verified before unpickling
labels = judges.label_from_judge(judge, z, 'O')                     # z: rows embedded with that operator
```

Notes on `label_from_judge`:
- It reproduces the nearest-centroid rule (`q > 1` strictly → `-1`), then the judge's refined
  group order and element offset. Labels are therefore global group ids:
  - use `n_groups = offset + number of groups` for that element;
  - or subtract the offset before building the record.
- It refuses, rather than relabels:
  - other backends;
  - an explicit classifier override;
  - an `abstain_radius` fit;
  - a judge without refined ids.
- Nothing is refitted, and λ is never altered.

**Other opt-in adapters.**
- `atom_hifi.experimental.periodic` (`wrap_atoms`, `corrected_center_fn`): integer-lattice
  wrapping for descriptor evaluation on unwrapped periodic coordinates. Fully periodic cells
  only. ASE constraints do not block the translation.
- `atom_hifi.experimental.descriptor`: ED-SOAP with independently set channel cutoffs.
- `judges.PCAPolicy`: `fixed_k:k` vs `native_auto:broken_stick` identities.

These never replace the production descriptor or the PCA defaults.

## The PIT-WATER example is an example

The mode was developed on a PIT-WATER interface trajectory (O atoms, 14,955 frames). That
study's outcomes are results of that system, its fresh reference cell (46.8533 Å, with an
unverified exact-run provenance) and its labels:
- a 4.8 Å cutoff;
- the O70 discussion;
- its fixed-21 and native-auto fronts.

They are **not** constants, defaults or recommendations of this mode. The generic package
contains no PIT thresholds, indices, frame counts or cells.

## Limitations

- Uniform sampling inside each declared segment only. Irregular timestamps are refused, never
  interpolated.
- Durations are in frames. `frame_interval` is metadata.
- Only affine source-frame selections are declarable (see Identity).
- `B/E` is descriptive. Its statistical uncertainty is not estimated, and fronts certify
  arithmetic only.
- Exact Level-2b escalation can be expensive for many groups with long duration supports. Bound
  it with `exact_budget`, or leave it off and keep `unresolved` pairs.