Configuration¶

Atom-HiFi is config-driven (ADR-0005, local-only). A single commented YAML template in atom_hifi.config is the source of truth for the public CLI surface: it is written verbatim by atom-hifi init, parsed for YAML defaults, and used as the strict allow-list for validation (unknown keys are rejected). Expert-only controls for ND grid scan and patient/OOD are accepted through the Python API, not YAML.

Build a config for your system¶

The first job of config.yaml is to translate your scientific question into an Atom-HiFi run: which structures define the library, which atoms define coverage, which descriptor measures local environments, and which clustering/search strategy should choose the representative set. Start from the generated template, fill in the system identity fields, then choose the backend and multi-element search path deliberately.

Generate a starter file:

atom-hifi init config.yaml

Then edit the minimum system block:

paths:
  lib_path: train_structs.xyz   # ASE-readable structure library
  focus_elements: [Ni, O]       # elements whose local environments matter
  output_dir: fr_results_NiO
descriptor:
  kind: eds                     # eds for built-in SOAP, ace for MACE ACE

Choose these four values from your system:

Question	Config key	First-run choice
Where is the structure library?	`paths.lib_path`	Path to an ASE-readable file such as `.xyz`, `.extxyz`, or trajectory input
Which atoms should define coverage?	`paths.focus_elements`	The species whose local environments you want represented; use `null` only when all species matter equally
Where should this run write caches and results?	`paths.output_dir`	A fresh directory name for this system and descriptor choice
Which descriptor should be used?	`descriptor.kind`	`eds` unless you already have a MACE model and want ACE descriptors

Applied example: multi-element adsorbate library¶

For a NOx/H adsorbate library on a surface, the scientifically relevant local states may be the adsorbate atoms rather than every support atom. A production config can therefore focus on N/O/H, use ED-SOAP, let HDBSCAN keep distinct noise environments, and use NSGA-II to pick separate c-factors for each element:

paths:
  lib_path: all_structures_cycle-3.xyz
  focus_elements: [N, O, H]
  output_dir: hdbscan_results
descriptor:
  kind: eds
  eds:
    species: null
    lengthscale: 1.1
    periodic: true
cluster:
  backend: hdbscan
  min_cluster_size: 2
selection:
  method: mu_tiebreak
microstate:
  reference: r100
eps_noise:
  method: thermal
scan:
  l_tol: 0.95
  n_coarse: 60
  n_fine: 60
nsga2:
  run: true
  n_gen: 50
  pop_size: 60
piecing:
  enabled: false
output:
  extract_surroundings: true

Read this as a set of modelling choices:

Config group	What it means in this example
`focus_elements: [N, O, H]`	Coverage is judged on adsorbate atoms. Surface/support atoms still appear in descriptors and output structures, but they do not define the set-cover targets.
`descriptor.kind: eds`	Use the built-in ED-SOAP descriptor. `lengthscale: 1.1` Å sets the physical scale for the short/long SOAP cutoffs, so it should be near a typical local bond length in the chemistry being resolved.
`cluster.backend: hdbscan`	Partition the ED-SOAP embedding with a distance-kind backend that can mark sparse/outlier environments as noise; Atom-HiFi deduplicates those noise environments and forces representatives for them.
`microstate.reference: r100`	Use an ε_noise-radius KMeans reference (or `r<NUM>` for a user-defined percentile). This is an opt-in physical assumption: environments inside ε_noise are treated as indistinguishable for the reference. Use the default `n2` when that assumption is uncertain.
`nsga2.run: true`	For three focus elements, choose independent c-factors for N/O/H instead of forcing one common resolution scale.
`output.extract_surroundings: true`	Write one representative environment per final group, useful for inspecting chemical motifs.

The remaining groups control runtime, restart behaviour, and inspection output:

Config group	Applied setting	Meaning
`n_jobs`, `compute.n_jobs_desc`, `compute.n_jobs_eps`	`n_jobs: 60`, `n_jobs_desc: 60`, `n_jobs_eps: null`	Use 60 workers for general parallel stages and descriptor computation; `n_jobs_eps: null` means ε_noise calibration also uses top-level `n_jobs`. Reduce only after checking memory pressure.
`log_level`, `seed`	`info`, `42`	`info` prints stage summaries and progress without debug noise. The seed fixes k-means, scan subsampling, and stochastic optimisation choices.
`scan`	`l_tol: 0.95`, `c_factor_range: null`, `n_coarse: 60`, `n_fine: 60`	Keep the default Likeness floor: selected points must preserve at least 95% of full-library microstate entropy. Set `l_tol: null` to disable that floor and auto-pick the maximum-F point. `c_factor_range: null` scans the standard `[0.1, 5.0]` range; coarse/fine counts set scan resolution.
`loop`	`run: false`	Skip active-learning and run on the full library. All built-in backends support the loop. OOD need not decay to zero, so a loop may consume the full library. HDBSCAN’s promoted noise representatives remain outside its deployable density targets.
`nsga2.restore`	`false`	Recompute the Pareto front. Set `true` only to reuse an existing compatible `pareto_front.csv` in the same output directory.
`piecing`	`enabled: false`	Treat each input structure as one selectable item. This is right for a normal many-structure adsorbate library.
`output.delta_pick`	`null`	Let Atom-HiFi pick the operating point automatically from the scan/NSGA-II result. Set manually only for deliberate refits.
`output.save_judge`	`true`	Write the fitted `Judge` to `judge_final.pkl` after the final fit, so the selection can be re-examined without a refit. The pickle omits the raw descriptor block, which only `extract_rep_surroundings` needs; rebuild it from `desc_lib.pkl` — a list of per-structure arrays, so stack it and keep the focus-element rows — and pass it to `judge.attach_descriptors`. Set `false` to skip the file entirely.
`surroundings`	`output_mode: full_structure`, `r_cut: null`, `n_per_group: 1`	Save the original periodic structure with center/neighbour/rest tags for one representative per group. `r_cut: null` uses the descriptor cutoff.
`refit.delta`	`null`	Run the full scan. Set to a c-factor from `fine_scan.out` only when re-fitting a saved run at a chosen point.

Choosing the backend¶

cluster.backend decides how local atomic states are formed from the embedded descriptor cloud. Pick it from the workflow you need, not from speed alone:

Backend	Use when	Avoid when
`decaf`	You want the default, deployable mean-shift path; you may need active-learning loop or patient/OOD classification.	Very large scans where speed is the main bottleneck and you do not need deployable classification.
`hdbscan`	You want density clustering with noise representatives for selection and frozen density attachment for deployment. See prediction semantics.	Noise representatives must be accepted deployment targets: promotion supplies no density support. NumPy 2.4+ remains refused for the positive-epsilon fitting path.
`kmeans`	You want a fast count-kind selector, compact covers, NSGA-II over per-element k-fractions, or nearest-centroid OOD/patient/loop classification.	You need grid c-factor optimisation or ε-anchored distance-bandwidth interpretation. Its loop may consume the full library when resolution-tracking OOD remains positive.

Distance-kind backends (decaf, hdbscan) sweep c_factor, meaning the actual per-element clustering bandwidth is δ_el = c_el × ε_noise_el. Count-kind backends (kmeans) sweep how many clusters to use, so c-factor logic does not apply; NSGA-II instead optimises an independent per-element k-fraction.

Multi-element search: why NSGA-II is the default¶

With more than one focus element, one shared granularity can be too coarse for one element and too fine for another. nsga2.run: true searches per-element c-factors for distance-kind backends or per-element k-fractions for kmeans, then tags two useful operating points in pareto_front.csv:

Output tag	Meaning
`selected`	Smallest redundancy among points with `L >= scan.l_tol`; this drives the final fit. With `l_tol: null`, it is the `fidelity_best` point.
`fidelity_best`	Highest `F = L/R` on the Pareto front; inspect as a candidate, not automatically the conservative choice.

Use NSGA-II for multi-element production runs with decaf, hdbscan, or kmeans. Set pop_size to at least 10 × N_focus_elements and at least 40; for N/O/H, pop_size: 60 is a reasonable budget. Increase n_gen if pareto_convergence.png has not plateaued. Use nsga2.restore: true only when you want to reuse an existing pareto_front.csv from the same output directory and compatible config.

Microstate reference and ε_noise¶

For a first run on an unfamiliar system, microstate.reference: n2 is the conservative default: per element, it uses roughly two atoms per microstate. Switch only when the physical assumption is clear:

Reference	Meaning	Use when
`n2`	Conservative k=N//2 microstate reference.	Default for unknown, soft, mobile, or mixed systems.
`r100`	Smallest resolved k whose KMeans radius is within ε_noise, searched through `N_atoms-1`. Results finer than n2 are kept with an advisory warning.	Known-rigid systems, or analyses where ε_noise is a valid identity radius.
`r<NUM>`	Same criterion at user percentile NUM (1–100); `r100` = the maximum. With an explicit `eps_noise.force` block (or `--f-tol`) it resolves against ε_F like `f_tol`, overriding that mode’s p95 default; force-resolution then requires thermal ε(σ) as well.	Tuning strictness deliberately. Hazard: the historical r90 criterion under-selected k ~3× (+56 % force error on PdMgO) — a thermal `r<NUM>` < 100 is an explicit user opt-in, not a default.
`f_tol`	Force-resolution reference from `eps_noise.force.f_tol`, searched up to `N_atoms-1`; requires thermal ε(σ).	Expert MLIP-training studies where force tolerance is the reference scale.

eps_noise.method: thermal is the default and measures descriptor motion under QHO thermal rattles. Use precomputed only when you are carrying known ε values from a compatible prior run. Use rdf when you deliberately want the library’s own first-neighbour peak width to set the displacement scale.

When to use piecing¶

piecing.enabled: true is for a specific failure mode: a library made of a few very large structures, often around 1000 atoms each, where every structure already contains almost every microstate. In that case set-cover can return |S| = 1, the c-factor inflates, and fidelity metrics saturate. Piecing cuts each large periodic structure into spatial voxel pieces and runs selection over pieces while keeping descriptors computed on the intact cells.

Use piecing when:

The library has few huge structures, not many ordinary structures.
A non-pieced run selects one or very few structures because each structure already contains all local states.
Per-element group counts collapse, especially for minority elements.

Do not use piecing for a normal molecular/surface-adsorbate library with many separate structures, and do not combine it with loop.run: true or loop.resume_from_iter. Tune piecing.target_atoms by checking per-element group counts at c*: lower target_atoms until each focus element resolves more than one meaningful local state.

If descriptor.kind: eds, the default ED-SOAP settings are meant to run immediately. The one value worth checking early is descriptor.eds.lengthscale: set it near a typical nearest-neighbour bond length in Å for your material. Keep descriptor.eds.periodic: true for crystals and slabs; set it to false for isolated molecules or clusters. If descriptor.kind: ace, set descriptor.ace.model_path to the MACE model file and choose device: cpu or cuda.

Run once before tuning:

atom-hifi run config.yaml 2>&1 | tee run.out

For a long run, launch with unbuffered output and record the process id:

setsid stdbuf -oL -eL atom-hifi run config.yaml > run.out 2>&1 & echo $! > run.pid

Check output artifacts first, then the log. Output files are more reliable completion signals than quiet stretches in the console output:

cat run.pid
ls -lh fr_results_NiO
test -s fr_results_NiO/representatives.xyz && echo "final representatives written"
tail -f run.out

For a standard run, expect these milestones in paths.output_dir:

Artifact	Meaning
`desc_lib.pkl`	Structure descriptors were computed or loaded from cache
`eps_noise_raw.npz`	ε_noise calibration finished, unless the chosen path skips it
`fine_scan.out`	Final L/R scan table was written
`hifi_final.png`	Final scan diagnostic plot was written
`representatives.xyz`	Final representative structures were selected
`judge_final.pkl`	The fitted `Judge` was saved (unless `output.save_judge: false`)

Optional stages add their own completion signals: learning_curve.png and descp_iter*.npy for loop.run: true, pareto_front.csv and Pareto plots for nsga2.run: true, and surroundings*.xyz when output.extract_surroundings: true.

Common failures usually point back to one config group:

Symptom	Check
`Unknown config key`	The key is not part of the YAML surface; remove it or use the Python API for expert-only controls
Input file error	`paths.lib_path` must point to an ASE-readable library from the run directory
Missing QHO-table element	Use `eps_noise.thermal.sigma_source: mass_only` or set `eps_noise.thermal.sigma_override` for that element
Out-of-memory or killed process	Check memory before reducing workers; descriptor work follows `compute.n_jobs_desc`, ε_noise calibration follows `compute.n_jobs_eps` or top-level `n_jobs`
Cache unreadable or incompatible	Atom-HiFi logs the cache problem and recomputes; keep caches only when descriptor, backend, and ε_noise settings match

For follow-up runs, keep the same paths.output_dir: descriptor and ε_noise caches are discovered there automatically when compatible. Use scan.cache_file only for a compatible prior el_fit_cache.npz from the same descriptor/backend/focus-element setup. Use loop.resume_from_iter only for active-learning loop checkpoints; it does not resume an arbitrary interrupted final scan. Current checkpoints include descp_iterNN_source_idx.npy and descp_iterNN_source_hash.npy, which record and verify the exact ordered source structures (including OOD-ranked batches); legacy checkpoints without both files must be regenerated rather than reconstructed by random-sampler replay.

Starter template¶

Generate a commented config.yaml with:

atom-hifi init            # writes ./config.yaml
atom-hifi init my.yaml    # or a chosen path

The template below is emitted by default_config_yaml() and documents every option inline.

atom_hifi.config.default_config_yaml()[source]

Return the canonical commented config template (verbatim init output).

Return type:: str

Loading and validation¶

atom_hifi.config.load_config(path)[source]

Load a user YAML config, merge it under the template defaults, and validate.

Strict: any key not present in the template tree raises ValueError; a custom descriptor kind is rejected with a pointer to the Python API.

Parameters:: path (str)
Return type:: dict

atom_hifi.config.fill_defaults(config)[source]

Return config deep-merged under runtime defaults (no YAML validation).

Used by atom_hifi.runner.run() so callers may pass a partial dict. Python API callers may use API-only controls such as grid and patient; those keys are deliberately rejected by load_config().

Parameters:: config (dict)
Return type:: dict

load_config() rejects YAML keys absent from the starter template, including the API-only controls grid, patient, paths.patient_path, and refit.grid_point. fill_defaults() is the Python API path and supplies hidden defaults for those controls before atom_hifi.runner.run() consumes the config.

Custom descriptor¶

descriptor.kind accepts eds (ED-SOAP) or ace (MACE ACE) in YAML. A custom descriptor is a Python callable and cannot be expressed in YAML — call atom_hifi.runner.run() from Python with config['descriptor']['custom_fn'] set to fn(List[Atoms]) -> ndarray.