---
title: "Auditing a SOM ensemble without forcing classes"
author: "Shaowen Ye"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Auditing a SOM ensemble without forcing classes}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

## The question comes before the class map

A self-organizing map can preserve an environmental gradient even when a hard
partition of its units is not reproducible. `SOMevidence` therefore keeps four
tasks separate:

1. represent multivariate structure;
2. audit mapping and topology;
3. test candidate partitions across an ensemble; and
4. make a conditional decision using requirements specified for the study.

The example uses simulated clusters so that the data-generating structure is
known. Real data never provide this kind of ground truth by default.

```{r data}
library(SOMevidence)

set.seed(1)
dat <- simulate_som_scenario(
  "clusters", n = 90, p = 6, seed = 10,
  id = paste0("example_", seq_len(90))
)
dat
```

The simulated class label is stored only in metadata. It is not part of the SOM
input and is not used below.

## Preserve the sampling design

Rows are subsampled here because they are independent by construction. For
field surveys, `group_subsample`, `block_subsample` or `leave_domain_out` should
be used when sites, transects, individuals, years or campaigns define the
independent unit.

```{r design}
resamples <- som_resamples(
  dat,
  method = "subsample",
  repeats = 3,
  prop = 0.8,
  seed = 20
)

spec <- som_spec(
  grids = list(c(3, 2), c(4, 3)),
  seeds = c(30, 31),
  rlen = 40,
  k = 2:4
)
```

This creates 12 attempted models: three resamples, two grids and two random
initializations. Every transformation, mean and standard deviation is learned
from the analysis rows of its own split.

```{r fit}
ensemble <- fit_som_ensemble(
  dat,
  spec,
  resamples,
  preprocess = som_preprocess("identity", center = TRUE, scale = TRUE)
)
ensemble
```

## Audit representation and partitions separately

```{r audit}
audit <- audit_som(ensemble)
audit
audit$grid_summary

successful_ids <- vapply(
  Filter(function(fit) isTRUE(fit$success), ensemble$fits),
  `[[`, character(1), "id"
)
representation <- audit_som_representation(
  ensemble,
  pairs = data.frame(
    fit_a = successful_ids[[1]],
    fit_b = successful_ids[[2]]
  ),
  neighbourhood_size = 8
)
representation$pairwise

partitions <- partition_som(ensemble)
partitions
partitions$stability

consensus <- consensus_som(partitions, k = 3)
consensus
consensus$cluster_summary

references <- fit_cross_models(
  ensemble,
  methods = c("kmeans", "ward"),
  k = 2:4
)
comparison <- compare_cross_models(partitions, references)
comparison$summary
```

Quantization error, topographic error and empty-unit rate describe different
properties of each representation. The experimental cross-fit audit describes
whether prespecified maps preserve similar sample topology; it supplies no
ranking or automatic selection. ARI describes agreement between partitions; it
is not classification accuracy and it does not establish ecological truth.

Pareto selection retains candidates that are not dominated across the stated
criteria. It does not collapse the evidence into a weighted score or a model
leaderboard. Because the fitted ensemble uses distinct resamples, fit-level
metrics are not compared directly. Configuration summaries must cover the same
successful resamples and random starts. All planned fits succeeded in this
worked example before the grid-level medians are compared.

```{r pareto}
stopifnot(audit$success_rate == 1)
pareto_candidates(
  audit$grid_summary,
  metrics = c(
    median_quantization_error = "min",
    median_topographic_error = "min",
    median_empty_unit_rate = "min"
  )
)
```

## A conclusion may be withheld

Without a prespecified gate, the package reports that defensibility has not
been assessed.

```{r no-gate}
assess_defensibility(audit, partitions, k = 3)
```

The following thresholds are deliberately permissive and serve only to show
the interface. They are not package defaults or general ecological standards.

```{r gate}
illustrative_gate <- som_gate(
  max_topographic_error = 0.5,
  max_empty_unit_rate = 0.5,
  min_median_ari = 0.2,
  min_success_rate = 0.95
)

assess_defensibility(
  audit,
  partitions,
  k = 3,
  gate = illustrative_gate,
  consensus = consensus,
  cross_model = comparison
)
```

A failed requirement yields `"abstain"`; unavailable evidence yields
`"uncertain"`. Passing an analyst-defined gate supports only the stated
partition under the stated analysis design. It does not convert an
unsupervised partition into a verified ecological type.

Because this simulation has known labels that were excluded from training, we
can compare them after the workflow. Real ecological categories should not be
treated as ground truth merely because they are available.

```{r external-labels}
evaluate_external_labels(consensus)
```

## Governed real-data examples

`open_data_registry()` lists versioned datasets selected for distinct purposes
in ecology, environmental monitoring and evolution. The package never
downloads these data during installation or examples.

```{r registry}
open_data_registry()[, c("id", "domain", "role", "license")]
```

## Compare prespecified analysis choices on shared samples

`run_som_sensitivity()` retains each evidence stream separately and also
compares scenario consensus labels on shared sample identifiers. A sample
enters this comparison only when at least two ensemble members assigned it in
both scenarios. The resulting ARI and AMI describe sensitivity to the stated
analysis choice; they are not accuracy measures. A second table asks whether
the set of observations clustered with each focal sample persists across the
same scenarios. The shared-universe Jaccard isolates changes in the partition
among jointly evaluable samples. The all-members Jaccard also includes eligible
members found in only one scenario, so it combines coverage and repartitioning
effects. Both are label-invariant, require at least two jointly evaluable
samples and need no preferred reference scenario.

```{r sensitivity}
sensitivity <- run_som_sensitivity(
  list(
    standardized = list(data = dat, spec = spec),
    unscaled = list(
      data = dat,
      spec = spec,
      preprocess = som_preprocess(center = FALSE, scale = FALSE)
    )
  ),
  cross_models = "ward",
  keep_workflows = FALSE
)

sensitivity$scenario_comparison
head(sensitivity$sample_summary)
plot(sensitivity)
```

`sample_summary` keeps the number and coverage of evaluable contrasts beside
both Jaccard summaries. `contrast_coverage` uses all prespecified scenario pairs
that requested the candidate `k`; `conditional_contrast_coverage` uses only
pairs for which global ARI/AMI were evaluable. Low coverage is missing evidence,
not stable membership. Likewise, a large value is conditional on the
prespecified scenarios and must not be interpreted as a probability that the
assignment is correct. Set `sample_profiles = FALSE` when only global scenario
comparisons are needed for a very large scenario-by-sample design.

When scenarios contain different feature sets, row subsets or reordered
observations, supply stable `id` values (or explicit non-positional row names)
to `som_data()`. Locally generated row numbers are not accepted as evidence
that observations match across different data objects.
For any legacy `som_data` object without `id_source`, rebuild the object with
`som_data(..., id = old$metadata$id)` to confirm that those identifiers are
intentional before comparing scenarios.