evaluate$select

evaluate$select() ranks the models of an existing evaluation, per outcome, on one metric or a weighted composite of several.

A data.table with columns outcome, model, score, rank and tied, one row per outcome and per model present in that outcome's scoring rows under any scoring metric, not only the ones requested. A model reachable only through a metric outside the composite still gets a row, with score and rank NA. model holds model names, not indices. It is a plain table, not a PolyGeniusEvaluation.

The steps, per outcome:

1. Scoring rows. Only rows with stratum == "all", no model.ref, no tier, analysis performance or incremental, and a registered direction of "higher" are used. A metric name resolves among these rows only, so delta.auc is incremental's. A requested metric that is not a performance or incremental metric, or that no scoring row carries, aborts. So does an empty metrics, and so does one resolving to more than one row per outcome and model, checked for every requested metric or, under the NULL default, every default metric of either type. So do results with no scoring rows. A requested metric not defined for an outcome's type is skipped for that outcome; an outcome with none of them defined gets no rows. A requested metric that is defined for an outcome's type but has no row for that particular outcome is kept in the composite, so every model of that outcome gets an NA score and rank (see Score). The NULL default never does this: it is restricted, per outcome, to the default metrics actually present for it.
2. Scaling. Each metric is min-max scaled across the models, (x - min) / (max - min). When max == min every model scores 1 on that metric.
3. Score. The weighted mean of the scaled metrics present for that outcome. A model missing any of them gets an NA score.
4. Rank. rank(-score, ties.method = "min"). Equal scores share the best rank; an NA score gives an NA rank.
5. Tied. TRUE for the top model, the first rank-1 model in the results' model order. For every other model, rank-1 ties included, TRUE or FALSE from a compare contrast on the "all" rows between it and the top model, in either orientation, with a non-NA interval: TRUE when lower <= 0 <= upper. NA when no such contrast exists. When a pair has several compare metrics, the first of delta.auc, delta.r2.liability and delta.r.squared is read. A model with an NA score has NA.

The score is relative to the candidate set: adding or removing a model rescales every metric. It is a ranking, not an estimand, and has no interval. The default metrics are correlated, so discrimination counts more than once. tied = TRUE means the contrast is not distinguishable from 0 at the conf.level the compare was run with, per comparison and unadjusted. It is not evidence of equivalence. Selecting and reporting on the same samples overstates the winner: select on one sample split and profile the selected names on a disjoint split. The split keeps relatives together, and split.by is not a substitute for it (see evaluate).

evaluate$compare() for the contrasts behind tied; evaluate$profile() to produce results; visualize$evaluate$profile() to plot the score beside its metrics.

Other evaluate-components: evaluate, evaluate.association(), evaluate.compare(), evaluate.incremental(), evaluate.performance(), evaluate.profile(), evaluate.redundancy(), evaluate.stratification()