core/dev.gmitch215.drift.rank/Weights

Weights

Every number the ranker uses to compare candidates, in one place. The evidence weights are hand-set guesses and are NOT calibrated: they order candidates and a score is not a probability. The fit on the DriftBench dev split (bench/data/calibration/fit.json) tried fitted dimension priors, a fitted probe and rule ratio, a fitted NONE and isotonic maps, and no fitted weight replaced these values, so they stay.

What the fit did support is a probability for a ranking whose candidates rest on their attribute change alone (probabilities): each candidate's share of the posterior against its rivals and NONE, read through the isotonic map, which gives the rate at which candidates with that share were the cause. The map is fit on every scenario of FIT and is not held out; out of fold, over the rankings that print a probability, it scored a Brier of 0.170 against 0.173 for saying one in n and 0.184 for the shares alone. The values are rates, not shares, and a ranking's values need not add up to one. A ranking with a probe, rule or spectrum item gets no probability: on rankings with a probe the shares with the hand-set probe ratio scored worse out of fold than saying one in n, and a fitted ratio scored no better. No dev scenario has a history, so spectrum weights are untested.

Properties

all

val all: List<Weight>

CALIBRATED

const val CALIBRATED: Boolean = false

dimensions

val dimensions: Map<Dimension, Weight>

FIT

const val FIT: String

const val ID: String

map

val map: List<Pair<Long, Long>>

The isotonic map from a share to a probability, both in micro-units: a share takes the value of the last step whose first share is at most it, the first step if none is.

MAX_RUNS

const val MAX_RUNS: Int = 3000

The most runs the Ochiai score takes; a longer history gets no spectrum.

MIN_CORRELATED

const val MIN_CORRELATED: Long

A spectrum score below this (half of perfect) never makes a candidate correlated.

MIN_PASSES

const val MIN_PASSES: Int = 2

Fewer included passing runs than this say nothing: one pass makes every change perfect.

NONE

const val NONE: Long

Weight of "no listed candidate is the cause" in the shares map reads; it is not fit.

probe

val probe: Weight

probeDimension

val probeDimension: Weight

probeObservational

val probeObservational: Weight

ruleObservational

val ruleObservational: Weight

rules

val rules: Map<Severity, Weight>

spectrum

val spectrum: Weight

Ratio of a perfect score at full coverage; lower scores and coverage scale it toward 1.

Functions

mapped

fun mapped(share: Long): Long

The value of map at share.

probabilities

fun probabilities(scores: List<Long>): List<Long>

shares of a ranking's scores, each read through map.

fun shares(scores: List<Long>, none: Long = NONE): List<Long>

Each score's share of the posterior, in micro-units: exp(score) over none plus the sum of exp over every score. The shares of a ranking add up to less than one.

spectrumRatio

fun spectrumRatio(score: Long, coverage: Long): Long

spectrum scaled by an Ochiai score and a history coverage, both in micro-units.

toJson

fun toJson(): JsonObject