core/dev.gmitch215.drift.plan

Package-level declarations

Types

data class Arm(val label: String, val flips: List<String>)

data class Belief(val masses: List<Mass>)

A distribution over hypotheses in micro-units that sums to exactly 1_000_000. UNKNOWN is the mass for "none of these", so the planner never assumes the cause is in the list.

Branch

data class Branch(val outcome: String, val probability: Long, val posterior: Belief)

Control

data class Control(val kind: InterventionClass, val reason: String)

ControlRow

data class ControlRow(val prefix: String, val environments: Set<Environment>, val reason: String)

Where an attribute whose path starts with ControlRow.prefix can be set, and why.

Controls

class Controls(rows: List<ControlRow>)

The attributes Drift can set itself, per environment, by longest path prefix. A path no row claims is never assumed controllable. The standard table is hand-set from the plan's intervention classes.

Cost

data class Cost(val minutes: Long, val runnerMinutes: Long, val trials: Long)

Minutes of waiting, runner minutes billed and trials run; whole units. The weights that turn these into one cost are in PlanWeights.

CostWeight

data class CostWeight(val id: String, val perUnit: Long, val reason: String)

Cost points per whole unit, in micro-points, and why.

Decision

enum Decision : Enum<Decision>

Environment

enum Environment : Enum<Environment>

Excluded

data class Excluded(val id: String, val reason: String)

Experiment

data class Experiment(val id: String, val kind: InterventionClass, val flips: List<String>, val dimensions: List<String>, val cost: Cost, val table: OutcomeTable, val arms: List<Arm>, val instructions: String)

Treatment arm: the passing side with Experiment.flips (hypothesis ids) set to the failing side's values; control arm: the passing side unchanged. Experiment.table holds what each hypothesis predicts.

Fisher

object Fisher

Frame

data class Frame(val kind: FrameKind, val passing: String, val failing: String, val environment: Environment)

What the two sides of the comparison are. Frame.passing and Frame.failing are labels used in the printed instructions; Frame.environment is where the arms run. The sides may be two commits in one place or one commit in two places (a local run against a CI run).

FrameKind

enum FrameKind : Enum<FrameKind>

Hypothesis

data class Hypothesis(val id: String, val members: List<String>)

A candidate cause flipped as one unit; Hypothesis.members are attribute paths or step: ids.

Information

object Information

Entropy and expected information gain in micro-bits (1_000_000 is one bit). Every figure is an integer: log2 is exact for powers of two and otherwise within 2 micro-bits of the true value (the error of FixedPoint.ln plus one division rounded to nearest), and each entropy term is rounded to nearest, ties up.

InterventionClass

enum InterventionClass : Enum<InterventionClass>

How an experiment runs, by who controls the manipulated attributes.

Mass

data class Mass(val id: String, val micro: Long)

Options

data class Options(val trialMinutes: Long, val failures: Int, val runs: Int, val alpha: Long = PlanWeights.ALPHA, val power: Long = PlanWeights.POWER, val maxPerArm: Int = Trials.MAX_PER_ARM, val unknown: Long = PlanWeights.UNKNOWN_PRIOR, val couplings: Map<String, String> = COUPLINGS)

Per-run inputs. Options.failures of Options.runs is the observed failure rate of the red side; Options.trialMinutes is the wall time of one trial of the failing command. Options.couplings maps an attribute to the hypothesis it always moves with, for one change that shows as several. A target step:NAME without a job names the first added step called NAME. The default merges the provisioner build date with its version and the Node and npm versions with Set up Node.

OutcomeTable

class OutcomeTable(val outcomes: List<String>, val rows: Map<String, List<Long>>)

What each hypothesis predicts for each outcome of an experiment: OutcomeTable.rows maps a hypothesis id to its outcome probabilities in micro-units (they sum to 1_000_000). A hypothesis with no row says nothing about the outcome and gets a uniform one.

Plan

data class Plan(val frame: Frame, val options: Options, val trials: TrialPlan, val hypotheses: List<Hypothesis>, val prior: Belief, val status: Status, val stuck: List<StuckReason>, val experiments: List<Planned>, val unseparated: List<Unseparated>, val excluded: List<Excluded>)

Experiments ranked by information gain per cost, with the posterior for every outcome of each. Plan.experiments lists the ranked ones first, then the rejected, then the manual ones. The full detail is always present; how much to show is the renderer's choice.

Planned

data class Planned(val experiment: Experiment, val gain: Long, val points: Long, val perCost: Long, val decision: Decision, val reason: String, val branches: List<Branch>)

Planned.perCost is micro-bits per cost point; Planned.points is the weighted cost in micro-points.

Planner

object Planner

One-step look-ahead experiment planner. Experiments come from Zeller's ddmin run over the controllable hypotheses: for every possible single culprit, the subsets ddmin would test are proposed, so the pool holds the full bundle, its halves, quarters and single members. Each arm sets a subset to the failing side's values and holds the rest at the passing side's. What cannot be set from here is listed as a manual experiment and never as an automatic one.

PlanWeights

object PlanWeights

Every number the planner chooses by hand, in one place, like the ranker's weights: reasoned guesses, not calibrated. They rank experiments against each other; a cost point is not money and an information gain from an uncalibrated prior is not a measured quantity.

PValue

data class PValue(val numerator: Long, val denominator: Long)

An exact probability numerator / denominator, both from binomial coefficients.

Status

enum Status : Enum<Status>

StuckCode

enum StuckCode : Enum<StuckCode>

StuckReason

data class StuckReason(val code: StuckCode, val detail: String)

TrialPlan

data class TrialPlan(val rate: Long, val alpha: Long, val targetPower: Long, val minimum: Int?, val perArm: Int?, val threshold: Int?, val power: Long)

The trial count for an arm pair. TrialPlan.perArm is null when no count up to the budget reaches TrialPlan.targetPower; TrialPlan.power is then the best reached, at TrialPlan.threshold failures for that count. TrialPlan.minimum is the fewest trials per arm at which alpha is reachable at all.

Trials

object Trials

Unseparated

data class Unseparated(val ids: List<String>, val reason: String)

Hypotheses no controllable experiment can tell apart, after every proposed one has run.