core/dev.gmitch215.drift.stats

Package-level declarations

Types

A percentile bootstrap on the ratio of medians. Each resample draws as many values as the side holds, with replacement, from the sorted samples; the ratio is the median of the second side over the median of the first, in permille, rounded half up. The generator is xorshift64 (shifts 13, 7 and 17) seeded from the SHA-256 of both sorted samples, so the interval is a function of the stored data alone and no clock or system random is read.

Cgroup

object Cgroup

What the cgroup counters say when two sets of samples differ: throttled CPU time and memory events summed over the samples of each side. A side that has no reading says nothing.

CliffsDelta

object CliffsDelta

Cliff's delta in permille: the share of pairs where the second side is larger, minus the share where it is smaller, scaled by 1000 and rounded half away from zero. The magnitude labels use the thresholds that Romano and others (2006) published for the absolute value: below 147 is negligible, below 330 small, below 474 medium, and anything else large.

ExactP

data class ExactP(val extreme: Long, val total: Long, val smallest: Long)

extreme of the total splits are at least as extreme as the observed one. smallest is the count at the widest possible distance, so smallest / total is the lowest p-value these sample sizes and ties can reach.

MannWhitney

data class MannWhitney(val first: Int, val second: Int, val greater: Long, val less: Long, val equal: Long, val exact: ExactP?)

The Mann-Whitney U statistic of two samples of whole numbers, counted from pairs: greater pairs have the second value above the first, less below, equal tied. A tie counts half in twoU, so every number stays an integer. exact is null above EXACT_LIMIT samples a side.

Metric

class Metric(val name: String, val unit: String, val verb: String, val what: String)

What one measurement is called, its unit, and how the larger side is described.

MetricComparison

class MetricComparison(val metric: Metric, val a: Spread, val b: Spread, val measuredA: Int, val measuredB: Int, val test: MannWhitney, val interval: RatioInterval)

One measurement of run A against run B: the spreads, Mann-Whitney U with its exact p-value, Cliff's delta and the bootstrap interval on the ratio of the medians (B over A). The verdict names a direction only when the interval leaves 1 and the exact p-value, when it is computed, is at most 0.05.

RatioInterval

data class RatioInterval(val low: Long, val high: Long)

A 95 percent interval on the ratio of two medians, in permille, from the bootstrap.

RunComparison

class RunComparison(val metrics: List<MetricComparison>, val cgroup: List<String>, val limits: List<String>)

Two runs that carry samples, compared on wall time and, where both sides measured them, CPU time and peak memory. CPU time is left out when both medians are 0 ms, because a ratio of two zeros says nothing. cgroup says when the container throttled or ran out of memory on one side and not the other; limits are the limits of drift diagnose, from the same code.