EvalTriage
class EvalTriage(manifest: CorpusManifest, seed: String = SEED, onClaim: (CorpusIssue, String, Int, String) -> Unit = { _, _, _, _ -> }, onMiss: (CorpusIssue, Fact, Set<String>) -> Unit = { _, _, _ -> })(source)
Scores the free-text extractor on the corpus. A form body (### Label sections, from a repo whose issue template is a form) states its environment under labelled headings; the facts read from those sections are the truth. The sections that gave a fact are cut out of the body and the extractor runs again on what is left, so the tool-output, key-value and pattern paths are judged against what the reporter typed into the form fields. Facts on keys the form did not label cannot be judged and are not scored. Heading-to-key mapping and value normalization are shared between the truth and the extractor, so the section path itself is not measured here.
Constructors
Link copied to clipboard
constructor(manifest: CorpusManifest, seed: String = SEED, onClaim: (CorpusIssue, String, Int, String) -> Unit = { _, _, _, _ -> }, onMiss: (CorpusIssue, Fact, Set<String>) -> Unit = { _, _, _ -> })