🧪 Consolidation Evaluation

Three exercises measure the sweep's quality: audit the discarded mentions (blind — the model's reason is revealed after you decide), the largest entities (over-merging) and the most similar entity pairs (under-merging); a fourth mode audits the discards of sweep prompt v004. Modes 5 and 6 audit the Step-2b repair that runs after the sweep: the merges prompt v001 proposed (mixed blind with pairs it refused) and the entities prompt v002 cut into pieces. Mode 8 audits sweep prompt v006 on France, Brazil, China and Angola, one question per screen. Modes 9 and 10 re-annotate the results of Step 2b on the production sweep v007: the merges and splits it applied, mixed blind with those it refused. Modes 11 and 12 (19 October) are the shorter version of 9 and 10 to annotate now: one pooled sample over four countries (choose ALL), lighter cards and one verdict per split. Items flagged shared are annotated by both annotators. Enter your name to start or resume.

Existing sessions

Loading…