The entire data-annotation cycle: gold data, inter-annotator agreement, crowdsourcing, validation, LLM annotation, and LLM-as-a-Judge.
The seminar was held for colleagues from several MIRIAI laboratories. The slides were intended as working materials and are not published on this website.
Topics included annotation-quality metrics, gold datasets, expert and crowdsourced annotation, multi-level validation, the use of LLMs, structured instructions, and LLM-as-a-Judge.
