Presenting “Evaluating Learning Language Models” at CLEF 2015

I presented a short paper at the 6th CLEF 2015 Conference and Labs of the Evaluation Forum in Toulouse on the deliberations from the workshop on “Evaluating Learning Language Representations” from last fall, where the challenges of evaluating the quality of a moving target was discussed by Jimmy Callin, Kevyn Collins-Thompson, Amaru Cuba Gyllensten, Ariel Ekgren, David Jürgens, Anna Korhonen, Fredrik Olsson, Magnus Sahlgren, Hinrich Schütze, and myself. The proposals from the workshop are in the paper – most importantly the basic principle that an evaluation of learning systems should evaluate the learning process rather than outcome of the learning process.

Advertisements