The WindowDiff evaluation measure [12] is becom- ing the standard criterion for evaluating text segmentation methods. Nevertheless, this metric is really not fair with re- gard to the characteristics of the methods and the results that it provides on different kinds of corpus are difficult to compare. Therefore, we first attempt to improve this mea- sure according to the risks taken by each method on differ- ent kinds of text. On the other hand, the production of a segmentation of reference being a rather difficult task, this paper describes a new evaluation metric that relies on the stability of the segmentations face to text transformations. Our experimental results appear to indicate that both pro- posed metrics provide really better indicators of the text seg- mentation accuracy than existing measures.
Citation:
Sylvain Lamprier, Tassadit Amghar, Bernard Levrat, Fr?d?ric Saubion, "On Evaluation Methodologies for Text Segmentation Algorithms," ictai, vol. 2, pp.19-26, 19th IEEE International Conference on Tools with Artificial Intelligence - Vol.2 (ICTAI 2007), 2007