Deep Parser Performance Evaluation
- Beyond PARSEVAL
- Workshop at LREC-98
- Workshop Series at Coling 2000
- School of Cognitive and Computing Sciences (COGS)
- Natural Language Theory and Technology (NLTT)
- Tsujii Laboratory
- Language Technology Lab (LT Lab)
- Institute for Research in Cognitive Science (IRCS)
- Gregor Erbach
- Daniel P. Flickinger
- Hans Uszkoreit
- Kentaro Torisawa
- Jun-ichi Tsujii
- Aravind K. Joshi
- John A. Carroll
- Bernd Kiefer
- Ulrich Callmeier
- Ann Copestake
- Ronald M. Kaplan
- Stephan Oepen
- XTAG
- EVALB
- XLE: Xerox Linguistics Environment
- incr tsdb()
- LILFES
- Robust Accurate Statistical Parsing Toolkit (RASP)
- PET
The task of parser evaluation is to measure the speed and accuracy of parsers with respect to a manually parsed test corpus. In the evaluation of so-called deep parsers, which are based on comprehensive linguistic theories, the assignment of the correct semantic analysis plays an important role. Another aspect is the handling and reduction of ambiguities, which is important not only from the linguistic point of view but because it plays a major role in the time and memory requirements of deep parsers. Test data can either be a treebank constructed from naturally occurring data, such as the "Wall Street Corpus", or a "test suite" containing constructed examples which cover all interesting linguistic phenomena. An important issue in deep parser evaluation is the comparison of parsers and grammars based on different linguistic theories.