Contact Addresses

There is a moderated mailing list mrp-users@nlpl.eu for task participants (or candidate future users of the data and evaluation software).  All interested parties are kindly asked to self-subscribe to this list.  The archives of the mailing list are open to the public (including all posts regarding the predecessor 2019 shared task), as the list is intended both for updates from the organizers and for general discussion among participants.

Additionally, the organizers of the MRP 2020 parsing task can be reached at the following address:

   mrp-organizers@nlpl.eu

News & Updates

August 30, 2020
Preliminary task results are now available; we are grateful to the eight teams who have submitted parser outputs.  It appears that MRP 2020 results will be advancing the state of the art for most of the frameworks.
July 30, 2020
Two deficiencies in the syntactic companion parses for the cross-lingual track have been reported: 417 missing German parses, and incorrect spacing in five Czech parses.  New files deu.mrp and ces.mrp are available for download.
July 20, 2020
The evaluation period will open on Monday, July 27, 2020 (one week from today), and will be hosted in the on-line CodaLab service.  We already invite prospective participants to register with CodaLab and make a trial submission there.
June 22, 2020
We hope that we have packaged the final version of the MRP 2020 training data, now available via the LDC.  To compensate for delays in producing some of the companion data, we have extended the evaluation period by one week, to July 27 through August 10, 2020.
June 18, 2020
The ‘white-list’ of third-party data, models, and tools that can be used in addition to the MRP 2020 training, validation, and companion data is now finalized.  Please observe the caveat regarding pre-trained, off-the-shelf syntactic parsers.
June 4, 2020
An updated MRP 2020 data package is now available for download via the LDC, providing new meaning representations graphs for additional languages and high-quality morpho-syntactic syntactic dependency parses.
May 25, 2020
We are running a few days late in wrapping up the cross-lingual training data, as well as the optional syntactic ‘companion’ dependency trees, for release via the LDC; we have pushed back the release date to Monday, June 1.
April 28, 2020
English training and validation data is now available for no-cost licensing through the Linguistic Data Consortium (LDC).  The pages summarizing data and tools and evaluation have been updated accordingly.
March 30, 2020
The initial task web site is on-line, a starting data package is available, and the first call for participation has been posted to major mailing list.  Please sign up for the task mailing list to receive continuous updates.

Task Co-Organizers

Acknowledgements

We are grateful to the Nordic Language Processing Laboratory (NLPL), which provides technical infrastructure for the MRP 2020 task.  Also, we warmly acknowledge the assistance of the Linguistic Data Consortium (LDC) in distributing the training data for the task to participants at no cost to anyone.

Several colleagues have assisted in designing the task and preparing its data and software resources.  Dan Flickinger (Stanford University) created fresh gold-standard annotations of some 1,000 WSJ strings, which form part of the EDS evaluation graphs in 2020.  Sebastian Schuster (Stanford University) advised on how to convert the gold-standard syntactic annotations from the venerable PTB and OntoNotes treebanks to Universal Dependencies, version 2.x, using ‘modern’ tokenization.  Milan Straka (Charles University in Prague) made available an enhanced version of his UDPipe parser and assisted in training Czech, English, and German morpho-syntacic parsing models (for the MRP companion trees).  Jayeol Chun (Brandeis University) provided invaluable assistance in conversion of the Chinese AMR annotations, preparation of the Chinese morpho-syntactic companion trees, and provisioning of companion alignments for the English AMR graphs. 

XHTML 1.0 | Last updated: 2020-10-04 (18:10)