Contact Addresses

There is a moderated mailing list mrp-users @ nlpl.eu for task participants (or candidate future users of the data and evaluation software). All interested parties are kindly asked to self-subscribe to this list. The archives of the mailing list are open to the public (including all posts regarding the predecessor 2019 shared task), as the list is intended both for updates from the organizers and for general discussion among participants.

Additionally, the organizers of the MRP 2020 parsing task can be reached at the following address:

mrp-organizers @ nlpl.eu

News & Updates

August 30, 2020: Preliminary task results are now available; we are grateful to the eight teams who have submitted parser outputs. It appears that MRP 2020 results will be advancing the state of the art for most of the frameworks.
July 30, 2020: Two deficiencies in the syntactic companion parses for the cross-lingual track have been reported: 417 missing German parses, and incorrect spacing in five Czech parses. New files deu.mrp and ces.mrp are available for download.
July 20, 2020: The evaluation period will open on Monday, July 27, 2020 (one week from today), and will be hosted in the on-line CodaLab service. We already invite prospective participants to register with CodaLab and make a trial submission there.
June 22, 2020: We hope that we have packaged the final version of the MRP 2020 training data, now available via the LDC. To compensate for delays in producing some of the companion data, we have extended the evaluation period by one week, to July 27 through August 10, 2020.
June 18, 2020: The ‘white-list’ of third-party data, models, and tools that can be used in addition to the MRP 2020 training, validation, and companion data is now finalized. Please observe the caveat regarding pre-trained, off-the-shelf syntactic parsers.
June 4, 2020: An updated MRP 2020 data package is now available for download via the LDC, providing new meaning representations graphs for additional languages and high-quality morpho-syntactic syntactic dependency parses.
May 25, 2020: We are running a few days late in wrapping up the cross-lingual training data, as well as the optional syntactic ‘companion’ dependency trees, for release via the LDC; we have pushed back the release date to Monday, June 1.
April 28, 2020: English training and validation data is now available for no-cost licensing through the Linguistic Data Consortium (LDC). The pages summarizing data and tools and evaluation have been updated accordingly.
March 30, 2020: The initial task web site is on-line, a starting data package is available, and the first call for participation has been posted to major mailing list. Please sign up for the task mailing list to receive continuous updates.

Task Co-Organizers

Acknowledgements

We are grateful to the Nordic Language Processing Laboratory (NLPL), which provides technical infrastructure for the MRP 2020 task. Also, we warmly acknowledge the assistance of the Linguistic Data Consortium (LDC) in distributing the training data for the task to participants at no cost to anyone.

Several colleagues have assisted in designing the task and preparing its data and software resources. Dan Flickinger (Stanford University) created fresh gold-standard annotations of some 1,000 WSJ strings, which form part of the EDS evaluation graphs in 2020. Sebastian Schuster (Stanford University) advised on how to convert the gold-standard syntactic annotations from the venerable PTB and OntoNotes treebanks to Universal Dependencies, version 2.x, using ‘modern’ tokenization. Milan Straka (Charles University in Prague) made available an enhanced version of his UDPipe parser and assisted in training Czech, English, and German morpho-syntacic parsing models (for the MRP companion trees). Jayeol Chun (Brandeis University) provided invaluable assistance in conversion of the Chinese AMR annotations, preparation of the Chinese morpho-syntactic companion trees, and provisioning of companion alignments for the English AMR graphs.

Definitions	Frameworks	Objectives	Data & Tools	Evaluation	Results	News & Contact
Contact Addresses There is a moderated mailing list `mrp-users` `@` `nlpl.eu` for task participants (or candidate future users of the data and evaluation software). All interested parties are kindly asked to self-subscribe to this list. The archives of the mailing list are open to the public (including all posts regarding the predecessor 2019 shared task), as the list is intended both for updates from the organizers and for general discussion among participants. Additionally, the organizers of the MRP 2020 parsing task can be reached at the following address: `mrp-organizers` `@` `nlpl.eu` News & Updates August 30, 2020 Preliminary task results are now available; we are grateful to the eight teams who have submitted parser outputs. It appears that MRP 2020 results will be advancing the state of the art for most of the frameworks. July 30, 2020 Two deficiencies in the syntactic companion parses for the cross-lingual track have been reported: 417 missing German parses, and incorrect spacing in five Czech parses. New files `deu.mrp` and `ces.mrp` are available for download. July 20, 2020 The evaluation period will open on Monday, July 27, 2020 (one week from today), and will be hosted in the on-line CodaLab service. We already invite prospective participants to register with CodaLab and make a trial submission there. June 22, 2020 We hope that we have packaged the final version of the MRP 2020 training data, now available via the LDC. To compensate for delays in producing some of the companion data, we have extended the evaluation period by one week, to July 27 through August 10, 2020. June 18, 2020 The ‘white-list’ of third-party data, models, and tools that can be used in addition to the MRP 2020 training, validation, and companion data is now finalized. Please observe the caveat regarding pre-trained, off-the-shelf syntactic parsers. June 4, 2020 An updated MRP 2020 data package is now available for download via the LDC, providing new meaning representations graphs for additional languages and high-quality morpho-syntactic syntactic dependency parses. May 25, 2020 We are running a few days late in wrapping up the cross-lingual training data, as well as the optional syntactic ‘companion’ dependency trees, for release via the LDC; we have pushed back the release date to Monday, June 1. April 28, 2020 English training and validation data is now available for no-cost licensing through the Linguistic Data Consortium (LDC). The pages summarizing data and tools and evaluation have been updated accordingly. March 30, 2020 The initial task web site is on-line, a starting data package is available, and the first call for participation has been posted to major mailing list. Please sign up for the task mailing list to receive continuous updates. Task Co-Organizers Omri Abend Lasha Abzianidze Johan Bos Jan Hajič Daniel Hershcovich Bin Li Stephan Oepen (chair) Tim O'Gorman Nianwen Xue Dan Zeman Acknowledgements We are grateful to the Nordic Language Processing Laboratory (NLPL), which provides technical infrastructure for the MRP 2020 task. Also, we warmly acknowledge the assistance of the Linguistic Data Consortium (LDC) in distributing the training data for the task to participants at no cost to anyone. Several colleagues have assisted in designing the task and preparing its data and software resources. Dan Flickinger (Stanford University) created fresh gold-standard annotations of some 1,000 WSJ strings, which form part of the EDS evaluation graphs in 2020. Sebastian Schuster (Stanford University) advised on how to convert the gold-standard syntactic annotations from the venerable PTB and OntoNotes treebanks to Universal Dependencies, version 2.x, using ‘modern’ tokenization. Milan Straka (Charles University in Prague) made available an enhanced version of his UDPipe parser and assisted in training Czech, English, and German morpho-syntacic parsing models (for the MRP companion trees). Jayeol Chun (Brandeis University) provided invaluable assistance in conversion of the Chinese AMR annotations, preparation of the Chinese morpho-syntactic companion trees, and provisioning of companion alignments for the English AMR graphs.
Definitions	Frameworks	Objectives	Data & Tools	Evaluation	Results	News & Contact
XHTML 1.0 \| Last updated: 2020-10-04 (16:10)