This repository contains the code and data for the paper Crowdsourcing Question-Answer Meaning Representations published in NAACL 2018. There is also a longer ArXiv version.
Question-Answer Meaning Representations are a new paradigm for representing predicate-argument structure, which makes use of free-form questions and their answers in order to represent a wide range of semantic phenomena. The semantic expressivity of QAMR compares to (and in some cases exceeds) that of existing formalisms, while the representations can be annotated by non-experts (in particular, using crowdsourcing).
We define QAMRs, develop a crowdsourcing pipeline for gathering them at scale on Mechanical Turk, gather a dataset of about 5,000 annotated sentences, perform a thorough analysis of this data, and run some intrinsic baselines on the dataset. In addition to this, concurrent work on Supervised Open Information Extraction (code) leverages the QAMR dataset to improve the performance of an Open IE system, especially on less commonly annotated predicates such as nominalizations.
data/: The officially released data.code/: Code and documentation for running the QAMR annotation pipeline and performing the data analysis presented in the NAACL paper.