Make TOP relation's value constant (#30)

* Add tests/ subdirectory with one test file

* Make TOP relation value constant

Fixes #25

* Clarify CHANGELOG entry

Author: goodmami

CHANGELOG.md

@@ -7,6 +7,9 @@
 * Add CI/CD configuration (see PR [#23][])
 * Better handle deinversion of special roles ([#10][])
 * Get `smatch-table.py` working again (part of PR [#27][])
+* Add `tests/` subdirectory and `test_top.py` ([#25])
+* Make TOP relation's value a constant string and not the top node's
+  concept to avoid double-penalizing different top concepts ([#25])
 
 ## [1.0.2][]
 
@@ -166,5 +169,6 @@ The following are taken from an old `update_log` file:
 [#19]: https://github.com/snowblink14/smatch/issues/19
 [#22]: https://github.com/snowblink14/smatch/issues/22
 [#23]: https://github.com/snowblink14/smatch/pull/23
+[#25]: https://github.com/snowblink14/smatch/pull/25
 [#27]: https://github.com/snowblink14/smatch/pull/27
 

amr.py

@@ -420,8 +420,8 @@ def update_triple(node_relation_dict, u, r, v):
             # each node has a relation list and attribute list
             relation_list.append(node_rel_list)
             attribute_list.append(node_attr_list)
-        # add TOP as an attribute. The attribute value is the top node value
-        attribute_list[0].append(["TOP", node_value_list[0]])
+        # add TOP as an attribute. The attribute value just needs to be constant
+        attribute_list[0].append(["TOP", 'top'])
         result_amr = AMR(node_name_list, node_value_list, relation_list, attribute_list)
         return result_amr
 

tests/README.md

@@ -0,0 +1,18 @@
+
+# smatch tests
+
+This directory contains test cases to verify the correct behavior of
+smatch. Run the tests with [pytest](https://pytest.org/). This will
+require you to install both pytest and smatch:
+
+```console
+$ pip install pytest
+$ pip install -e .  # current directory is smatch
+$ pytest
+```
+
+**Note:** As smatch is inherently non-deterministic due to its
+hill-climbing implementation, the tests can be "flaky" (i.e.,
+sometimes pass, sometimes fail). To mitigate the possibility of flaky
+tests, test cases should use *minimal* AMRs so it becomes trivial for
+smatch to get the optimal solution.

tests/test_top.py

@@ -0,0 +1,57 @@
+
+"""
+The top node in an AMR determines the focus of the encoded
+meaning. For example, the following have the same dependencies but a
+different node at the top:
+
+    (w / white-03
+       :ARG1 (m / marble))
+
+The above AMR means "the marble is white" or "the whiteness of the
+marble".
+
+    (m / marble
+       :ARG1-of (w / white-03))
+
+This AMR means "the marble that is white" or "the white marble".
+
+For this reason, AMRs that differ only in which node is the top will
+get smatch scores less than 1.0.
+
+For more information see:
+
+    https://github.com/amrisi/amr-guidelines/blob/master/amr.md#focus
+"""
+
+import smatch
+
+a = '(a / alpha :ARG0 (b / beta))'
+b = '(a / alternative :ARG0 (b / beta))'
+c = '(a / alpha :ARG0 (b / b-side))'
+d = '(b / beta :ARG0-of (a / alpha))'
+
+
+def get_amr_match(amr1, amr2):
+    vals = smatch.get_amr_match(amr1, amr2)
+    smatch.match_triple_dict.clear()
+    return vals
+
+
+def test_same():
+    assert get_amr_match(a, a) == (4, 4, 4)
+    smatch.match_triple_dict.clear()
+
+
+def test_same_top_different_top_concept():
+    assert get_amr_match(a, b) == (3, 4, 4)
+    smatch.match_triple_dict.clear()
+
+
+def test_same_top_different_dependent_concept():
+    assert get_amr_match(a, c) == (3, 4, 4)
+    smatch.match_triple_dict.clear()
+
+
+def test_same_different_top():
+    assert get_amr_match(a, d) == (3, 4, 4)
+    smatch.match_triple_dict.clear()