Update synonym list from CourtListener #14
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a big addition to our synonym list. The source for this data is mostly from the Indigo Book, which has many tables of abbreviations, such as:
On top of that, I added a few things:
From there I did the following:
Remove duplicates (of which there were many)
Removed phrases (Solr is bad at this)
Removed anything with a period or apostrophe in the middle, like,
U.S.
, because those things get split by Solr anyway.Cleaned up a bunch of items that had brackets, like
trans[lator, lation], trans
. In the case that the expanded word lists were semantically different, I made them into mappings. Else, I made them into synonyms. For example, here's what trans maps to:This means that when you type
trans
the system doesn't know if you meantranslator
ortransgender
. OTOH, something else might just be:Because they're all essentially the same semantically.
Eliminated one-letter abbreviations (they're not likely to be useful)
Removed real words that'd cause trouble. For example, we wouldn't want every search for
cat
to turn up results forcategory
.All in all, I think she's a fine list. Definitely a conservative one, which lawyers like, but also one that should make lots of searches (especially those involving abbreviations) work better.