The following is a list of currently supported interfaces along-with sample transformations. (Note that the same interfaces can be used for creating new filters by implementing ".filter()" instead of ".generate()")
Interface | Description | Tasks | Example Transformation | Default Evaluation Models & Datasets |
---|---|---|---|---|
SentenceOperation |
Expects a text as input and return of a transformed piece of text. | Text Classification, T2T Generation | BackTranslation |
("textattack/roberta-base-imdb", "imdb") |
SentenceAndTargetOperation |
Expects a source and a target text as inputs and return of their transformations. | Text Classification, T2T Generation | ChangeTwoWayNamedEntities |
("sshleifer/distilbart-xsum-12-6", "xsum") |
KeyValuePairsOperation |
Expects key-value pairs as input and returns a piece of text as output. | AMR-to-Text, E2E Task | ----- | |
RDFOperation |
Expects an RDF triplet as input and returns a piece of RDF triplet as output. | RDF-to-Text Generation | ----- | |
QuestionAnswerOperation |
Expects a question answering example as input and returns its transformation. | QA, QG | RedundantContextForQa |
("mrm8488/bert-tiny-5-finetuned-squadv2", "squad") |
TaggingOperation |
Expects a list of tokena and a list of tags as input and returns its transformation. | Tagging | LongerNamesNer |
("dslim/bert-base-NER", "conll2003") |
SentencePairOperation |
Expects a text pair and label as input and returns its transformation. | Paraphrase_Detection, Entailment | LexicalCounterfactualGenerator |
----- |
We also welcome pull-requests of newer interfaces. To add a new interface, follow the below steps:
- Create a new python file - "YourInterface.py" in the interfaces folder
- Inside this python file, define a class with the appropriate inputs for the generate and the filter functions.
A good idea would be to look at existing interfaces like
from interfaces.Operation import Operation class YourInterface(Operation):
SentenceOperation
andQuestionAnswerOperation
- [Optional] Now, you can create a transformation and a filter corresponding to
YourInterface
with the usual steps mentioned on the main page - [Optional] To gauge the effectiveness of your transformation, you can call a HuggingFace model and evaluate it over a HuggingFace dataset with the following command
That's it! You can now measure your transformation with this simple command!
python evaluate.py -t NameOfTransformationClass