Skip to content

Latest commit

 

History

History
 
 

interfaces

README.md

Supported Interfaces

The following is a list of currently supported interfaces along-with sample transformations. (Note that the same interfaces can be used for creating new filters by implementing ".filter()" instead of ".generate()")

Interface Description Tasks Example Transformation Default Evaluation Models & Datasets
SentenceOperation Expects a text as input and return of a transformed piece of text. Text Classification, T2T Generation BackTranslation ("textattack/roberta-base-imdb", "imdb")
SentenceAndTargetOperation Expects a source and a target text as inputs and return of their transformations. Text Classification, T2T Generation ChangeTwoWayNamedEntities ("sshleifer/distilbart-xsum-12-6", "xsum")
KeyValuePairsOperation Expects key-value pairs as input and returns a piece of text as output. AMR-to-Text, E2E Task -----
RDFOperation Expects an RDF triplet as input and returns a piece of RDF triplet as output. RDF-to-Text Generation -----
QuestionAnswerOperation Expects a question answering example as input and returns its transformation. QA, QG RedundantContextForQa ("mrm8488/bert-tiny-5-finetuned-squadv2", "squad")
TaggingOperation Expects a list of tokena and a list of tags as input and returns its transformation. Tagging LongerNamesNer ("dslim/bert-base-NER", "conll2003")
SentencePairOperation Expects a text pair and label as input and returns its transformation. Paraphrase_Detection, Entailment LexicalCounterfactualGenerator -----

We also welcome pull-requests of newer interfaces. To add a new interface, follow the below steps:

  1. Create a new python file - "YourInterface.py" in the interfaces folder
  2. Inside this python file, define a class with the appropriate inputs for the generate and the filter functions. 
    from interfaces.Operation import Operation

class YourInterface(Operation):
    A good idea would be to look at existing interfaces like SentenceOperation and QuestionAnswerOperation
  3. [Optional] Now, you can create a transformation and a filter corresponding to YourInterface with the usual steps mentioned on the main page
  4. [Optional] To gauge the effectiveness of your transformation, you can call a HuggingFace model and evaluate it over a HuggingFace dataset with the following command 
    python evaluate.py -t NameOfTransformationClass
    That's it! You can now measure your transformation with this simple command!