Tralnslation-using-Transfors

Educational repository, implementing encoder/decoder transformer model to perform translation

Attention Is All You Need

• Link to paper

Model

• Model itself is composed of an encoder and a decoder. Based on the paper, each-one is consists of a few modules which mainly are multi-head attention, residual connections plus layer normalisation and feedforward network.
• tokenizers package is used to perform tokenization

Dataset

• Thanks to hugging face , datasets package is used to load 'opus_books' dataset

Evaluation