This is my tensorflow implementation of the Dynamic Coattention Network applied to question answering for the SQuAD database (tested with tensorflow version 1.1 and 1.2). The network gets a Wikipedia article and a question as inputs and should predict a segment (or span) of the article that answers the question.
The data in the data/squad folder was downloaded and preprocessed via the starter code from assignment 4 of the Stanford Course CS224n: Natural Language Processing with Deep Learning.
If you just want to have a look at the DCN implementation check out DCN_model.py, it is only around 200 lines long.
To implement the model I had to explore some tensorflow functions like tf.gather_nd and tf.map_fn. I did my experiments with these functions on toy data in this notebook in the Experimentation_Notebooks folder.
The best result so far is 48% EM (exact match) and 64% F1 score on the validation set. Training was started via
python code/train.py --rnn_state_size=150Note:
- You will need the tqdm package to run the code
- Right now the project is on ice, due to the high costs for training on AWS instances. I might continue the project once I get a proper graphics card.
TODO:
- The hyperparameter search is not finished (e.g.: How much can using 300 dimensional word vectors improve performance compared to 100 dimensional word vectors?)
- Check influence of LSTM vs GRU