LowFER.RuWordNet

Testing the algorithm proposed in the article "LowFER: Low-rank Bilinear Pooling for Link Prediction", ICML 2020 on the Russian-language thesaurus of the WN format - RuWordNet for 2021.

Table of content

• LowFER.RuWordNet
  • Table of content
  • Launch
  • Results
  • Requirements
  • Authors
  • Licences
  • Citing

Launch

1. Original WN18 data launch with hyperparams tuned by Saadullah Amin:

   python main.py --dataset WN18 --num_iterations 30 --batch_size 128 --lr 0.005 --dr 0.995 --edim 200 --rdim 30 --input_dropout 0.2 --hidden_dropout1 0.1 --hidden_dropout2 0.2 --label_smoothing 0.1 --k 10

2. Our RuWordNet-2021 data launch:

   python main.py --dataset rwn-2021 --num_iterations 30 --batch_size 128 --lr 0.005 --dr 0.995 --edim 200 --rdim 30 --input_dropout 0.2 --hidden_dropout1 0.1 --hidden_dropout2 0.2 --label_smoothing 0.1 --k 10

Results

Dataset	MRR	Hits@10	Hits@3	Hits@1
Original Data
WN18	0.953	0.958	0.955	0.949
FB15k	0.795	0.892	0.833	0.741
WN18RR	0.470	0.526	0.482	0.443
FB15k-237	0.358	0.544	0.394	0.266
Our Data
RuWN21*	0.91	0.94	0.94	0.92

*after 30 iterations

Requirements

The original codebase was implemented in Python 3.6.6. Required packages are:

python     3.6.6
numpy      1.15.1
pytorch    1.0.1

Authors

2022, Grandilevskii Aleksei, software engineer, github: @zer0deck, email: [email protected], website: zer0deck.com

2022, Sorokin Mikhail, ML engineer, github: @Mikha1lSorokin, email: [email protected]

Licences

1. RuWordNet is licensed as CCANSA 3.0 Licence

   
   This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

2. LowFER itself is licenced as MIT License.

Citing

This project is provided by LowFER model.

1.  @inproceedings{pmlr-v119-amin20a,
      title ={{L}ow{FER}: Low-rank Bilinear Pooling for Link Prediction},
      author = {Amin, Saadullah and Varanasi, Stalin and Dunfield, Katherine Ann and Neumann, G{\"u}nter},
      booktitle = {Proceedings of the 37th International Conference on Machine Learning},
      pages = {257--268},
      year = {2020},
      editor = {III, Hal Daumé and Singh, Aarti},
      volume = {119},
      series = {Proceedings of Machine Learning Research},
      month = {13--18 Jul},
      publisher = {PMLR},
      pdf = {http://proceedings.mlr.press/v119/amin20a/amin20a.pdf},
      url = {https://proceedings.mlr.press/v119/amin20a.html}
    }

2.  @inproceedings{dikeoulias-etal-2022-temporal,
        title = "Temporal Knowledge Graph Reasoning with Low-rank and Model-agnostic Representations",
        author = {Dikeoulias, Ioannis and Amin, Saadullah and Neumann, G{\"u}nter},
        booktitle = "Proceedings of the 7th Workshop on Representation Learning for NLP",
        month = may,
        year = "2022",
        address = "Dublin, Ireland",
        publisher = "Association for Computational Linguistics",
        url = "https://aclanthology.org/2022.repl4nlp-1.12",
        doi = "10.18653/v1/2022.repl4nlp-1.12",
        pages = "111--120",
    }

3. @inproceedings{balazevic2019tucker,
      title={TuckER: Tensor Factorization for Knowledge Graph Completion},
      author={Bala\v{z}evi\'c, Ivana and Allen, Carl and Hospedales, Timothy M},
      booktitle={Empirical Methods in Natural Language Processing},
      year={2019}
    }