Skip to content

This repository is associated with the paper "Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling", accepted at EACL 2023.

Notifications You must be signed in to change notification settings

AdhyaSuman/NTMs_Dropout_Analysis

Repository files navigation

VAE-NTMs

VAE-NTMs is a class of neural topic models (NTMs) that uses amortized variational inference technique to compute the approximate posterior distribution.

Models

In this paper, we have analyzed the effect of dropout in VAE-based neural topic models. We have chosen three widely used neural topic models, which are as follows:

right-aligned logo in README

Datasets

We have used the following datasets:

Tutorials

Name Link
Quantitative evaluation of topic quality and document classification Open In Colab

Acknowledgment

All experiments are conducted using OCTIS which is an integrated framework for topic modeling.

OCTIS: Silvia Terragni, Elisabetta Fersini, Bruno Giovanni Galuzzi, Pietro Tropeano, and Antonio Candelieri. (2021). OCTIS: Comparing and Optimizing Topic models is Simple!. EACL. https://www.aclweb.org/anthology/2021.eacl-demos.31/

How to cite this work?

This work has been accepted at EACL 2023!

Read the paper:

  1. ACL Anthology
  2. ArXiv

If you decide to use this resource, please cite:

@inproceedings{adhya-etal-2023-neural,
title = "Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling",
author = "Adhya, Suman  and
  Lahiri, Avishek  and
  Sanyal, Debarshi Kumar",
booktitle = "Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics",
month = may,
year = "2023",
address = "Dubrovnik, Croatia",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.eacl-main.162",
pages = "2220--2229",
abstract = "Dropout is a widely used regularization trick to resolve the overfitting issue in large feedforward neural networks trained on a small dataset, which performs poorly on the held-out test subset. Although the effectiveness of this regularization trick has been extensively studied for convolutional neural networks, there is a lack of analysis of it for unsupervised models and in particular, VAE-based neural topic models. In this paper, we have analyzed the consequences of dropout in the encoder as well as in the decoder of the VAE architecture in three widely used neural topic models, namely, contextualized topic model (CTM), ProdLDA, and embedded topic model (ETM) using four publicly available datasets. We characterize the dropout effect on these models in terms of the quality and predictive performance of the generated topics."
}

About

This repository is associated with the paper "Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling", accepted at EACL 2023.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published