# AG News Classification With Bert Model

Multi-label Classification using the BERT (Bidirectional Encoder Representations from Transformers) model. This code demonstrates how to preprocess text data, train a BERT-based multi-label classification model, and evaluate its accuracy. 


## Introduction

This project uses the BERT model to perform classification on news title and text data. It includes data preprocessing, model training, and evaluation steps. Sentiment analysis is the process of determining the sentiment or emotional tone of a piece of text, such as positive, negative, or neutral.

## Prerequisites

Before running the code, you need to ensure that you have the following prerequisites:

    Python 3.7+
    PyTorch
    Hugging Face Transformers library
    pandas
    CUDA-enabled GPU (optional but recommended for faster training)

## Installation

To install the required Python libraries, you can use the following commands:

    pip install torch
    pip install transformers
    pip install pandas

## Usage

### Data Preprocessing

The code performs the following data preprocessing steps:

    Imports necessary libraries and checks for GPU availability.
    Loads training and testing data from CSV files (train.csv and test.csv).
    Combines the "Title" and "Description" columns into a single "Text" column.
    Converts class labels to the correct format (e.g., mapping 1 to 0, 2 to 1, etc.).
    Tokenizes the text data using the BERT tokenizer, "Encode_plus" is used to convert the text into format suitable for Bert model.
    Creates PyTorch Datasets and DataLoaders for training and testing.

### Model Details

The code uses the bert-base-cased pre-trained BERT model and fine-tunes it for text classification with a specified number of output classes (in this case, 4 classes). 



SentimentClassifier is a PyTorch module, and it inherits from torch.nn.Module.

In the constructor __init__, the following components are initialized:
    self.bert: This loads the pre-trained BERT model ('bert-base-cased') using BertModel.from_pretrained. This BERT model will be used for feature extraction.
    self.drop: A dropout layer with a dropout probability of 0.45. Dropout is used for regularization to prevent overfitting.
    self.out: A linear layer for the final classification. The input size is the hidden size of the BERT model (self.bert.config.hidden_size), and the output size is n_classes, which is the number of classes for sentiment classification.

The forward method defines the forward pass of the model:
    The input input_ids and attention_mask are passed to the BERT model (self.bert).
    The pooled output from BERT is passed through the dropout layer (self.drop).
    Finally, the output from the dropout layer is passed through the linear layer (self.out), which produces the logits for each class.

This SentimentClassifier class is designed to be used as a part of a larger pipeline for sentiment classification. You can instantiate this class and use it for training, evaluation, and inference on sentiment analysis tasks.

### Training

Hyperparameter
    batch_size_train= 64
    batch_size_test= 64
    max_len_train=80
    max_len_test=80
    lr=2e-5
    epochs= 1   

The training section of the code includes the following steps:

    Defines the training model.
    Moves the model to the GPU if available.
    Defines the loss function and optimizer.
    Trains the model for a specified number of epochs.
    Prints training accuracy after each epoch.

### Evaluation

The evaluation section of the code includes the following steps:

    Switches the model to evaluation mode.
    Evaluates the model's performance on the testing dataset.
    Prints testing accuracy after each epoch.

## Credits
reference link:
    https://blog.51cto.com/u_15127680/3841198
    https://leemeng.tw/attack_on_bert_transfer_learning_in_nlp.html?fbclid=IwAR3R0oInXiENBS5YSm6DklSXG_d-ehPNDZdSzGusA9nD_0L_GZ0xq-iQ3Mw
    https://www.kaggle.com/code/rushinaik/multiclass-bert


## License
This project is licensed under the MIT License - see the LICENSE file for details.