Toxicity Classification:

1. Business Problem:

Source: https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification

Description: https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/overview/description

Problem Statement: Given a comment made by the user, predict the toxicity of the comment.

2. Machine Learning Problem Formulation:

2.1 Data:

• Source: https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/data
• We have one single csv file for training and one cvs file to test.
• Columns in train data:
  • Comment_text: This is the data in string format which we have to use to find the toxicity.
  • target: Target values which are to be predicted (has values between 0 and 1)
  • Data also has additional toxicity subtype attributes: (Model does not have to predict these)
    • severe_toxicity
    • obscene
    • threat
    • insult
    • identity_attack
    • sexual_explicit
  • Comment_text data also has identity attributes carved out from it, some of which are:
    • male
    • female
    • homosexual_gay_or_lesbian
    • christian
    • jewish
      • muslim
    • black
    • white
    • asian
    • latino
    • psychiatric_or_mental_illness
  • Apart from above features the train data also provides meta-data from jigsaw like:
    • toxicity_annotator_count
    • identity_anotator_count
    • article_id
    • funny
    • sad
    • wow
    • likes
    • disagree
    • publication_id
    • parent_id
    • article_id
    • created_date

2.2 Example Datapoints and Labels:

Comment: i'm a white woman in my late 60's and believe me, they are not too crazy about me either!!

• Toxicity Labels: All 0.0
• Identity Mention Labels: female: 1.0, white: 1.0 (all others 0.0)

Comment: Why would you assume that the nurses in this story were women?

• Toxicity Labels: All 0.0
• Identity Mention Labels: female: 0.8 (all others 0.0)

Comment: Continue to stand strong LGBT community. Yes, indeed, you'll overcome and you have.

• Toxicity Labels: All 0.0
• Identity Mention Labels: homosexual_gay_or_lesbian: 0.8, bisexual: 0.6, transgender: 0.3 (all others 0.0)

2.3 Type of Machine Learning Problem:

We have to predict the toxicity level(target attribute). The values range from 0 to 1 inclusive. This is a regression problem. It can also be treated as a classification problem if we take every value below 0.5 to be non-toxic and above it to be toxic, we would then get a binary classification problem.

2.4 Performance Metric:

The competition will use ROC_AUC as the metric after converting the numeric target variable into a categorical variable by using a threshold of 0.5. Any comment above 0.5 will be assumed to be toxic and below it non-toxic. For our training and evaluation we will use the MSE(Mean Squared Error). More on evaluation: https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/overview/evaluation

2.5 Machine Learning Objectives and Constraints:

Objectives: Predict the toxicity of a comment made by the user. (0 -> not toxic, 1 -> highest toxicity level)

Constraints:

• The model should be fast to predict the toxicity rating.
• Interpretability is not needed.