HarmonyBot is an AI-powered tool developed to detect and mitigate offensive language in real time, aiming to create safer online spaces, particularly for vulnerable groups such as women and girls. The bot leverages machine learning, sentiment analysis, and Google’s Perspective API to identify toxic language, assess its severity, and suggest positive alternatives, fostering a more respectful digital environment.
- Inspiration
- What HarmonyBot Does
- Technologies Used
- System Architecture
- How It Works
- Challenges and Solutions
- Accomplishments
- Future Enhancements
- How to Test
- Contribution Towards Gender Equality
The idea for HarmonyBot stemmed from the need to address the high prevalence of online harassment, especially targeting women and girls. The project aims to foster safer digital spaces by mitigating abusive language, empowering users to engage confidently without fear of harassment.
HarmonyBot analyzes messages and offers a detailed analysis that includes:
- Offensive Language Detection: Classifies content as “offensive” or “non-offensive” using a machine learning model.
- Sentiment Analysis: Assesses emotional tone to determine the negativity of the content.
- Toxicity Scoring: Uses the Google Perspective API to quantify the message’s toxicity.
- Positive Alternatives: Suggests respectful alternatives for flagged language to encourage positive communication.
- Real-Time Feedback: Provides instant insights on the offensive content, severity, toxicity, and alternatives to the user.
- Programming Language: Python
- Web Framework: Streamlit (for UI)
- Machine Learning: Pre-trained model with TextBlob for sentiment analysis
- API Integration: Google Perspective API for toxicity analysis
- Environment Management: dotenv (to manage API keys securely)
- Libraries: TextBlob, Pickle, os, requests, subprocess
The HarmonyBot architecture includes:
- Frontend: Streamlit UI for user input, feedback display, and results visualization.
- Backend:
- Model: Machine learning model trained to classify offensive content.
- Sentiment Analyzer: TextBlob to evaluate message sentiment.
- Toxicity API: Google Perspective API for toxicity scoring.
- Database: Not applicable in the current version, as data is not stored.
- User Input: Users enter a message in the Streamlit UI.
- Offensive Language Detection: The pre-trained model classifies the message as “offensive” or “non-offensive.”
- Sentiment Analysis: TextBlob assesses the emotional tone of the message.
- Toxicity Scoring: Google Perspective API provides a toxicity score for further evaluation.
- Suggestions: Offensive words are replaced with respectful alternatives.
- Feedback: Results on offensiveness, sentiment, severity, toxicity score, and suggested alternatives are displayed to the user.
- Perspective API Integration: Managing API responses and ensuring accuracy required careful setup.
- Data Sensitivity: Handled API keys securely with environment variables and dotenv.
- False Positives: Fine-tuned the model to better identify true offensive content and minimize false positives.
- Successfully integrated offensive language detection with toxicity and sentiment analysis.
- Created a real-time, interactive UI for instant feedback on language use.
- Developed a dictionary for respectful alternatives, promoting positive communication.
- Multilingual Support: Expand offensive language detection across multiple languages.
- Advanced Sentiment Analysis: Use a more sophisticated model to assess nuanced contexts.
- Live Chat Monitoring: Enable real-time chat monitoring and automated warnings.
- User Customization: Allow users to adjust sensitivity levels for personal preference.
- Browser Extension: Develop a browser extension for real-time content filtering.
- Bias Testing: Continuously test and adjust the model for biases and fairness.
- Launch HarmonyBot: Open the Streamlit app or run locally.
- Enter Test Sentences:
- Test with both offensive and non-offensive sentences.
- Check for sentiment accuracy with positive, neutral, and negative tones.
- Confirm toxicity scores with varying levels of abusive language.
- Observe the alternatives suggested for flagged language.
- Analyze Results: Verify that flagged messages, sentiment, toxicity scores, and suggested phrases align with expectations.
HarmonyBot is designed to combat online harassment, which disproportionately impacts women and girls. By filtering harmful language and encouraging positive alternatives, it fosters a safer online environment that promotes gender equality. HarmonyBot empowers women to participate freely in digital spaces and provides data insights to support gender-sensitive policies.