Practical_13_Ethics.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "orig_nbformat": 2,
    "deepnote_notebook_id": "cc9f7fb7-8f5d-43e4-954d-1c7504f268ba",
    "deepnote_execution_queue": [],
    "colab": {
      "name": "Practical_13_Ethics.ipynb",
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00000-37dc0ff3-e2cf-496e-beac-857fe8af5521",
        "deepnote_cell_type": "markdown",
        "id": "tx0zfTBIMCp2"
      },
      "source": [
        "# Ethics: NLP should not do harm"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00001-82d4babc-9e2b-4cab-b8ff-087ae664348b",
        "deepnote_cell_type": "markdown",
        "id": "CXBcCx8VMCp5"
      },
      "source": [
        "Last week, December 2, 2020, Timnit Gebru, the **co-lead of Google’s ethical AI team** and a widely respected leader in **AI ethics research**, announced via Twitter that the company had forced her out. The pretext for this was her email to her colleagues in which she \"pushed back against Google’s censorship of her (and her colleagues’) research, which focused on examining the **environmental and ethical implications of large-scale AI language models (LLMs)** used in many Google products.\n",
        "(More on this: https://www.bbc.com/news/technology-55164324)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00002-f976a8dc-81a3-49d2-8366-7c26fc0e3cc1",
        "deepnote_cell_type": "markdown",
        "id": "419rvxv1MCp6"
      },
      "source": [
        "Gebru's paper that was first accepted for publication but then unexpectedly withdrawn by Google without clear explanation was talking about many ethical issues in NLP which we will try to cover (at least partially) in today's tutorial."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00003-f1080566-b8a6-431b-99f3-f4b657bbb815",
        "deepnote_cell_type": "markdown",
        "id": "wJeB_ZwhMCp6"
      },
      "source": [
        "**\"If we can't talk freely about the ethical challenges posed by AI systems, we will never build ethical systems.\" **\n",
        "\n",
        "*DeepMind research scientist Iason Gabriel (Twitter)*\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00002-096d6e97-33b3-481b-ae12-364ec01dba0b",
        "deepnote_cell_type": "markdown",
        "id": "WbmuoKF-MCp7"
      },
      "source": [
        "## What is ethics?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00003-bd529805-e55f-4e94-b325-4762ed321170",
        "deepnote_cell_type": "markdown",
        "id": "p-ugA9OTMCp7"
      },
      "source": [
        "**Ethics** is moral principles that govern a person's behaviour or the conducting of an activity.\n",
        "When we discuss ethics, we think about **how to live one's life**, what is **good** and what is **bad**, and how to **act responsibly** with due considerations to the **outcomes** of your actions. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00004-42cd8f4d-3aee-43e3-a937-83a8d93151cb",
        "deepnote_cell_type": "markdown",
        "id": "09YBCmhMMCp7"
      },
      "source": [
        "Professionals deal with the questions of ethics all the time. \n",
        "Think about:\n",
        "- Engineers who develop weapons - is it ethical? Is it \"good\"?\n",
        "- Medical ethics (e.g.Nazi medical experiments; Tuskegee Syphilus Study (https://en.wikipedia.org/wiki/Tuskegee_Syphilis_Study)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00005-4daf4236-30fc-4107-a245-613abfef748d",
        "deepnote_cell_type": "markdown",
        "id": "347xlnawMCp8"
      },
      "source": [
        "Optional reads (some codes of ethics that IT professionals can refer to in their daily practice):\n",
        "\n",
        "- Google's Code of AI ethics: https://ai.google/principles\n",
        "- IBM code of AI ethics: https://www.ibm.com/watson/assets/duo/pdf/everydayethics.pdf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00006-4526a8e4-9a0d-40a1-b78a-e5564e311742",
        "deepnote_cell_type": "markdown",
        "id": "9nJgMqIFMCp8"
      },
      "source": [
        "## Questions to ask ourselves when developing NLP applications"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00006-95bfd6aa-14a9-4e8c-874f-9baed9a6f1df",
        "deepnote_cell_type": "markdown",
        "id": "XelXUIcDMCp8"
      },
      "source": [
        "The Belmont Report (https://www.hhs.gov/ohrp/sites/default/files/the-belmont-report-508c_FINAL.pdf) outlines some important ethical principles and guidelines for research involving **human subjects**. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00007-cdbd8b2e-5a1b-44d1-b9c2-c7755f0b54db",
        "deepnote_cell_type": "markdown",
        "id": "dEjo1-iAMCp8"
      },
      "source": [
        "**REFLECT AND WRITE (1)**: Why are we talking about human subjects when we talk about Natural Language Processing? (1-2 sentences)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00011-54d72c46-e402-4286-a83d-3989cf6caf05",
        "deepnote_cell_type": "markdown",
        "id": "xKuZRg4yMCp9"
      },
      "source": [
        "### Write your answer here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00008-f1d7f992-be20-424e-9bd5-682c391d1807",
        "deepnote_cell_type": "markdown",
        "id": "pCp_WL3DMCp9"
      },
      "source": [
        "Below are some fundamental ethical principles from Belmont Report that can and should be applied to NLP research, along with other AI and IT-related ethics principles:\n",
        "\n",
        " - **Respect for persons**: protecting the autonomy of all people and treating them with courtesy and respect and allowing for informed consent. Researchers must be truthful and conduct no deception.\n",
        "\n",
        "Are we respecting the autonomy of the humans in the research (authors, labelers, other participants)?\n",
        "\n",
        "\n",
        " - **Beneficence**: the philosophy of \"Do no harm\" while maximizing benefits for the research project and minimizing risks to the research subjects.\n",
        "\n",
        "Who could be harmed? By data or by prediction errors?\n",
        "\n",
        " - **Justice**: ensuring reasonable, non-exploitative, and well-considered procedures are administered fairly — the fair distribution of costs and benefits to potential research participants — and equally.\n",
        "\n",
        "Is the training data representative? Are we being fair to everyone?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00009-4eac373b-8bdb-4c6b-85cf-1f542bad87a5",
        "deepnote_cell_type": "markdown",
        "id": "IotB3ygkMCp9"
      },
      "source": [
        "Many NLP-specific studies emphasize the impact of NLP on **social justice** - equal opportunities for individuals and groups\n",
        "(such as minorities) within society to access resources, get their voice heard, and be represented\n",
        "in society. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00010-786da2e3-a3fa-4f47-b17c-481c49c674f0",
        "deepnote_cell_type": "markdown",
        "id": "l2zfpE-FMCp-"
      },
      "source": [
        "Two more important principles discussed in relation to AI in general and thus applicable to NLP:\n",
        "- **transparency** (\"AI should be designed for humans\n",
        "to easily perceive, detect, and\n",
        "understand its decision process\" (IBM))\n",
        "- **accountability** (\"Every person\n",
        "involved in the creation of AI at any\n",
        "step is accountable for considering\n",
        "the system’s impact in the world,\n",
        "as are the companies invested in its\n",
        "development\" (IBM))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00011-44b94a8d-1732-40f8-b948-4ad6d6b21994",
        "deepnote_cell_type": "markdown",
        "id": "2fElx0DrMCp-"
      },
      "source": [
        "Let's have a look at some cases and think about them in terms of ethics. Can these applications of language technology be harmful? In what ways and to whom? Can it be prevented and how?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00015-4341663b-c622-449f-8938-129895b5b76a",
        "deepnote_cell_type": "markdown",
        "id": "fPyLZ1feMCp-"
      },
      "source": [
        "## 1. Dangers of very large data in NLP"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00016-e60ad8a2-597d-43f3-91b7-386a8cc589a7",
        "deepnote_cell_type": "markdown",
        "id": "yQT3I4tJMCp-"
      },
      "source": [
        "Language models (e.g. GPT-2, GPT-3, BERT etc) are often trained on huge amounts of data from the internet. Common Crawl dataset is commonly used for training such models; it is \n",
        "“petabytes of data collected over 8 years of web crawling”. \n",
        "\n",
        "First thought you may get: awesome! The more, the better! But think twice. \n",
        "\n",
        "When we produce language,what we say or write inevitably reflects our worldviews and biases.\n",
        "\n",
        "Who posts on the Internet? Who uses Twitter and Reddit? What colour is their skin, what countries do they come from? Whose viewpoints are more represented and whose are less represented or ingored?\n",
        "All this will be eventually encoded in your model, which will be then used for real-life applications potentially impacting lives.\n",
        "\n",
        "We can simply start from the fact that Internet access is far from being distributed evenly, therefore Internet data is skewed and overpresesents younger people from developed countries. \n",
        "Models trained on \"all-Web\" data does not represent all of us equally and thus may make the world's inequality worse."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00018-5d56c904-9db6-4bcf-a53c-afc487c1ea55",
        "deepnote_cell_type": "markdown",
        "id": "PV_xDsycMCp_"
      },
      "source": [
        "Let's see how worldviews and biases can be implicit in the data and as a result encoded in the models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00014-1af02ade-db0d-4192-9c96-bc33f4b08410",
        "deepnote_cell_type": "markdown",
        "id": "jJ_J0fwEMCp_"
      },
      "source": [
        "## 2. Gender bias encoded in NLP models"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00015-8e816cc0-0555-4c45-adc6-d00f4815698a",
        "deepnote_cell_type": "markdown",
        "id": "ihrRxHzUMCp_"
      },
      "source": [
        "a. **Biased word embeddings**\n",
        "\n",
        "Word embedding models are known for containing gender and racial bias. It is concerning as the use of biased models amplify the biases already present in the society. What society do we want to leave in?\n",
        "The word2vec model trained on Google News (a very widely used model!) produces reasonable analogies like \"Paris to France = Tokyo to Japan\" and extremely biased ones like:\n",
        "**\"computer programer to man = homemaker to woman\"**.\n",
        "The male professions, according to the models, are\n",
        "- maestro\n",
        "- skipper\n",
        "- protege\n",
        "- philosopher\n",
        "- captain\n",
        "- architect\n",
        "- financier\n",
        "- warrior\n",
        "- broadcaster\n",
        "- magician\n",
        "\n",
        "while female occupations are \"homemaker\", \"nurse\", \"receptionist\", \"librarian\". In this model, men do \"carpentry\" while women do \"sewing\". \n",
        "These embeddings reflect biases present in the broader society. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00012-e66c8301-6fef-45b0-9c30-0e5e94a7b608",
        "deepnote_cell_type": "markdown",
        "id": "REk_S8MVMCp_"
      },
      "source": [
        "**b. Biases in machine translation**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00016-098372fd-02aa-4e0f-86fe-b0ef07ee8502",
        "deepnote_cell_type": "markdown",
        "id": "u_HjTzDSMCqA"
      },
      "source": [
        "\n",
        "Gender bias is a problem in machine translation where there are more examples (on which the models are trained) featuring males than females, which results in better translation for male-featuring sentences. \n",
        "\n",
        "Example: \n",
        "\"The **doctor** told the nurse that **she** had been busy\".\n",
        "\n",
        "Think for a moment: who is \"SHE\", from the sentence structure? Doctor or nurse?\n",
        "\n",
        "A human translator carrying out coreference resolution would understand that **she** refers to the doctor,\n",
        "and correctly translate the entity to German as **Die Arztin**.  An NMT model trained on a biased dataset in which most doctors are male might incorrectly default to the masculine form, **Der Arzt**.\n",
        "\n",
        "(Source: https://arxiv.org/pdf/2004.04498.pdf)\n",
        "\n",
        "Another example: **The doctor had been busy** - this would likely be translated with a masculine\n",
        "entity according to the model's bias."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00019-319fdc52-d675-4ce4-aa77-d25c6184e1b8",
        "deepnote_cell_type": "markdown",
        "id": "qsGlH9UlMCqA"
      },
      "source": [
        "**TRY IT OUT AND WRITE THE RESULTS (1-2 sentences) (2) **: Try Google Translate (https://translate.google.com/) to translate into a language of your choice some sentences (make up your a couple of your own) using the names of occupations listed earlier as \"masculine\" or \"feminine\" (according to the word2vec model) and see if they are translated to that language with a masculine or feminine word. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00024-00d3ea45-5e42-4f4f-9083-cf1c9eea69de",
        "deepnote_cell_type": "markdown",
        "id": "aawEg5axMCqB"
      },
      "source": [
        "### write your answer here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00015-f2a10fc4-7b55-4222-8b11-000e52718c13",
        "deepnote_cell_type": "markdown",
        "id": "Gjv4WAz6MCqB"
      },
      "source": [
        "## 3. Racial bias in NLP models and data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00019-22061960-ada3-4b57-a00e-62738e35cde2",
        "deepnote_cell_type": "markdown",
        "id": "L-XHk_kmMCqB"
      },
      "source": [
        "### Case 1. Spell-checker and Black names"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00021-9e5f143b-a4a3-4869-96de-b3880fd8aa77",
        "deepnote_cell_type": "markdown",
        "id": "O7FxVKKOMCqB"
      },
      "source": [
        "![] (https://kottke.org/plus/misc/images/deborah-roberts-pluralism.jpg)\n",
        "\n",
        "The piece above is part of a series called Pluralism by artist Deborah Roberts — it’s a collage of dozens of Black names marked as misspelled by Microsoft Word’s built-in spell checker.  \n",
        "It is meant to make us think about the \"neutrality of technology, how software is built, who builds it, and for whom it is designed\".\n",
        "\n",
        "https://kottke.org/20/10/the-spell-checkers-agenda"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00020-2959a070-be66-4c25-9cb7-68682873fd9d",
        "deepnote_cell_type": "markdown",
        "id": "H7BISRu-MCqC"
      },
      "source": [
        "### Case 2: Twitter bot Tay"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00022-c1e13e15-0e0c-4d5b-9fd1-f9adbe7a2a33",
        "deepnote_cell_type": "markdown",
        "id": "Zzq8Wwa8MCqC"
      },
      "source": [
        "![] (https://i.guim.co.uk/img/media/59900576343e3eb9c228925499c3d03a76b3a7cd/16_0_973_584/master/973.jpg?width=700&quality=85&auto=format&fit=max&s=66da60a6ab8773cb32b849774d9f0ff1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00023-b141b7b3-0934-4429-96da-69e7fe0a0c5d",
        "deepnote_cell_type": "markdown",
        "id": "RPm6W9xdMCqC"
      },
      "source": [
        "![](https://cbsnews2.cbsistatic.com/hub/i/2016/03/24/82a90c5d-1ce6-4fa2-8649-9ed7e5186529/1812195fd16a02cac0cd9f0e7223730b/taytweet-resized.jpg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00021-77ae40b7-2eb1-4f8e-87d5-84024c621b62",
        "deepnote_cell_type": "markdown",
        "id": "7fnJ6aNrMCqC"
      },
      "source": [
        "In 2016, less than a day after Microsoft launched its new **AI bot Tay**, she had to be suspended from tweeting after posting a series of racist statements, including \"Hitler was right I hate the jews.\"  The bot was designed to communicate with \"18 to 24 year olds in the U.S\" and \"experiment with and conduct research on conversational understanding.\" It appears some of her racist replies were simply regurgitating the statements trolls tweeted at her. What happened? The ML system was learning from the conversation it had with people, and its vocabulary and the worldview would develop based on these conversations. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00022-4680a52e-e35d-49ff-8024-268f9ddd37f5",
        "deepnote_cell_type": "markdown",
        "id": "GS-aroDXMCqC"
      },
      "source": [
        "Some of the infamous Tay tweets and their \"learning\" evolution enacted in this video:https://www.youtube.com/embed/pc0rd_K22w8"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00026-8f0e40c8-1b4e-4d3b-a0be-92bf506a6ee0",
        "deepnote_cell_type": "markdown",
        "id": "pACTyBOCMCqD"
      },
      "source": [
        "And the whole story: https://www.youtube.com/watch?v=HsLup7yy-6I"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00024-e358e65e-d8c4-424b-97aa-f39ea9435198",
        "deepnote_cell_type": "markdown",
        "id": "1yS1ZjYwMCqD"
      },
      "source": [
        "\"When asked for comment, Microsoft sent this statement: \"The AI chatbot Tay is a machine learning project, designed for human engagement. It is as much a social and cultural experiment, as it is technical. Unfortunately, within the first 24 hours of coming online, we became aware of a coordinated effort by some users to abuse Tay's commenting skills to have Tay respond in inappropriate ways. As a result, we have taken Tay offline and are making adjustments.\"\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00025-5c0799b2-270c-4098-b661-f29fe0ac5b69",
        "deepnote_cell_type": "markdown",
        "id": "L2KLIZNuMCqD"
      },
      "source": [
        "Why was it released to the public without a mechanism that would have protected the bot from such abuse, blacklisting contentious language? \n",
        "Could the situation be presented if Microsoft had filtered words like the n-word and \"holocaust\", for example?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00026-24c871b3-e6f1-4777-bd9d-5525c5728362",
        "deepnote_cell_type": "markdown",
        "id": "lR_9zFJIMCqE"
      },
      "source": [
        "**REFLECT AND WRITE (3):** What strategy can you think of that Microsoft could have taked to prevent such a disasterous situation?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00037-22a0aad0-27e6-4cbd-ba3a-49d5407a0d0f",
        "deepnote_cell_type": "markdown",
        "id": "aPTbDDuKMCqE"
      },
      "source": [
        "### Write your answer here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00022-6c7d7fb8-be94-4dd2-ba14-768153e68030",
        "deepnote_cell_type": "markdown",
        "id": "dT0eN4GbMCqE"
      },
      "source": [
        "## 4. Green NLP"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00029-6ec9de3e-e305-4417-8794-336fb71331ee",
        "deepnote_cell_type": "markdown",
        "id": "WLVBszWLMCqF"
      },
      "source": [
        "![](https://miro.medium.com/max/2412/0*usrjviODaRsgJVFg.png)\n",
        "With the compute and energy demands of many modern ML methods growing\n",
        "exponentially, ML systems have the potential to significantly contribute to **carbon\n",
        "emissions**: https://www.technologyreview.com/2019/06/06/239031/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/\n",
        "\n",
        "Important steps for the NLP community: **calculate and report in the papers/experiment reports the energy consumption and carbon emissions**. \n",
        "\n",
        "A calculator has been developed, including a generator producing a text to insert in a report as a **push for more transparency and accountability**: https://mlco2.github.io/impact/#home"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00041-621ce8eb-3d9c-4c4f-be5e-96fcf7697b90",
        "deepnote_cell_type": "markdown",
        "id": "C3YAaR9DMCqF"
      },
      "source": [
        "**REFLECT AND WRITE (4) (1-3 sentences)**: Check out this webpage with the calculator and calculate carbon emissions for your imaginary ML project: https://mlco2.github.io/impact/#compute \n",
        "Scroll down to \"What you can do section\", read through the options and let us know if you had thought about these aspects of ML before coming to today's tutorial :) "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00042-806ce9d7-6368-4817-8d0a-9b445512d29e",
        "deepnote_cell_type": "markdown",
        "id": "zF_PwcX0MCqF"
      },
      "source": [
        "### write your answer here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00024-a9adb44d-5100-46ff-bead-acc04af119bd",
        "deepnote_cell_type": "markdown",
        "id": "adB4cC_4MCqF"
      },
      "source": [
        "## 5. Privacy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00025-a091e12e-1707-405c-90b1-75bdc357cd39",
        "deepnote_cell_type": "markdown",
        "id": "FD8OdMY9MCqF"
      },
      "source": [
        "There is a software marketed as a system “that can mitigate the effects of the COVID-19 pandemic across jail and prison facilities” by **alerting prison authorities to sickness-related conversations between inmates and the outside world**.\n",
        "It scans prisoners' phone conversations, searching for relevant keywords. “It **automatically downloads, analyzes, and transcribes all recorded inmate calls**, proactively flagging them for review,” explains the product brochure, which also claims this “near real-time intelligence” can be used to identify sick inmates, help allocate personnel in understaffed prisons, and even prevent “COVID-19 related murder.”"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00040-b6b6c771-758e-4d4c-9380-e27391f5d1c7",
        "deepnote_cell_type": "markdown",
        "id": "1DyBWRbUMCqG"
      },
      "source": [
        "Think about the following aspects of the technology:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00027-34d422f0-e599-457d-b7fd-48cb47f307b6",
        "deepnote_cell_type": "markdown",
        "id": "GSU9HTuzMCqG"
      },
      "source": [
        " - Can it also be used to suppress news of the inmates' sickness? Or to retaliate against those raising alarms about prison conditions?\n",
        " - If a person talks about COVID-19 on the phone  necessarily mean they are infected with COVID-19?\n",
        " - How about non-covid-related conversations (e.g with inmates' attorneys?)? Privacy issues?\n",
        " - Is voice recognition technology 100% accurate? Absolutely non-biased?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00028-ec6e09ef-4279-4090-a312-7a6f032513bd",
        "deepnote_cell_type": "markdown",
        "id": "357h_i76MCqG"
      },
      "source": [
        "If you are interested, read more: https://theintercept.com/2020/04/21/prisons-inmates-coronavirus-monitoring-surveillance-verus/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00035-a24cc427-b77d-4ef9-9667-1e7e90729b08",
        "deepnote_cell_type": "markdown",
        "id": "aKIo4E0cMCqG"
      },
      "source": [
        "## 6. Crowdsourcing ethics in NLP annotation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00036-5d3c4b17-56a4-44df-b54c-ef4b131a34ce",
        "deepnote_cell_type": "markdown",
        "id": "CwwSQzQNMCqG"
      },
      "source": [
        "It is not uncommon to hear NLP researchers saying something like \"It cost me less than one hundred bucks to\n",
        "annotate this using Amazon Mechanical Turk!” But is this something that a researcher can be proud about? \n",
        "\n",
        "**Amazon Mechanical Turk (MTurk)** is a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually.\n",
        "Using Turks, one can produce language resources at a fractions of the standard price (e.g. US$0.05 to translate a sentence). \n",
        "\n",
        "The mean wage of a Turker is below 2USD/hour. There is a common assumption that the only people who could accept such a low-paid job is stay-at-home moms or US students with plenty of time to kill. However, research suggests that it is not the case: for example 20% of Indian Turkers said they do the work to meet the basic needs. For many Turkers it is a workplace, but this labour market does not offer a workers' union or any other protection from the employers' wrongdoings. \n",
        "\n",
        "Related paper (optional read): https://www.aclweb.org/anthology/J11-2010.pdf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00035-bee40b18-7512-4669-9d7d-484fcdac0d30",
        "deepnote_cell_type": "markdown",
        "id": "p3F0qsPSMCqG"
      },
      "source": [
        "## 7.Text generations ethics (GPT-2 & GPT-3)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00046-5d5844b1-de0d-43bd-aa60-fdbc7a849098",
        "deepnote_cell_type": "markdown",
        "id": "9vTfZvvGMCqG"
      },
      "source": [
        "You may have heard about the GPT-2 and more recent GPT-3 models trained on the massive Common Crawl web dataset and generating text so well that it often believed to be written by a human."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00047-12d34564-5a1f-4079-9378-54ebf8457e6b",
        "deepnote_cell_type": "markdown",
        "id": "m5zLe9LEMCqH"
      },
      "source": [
        "Try GPT-2 here: https://transformer.huggingface.co/doc/gpt2-large"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00048-6365943d-057a-4981-82e6-7840d2cd58da",
        "deepnote_cell_type": "markdown",
        "id": "LhrWI0LSMCqH"
      },
      "source": [
        "Check out a piece written by GPT-3 (not a graded exercise but try generating something to check if it has implicit racist/gender bias!): https://drive.google.com/file/d/1qtPa1cGgzTCaGHULvZIQMC03bk2G-YVB/view"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00049-c85121f8-c33e-492b-a4a1-0262a2ad78b8",
        "deepnote_cell_type": "markdown",
        "id": "JEm1iZZ_MCqH"
      },
      "source": [
        "If a model can generate a human-like text who would be interested in using it?\n",
        "Without ethics, one could easily:\n",
        "\n",
        "- create high-quality spam, fake news, desinformation, propaganda\n",
        "- generate content for trolling and abusive bots on social media"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00050-6cfa6d7c-f29c-4432-ad66-32e958d03121",
        "deepnote_cell_type": "markdown",
        "id": "ES3vNHO8MCqH"
      },
      "source": [
        "The creators of GPT-2 were hesitant to publicly release the model because it was \"too dangerous\". GPT-3 has not been released to public. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00052-da8e7189-c66a-495b-a938-c18929998c7d",
        "deepnote_cell_type": "markdown",
        "id": "Hzd99Sm_MCqH"
      },
      "source": [
        "Those who had access to the model report that it is biased (gender/race): it \n",
        "- tends \"to associate occupations requiring higher levels of education (banker, professor, legislator) and those requiring more physical labor (mason, millwright, etc.) with males and occupations such as nurse, receptionist, midwife, and housekeeper with females\".\n",
        "- makes  \"concerning associations with gender and adjectives\"\n",
        "![](https://matthewpburruss.com/assets/resources_GTP_2/gender_table2.png)\n",
        "- propagates stereotypes existing in the society:\n",
        "![](https://matthewpburruss.com/assets/resources_GTP_2/religion_table.png)\n",
        "(Source: https://matthewpburruss.com/post/the-unethical-story-of-gpt-3-openais-million-dollar-model/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00056-74e03506-5768-43fe-842a-88edd98fd1ab",
        "deepnote_cell_type": "markdown",
        "id": "gfgZPl0IMCqH"
      },
      "source": [
        "**REFLECT AND WRITE (5)**: What's wrong with releasing GPT-3 to public? Why are such models viewed as potentially dangerous?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00058-92b4d9a8-05d0-4b3b-9d70-229a2c4f7e69",
        "deepnote_cell_type": "markdown",
        "id": "xP4FYJPVMCqH"
      },
      "source": [
        "### write your answer here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00036-29f56adb-78e1-4b8d-b363-821e2cca62f9",
        "deepnote_cell_type": "markdown",
        "id": "pP6_szgZMCqH"
      },
      "source": [
        "## 8. Unethical chatbots\n",
        "\n",
        "In addition to the racist Tay, here is another case: https://www.theguardian.com/technology/2015/feb/12/randomly-generated-tweet-by-bot-investigation-dutch-police\n",
        "\n",
        "\"When Twitter user @jeffrybooks tweeted saying “I seriously want to kill people” at a fashion and cosmetics convention happening at Amsterdam, the Dutch police took the threat seriously.\"\n",
        "The bot was a  markov chain generator using a simple algorithm to create vaguely coherent sentences from a corpus of text. Oops!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00043-fbd48f10-54c4-48fd-93dc-4fac63001d57",
        "deepnote_cell_type": "markdown",
        "id": "-4ZwtmC5MCqI"
      },
      "source": [
        "# How can we use NLP for common good?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00044-de3dd14f-07d1-4485-a99f-a111dd75edff",
        "deepnote_cell_type": "markdown",
        "id": "DwQI0QaxMCqI"
      },
      "source": [
        " - Fact-Checking/Fake News Detection\n",
        " - Studying Propaganda and Political Misinformation\n",
        " - Detecting bias (inc. law and justice applications)\n",
        " - Identifying and fighting toxicity/hate/abuse/suidical behaviour (e.g. in social media and online conversations)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00041-924a0fe7-2a3b-4a42-8b51-84cf1206a2c8",
        "deepnote_cell_type": "markdown",
        "id": "0TnaXhCKMCqI"
      },
      "source": [
        "**Example 1**. Creating a dataset of anti-Asian hate speech in Twitter related to COVID and conducting a longitudinal analysis; discovering that\n",
        "-  hate is contagious and nodes are highly likely to become hateful after being exposed to hateful content\n",
        "- counterhate messages can discourage users from turning hateful in the first place\n",
        "- show that hateful bots are more successful in attracting followers compared to counterhate bots"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00041-28333d07-4298-4b33-93b4-fe784dbf79dd",
        "deepnote_cell_type": "markdown",
        "id": "YxAsB-krMCqI"
      },
      "source": [
        "**Example 2**. Facebook has developed algorithms that spot warning signs in users' posts and the comments their friends leave in response. After confirmation by Facebook's human review team, the company contacts those thought to be at risk of self-harm to suggest ways they can seek help."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00063-d04d6b6c-b806-4729-a9b7-a14dfc19d2d2",
        "deepnote_cell_type": "markdown",
        "id": "ATFucGyEMCqI"
      },
      "source": [
        "**REFLECT AND WRITE (6)**: Give one real-life or imaginary example of ethical application of NLP"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00065-48ecee1b-411e-41d3-8199-4041a4265677",
        "deepnote_cell_type": "markdown",
        "id": "EGJerFCXMCqI"
      },
      "source": [
        "### write your answer here\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00041-a4edf729-d368-43d0-bd59-2ba39b220847",
        "deepnote_cell_type": "markdown",
        "id": "LnWCiBSKMCqI"
      },
      "source": [
        "# Debiasing language data for training models"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00042-33a5c8af-8587-480a-921d-ba4cb80ab22d",
        "deepnote_cell_type": "markdown",
        "id": "IYRPu9d5MCqJ"
      },
      "source": [
        "As an example of a debiasing strategy, we will have a look at several ways to eliminate gender bias in language data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00043-2eec9803-95a2-4638-804b-920237b202d1",
        "deepnote_cell_type": "markdown",
        "id": "gGfwIG5LMCqJ"
      },
      "source": [
        "https://arxiv.org/pdf/1906.08976.pdf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00044-6b9d98d6-6ed8-42d6-9940-045c6850b4ec",
        "deepnote_cell_type": "markdown",
        "id": "khk8oiu6MCqJ"
      },
      "source": [
        "1. **Data augmentation**\n",
        "\n",
        "When a dataset contains a disproportionate number of references to one gender, we can create an augmented\n",
        "data set identical to the original data set but biased towards the opposite gender and to train a model\n",
        "on the union of the original and data-swapped\n",
        "sets. \n",
        "\n",
        "The augmented data set is created using gender-swapping. The goal of data\n",
        "augmentation is to debias predictions by training the model on a gender-balanced data set.\n",
        "\n",
        "- for every sentence in the original data set, create that sentence’s gender-swapped equivalent (e.g. “He went to the park” vs “She\n",
        "went to the park”.)\n",
        "-  apply name anonymization to every original sentence and its gender-swapped equivalent - replace all named entities with\n",
        "anonymized entities, such as “E1” (e.g. \"Mary likes her mother Jan\" >>> \"E1 likes his father E2\" \n",
        "This removes gender associations with named entities in sentences. \n",
        "- train the model on the union of the original data set with name-anonymization\n",
        "and the augmented data set. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00045-35f60bbb-8f78-4d9d-85e4-dc124ee86635",
        "deepnote_cell_type": "markdown",
        "id": "7q89Q8HyMCqK"
      },
      "source": [
        "2. **Gender tagging**\n",
        "\n",
        "Consider Machine Translation where mixing up the gender of the speaker can lead to inaccurate predictions. \n",
        "Current MT models predict the source to be male a disproportionate amount of time -  because training sets are dominated by male-sourced\n",
        "data points, so the models learn skewed statistical relationships and are thus more likely to predict the speaker to be male when the gender of\n",
        "the source is ambiguous. \n",
        "\n",
        "**Gender tagging** mitigates this by adding a tag indicating the gender of the source of the data\n",
        "point to the beginning of every data point. \n",
        "\n",
        "E.g. “I’m happy” would change to “MALE I’m happy.” In theory, encoding gender information in\n",
        "sentences could improve translations in which the gender of the speaker affects the translation (i.e.\n",
        "“I am happy” could translate to “Je suis heureux” [M] or “Je suis heureuse” [F]), since English does\n",
        "not mark the gender of the speaker in this case.\n",
        "\n",
        "Problems:\n",
        "\n",
        "- can be expensive: knowing the gender of the source of a data point requires meta-information, and obtaining\n",
        "this could be costly in terms of memory usage and\n",
        "time. \n",
        "- MT models may need to be\n",
        "redesigned to correctly parse the gender tags."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00046-f40f9ccd-7550-4f75-86b2-c6c4e7629f6d",
        "deepnote_cell_type": "markdown",
        "id": "IF0F3H9kMCqK"
      },
      "source": [
        "3. **Debiasing word embeddings**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00047-01f61341-85f9-469a-a7b6-200c71d11494",
        "deepnote_cell_type": "markdown",
        "id": "HGiq5HFPMCqK"
      },
      "source": [
        "https://www.youtube.com/watch?v=fg8ijSPHyx0 (11 min)"
      ]
    }
  ]
}