autoauto- [深度学习](#深度学习)auto - [[理论、正则化](notes/DeepLearning.md)](#理论正则化notesdeeplearningmd)auto - [[少样本学习](notes/ZeroShotLearning.md)](#少样本学习noteszeroshotlearningmd)auto - [[压缩、剪枝、蒸馏、AUTOML](notes/Automl.md)](#压缩剪枝蒸馏automlnotesautomlmd)auto- [自然语言处理](#自然语言处理)auto - [[语言模型](notes/LanguageModel.md)](#语言模型noteslanguagemodelmd)auto - [[文本分类](notes/Classification.md)](#文本分类notesclassificationmd)auto - [[语法分析:分词、实体识别、依存分析、指代消解等](notes/NlpGrammaticalAnalysis.md)](#语法分析分词实体识别依存分析指代消解等notesnlpgrammaticalanalysismd)auto - [[表示学习：词向量、句向量](notes/Representation.md)](#表示学习词向量句向量notesrepresentationmd)auto - [[文本匹配](notes/Match.md)](#文本匹配notesmatchmd)auto - [[数据增强](notes/DataAugmentation.md)](#数据增强notesdataaugmentationmd)auto - [[迁移学习](notes/TransferLearning.md)](#迁移学习notestransferlearningmd)auto - [[关键词抽取](notes/KeyPhrase.md)](#关键词抽取noteskeyphrasemd)auto - [[文本摘要](notes/Summarization.md)](#文本摘要notessummarizationmd)auto - [[聊天机器人](notes/ChatBot.md)](#聊天机器人noteschatbotmd)auto - [[假新闻检测](notes/FakeNewsDetection.md)](#假新闻检测notesfakenewsdetectionmd)auto - [[知识图谱](notes/KnowlegeGraph.md)](#知识图谱notesknowlegegraphmd)auto - [[机器翻译](notes/MachineTranslation.md)](#机器翻译notesmachinetranslationmd)auto - [[自然语言生成](notes/NaturalLanguageGeneration.md)](#自然语言生成notesnaturallanguagegenerationmd)auto - [[自然语言推理](notes/NaturalLanguageInference.md)](#自然语言推理notesnaturallanguageinferencemd)auto - [[预训练语言模型](notes/PretrainedLangugeModel.md)](#预训练语言模型notespretrainedlangugemodelmd)auto - [[阅读理解](notes/ReadingComprehension.md)](#阅读理解notesreadingcomprehensionmd)auto- [[推荐](notes/Recommendation.md)](#推荐notesrecommendationmd)auto- [工程](#工程)autoauto

深度学习

理论、正则化

• 列表汇总

---

激活函数

• 1606.08415 Gaussian Error Linear Units (GELUs) 高斯误差线性单位(GELUs)

---

正则化

---

损失函数

• [1911.02855 Dice Loss for Data-imbalanced NLP Tasks]

• 201801 AMSoftmax: Additive Margin Softmax for Face Verification

• 201708 Focal Loss for Dense Object Detection

少样本学习

• 列表汇总

压缩、剪枝、蒸馏、AUTOML

• 列表汇总

• 2002.12620 TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing

自然语言处理

• 202003 Natural Language Processing Advancements By Deep Learning: A Survey
  • https://arxiv.org/abs/2003.01200

语言模型

• 列表汇总

• 201911 Generalization through Memorization: Nearest Neighbor Language Models

文本分类

• 列表汇总

---

• 2103.12407 Detecting Hate Speech with GPT-3 用 GPT-3检测仇恨言论

---

• 202004 Deep Learning Based Text Classification: A Comprehensive Review

• 2014 Convolutional Neural Networks for Sentence Classification

• 201412 Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

• 2016 Hierarchical Attention Networks for Document Classification

• 2016 Recurrent Neural Network for Text Classification with Multi-Task Learning

• 201607 Bag of Tricks for Efficient Text Classification

• 2017ACL Deep Pyramid Convolutional Neural Networks for Text Categorization

• 2018 Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN

• Baselines and Bigrams: Simple, Good Sentiment and Topic Classification

语法分析:分词、实体识别、依存分析、指代消解等

• 列表汇总

---

序列标注

• 201908 - LAN - Hierarchically-Refined Label Attention Network for Sequence Labeling

---

分词

• 201704 Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF
• 2019ACL Is Word Segmentation Necessary for Deep Learning of Chinese Representations?

---

命名实体识别

• 2101.00396 Lex-BERT: Enhancing BERT based NER with lexicons Lex-BERT: 用词典增强基于 BERT 的 NER
• 2101.11420 Recent Trends in Named Entity Recognition (NER) 命名实体识别(NER)的最新发展趋势
• 202002 Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
• 202002 Zero-Resource Cross-Domain Named Entity Recognition
• 201812 A Survey on Deep Learning for Named Entity Recognition

---

指代消解

---

语义角色

---

依存分析

---

NLU

• 2012.15639 TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis 一个用于细粒度 NER 和增强语义分析的文本理解系统

表示学习：词向量、句向量

• 列表汇总

---

词向量

• 2014 Glove: Global Vectors for Word Representation
• 2013 Efficient Estimation of Word Representations in Vector Space
• 2013 Distributed Representations of Words and Phrases and their Compositionality

---

句向量

• 2015 From Word Embeddings To Document Distances
• 2014 Doc2Vec Distributed Representations of Sentences and Documents

文本匹配

• 列表汇总

• 201908 Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

• 2019 Keyword-Attentive Deep Semantic Matching

• 201908 ACL RE2: Simple and Effective Text Matching with Richer Alignment Features

• 201905 FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance

• 201904 Understanding the Behaviors of BERT in Ranking

• 201804 SAN - Stochastic Answer Networks for Natural Language Inference

• 201609 ESIM Enhanced LSTM for Natural Language Inference

• 2013 DSSM Learning Deep Structured Semantic Models for Web Search using Clickthrough Data

• 201606 A Decomposable Attention Model for Natural Language Inference

数据增强

• 列表汇总

• 202004 When does data augmentation help generalization in NLP?

• 1904.12848 Unsupervised Data Augmentation 无监督数据增强

• 201901 EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification

迁移学习

• 列表汇总

• 202005 Cross-lingual Transfer of Twitter Sentiment Models Using a Common Vector Space

关键词抽取

• 列表汇总

---

Review

• 2009.10229 An Empirical Study on Neural Keyphrase Generation 神经关键词生成的实证研究

• 201905 A review of keyphrase extraction

---

Unsupervised

• 2004.13639 Joint Keyphrase Chunking and Salience Ranking with BERT 基于BERT联合关键词分组和显著性排序

• 202002 SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-Trained Language Model 基于预训练语言模型的无监督关键词提取基线 SIFRank

• 2018 ACL EmbedRank++ Simple Unsupervised Keyphrase Extraction using Sentence Embeddings

---

Supervised

文本摘要

• 列表汇总

---

General

• 201907 Simple Unsupervised Summarization by Contextual Matching

---

Extractive

• 201903 Fine-tune BERT for Extractive Summarization
• 2019ACL Single Document Summarization as Tree Induction

---

Abstractive

聊天机器人

• 列表汇总

---

Task-oriented

• 202003 Recent Advances and Challenges in Task-oriented Dialog System

---

Dialogue

• 201911 DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

---

意图识别与槽抽取

• 列表

• 2103.15871 Industry Scale Semi-Supervised Learning for Natural Language Understanding 自然语言理解的工业规模半监督学习

• 201902 BERT for Joint Intent Classification and Slot Filling

假新闻检测

• 列表汇总

知识图谱

• 列表汇总

---

Entity Linking

---

Relation Extraction

• 2102.01373 An Improved Baseline for Sentence-level Relation Extraction 一种改进的句级关系抽取基线

• 2102.01156 Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings 基于 bert 标记和实例嵌入的远监督关系抽取

• 2010.12812 A Frustratingly Easy Approach for Entity and Relation Extraction 一种简单得令人沮丧的实体和关系抽取方法

• 202004 Review and Outlook for Relation Extraction

• 1711.07010 A Discourse-Level Named Entity Recognition and Relation Extraction Dataset for Chinese Literature Text 中文文学作品命名实体识别与关系抽取数据集

---

Event Extraction

机器翻译

• 列表汇总

• 201409 Neural Machine Translation by Jointly Learning to Align and Translate

• 201409 Sequence to Sequence Learning with Neural Networks

自然语言生成

• 列表汇总

---

Chatbot

• 202004 Few-Shot Natural Language Generation by Rewriting Templates

---

诗歌

• 2019 基于神经网络的集句诗自动生成

---

新闻评论

• 201909 Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation
• 201906 Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
• 201809 Unsupervised Machine Commenting with Neural Variational Topic Model
• 201805 Automatic Article Commenting: the Task and Dataset

---

评论评价

---

Pretrain ML

• 201909 A Conditional Transformer Language Model for Controllable Generation

自然语言推理

• 列表汇总

预训练语言模型

• 列表汇总

• 2103.06874 CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation

• 2103.04350 Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

---

模型

• 2103.10360 All NLP Tasks Are Generation Tasks: A General Pretraining Framework 所有 NLP 任务都是生成任务: 一个通用的预训练框架

• 2103.04350 Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

• 2103.06874 CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation

• 202004 DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

• 202004 Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

• 202003 Pre-trained Models for Natural Language Processing: A Survey

• 2020 ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

• 202001 ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training

• 201909 ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

• 201909 NEZHA: NEURAL CONTEXTUALIZED REPRESENTATION FOR CHINESE LANGUAGE UNDERSTANDING

• 201907 ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding

• 201906 RoBERTa: A Robustly Optimized BERT Pretraining Approach

• 201906 ERNIE: Enhanced Language Representation with Informative Entities（THU/ACL2019）

• 1905.03197 Unified Language Model Pre-training for Natural Language Understanding and Generation

• 201904 ERNIE: Enhanced Representation through Knowledge Integration

• 2018 ACL Deep contextualized word representations: ELMO

• 2018 GPT Improving Language Understanding by Generative Pre-Training

• 201801 Universal Language Model Fine-tuning for Text Classification

• 201706 attention is all you need

---

蒸馏

• 1909.10351 TinyBERT: Distilling BERT for Natural Language Understanding
• 1910.01108 DistilBERT,adistilledversionofBERT:smaller, faster,cheaperandlighter
• 1908.08962 Well-Read Students Learn Better: On the Importance of Pre-training Compact Models

---

应用

• 2101.10642 Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks BERT 和 ALBERT 句子嵌入对下游自然语言处理任务的影响
• 2010.05522 Pre-trained Language Model Based Active Learning for Sentence Matching 基于预训练语言模型的主动学习句子匹配
• 2103.10385 GPT Understands, Too GPT 也能理解
• 2103.10673 Cost-effective Deployment of BERT Models in Serverless Environment 无服务环境下 BERT 模型的性价比部署
• 2004.02288 Continual Domain-Tuning for Pretrained Language Models 预训练语言模型的持续域微调
• 202004 Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning
• 202002 REALM: Retrieval-Augmented Language Model Pre-Training

---

情感分类

• 2103.07098 A Weakly Supervised Approach for Classifying Stance in Twitter Replies

• 202005 SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis

---

原理

• 202003 What the MASK? Making Sense of Language-Specific BERT Models
• 202002 A Primer in BERTology: What we know about how BERT works
• 201909 How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings
• 201910 T5 Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

阅读理解

• 列表汇总

---

综述

---

开放领域

---

KBQA

• 2104.00218 Integrating Subgraph-aware Relation and Direction Reasoning for Question Answering 基于子图感知关系和方向推理的问答系统

推荐

• 列表汇总

---

概念挖掘

---

模型

• 202011 Graph Neural Networks in Recommender Systems: A Survey

• 1703.04247 DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

• 201606 Wide & Deep Learning for Recommender Systems

• 1601.02376 Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction

• 2010 Factorization Machines

---

新闻推荐

• 1801.08284 DKN: Deep Knowledge-Aware Network for News Recommendation

• KDD2017 Embedding-based News Recommendation for Millions of Users

工程