Merge branch 'master' of https://github.com/nreimers/deeplearning4nlp…

…-tutorial.git

2016-11_Seminar/README.md

@@ -28,19 +28,70 @@ The following is a short list with good introductions to different aspects of de
 ## Theory 1 - Introduction to Deep Learning 
 **Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/1_Theory_Introduction.pdf)
 
+The first theory lesson covers the fundamentals of deep learning. 
+
 ## Theory 2 - Introduction to Deep Learning Frameworks
 **Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/2_Theory_Frameworks.pdf)
 
+The second lesson gives an overview of deep learning frameworks. Hint: Use [Keras](http://keras.io) and have a look at Theano and TensorFlow.
+
 ## Code Session 1 - SENNA Architecture for Sequence Classification
+**Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/Session%201%20-%20SENNA/SENNA.pdf)
+
+**Code:** See folder [Session 1 - SENNA](https://github.com/UKPLab/deeplearning4nlp-tutorial/tree/master/2016-11_Seminar/Session%201%20-%20SENNA)
+
+The first code session is about the SENNA architecture ([Collobert et al., 2011, NLP (almost) from scratch](https://arxiv.org/abs/1103.0398)). In the folder you can find Python code for the preprocessing as well as Keras code to train and evaluate a deep learning model. The folder contains an example for Part-of-Speech tagging, which require the English word embeddings from [Levy et al.](https://levyomer.wordpress.com/2014/04/25/dependency-based-word-embeddings/). 
+
+You can find in this folder also an example for German NER, based on the [GermEval 2014 dataset](https://sites.google.com/site/germeval2014ner/). To run the German NER code, you need the [word embeddings for German from our website](https://www.ukp.tu-darmstadt.de/research/ukp-in-challenges/germeval-2014/).
+
+**Recommended Readings:**
+ * [CS224d - Lecture 2](https://www.youtube.com/watch?v=T8tQZChniMk)
+ * [CS224d - Lecture 3](https://www.youtube.com/watch?v=T1j2Q9_FgTM)
 
 ## Theory 3 - Introduction to Convolutional Neural Networks
 **Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/3_Theory_Convolutional_NN.pdf)
 
+This is an introduction to Convolutional Neural Networks.
+
+**Recommended Readings:**
+ * [CS224d - Lecture 13](https://www.youtube.com/watch?v=EevTPpQvxiU)
+ * [Kim, 2014, Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1408.5882)
+
+
 ## Code Session 2 - Convolutional Neural Networks for Text Classification
+**Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/Session%202%20-%20Sentence%20CNN/Sentence_CNN.pdf)
+
+**Code:** See folder [Session 2 - Sentence CNN](https://github.com/UKPLab/deeplearning4nlp-tutorial/tree/master/2016-11_Seminar/Session%202%20-%20Sentence%20CNN)
+
+This is a Keras implementation of the [Kim, 2014, Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1408.5882). We use the same preprocessing as provided by Kim in his [github repository](https://github.com/yoonkim/CNN_sentence) but then implement the rest using Keras.
+
 
 ## Code Session 3 - Convolutional Neural Networks for Relation Extraction
+**Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/Session%203%20-%20Relation%20CNN/Relation_CNN.pdf)
+
+**Code:** See folder [Session 3 - Relation CNN](https://github.com/UKPLab/deeplearning4nlp-tutorial/tree/master/2016-11_Seminar/Session%203%20-%20Relation%20CNN)
+
+This is an implementation for relation extraction. We use the [SemEval 2010 - Task 8](https://docs.google.com/document/d/1QO_CnmvNRnYwNWu1-QCAeR5ToQYkXUqFeAJbdEhsq7w/preview) dataset on semantic relations. We model the task as a pairwise classification task.
+
+**Recommended Readings:**
+ * [Zeng et al., 2014, Relation Classification via Convolutional Deep Neural Network](http://www.aclweb.org/anthology/C14-1220)
+ * [dos Santos et al., 2015, Classifying Relations by Ranking with Convolutional Neural Networks](https://arxiv.org/abs/1504.06580)
+
 
 ## Theory 4 - Introduction to LSTM
 **Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/4_Theory_Recurrent Neural Networks.pdf)
 
+**Code:** See folder [Session 4 - LSTM Sequence Classification](https://github.com/UKPLab/deeplearning4nlp-tutorial/tree/master/2016-11_Seminar/Session%204%20-%20LSTM%20Sequence%20Classification)
+
+LSTMs are a powerful model and became very popular in 2015 / 2016. 
+
+**Recommended Readings:**
+  * [RNN Effectivness](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
+    * [RNN Effectivness - Video](https://skillsmatter.com/skillscasts/6611-visualizing-and-understanding-recurrent-networks)
+  * [Understanding LSTMs](http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
+  * [C224d Lecture 7](https://www.youtube.com/watch?v=rFVYTydGLr4)
+
 ## Code Session 4 - LSTM for Sequence Classification
+**Slides:** [pdf](https://github.com/UKPLab/deeplearning4nlp-tutorial/raw/master/2016-11_Seminar/Session%204%20-%20LSTM%20Sequence%20Classification/LSTM%20for%20Sequence%20Classification.pdf)
+
+The folder contains a Keras implementation to perfrom sequence classification using LSTM. We use the [GermEval 2014 dataset](https://sites.google.com/site/germeval2014ner/) for German NER. But you can adapt the code easily to any other sequence classification problem (POS, NER, Chunking etc.). Check the slides for more information.

README.md

@@ -1,19 +1,22 @@
 # Deep Learning for NLP - Tutorial
 Hands-on tutorial on deep learning with a special focus on Natural Language Processing (NLP).
 
->This GIT repository accompanies the UKP lecture on Deep Learning for Natural Language Processing.
-In contrast to other lectures, this lecture focuses on the usage of deep learning methods. 
-As programming infrastructure we use Python in combination with Keras and Lasagne.
-This lecture is structured into 6 parts. Each parts contains some recommended readings, 
-which are supposed to be read before class. In class (video will be streamed and recorded) we 
-will discuss the papers and provide some more background knowledge. With the start of the second lecture, 
-each lecture will contain some practical exercise, in the most cases to implement a certain deep neural
-network to do a typical NLP task, for example Named Entity Recognition, Genre Classifcation of Sentiment 
-Classification. The lecture is inspired by an engineering mindset: The beautiful math and complexity of 
-the topic is sometimes neglected to provide instead an easy-to-understand and easy-to-use approach to 
-use Deep Learning for NLP tasks (we use what works without providing a full background on every aspect).
-At the end of the lecture you should be able to understand the most important aspect of deep learning 
-for NLP and be able to programm and train your own deep neural networks.
+This GIT repository accompanies the UKP lectures and seminars on Deep Learning for Natural Language Processing. In contrast to other tutorials, this tutorial focuses on the usage of deep learning methods. 
+
+# Deep Learning for NLP - Seminar - Nov. 2016
+In November 2016 I gave a seminar at the University of Duisburg-Essen. The slides as well as the source code is available in the folder **[2016-11_Seminar](2016-11_Seminar/)**. In the seminar I use Python 2.7, Theano 0.8.2, and Keras 1.1.1 to model four different deep learning models for NLP:
+
+1. Feed Forward Architecture for Sequence Classification (e.g. POS, NER, Chunking)
+2. Convolutional Neural Network for Sentence / Text Classification (e.g. sentiment classification)
+3. Convolutional Neural Network for Relation Extraction (e.g. semantic relation extration)
+4. Long-Short-Term-Memory (LSTM)-Networks for Sequence Classificaiton 
+
+
+# Deep Learning for NLP - Lecture - Oct. 2015
+In October 2015 I gave a lecture for the UKP Department at the Technical University of Darmstadt. The lecture is structured in six parts and covers the basics about deep learning. In the lecture I use Python 2.7, Theano 0.6.0 and Keras 0.3.0 to model different applications of deep learning for NLP. The slides, the source code, and video recordings are available in the folder **[2015-10_Lecture](2015-10_Lecture/)**.
+
+
+## Contact
 
 Contact person: Nils Reimers, [email protected]
 
@@ -24,4 +27,4 @@ http://www.tu-darmstadt.de/
 
 Don't hesitate to send us an e-mail or report an issue, if something is broken (and it shouldn't be) or if you have further questions.
 
-> This repository contains experimental software and is published for the sole purpose of supporting the lectures. 
+> This repository contains experimental software and is published for the sole purpose of supporting the lectures.