Lecture 12: Lexical Semantics (part I) - Word Representations and Word Embeddings.

• Co-occurence Matrix
• Geometric Interpretation
• Cosine Similarity
• Distributional Semantic Models
• Word Embeddings
• SVDF
  • Use matrix decomposition on co-occurence matrix, for example Singular Value Decomposition (SVD).
  • Learn embeddings using neural networks. Minimal feature- engineering required.
• Skip-Gram Model: loss

Lecture 13: Machine Learning: Linear and Log-Linear Models

Lecture 14: Machine Learning: Feed-forward Neural Networks, Autoencoders/embeddings, Dense networks

Autoencoders Embeddings (Word level semantics)

Skip-Gram Model

Continuous Bag-of-Words Model (CBOW)

output？？
loss？？

Lecture 15: Neural Models for NLP

Doc2vec/paragraph2vec

CNN

Lecture 16: Neural Machine Translation, Attention and Self-attention

• Attention
• Computing BLEU

Lecture 17: Transformer and BERT