Sebastian Ruder

Deep Learning for NLP Best Practices

The secret ingredients of word2vec (2016)

Word2vec is a pervasive tool for learning word embeddings. Its success, however, is mostly due to particular architecture choices. Transferring these choices to traditional distributional methods makes them competitive with popular word embedding methods.

Word embeddings in 2017: Trends and future directions

Do NLP Beyond English

7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English.

An overview of gradient descent optimization algorithms (2016)

Gradient descent is the preferred way to optimize neural networks and many other machine learning algorithms but is often used as a black box. This post explores how many of the most popular gradient-based optimization algorithms such as Momentum, Adagrad, and Adam actually work.

The State of Transfer Learning in NLP

Optimization for Deep Learning Highlights