[latexpage] On Thursday, OpenAI announced that they had trained a language model. They used a large training dataset and showed that the resulting model was useful for downstream tasks where training data is scarce. They
Unless you have been out of touch with the Deep Learning world, chances are that you have heard about BERT — it has been the talk of the town for the last one year. At the end of 2018 researchers …
Tim Rocktäschel — Co-Founder and Chief Scientist of Recursive; Professor of AI at UCL. Artificial General Intelligence, Open-Endedness, and Recursive Self-Improvement.
Machine learning is the new centre of tech, and like all big new things there are issues. ‘AI bias’ is much-discussed right now: machine learning finds patterns but sometimes it finds the wrong one, and it can be hard to tell.
Over the past few years, deep learning has been taking by storm many industries. From voice recognition to image analysis and synthesis, neural networks have turned out to be very efficient at solv…
I’ve done several experiments with a text-generating neural network called GPT-2. Trained at great expense by OpenAI (to the tune of tens of thousands of dollars worth of computing power), GPT-2 learned to imitate all kinds of text from the internet. I’ve interacted with the basic model, discovering its abilities to…
Google presented a vision of ambient computing that goes beyond the smartphone. The company is well-placed, but faces challenges both in the marketplace and in the mirror.
In the previous article in this series we distinguished between two kinds of unsupervised learning (cluster analysis and dimensionality reduction) and discussed the former in some detail. In this installment we turn our attention to the later. In dimensionality reduction we seek a function $f : \mathbb{R}^n \mapsto…
It's been 7 months since my last commentary on the field, and as it became regular appearance in this blog (and in fact many people apparently enjoy this form and keep asking for it),
I wrote earlier about a neural net-powered dungeon crawling text adventure game called GPT-2-Adventure in which gameplay is incoherent and dreamlike, as you encounter slippery sign text, circular passages, and unexpected lozenge rooms. A PhD student named Nathan trained the neural net on classic dungeon crawling…
Machine learning systems are stuck in a rut Barham & Isard, HotOS’19 In this paper we argue that systems for numerical computing are stuck in a local basin of performance and programmabil…
Neural networks can be good at naming things, I’ve discovered. Recently I’ve been experimenting with a neural network called GPT-2, which OpenAI trained on a huge chunk of the internet. Thanks to a colab notebook implementation by Max Woolf, I’m able to fine-tune it on specific lists of data -
We should expect Apple to lead the industry on this front, but in fact, they’re far behind.
neural networks , convolutional neural networks , convolution , math , probability
When making a start in a new field it is common to feel overwhelmed. You may lack confidence or feel as though you are not good enough or that you are lacking some prerequisite. You will explore these issues in this post and learn that such feelings can lead to actions that can consume a […]
A 2017 cartoon from this post. After the embargo update (Oct 25): Now that I have some answers from the people involved let me make a quick update: 1) I still find the paper unconvincing, specific…
Learning to prove theorems via interacting with proof assistants Yang & Deng, ICML’19 Something a little different to end the week: deep learning meets theorem proving! It’s been a while …
A simple walkthrough of deriving backpropagation for CNNs and implementing it from scratch in Python.
30% less disk required compared to using the single best strategy
In this article I'll be explaining how GauGAN converts simple blobs of color into photorealistic paintings of landscapes.
First part of this series | Previous | Next |Index In the latter part of the last episode, I outlined two problems I want to solve next. The first is to weigh in the direction of my target when fin…
