I’ll start with a concept of 1990 that has become popular: unsupervised learning without a teacher through two adversarial neural networks (NNs) that duel in a minimax game, where one NN minimizes the objective function maximized by the other. The first NN generates data through its output actions, the second NN predicts the data. The second NN minimizes its error, thus becoming a better predictor. But it is a zero sum game: the first NN tries to find actions that maximize the error of the second NN. The system exhibits what I called “artificial curiosity” because the first NN is motivated to invent actions that yield data that the second NN still finds surprising, until the data becomes familiar and eventually boring. A similar adversarial zero sum game was used for another unsupervised method called "predictability minimization," where two NNs fight each other to discover a disentangled code of the incoming data (since 1991), remarkably similar to codes found in biological brains. I’ll also discuss passive unsupervised learning through predictive coding of an agent’s observation stream (since 1991) to overcome the fundamental deep learning problem through data compression. I’ll offer thoughts as to why most current commercial applications don’t use unsupervised learning, and whether that will change in the future.
Biography: Since age 15, the main goal of Jürgen Schmidhuber has been to build a self-improving Artificial Intelligence (AI) smarter than himself, then retire. His lab's deep learning neural networks have revolutionized machine learning. As of 2017, they are on over 3 billion devices, and used billions of times per day, for Facebook's automatic translation, Google's speech recognition and Google Translate, Apple's Siri & QuickType, Amazon's Alexa, etc. He also pioneered unsupervised adversarial networks, artificial curiosity and meta-learning machines that learn to learn. He is recipient of numerous awards, and chief scientist of the company NNAISENSE, which aims at building the first practical general purpose AI. He is also advising various governments on AI strategies.