RNNs are computationally more powerful and biologically more plausible than other adaptive approaches such as Hidden Markov Models (no continuous internal states), feedforward networks and Support Vector Machines (no internal states at all). Our recent applications include adaptive robotics and control, handwriting recognition, speech recognition, keyword spotting, music composition, attentive vision, protein analysis, stock market prediction, and many other sequence problems.

Early RNNs of the 1990s could not learn to look far back into the past. Their problems were first rigorously analyzed on Schmidhuber's RNN long time lag project by his former PhD student Hochreiter (1991). A feedback network called "Long Short-Term Memory" (LSTM, Neural Comp., 1997) overcomes the fundamental problems of traditional RNNs, and efficiently learns to solve many previously unlearnable tasks involving:

• Recognition of temporally extended patterns in noisy input sequences
• Recognition of the temporal order of widely separated events in noisy input streams
• Extraction of information conveyed by the temporal distance between events
• Stable generation of precisely timed rhythms, smooth and non-smooth periodic trajectories
• Robust storage of high-precision real numbers across extended time intervals.

LSTM has transformed machine learning and Artificial Intelligence (AI), and is now available to billions of users through the world's four most valuable public companies: Apple (#1 as of March 31, 2017), Google (Alphabet, #2), Microsoft (#3), and Amazon (#4).