Unsupervised Learning, Recommenders, Reinforcement Learning

Course 3 notes on unsupervised learning, recommender systems, and reinforcement learning, from clustering and anomaly detection to a simulated moon lander.

Topics

Course 3 of 3 in the Machine Learning Specialization by Andrew Ng (DeepLearning.AI / Stanford).

The first two courses, Supervised Machine Learning and Advanced Learning Algorithms, spent a lot of time on supervised learning. This third and final course goes beyond it, with a new set of powerful techniques in three parts.

• Unsupervised learning. Clustering algorithms, a way of grouping data into clusters, as well as anomaly detection. Both are techniques used by many companies today in important commercial applications.
• Recommender systems. When you go to an online shopping website or a video streaming website, how does it recommend products or movies to you? Recommender systems are one of the most commercially important machine learning technologies, moving many billions of dollars of value around, yet they receive surprisingly little attention from academia. The same ideas also give a sense of how large online ad-tech companies decide which ads to show.
• Reinforcement learning. Famous for playing a variety of video games, even outperforming humans, and used to control many kinds of robots. It is a newer, emerging technology, with fewer commercial applications so far than the techniques above, but it is opening a new frontier for what learning algorithms can do, and these notes culminate in using it to land a simulated moon lander.

Notes

These pages follow the Coursera videos and labs, rewritten for high school readers with review questions and plots where they help.