Deep Learning
Deep learning has already transformed traditional internet businesses like web search and advertising. But it is also enabling brand new products, businesses, and ways of helping people. Applications range from better healthcare (where deep learning is getting very good at reading X-ray images), to delivering personalized education, to precision agriculture, to self-driving cars, and many others.
These notes cover the Deep Learning Specialization by Andrew Ng (DeepLearning.AI).
Course material ages, so every course below records when it was first published and when its publisher last revised it. The specialization launched in August 2017 and received one major revision in April 2021, which moved the programming assignments to TensorFlow 2 and added MobileNet and U-Net to Course 4 and Transformers to Course 5. Everything else reflects deep learning practice as it stood in 2017 and 2018, so read framework APIs, benchmark numbers, and any “state of the art” framing as historical rather than current. The notes follow the course as taught. Lab code is the exception, ported to the current Python stack, with a callout on each lab page naming what changed.
AI is the New Electricity
Starting about 100 years ago, the electrification of society transformed every major industry: transportation, manufacturing, healthcare, communications, and more. Today, we see a surprisingly clear path for AI to bring about an equally big transformation.
The part of AI that is rising rapidly and driving a lot of these developments is deep learning. It is one of the most highly sought-after skills in the technology world today.
Specialization Roadmap
The Deep Learning Specialization consists of five courses.
Course 1: Neural Networks and Deep Learning
The foundational course. It covers the foundations of neural networks, how to build a neural network (including a deep neural network), and how to train it on data. By the end, you will be able to build a deep neural network to recognize cats (following a long-standing tradition in the deep learning community).
Reference: https://www.coursera.org/learn/neural-networks-deep-learning
Published: August 2017. Last major update: April 2021 (assignments moved to TensorFlow 2).
Course 2: Improving Deep Neural Networks
Once you have built a neural network, how do you actually get it to perform well? This course covers the practical aspects, including hyperparameter tuning, regularization, diagnosing bias and variance, and advanced optimization algorithms (momentum, RMSProp, Adam). A lot of building neural networks can feel like black magic. This course demystifies that process.
Reference: https://www.coursera.org/learn/deep-neural-network
Published: August 2017. Last major update: April 2021 (assignments moved to TensorFlow 2).
Course 3: Structuring Machine Learning Projects
The strategy for building a machine learning system has changed in the era of deep learning. This course covers how to split data into train, development (dev/holdout/cross-validation), and test sets using modern best practices, how to handle training and test sets that come from different distributions, and when to use (and when not to use) end-to-end deep learning. It draws from hard-won lessons in building and shipping deep learning products in production.
Reference: https://www.coursera.org/learn/machine-learning-projects
Published: August 2017. Last major update: none announced. The April 2021 revision covered Courses 1, 2, 4, and 5, and this course carries no programming assignments.
Course 4: Convolutional Neural Networks
Convolutional neural networks are often applied to images. This course covers how to build these models and apply them to computer vision tasks, including image classification, object detection, face recognition, and neural style transfer.
Reference: https://www.coursera.org/learn/convolutional-neural-networks
Published: November 2017. Last major update: April 2021 (TensorFlow 2, plus new MobileNet and U-Net material).
Course 5: Sequence Models
Sequence models handle sequential data like text and audio. This course covers Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) models, and applications to natural language processing, speech recognition, and music generation. Natural language is just a sequence of words, and these models are specifically designed to work with that kind of sequential structure.
Reference: https://www.coursera.org/learn/nlp-sequence-models
Published: early 2018. Last major update: 2021 (TensorFlow 2, plus a new week on Transformers).