Calculus [Online course from MIT]
Linear Algebra [CS6015 or equivalent]  [Online course from MIT]
Probability Theory [CS6015 or equivalent]  [Online course from MIT]
Nonlinear Optimization [CS5020 or equivalent]  [First Course in Optimization by Prof. Soman (IITB) available on CDEEP]
Pattern Recognition and Machine Learning [CS5691 or equivalent]  [Andrew Ng's ML course]
Instructor: Mitesh M. Khapra
When: JanMay 2019
Lectures: Slot E
Where: Online (cisco webex link posted on moodle)
Teaching Assistants: To be announced
Name  Lab  Office hours  Days  

Kaarthik Raja M V  SMT Lab  cs16b108@smail.iitm.ac.in  34 pm  Tuesday, Thursday 
Priya Jain  SMT Lab  cs19m049@smail.iitm.ac.in  121 pm  Tuesday, Thursday 
Janakiram Yellapu  VP Lab  cs19m028@smail.iitm.ac.in  24 pm  Wednesday 
Sasank Annavarapu  NA  cs19m058@smail.iitm.ac.in  34 pm  Tuesday, Wednesday 
Ankit Singh  CV Lab  singh.ankit@smail.iitm.ac.in  34 pm  Monday, Friday 
Himasagar Mallepalli  VP Lab  cs19m036@smail.iitm.ac.in  13 pm  Wednesday 
Shivani Baranwal  SMT Lab  cs19m062@smail.iitm.ac.in  12 pm  Tuesday, Friday 
Bethu Sai Sampath  NA  cs19m018@smail.iitm.ac.in  24 pm  Wednesday 
B Krishnanjali  NA  cs19m017@smail.iitm.ac.in  34 pm  Monday, Friday 
Bikash Kumar Behera  NA  cs19m019@smail.iitm.ac.in  46 pm  Tuesday 
Lecture#  Contents  Lecture Slides  Lecture Videos  Extra Reading Material 

Lecture 0  Syllabus, Logistics  Slides     
Lecture 1  (Partial) History of Deep Learning, Deep Learning Success Stories  M1  M2 M3  M4  M5 M6  M7  M8 M9  Juergen Schmidhuber, Deep Learning in Neural Networks: An Overview. 2014  
Lecture 2  McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs  M1  M2 M3  M4  M5 M6  M7  M8  Chapters 1,2,3,4 from Neural Networks by Rojas  
Lecture 3  Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks  M1  M2 M3  M4  M5  http://neuralnetworksanddeeplearning.com/chap4.html  
Lecture 4  Feedforward Neural Networks, Backpropagation  M1  M2  M3  M4  M5  M6  M7  M8  M9  M10  See Lecture 2 [Training Neural Networks] by Hugo Larochelle  
Lecture 5  Gradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam  T  H  M1 and M2  M3  M4  M5  M6  M7  M8  M9  M9 part 2  
Lecture 6  Eigenvalues and eigenvectors, Eigenvalue Decomposition, Basis, Principal Component Analysis and its interpretations, Singular Value Decomposition  T  H  M1  M2 M3  M4  M5 M6  M6 part 2  M7 M8  
Lecture 7  Autoencoders and relation to PCA, Regularization in autoencoders, Denoising autoencoders, Sparse autoencoders, Contractive autoencoders  T  H  M1  M2 M3  M4  M5 M6  
Lecture 8  Bias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout  T  H  M1  M2 M2 part 2  M3  M4 M5  M6  M7  M8  M9  M10 M11  
Lecture 9  Greedy Layerwise Pretraining, Better activation functions, Better weight initialization methods, Batch Normalization  T  H  M1  M2 M3  M4  M5  
Lecture 10  Learning Vectorial Representations Of Words  T  H  M1  M2 M3  M3 part 2  M4 M5  M5 part 2  M6  M7  M8  M9  M10  
Lecture 11  Convolutional Neural Networks, LeNet, AlexNet, ZFNet, VGGNet, GoogLeNet, ResNet  T  H  M1  M2 M3  M3 part 2  M4 M4 part 2  M5  
Lecture 12  Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks  T  H  M1  M2 M3  M4  M5 M6  M7  M8  M9  M10 

Lecture 13  Recurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT  T  H  M1  M2 M3  M4  M5  
Lecture 14  Gated Recurrent Units (GRUs), Long Short Term Memory (LSTM) Cells, Solving the vanidhing gradient problem with LSTMs  T  H  M1  M2 M3  M3 part 2  
Lecture 15  Encoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical Attention  T  H  M1  M2 M3  M3 part 2  M4 M5  
Lecture 16  Directed Graphical Models  T  H  Will be available soon  
Lecture 17  Markov Networks  T  H  Will be available soon  
Lecture 18  Using joint distributions for classification and sampling, Latent Variables, Restricted Boltzmann Machines, Unsupervised Learning, Motivation for Sampling  T  H  Will be available soon  
Lecture 19  Markov Chains, Gibbs Sampling for training RBMs, Contrastive Divergence for training RBMs  T  H  Will be available soon  
Lecture 20  Variational autoencoders  T  H  Will be available soon  
Lecture 21  Autoregressive Models: NADE, MADE, PixelRNN  T  H  Will be available soon  
Lecture 22  Generative Adversarial Networks (GANs)  T  H  Will be available soon 
Topics  Resources  Release Date  Submission Date  

Assignment 0  History of DL  05Feb2021  15May2021  
Assignment 1  Feedforward Neural Networks  Link  20Feb2021  13Mar2021 
Assignment 2  Convolutional Neural Networks  Link  13Mar2021  03Apr2021 
Assignment 3  Recurrent Neural Networks  Link  03Apr2021  24Apr2021 
Assignment 4  Deep Generative Models  Link  28Apr2021  25May2021 
Topics  Resources  Release Date  Submission Date  

Tutorial 1  Calculus  13Feb2021  
Tutorial 2  Linear Algebra  13Feb2021  
Tutorial 3  MP Neurons, Perceptrons  20Feb2021  
Tutorial 4  Sigmoid Neurons, Gradient Descent  26Feb2021  
Tutorial 5  Feedforward Neural Networks, Backpropagation  26Feb2021 
Deep Learning for Computer Vision [from Stanford]
Deep Learning for NLP [from Stanford]