Week 10, 11, 12
Probabilistic Graphical Models
"Graphical Models and Message Passing Algorithms: Some Introductory Lectures" by Martin J. Wainwright
"An Introduction to Restricted Boltzmann Machines", by Asja Fischer and Christian Igel
"An Introduction to Conditional Random Fields", by Charles Sutton, Andrew McCallum
Week 8 and 9
Recurrent Neural Networks
Understanding LSTM Networks (blog by Christopher Olah)
Long short-term memory, by Hochreiter, & Schmidhuber, in Neural computation, 9(8), 1997, 1735-1780.
Learning to forget: Continual prediction with LSTM, by Gers, Schmidhuber, & Cummins in Neural computation, 12(10), 2000, 2451-2471.
Week 7
Visualizing what a CNN learns
Stanford CS231n: Visualizing What ConvNets Learn
Visualizing Higher-Layer Features of a Deep Network, by Erhan et al.
Visualizing and Understanding Convolutional Networks, by Zeigler and Fergus
Understanding Neural Networks Through Deep Visualization, by Yosinski et al.
Week 6
Why deep learning works
On the Number of Linear Regions of Deep Neural Networks
Escaping From Saddle Points – Online Stochastic Gradient for Tensor Decomposition
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
The Loss Surfaces of Multilayer Newww.jmlr.org/proceedings/papers/v38/choromanska15.pdftworks
Week 5
Some Specialized Convolutional Neural Networks for Image Classification
AlexNet
VggNet
Deeply Supervised Nets
GoogLeNet and Inception Module
Deep Residual Nets
Week 4
Simple improvements to Convolutional Neural Networks
"Stochastic Pooling for Regularization of Deep Convolutional Neural Networks" by Zeiler and Fergus, in ICLR 2013.
"Dropout: A Simple Way to Prevent Neural Networks from Overfitting", by Srivastava, Hinton, Krizhevsky, Sutskever, Salakhutdinov, in JMLR 2014.
"Regularization of Neural Networks using DropConnect" by Wan, Zeiler, Zhang, LeCun, Fergus, in ICML2013.
Week 3
Introduction to Convolutional Neural Networks
"Gradient-based learning applied to document recognition" by Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, in Proceedings of the IEEE, November 1998.
"Deep Learning" by Michael Nielsen.
Week 2
Neural Networks
"Artificial Neural Networks: A Tutorial" by AK Jain, J Mao, KM Mohiuddin, in IEEE Computer, 1996.
"Universal approximation using FNNs: A survey of some existing methods, and some new results" by F Scarselli, AC Tsoi in Neural networks, 1998.
"Using neural networks to recognize handwritten digits" by M Nielson (Consider donating to his site, because it seems to be a lot of work, and awesome)
Week 1
Logistic Regression
Logisitc Regression on Wikipedia
"Logistic Regression Tutorial for ML" by J Brownlee
"Why the logistic function? A tutorial discussion on probabilities and neural networks" by MI Jordan
LASSO
"Regression Shrinkage and Selection via the LASSO" by R Tibshriani, in J R Stat Soc B, 1996.
Elastic Net
"Regularization and variable selection via the elastic net" by H Zou, T Hastie, in J R Stat Soc B, 2005.
SCAD Penalty
"Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties" by J Fan, R Li, in J Am Stat Assoc, 2001.
Elastic SCAD SVM for HDLSS Binary Classification by N Becker, G Toedt, P Lichter, A Benner, in BMC Bioinformatics 2011.
Probabilistic Graphical Models
"Graphical Models and Message Passing Algorithms: Some Introductory Lectures" by Martin J. Wainwright
"An Introduction to Restricted Boltzmann Machines", by Asja Fischer and Christian Igel
"An Introduction to Conditional Random Fields", by Charles Sutton, Andrew McCallum
Week 8 and 9
Recurrent Neural Networks
Understanding LSTM Networks (blog by Christopher Olah)
Long short-term memory, by Hochreiter, & Schmidhuber, in Neural computation, 9(8), 1997, 1735-1780.
Learning to forget: Continual prediction with LSTM, by Gers, Schmidhuber, & Cummins in Neural computation, 12(10), 2000, 2451-2471.
Week 7
Visualizing what a CNN learns
Stanford CS231n: Visualizing What ConvNets Learn
Visualizing Higher-Layer Features of a Deep Network, by Erhan et al.
Visualizing and Understanding Convolutional Networks, by Zeigler and Fergus
Understanding Neural Networks Through Deep Visualization, by Yosinski et al.
Week 6
Why deep learning works
On the Number of Linear Regions of Deep Neural Networks
Escaping From Saddle Points – Online Stochastic Gradient for Tensor Decomposition
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
The Loss Surfaces of Multilayer Newww.jmlr.org/proceedings/papers/v38/choromanska15.pdftworks
Week 5
Some Specialized Convolutional Neural Networks for Image Classification
AlexNet
VggNet
Deeply Supervised Nets
GoogLeNet and Inception Module
Deep Residual Nets
Week 4
Simple improvements to Convolutional Neural Networks
"Stochastic Pooling for Regularization of Deep Convolutional Neural Networks" by Zeiler and Fergus, in ICLR 2013.
"Dropout: A Simple Way to Prevent Neural Networks from Overfitting", by Srivastava, Hinton, Krizhevsky, Sutskever, Salakhutdinov, in JMLR 2014.
"Regularization of Neural Networks using DropConnect" by Wan, Zeiler, Zhang, LeCun, Fergus, in ICML2013.
Week 3
Introduction to Convolutional Neural Networks
"Gradient-based learning applied to document recognition" by Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, in Proceedings of the IEEE, November 1998.
"Deep Learning" by Michael Nielsen.
Week 2
Neural Networks
"Artificial Neural Networks: A Tutorial" by AK Jain, J Mao, KM Mohiuddin, in IEEE Computer, 1996.
"Universal approximation using FNNs: A survey of some existing methods, and some new results" by F Scarselli, AC Tsoi in Neural networks, 1998.
"Using neural networks to recognize handwritten digits" by M Nielson (Consider donating to his site, because it seems to be a lot of work, and awesome)
Week 1
Logistic Regression
Logisitc Regression on Wikipedia
"Logistic Regression Tutorial for ML" by J Brownlee
"Why the logistic function? A tutorial discussion on probabilities and neural networks" by MI Jordan
LASSO
"Regression Shrinkage and Selection via the LASSO" by R Tibshriani, in J R Stat Soc B, 1996.
Elastic Net
"Regularization and variable selection via the elastic net" by H Zou, T Hastie, in J R Stat Soc B, 2005.
SCAD Penalty
"Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties" by J Fan, R Li, in J Am Stat Assoc, 2001.
Elastic SCAD SVM for HDLSS Binary Classification by N Becker, G Toedt, P Lichter, A Benner, in BMC Bioinformatics 2011.