Deep learning enables computers to process information like human brain. McCulloch and Pitts created first neural network concept in 1940s. Rosenblatt's Mark I Perceptron demonstrated practical neural network use in 1957
Born in 1928 in New Rochelle, New York to Jewish parents. Graduated from Cornell University with A.B. in 1950 and Ph.D. in 1956. Built custom computer EPAC for psychometric analysis in 1951-1953
AlexNet was created by Krizhevsky, Sutskever, and Hinton in 2012. Network contained 60 million parameters and 650,000 neurons. Achieved 15.3% top-5 error in ImageNet competition. Used non-saturating ReLU activation function
LSTM is a recurrent neural network designed to overcome the vanishing gradient problem. It consists of a cell and three gates: input, output, and forget gates. The forget gate determines what information to discard from the previous state. The input gate controls which new information to store. The output gate determines which information to output
Deep learning uses interconnected layers of nodes to process numeric data. Deep neural networks consist of multiple hidden layers for complex data processing. Training involves saving numeric state (weights) in network nodes
ResNet is a 50-layer deep convolutional neural network introduced in 2015. ResNet won ILSVRC 2015 classification competition with 3.57% training error. Deeper neural networks face challenges with training and accuracy degradation