# Deep Learning at a Glance ## Purpose of these notes These notes are born out of my preparations for co-developing and co-teaching the Introduction to Deep Learning course at Binghamton University in the Fall of '22. So this is not a book and doesn't provide a comprehensive treatment of the material from the ground up. For a topic like deep learning, that may be an insurmountable task. But rather, it aims to hit the most important points in a short amount of time, as you would do in a lecture. The goal is to develop an intuition for the subject and get students working with models, libraries, and code as quickly as possible. ## What is Deep Learning? Defining deep learning can be a tricky topic. Deep learning models are based on neural networks combined together in hierarchical models. The usual criteria for deep learning algorithms are based on neural networks with 3 or more layers. The depth of the neural network does play an important role in the effectiveness of the model. But, such definitions can seem a bit arbitrary and don't provide insight into the behavior of learning algorithms and why they are so powerful. ## Deep Learning vs Machine Learning So, as we see from the definition above, it is more instructive to understand deep learning (and its associated algorithms) by comparing and contrasting its capabilities, especially with the more traditional machine learning algorithms. Deep learning is a subset of machine learning, but some key differentiating factors are: 1. **Based on Neural Networks (parametric):** DL models are parametric methods. Their weights parameterize them! Contrast this with non-parametric models such as kNNs, k-Means, Gaussian processes, etc. 2. **No feature engineering:** DL models generally operate on "raw" data like images and sentences. Instead of manual feature extraction, deep learning algorithms are designed to automatically extract the relevant features. ![A venn diagram of the learning methods](/files/C7UJPHSdDRjQlAYQQSaH) ## Why Deep Learning? Deep learning presents a significant paradigm shift to the status quo of traditional machine learning. Traditional machine learning models rely heavily on feature engineering. Feature engineering that a human usually does with intimate domain knowledge. Researchers painstakingly went through the raw input data and combined and transformed the data to effectively summarize the data. This involved a significant amount of trial and error. Real-world data is difficult to handle. It is high-dimensional and often very noisy. There is a high variance in the data, and there is a LOT of data. Feature engineering for real-world applications is difficult. Deep learning solves this problem. In the case of DL, the features are learned automatically! Data directly from the sensors can be fed to deep learning models with minimal transformations, and significantly in the absence of a human expert. Furthermore, since domain-specific feature engineering is not needed, the algorithms can be used in many different domains, and are robust to variations in the data. Features are learned from data, so while the traditional machine learning models' performance is saturated as the amount of data increases, deep learning models tend to perform better and better with more data. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://szaman.gitbook.io/intro-to-deep-learning/readme.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.