# Autograd Demo

[(Assuming you've gone through the previous set of notes)](/intro-to-deep-learning/deep-learning-engineering/getting-started-with-pytorch.md)\
So autograd is pretty cool. But apart from week 1 calculus homework, what else is it good for? Well, let's try to put what we know about SGD, tensors, and autograd and learn a function from data!

#### We start with some random looking data points (features, and labels)

```python
x_train = torch.tensor([.0077036, .050490, .13341, .47190, 18.772, 115.76])
y_train = torch.tensor([.058864, .37950, 1.0000, 3.5396, 140.61, 860.09])


print(x_train.shape)
print(y_train.shape)
```

We want to learn a simple linear\
function such that,

$$
y = Wx
$$

Those familiar with linear modelling can see that's just a linear regression model.

Ok, next. Let's define a `loss_fun` that is the mean squared loss (MSE):

$$
\mathcal{L} = \frac{1}{N} \sum\_{i}^{N} ||y\_{pred}-y ||^2
$$

We'll also need a way to update W, so we'll use gradient descent.

$$
W^{i+1} = W^{i} - \eta \frac{\partial \mathcal{L}}{ \partial W^i}
$$

We can calculate the derivative by hand, and get the formula

$$
W^{i+1} = W^{i} - 2\eta x (y\_{pred}-y)
$$

But we won't go to the trouble, and use `autograd` instead.<br>

```python
def predictor(_input, parameter):
  y_pred = parameter * _input
  return y_pred

def loss_func(y_pred, y):
  return torch.mean((y_pred - y) ** 2)

# Initialize parameter with a random number [0, 1)
# with gradient tracking turned on
# The actual value we start with is not particularly relevant.
W = torch.rand([1], requires_grad=True) 

learning_rate = 1e-6

for iteration in range(2000):
  if W.grad is not None:
    # Let's 0 the gradient just in case the gradient has been accumulated
    W.grad.zero_() 
  
  y_pred = predictor(x_train, W)
  loss = loss_func(y_pred,y_train)
  loss.backward()
  with torch.no_grad(): # Turn off gradient for the update
    # Just making sure the value we take out is 
    # detached from the computation graph
    old_param = W.clone().detach() 
    grad = W.grad
    W = W - learning_rate * grad # Update
    W.requires_grad_()
  
  if (iteration % 100 == 0):
    # Print out the state of our calculations
    print(f"Epoch {iteration}, Loss: {loss.item():.4f} \t Gradient "+
          f"{grad.item():.4f} \t W_t: {old_param.item():.4f} "+
          f"\t W_t+1 : {W.item():.4f}") 

```

***

**TASK:**

a) Generate 20 random numbers between 0 to 100. Let's say these are temperature measurements in Fahrenheit. Also, generate the Celsius counterparts of these temperatures as labels.

b) Can you use the above formalism to obtain a model for a function that takes a temperature in units of Fahrenheit, and outputs the temperature in Celsius?

c) How did your model perform? What modifications did you have to make?


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://szaman.gitbook.io/intro-to-deep-learning/deep-learning-engineering/getting-started-with-pytorch/autograd-demo.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.