Utkarsh ERA V6 Part1 Backpropogation
- Nodes:
i1
,i2
- Description: The input layer is where the network receives its input data. There are two input nodes in this network.
Weights between Input and Hidden Layer
- Weights:
w1
,w2
,w3
,w4
- Description: Each connection between the input and hidden layer nodes has an associated weight. These weights adjust the input values as they are passed to the next layer.
Hidden Layer
- Nodes:
h1
,h2
- Description: The hidden layer processes the inputs received from the input layer. It combines the weighted inputs and applies a nonlinear sigmoid activation function. The diagram connections (
a_h1
,a_h2
).
Weights between Hidden and Output Layer
- Weights:
w5
,w6
,w7
,w8
- Description: These weights modulate the signals being passed to the output layer from the hidden layer.
- Nodes:
o1
,o2
- Description: The output layer receives the processed signals from the hidden layer and generates the output of the network. Each output node represents a specific output value.
- Activations:
a_o1
,a_o2
- Description: These denote the activation functions applied at the output nodes, determining the final output of each node.
- Errors:
E1
,E2
- Description: The network calculates the error for each output node by comparing the actual output with the target output. Error formula suggests L2 Loss
- Node:
E_Total
- Description: The total error of the network is calculated by summing the errors from all output nodes. This value is crucial during the training process.
The network is trained using a backpropagation algorithm to adjust the weights based on the calculated error. The goal is to minimize E_Total
by iteratively updating the weights to improve the model's predictions.
Loss function reduces linearly
Loss function reduces more steeply
![image](https://private-user-images.githubusercontent.com/36775837/309418685-4f3c8ef0-96e4-4354-956f-dc77294e62ff.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NjI2NTEsIm5iZiI6MTcyNDg2MjM1MSwicGF0aCI6Ii8zNjc3NTgzNy8zMDk0MTg2ODUtNGYzYzhlZjAtOTZlNC00MzU0LTk1NmYtZGM3NzI5NGU2MmZmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDE2MjU1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTI2MzI1ZjJlMGMyYjg2YWIzZDlhYjEyODcxMjM3MDgzY2IxODlkYTJmZWYwNDQxZjIyZmU1OGJjYzUwNjQwMmImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.-D_ToFdByDH7HRXJHg9JU75piQ3XI4SbX57SAFdPFUw)
Loss function reduces more steeply compared to 0.2
![image](https://private-user-images.githubusercontent.com/36775837/309418911-d0004646-e2a2-4518-b0aa-d8cd37d29600.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NjI2NTEsIm5iZiI6MTcyNDg2MjM1MSwicGF0aCI6Ii8zNjc3NTgzNy8zMDk0MTg5MTEtZDAwMDQ2NDYtZTJhMi00NTE4LWIwYWEtZDhjZDM3ZDI5NjAwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDE2MjU1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWY2NTViOGZkNzM2ZjIwNTk0NWM5NTU4N2MxODViMzkwMWIwZWE5YzY2NmE4Y2EzZTNjMjk3ZTgwNDU5YWU3NjMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ka_3fdwqJX9KnAf7ffSTMDeHTaR5jH_yhygY-HNbveQ)
Loss function reduces more steeply compared to 0.5
![image](https://private-user-images.githubusercontent.com/36775837/309418976-f6b11b18-f5aa-4189-9fa0-e3644ff9c00e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NjI2NTEsIm5iZiI6MTcyNDg2MjM1MSwicGF0aCI6Ii8zNjc3NTgzNy8zMDk0MTg5NzYtZjZiMTFiMTgtZjVhYS00MTg5LTlmYTAtZTM2NDRmZjljMDBlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDE2MjU1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI5YjFlMzliNDc0MmMxZmRkMzQ0NzdmZGEzMDg4OWUzY2VhZjE2ZWJjOTRjZTUzZDdlMTVkZGViMDRlOGU0MDYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.WO9K7mKIyS0N_thXdj1F2b0x9D57cQdJIqKwV2jMx7s)
![image](https://private-user-images.githubusercontent.com/36775837/309419059-f3643ca7-8b49-4458-a073-925149186308.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NjI2NTEsIm5iZiI6MTcyNDg2MjM1MSwicGF0aCI6Ii8zNjc3NTgzNy8zMDk0MTkwNTktZjM2NDNjYTctOGI0OS00NDU4LWEwNzMtOTI1MTQ5MTg2MzA4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDE2MjU1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk0NmI2MTRlNzc5ZWI0MmY1MWIyZmVhM2Q1ZWFlM2Q1NjE4M2MyZTk1OWNlYzU1NWMwNWQ5OTQxNDNlMjZmMGYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.8jP33MtVkZb_0f315dXatTULja0zEmGJKru305qGOxs)
![image](https://private-user-images.githubusercontent.com/36775837/309419140-36863de2-4dfb-404b-b936-215f5414b854.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NjI2NTEsIm5iZiI6MTcyNDg2MjM1MSwicGF0aCI6Ii8zNjc3NTgzNy8zMDk0MTkxNDAtMzY4NjNkZTItNGRmYi00MDRiLWI5MzYtMjE1ZjU0MTRiODU0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDE2MjU1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTc1ZmRjODczNTM0OTljOTg2N2Y0ZjVkZDIyYzcwNTI0NGI0ZjE3ZWE1ZGRkMmM0ZGQ1YmE3ZGM2MDBjZWRhYjcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.uEbRqlJqns8TqtyxojvnTQr1R6Gf38-eU9E6qS9E7Jk)