Neural Network
Yurasov Nikita
Computer Security
Higher School of Economics
2020
Yurasov Nikita (HSE)
Neural Network
2020
1/8
Plan
1
Single Neuron
2
Combining Neurons into a Neural Network
3
Training neural network
4
Loss minimization
5
Problems of Neural Networks
6
List of figures
Yurasov Nikita (HSE)
Neural Network
2020
2/8
Single Neuron
Working of neuron:
x1 → x1 ∗ w 1
x2 → x2 ∗ w 2
Figure 1: Single Neuron
(x1 ∗ w1 ) + (x2 ∗ w2 ) + b
y = f (x1 ∗ w1 + x2 ∗ w2 + b)
Figure 2: Sigmoid function
Yurasov Nikita (HSE)
Neural Network
2020
3/8
Single Neuron
Working of neuron:
x1 → x1 ∗ w 1
x2 → x2 ∗ w 2
Figure 1: Single Neuron
(x1 ∗ w1 ) + (x2 ∗ w2 ) + b
y = f (x1 ∗ w1 + x2 ∗ w2 + b)
Figure 2: Sigmoid function
Yurasov Nikita (HSE)
Neural Network
2020
3/8
Single Neuron
Working of neuron:
x1 → x1 ∗ w 1
x2 → x2 ∗ w 2
Figure 1: Single Neuron
(x1 ∗ w1 ) + (x2 ∗ w2 ) + b
y = f (x1 ∗ w1 + x2 ∗ w2 + b)
Figure 2: Sigmoid function
Yurasov Nikita (HSE)
Neural Network
2020
3/8
Combining Neurons into a Neural Network
Feedforward propagation:
Let’s use w = [0, 1], x = [2, 3], b = 0
h1 = h2 = f (w ∗ x + b) =
= f ((0 ∗ 2) + (1 ∗ 3) + 0) =
= f (3) = 0.9526
Figure 3: Complete NN
o1 = f (w ∗ [h1 , h2 ] + b) =
= ((o ∗ h1 ) + (1 ∗ h2 ) + b) =
= f (0.9526) = 0.7216
Yurasov Nikita (HSE)
Neural Network
2020
4/8
Training neural network
Example of data:
Name
Nikita
Alex
Vika
...
Gender
0
0
1
...
Height
185
190
167
...
Weight
70
87
56
...
Loss function for one observation
1
1󰁛
M SE =
(ytrue − ypred )2 = (ytrue − ypred )2
1
i=1
Yurasov Nikita (HSE)
Neural Network
2020
5/8
Loss minimization
Loss function is used only with a single training example. But cost function
is the cost of the parameters.
Cost function
M
M
1 󰁛
1 󰁛
(i) (i)
J(w, b) =
L(ŷ , y ) =
(ytrue − ypred )2
M
M
i=1
i=1
Minimization task
w, b = arg min J(w, b)
w,b
Solution: gradient descent (stochastic)
Yurasov Nikita (HSE)
Neural Network
2020
6/8
Problems of Neural Networks
Figure 4: Big NN
Yurasov Nikita (HSE)
Neural Network
2020
7/8
List of figures
Figure 1: https://towardsdatascience.com/machine-learning-forbeginners-an-introduction-to-neural-networks-d49f22d238f9
Figure 2: https://towardsdatascience.com/machine-learning-forbeginners-an-introduction-to-neural-networks-d49f22d238f9
Figure 3: https://towardsdatascience.com/machine-learning-forbeginners-an-introduction-to-neural-networks-d49f22d238f9
Figure 4: https://sci-news.ru/2019/oblasti-primenenija-trendy-itehnologii-iskusstvennogo-intellekta/
Yurasov Nikita (HSE)
Neural Network
2020
8/8