## OPTIMIZATION Course

# Hessian/Gradient

The gradient (first derivative) and the Hessian (second derivative), will be used quite a lot in unconstrained optimization.

## Gradient

We are calling **gradient**, noted nabla ($\nabla$), the "derivative" of $f$.

That's not a simple derivative, as, in most cases, $f$ usually has multiples parameters. We will calculate partial derivatives. Example:

- $f(x,y)$
- we got two variables
- derivative of x, y is a constant: $\frac{\partial f}{\partial x}$
- derivative of y, x is a constant: $\frac{\partial f}{\partial y}$

And our gradient would be

## Hessian

Following what you learned for the gradient, the **hessian** is the derivative of a gradient. We note it $Hf$ if $f$ is our function.

## Exercise

What's the gradient and the hessian of this function?

\[ f(x, y) = 2x^2 + y^2 − 2xy + 4x \]

Let's calculate our gradient

- \(\frac{\partial f}{\partial x} = 4x + 0 + - 2 * 1 * y + 4 = 4x -2y + 4\)
- \(\frac{\partial f}{\partial y} = 2 * 0 + 2y + - 2 * x * 1 + 0 = 2y-2x\)
So we have the gradient \[ \nabla f(x,y) = \begin{pmatrix}4x -2y + 4\\2y-2x\end{pmatrix} \]

Then round 2, the hessian

- 0,0 in x: $4−0+0 = 4$
- 0,0 in y: $0−2∗1 = -2$
- 1,0 in x: $0 -2 * 1 + 0 = -2$
- 1,0 in y: $2*1-0 = 2$
So we have the hessian \[ Hf(x,y)=\begin{pmatrix}4&-2\\-2&2\end{pmatrix} \] I hope you took the time to do this easy one!