SciPy Optimizers

What scipy.optimize Does

scipy.optimize is SciPy's subpackage for finding the input that minimizes a function, or the input where a function crosses zero. Those two problems - minimization and root-finding - cover an enormous range of practical tasks, from fitting a curve to data to solving an equation that has no neat algebraic solution.

• minimize() - find the input that minimizes a scalar function of one or more variables
• minimize_scalar() - the same idea, specialized for a single-variable function
• root() - find where a function of one or more variables equals zero
• root_scalar() - the same idea, specialized for a single-variable function
• curve_fit() - fit a function with adjustable parameters to a set of data points
• linprog() - solve linear programming problems with linear constraints

Minimizing a Function

from scipy.optimize import minimize

def f(x):
    return x**2 + x + 2

result = minimize(f, x0=0)

print(result.x)    # array([-0.5])
print(result.fun)  # 1.75

Calculus tells us the minimum of x**2 + x + 2 sits where its derivative, 2x + 1, equals zero - at x = -0.5, giving a value of 1.75. minimize() finds this numerically without you ever writing the derivative yourself, starting from the initial guess x0 and refining it step by step. The returned result object carries the location in .x, the function value in .fun, and a .success flag telling you whether the algorithm converged.

Finding a Root

from scipy.optimize import root
import numpy as np

def equation(x):
    return x + np.cos(x)

solution = root(equation, x0=0)

print(solution.x)            # array([-0.73908513])
print(equation(solution.x))  # very close to 0

There is no simple formula for solving x + cos(x) = 0 by hand, but root() locates it numerically: starting from x0=0, it iterates until the function's value is close enough to zero. Plugging the result back into equation() confirms the residual is essentially zero.