12.3. minimize()
and objective Functions¶
As mentioned above, the objective function returns an array calculated from given a group of parameters. This array will be minimized in the least-squares sense in the fitting process. For most fits, the objective function should return the residual array (data - model), given a group of parameters and optional inputs. You’ll note that we didn’t explicitly mention any data in describing the objective function. This is because, formally, the minimization process may be looking for a solution to a purely mathematical problem, not just fitting to data. Even when the objective function does return the difference of data and model, the data to be modeled may be quite complex. It might, for example, be contained in two or more arrays – perhaps what you want to model is the difference of two image arrays, or the fourier filtered average of ten spectra. Because of such complexities, the reliance of optional arguments appears to be the best approach.
A simple objective function that models data as a line might look like this:
params = param_group(offset = param(0., vary=True),
slope = param(200, min=0, vary=True))
def residual(pars, xdata=None, ydata=None):
model = pars.offset + pars.slope * xdata
diff = ydata - model
return diff
enddef
Here params
is a Group containing two Parameters as defined by
_math.param()
, discussed earlier.
To actually perform the fit, the minimize()
function must be
called. This takes the objective function as its first argument, and
the group containing all the Parameters as its second argument,
keyword arguments for the optional arguments for the objective
function, and several keyword arguments to alter the fitting process
itself. Here is the function call using the objective function
defined above, assuming you have a group called data
containing
the data you are trying to fit:
result = minimize(residual, params, kws={'xdata': data.x, 'ydata':data.y})
As the fit proceeds, the values the Parameter values will be updated, and the objective function will be called to recalculate the residual array. Thus the objective function may be called many times before the fitting procedure decides it has found the best solution that it can.
Changed in version 0.9.34: minimize()
returns a result group containing fit statistics.
- minimize(fcn, paramgroup, args=None, kws=None, method='leastsq', **extra_kws)¶
find the best-fit values for the Parameters in
paramgroup
such that the output array from the objective functionfcn()
has minimal sum-of-squares.- Parameters:
fcn – objective function, which must have signature and output as described below.
paramgroup – a Group containing the Parameters used by the objective function. This will be passed as the first argument to the objective function. The Group can contain other components in addition to the set of Parameters for the model.
args – a tuple of positional arguments to pass to the objective function. Note that a single argument tuple is constructed by following a value with a comma (it is not sufficient to enclose a single value in parentheses)
kws – a dictionary of keyword/value arguments to pass to the objective function.
method – name (case insensitive) of minimization method to use (default=’leastsq’)
extra_kws – additional keywords to pass to fitting method.
- Returns:
a Group containing several fitting statisics and best-fit parameters.
The
method
argument gives the name of the fitting method to be used. Several methods are available, as described below in Table of Fitting Methods.
Table of Fitting Methods.
Listed are the names and description of fitting methods available to the
minimize()
function. The leastsq method is the default, and the only method for which uncertainties and correlations are automatically calculated.
method name
Description
Leastsq
Levenberg-Marquardt.
Nelder-Mead
Nelder-Mead downhill simplex.
Powell
Powell’s method.
BFGS
quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno.
CG
Conjugate Gradient.
LBFGSB
Limited-Memory BFGS Method with Constraints.
TNC
Truncated Newton method.
COBYLA
Constrained Optimization BY Linear Approximation.
SLSQP
Sequential Least SQuares Programming.
Further information on these methods, including full lists of extra parameters that can be passed to them, can be found in the lmfit documentation.
It should be noted that the Levenberg-Marquardt algorithm is almost always the fastest of the methods listed (often by 10x), and is generally fairly robust. It is sometimes criticized as being sensitive to initial guesses and prone to finding local minima. The other fitting methods use very different algorithms, and so can be used to explore these effects. Many of them are much slower – using more than ten times as many evaluations of the objective function is not unusual. This does not guarantee a more robust answer, but it does allow one to try out and compare the results of the different methods.
While the TNC, COBYLA, SLSQP, and LBFGSB methods are supported, their principle justification is that the underlying algorithms support constraints. For Larch, this advantage is not particularly important, as all fitting methods can have constraints applied through Parameters, and the mechanism used by the native methods is not actually even supported with Larch. That said, all these methods are still interesting to explore.
Extra keywords for the leastsq method include:
extra_kw
arg formethod='leastsq'
Default Value
Description
xtol
1.e-7
Relative error in the approximate solution
ftol
1.e-7
Relative error in the desired sum of squares
maxfev
2000*(nvar+1)
maximum number of function calls (nvar= # of variables)
Dfun
None
function to call for Jacobian calculation
By default, numerical derivatives are used, and the following arguments are used.