Problem with fminunc in R14

I recently switched to Matlab 7.0.4 (R14SP2) and I find that fminunc
in R14 fails to work with the same minimization problem that I could
use fminunc in R13 to solve.

The symptom seems to be that right after the first iteration (I
optimset Display on Iter), the fminunc in R14 will move the variable
too far away from the starting point and my objective function will
generate overflow.

I had no trouble with the same minimization problem, using fminunc
under R13 with the same starting point.

I tried to look into fminunc and realize there is quite an overhaul
of the codes. Could anyone give me some practical suggestion? So
far I am trying to optimset DiffMaxChange to a smaller value, but it
has not been useful.

The algorithm in the *medium scale version* of fminunc was
reimplemented in R14.

The older algorithm (the one you're running in MATLAB
R13) can be less efficient on smooth problems but is
more tolerant in the sense that it can handle nonsmooth
problems better.

The newer one (the one you're running under R14SP2) is
a more modern, standard quasi-Newton implementation, and
tends to be faster than the old one when the smoothness
assumptions on the objective are met.

You found that your objective overflows at the first
iteration.

A couple of things to try:
1.
One way to limit the length of the first
step is via the initial quasi-Newton matrix (these
matrices approximate the inverse of the Hessian).

InitialHessType = 'user-supplied'
InitialHessMatrix = alpha

where alpha is a scalar (that defaults to 1). You can
experiment with this value alpha: the larger you set it,
the shorter the initial step will be.

2.
Another approach that people have used is to add a line
of code in the objective that sets the objective value
to a high (but finite) number whenever the objective
becomes +infinity. This approach is in general not
recommended, but I've seen it work OK as long as this
safeguard is active only a few times, during the
early iterations of the algorithm.

-marcelo

Namo wrote:
> I recently switched to Matlab 7.0.4 (R14SP2) and I find that fminunc
> in R14 fails to work with the same minimization problem that I could
> use fminunc in R13 to solve.
>
> The symptom seems to be that right after the first iteration (I
> optimset Display on Iter), the fminunc in R14 will move the variable
> too far away from the starting point and my objective function will
> generate overflow.
>
> I had no trouble with the same minimization problem, using fminunc
> under R13 with the same starting point.
>
> I tried to look into fminunc and realize there is quite an overhaul
> of the codes. Could anyone give me some practical suggestion? So
> far I am trying to optimset DiffMaxChange to a smaller value, but it
> has not been useful.