# NOTE: The content of this page has been moved into the Ipopt documentation

# Ipopt Additional Features

This Wikipage is intended to describe additional features of IPOPT. If the user would like to contribute with any ideas please contact the IPOPT project leader Andreas Wächter and, if possible, please include some documentation in this page explaining the use of the feature. Some examples of potentially useful features include different algorithmic strategies, warm-starting strategies, post-optimal analysis and extraction of information from the primal-dual system.

Victor M. Zavala

Department of Chemical Engineering

Carnegie Mellon University

vzavala@…

## Warm-Starting Capabilities from AMPL

Warm-starting an interior-point algorithm is an important issue. One of the main difficulties arises from the fact that full-space variable information is required to generate the warm-starting point. While IPOPT is currently equipped to retrieve and receive this type of information through the TNLP interface, there exist some communication barriers in the AMPL interface. When the user solves the AMPL problem:

min f(x) x in R^n s.t. g_L <= g(x) <= g_U x_L <= x <= x_U

IPOPT will only return the optimal values of the primal variables `x` and of the constraint multipliers corresponding to the active bounds of `g(x)`. The constraint multiplier values can be accessed through the `dual` suffix or through the `.sol` file. If this information is used to solve the *same* problem again, you will notice that IPOPT will take some iterations in finding the *same* solution. The reason for this is that we are missing the input information of the multipliers `zL` and `zU` corresponding to the variable bounds. For some details on the notation used in this page please refer to this document.

IPOPT will now retrieve the value of the bound multipliers `zL` and `zU` to AMPL. This will be communicated to the AMPL user through the suffixes `ipopt_zL_out` and `ipopt_zU_out`, respectively. The user does not need to declare these suffixes, they will be generated automatically in the AMPL interface. The user can use the suffix values to initialize the bound multipliers for subsequent calls. In order to pass this information to IPOPT, the user will need to declare and assign values to the suffixes `ipopt_zL_in` and `ipopt_zU_in`. For instance, for a given variable `x[i]`, this can be done by setting:

let x[i].ipopt_zL_in := x[i].ipopt_zL_out; let x[i].ipopt_zU_in := x[i].ipopt_zU_out;

If the user does not specify some of these values, IPOPT will set these multipliers to `1.0` (as before). In order to make the warm-start effective, the user now has control over the following options from AMPL:

warm_start_init_point warm_start_bound_push warm_start_mult_bound_push

please refer to the IPOPT documentation for more information on the use of these options.

Note that the use of this feature is far from solving the complicated issue of warm-starting interior-point algorithms. As a general advice, this feature will be useful if the user observes that the solution of subsequent problems (i.e.; for different data instances) preserves the same set of active inequalities and bounds (monitor the values of `zL` and `zU` for subsequent solutions). In this case, initializing the bound multipliers and setting `warm_start_init_point=YES` and `warm_start_bound_push`, `warm_start_mult_bound_push` and `mu_init` to a small value (`1e-6` or so) will reduce significantly the number of iterations. This is particularly useful in setting up on-line applications and high-level optimization strategies in AMPL.

If active-set changes are observed between subsequent solutions, then this strategy might not decrease the number of iterations (in some cases, it might even tend to increase the number of iterations). Please refer to these slides for a simple conceptual explanation of this.

You might also want to try the adaptive barrier update (instead of the default monotone one where above we chose the initial value `1e-6`) when doing the warm start. This can be activated with the `mu_strategy=adaptive` option, and the option `mu_oracle` gives some alternative choices. In general, the adaptive choice often leads to less iterations, but the computational cost per iteration might be higher.

### AMPL Model Example

This illustrates the use of the warm-start feature on problem hs071_ampl.mod in the IPOPT documentation:

# tell ampl to use the ipopt executable as a solver # make sure ipopt is in the path! option solver ipopt; # declare the variables and their bounds, # set notation could be used, but this is straightforward var x1 >= 1, <= 5; var x2 >= 1, <= 5; var x3 >= 1, <= 5; var x4 >= 1, <= 5; # specify the objective function minimize obj: x1 * x4 * (x1 + x2 + x3) + x3; # specify the constraints s.t. inequality: x1 * x2 * x3 * x4 >= 25; equality: x1^2 + x2^2 + x3^2 +x4^2 = 40; # specify the starting point let x1 := 1; let x2 := 5; let x3 := 5; let x4 := 1; # solve the problem solve; # ipopt returns the bound multipliers through # the suffixes ipopt_zL_out and ipopt_zU_out # the user does not need to specify them # print the solution and variable bounds multipliers display x1; display x2; display x3; display x4; display x1.ipopt_zL_out; display x1.ipopt_zU_out; display x2.ipopt_zL_out; display x2.ipopt_zU_out; display x3.ipopt_zL_out; display x3.ipopt_zU_out; display x4.ipopt_zL_out; display x4.ipopt_zU_out; # define initial conditions for bound multipliers # to solve new problem suffix ipopt_zL_in, IN; suffix ipopt_zU_in, IN; let x1.ipopt_zL_in := x1.ipopt_zL_out; let x1.ipopt_zU_in := x1.ipopt_zU_out; let x2.ipopt_zL_in := x2.ipopt_zL_out; let x2.ipopt_zU_in := x2.ipopt_zU_out; let x3.ipopt_zL_in := x3.ipopt_zL_out; let x3.ipopt_zU_in := x3.ipopt_zU_out; let x4.ipopt_zL_in := x4.ipopt_zL_out; let x4.ipopt_zU_in := x4.ipopt_zU_out; # set options for warm-start option ipopt_options "warm_start_init_point yes warm_start_bound_push 1e-6 warm_start_mult_bound_push 1e-6 mu_init 1e-6"; # solve the problem again solve;

To appreciate the importance of initializing the bound multipliers, the user might want to modify the AMPL code by making the second `solve` call without initialization of `zL` and `zU` and setting `mu_init 1e-6`.

### Ideas for Future Work

In the case of active-set changes, it has been observed that using an intermediate iterate might be beneficial in warm-starting a subsequent problem (this option is currently used in the MINLP solver Bonmin). An alternative to enable this feature from AMPL would be to specify an option like `retrieve_iteration=#` that would return the `#-th` iterate before the last one and communicated back to AMPL through suffixes. Note that, in this case, both multipliers of `g(x)` would be required. The user could then use this intermediate iterate to warm-start a subsequent problem.

Further features for warm-starting are also part of the post-optimal analysis toolbox of IPOPT which is currently under development.