# Using machine learning in CPLEX 12.10

CPLEX is proud to announce that in the newest release, version 12.10, for the first time, a commercial Mathematical Optimization solver implements a Machine-Learning-based classifier to make automatic decisions over some algorithmic choices.

In particular, the Machine-Learning-based classifier is invoked by default to decide if the binary component of a Mixed-Integer Quadratic Optimization problem should benefit from the application of a linearization step in which, essentially, the problem is transformed in a Mixed-Integer Linear Optimization problem and solved in that form.

The implemented classifier (based on the early work [1]) is applied after the model is presolved in linearized form and uses 21 features plus a regression estimation of the objective function difference of the two formulations. Based on those features, the classifier makes the decision and, in case it is decided to solve the problem with the original formulation, the preprocessing is done again.

The implemented classifier is a Support Vector Machine model. It was trained using a set of MIQP models composed of: CPLEX internal set, problems submitted to be solved by CPLEX through the NEOS cloud based services and randomly generated models. The total set had 1674 models, 1381 of which were used for training and the remainder for validation. On the validation set, the final classifier has an accuracy of 79%, a precision of 78% and a recall of 79%. Using this classifier, the solution time decreases by 28% for the whole validation set and 49% for models in the validation set that take at least 1 second to solve. The scatter plot below shows the running times in logarithmic scale on the validation set for the previous version of CPLEX (12.9.0) and the new one (12.10.0) that uses the classifier.

We consider the result a significant success from a performance standpoint and in the direction of introducing statistical methods by Machine Learning within Mathematical Optimization solvers. We believe the methodology developed to achieve that could be beneficial to a variety of other tasks in CPLEX, so stay tuned for more progress thanks to Machine Learning in Decision Optimization!

[1] P. Bonami, A. Lodi, G. Zarpellon, *Learning a Classification of Mixed-Integer Quadratic Programming Problems*, in W.-J. van Hoeve, Ed., *Integration of Constraint Programming, Artificial Intelligence, and Operations Research – CPAIOR 2018*, Lecture Notes in Computer Science 10848, Springer-Verlag, Berlin Heidelberg, 2018, 595–604.