A method for constructing fully interpretable linear regressions with statistically significant estimates according to the Student’s criterion and insignificant intercorrelation coefficients

Abstract

The article is devoted to the urgent problem of constructing interpretable machine learning models, namely, multiple linear regression models. To estimate their unknown parameters, ordinary least squares is used. A probabilistically-statistical definition of fully interpretable linear regression is formulated. Its construction involves selecting the optimal number of most informative regressors based on determination coefficients in order to ensure consistency between the signs of regression coefficients and the substantive meanings of variables. Additionally, estimates are ensured to be significant and intercorrelation coefficients insignificant by using Student’s t-test. To construct fully interpretable regressions, we propose a method that uses mixed 0-1 integer linear programming. Both strict and non-strict versions of this method are considered. Computational experiments were carried out, which in most cases showed the effectiveness of the proposed method compared to the generating all subsets method.