Estimation of Uncertainty in the Regression Problem with Bayesian Regularization of the Solution

Abstract

The work considers the estimation of uncertainty of the regression model. The regression model is sought as a linear combination of basic functions. Coefficients in the linear combination are selected by minimizing the sum of the root-mean-square error of the approximation on the training set and the regularization term, which imposes restrictions on the set of solutions. Using a regularization term is one of the ways to combat the mathematical incorrectness of the problem. The regularization term consists of a penalty function and a regularization factor, which is an additional parameter of the regression model. The Bayesian approach allows to estimate the optimal value of the regularization factor directly from the data as being the most plausible. Constructing a regression model as a linear combination of basic functions (from a predefined set) allows us to reduce the high computational cost of the Bayesian approach by replacing the general iterative procedure by analytical expressions. A byproduct of estimating the regularization factor by Bayessian approach is the uncertainty of the regression model. The correctness of this assessment is the main subject of this research. The proposed approach of the uncertainty estimation is tested on the synthetic artificially noisy data. The proposed method estimated the noise magnitude close to the value used during data generation. The accuracy of the proposed method has outperformed the accuracy of the Gamma test, approach widely used to estimate uncertainty incorporated into the set of data.