The paper presents analytic and stochastic methods of structure parameters estimation for a model selection problem. Structure parameters are covariance matrices of parameters of linear and non-linear regression models. To optimize model parameters and structure parameters we maximize a model evidence, a convolution of a data likelihood with a prior distribution of model parameters. The analytic methods are based on the derivatives computation of the approximated model evidence. The stochastic methods are based on the model parameters sampling and data cross-validation. The proposed methods are tested and compared on the synthetic and real data.