ML IALGO LINREG: Difference between revisions

Revision as of 09:16, 29 March 2023

ML_IALGO_LINREG = [integer]
Default: ML_IALGO_LINREG = 1

Description: This tag determines the algorithm that is employed to solve the system of linear equations in the ridge regression method for machine learning.

In the ridge regression method for machine learning one needs to solve for the unknown weights $\mathbf{w}$ minimizing

${\displaystyle || \mathbf{Y} - \mathbf{\Phi} \mathbf{w} || \rightarrow \mbox{min} }$

For more details please see here.

The following options are available to solve for $\mathbf{w}$ :

ML_IALGO_LINREG=1: Bayesian linear regression (see here). Recommended for NSW $\ge$ 1. Usable with ML_MODE = TRAIN, SELECT, and REFITBAYESIAN.
ML_IALGO_LINREG=2: QR factorization. Usable with ML_MODE = REFIT and REFITBAYESIAN.
ML_IALGO_LINREG=3: Singular value decomposition. Usable with ML_MODE = REFIT and REFITBAYESIAN.
ML_IALGO_LINREG=4: Singular value decomposition with Tikhonov regularization. Usable with ML_MODE = REFIT and REFITBAYESIAN.

For on the fly learning ML_MODE = TRAIN and reselection of local reference configurations ML_MODE = SELECT, it is strictly necessary to use Bayesian regression (ML_IALGO_LINREG=1), since uncertainty estimates are only available for Bayesian regression.

Refitting: Although the above modes result in an ML_FFN file that could be used for production runs, we strongly advise to refit the ML_ABN files. For that copy the ML_ABN file to the ML_AB file and use ML_MODE= REFIT (if Bayesian error estimation is required during production runs ML_MODE= REFITBAYESIAN is an option). This mode employs ML_IALGO_LINREG=4 by default.

From ML_IALGO_LINREG>1, ML_IALGO_LINREG=3 and 4 are the most tested approaches and we use ML_IALGO_LINREG=4 routinely before employing a machine learned force field. ML_IALGO_LINREG=4 gives more stable force fields and better fitting accuracy than ML_IALGO_LINREG=3, due to the regularization term employed.

ML_IALGO_LINREG=4 dramatically improves the condition number of the fitting compared to ML_IALGO_LINREG=1 since it directly uses the design matrix. In contrast ML_IALGO_LINREG=1 requires to use the covariance matrix (square of the design matrix), which effectively doubles the condition number. However, it also needs significantly more memory than ML_IALGO_LINREG=1 (at least twice that much). Please always monitor the memory estimates in ML_LOGFILE! It also takes longer to finish than ML_IALGO_LINREG=1, but still the time needed for refitting compared to the time for training is negligible.

A description how to choose the right algorithm for accurate force fields is given here.

@@ Line 21: / Line 21: @@
 For on the fly learning {{TAG|ML_MODE}} = ''TRAIN'' and reselection of local reference configurations {{TAG|ML_MODE}} = ''SELECT'', it is strictly necessary to use Bayesian regression ({{TAG|ML_IALGO_LINREG}}=1), since uncertainty estimates are only available for Bayesian regression.
-All other methods are used to read an existing {{TAG|ML_AB}} data base file ({{TAG|ML_ISTART}}=1) and create a final {{TAG|ML_FFN}} force field file (postprocessing). During on the fly training, the database {{TAG|ML_ABN}} file is created, and before postprocessing the user needs to copy the {{TAG|ML_ABN}} file to the {{TAG|ML_AB}} file. Among the postprocessing methods, {{TAG|ML_IALGO_LINREG}}=3  is the best tested approach and we use it routinely before employing a machine learned force field. It should be also noted that this postprocessing step is computationally somewhat more demanding than the Bayesian linear regression, but typically this post processing step still requires between a few minutes and an hour. So usually the extra cost is negligible compared to the original training.
+'''Refitting''':
+Although the above modes result in an {{TAG|ML_FFN}} file that could be used for production runs, we strongly advise to refit the {{TAG|ML_ABN}} files. For that copy the {{TAG|ML_ABN}} file to the {{TAG|ML_AB}} file and use {{TAG|ML_MODE}}= ''REFIT'' (if Bayesian error estimation is required during production runs {{TAG|ML_MODE}}= ''REFITBAYESIAN'' is an option). This mode employs {{TAG|ML_IALGO_LINREG}}=4 by default.
+From {{TAG|ML_IALGO_LINREG}}>1, {{TAG|ML_IALGO_LINREG}}=3 and 4 are the most tested approaches and we use {{TAG|ML_IALGO_LINREG}}=4 routinely before employing a machine learned force field. {{TAG|ML_IALGO_LINREG}}=4 gives more stable force fields and better fitting accuracy than {{TAG|ML_IALGO_LINREG}}=3, due to the regularization term employed.
+{{TAG|ML_IALGO_LINREG}}=4 dramatically improves the condition number of the fitting compared to {{TAG|ML_IALGO_LINREG}}=1 since it directly uses the design matrix. In contrast {{TAG|ML_IALGO_LINREG}}=1 requires to use the covariance matrix (square of the design matrix), which effectively doubles the condition number.
+However, it also needs significantly more memory than {{TAG|ML_IALGO_LINREG}}=1 (at least twice that much). Please always monitor the memory estimates in {{TAG|ML_LOGFILE}}! It also takes longer to finish than {{TAG|ML_IALGO_LINREG}}=1, but still the time needed for refitting compared to the time for training is negligible.
 A description how to choose the right algorithm for accurate force fields is given [[Machine learning force field: Theory#Sparsification of local reference configurations|here]].

Revision as of 09:16, 29 March 2023

Related tags and articles