ML IALGO LINREG: Difference between revisions

Latest revision as of 15:52, 3 July 2023

ML_IALGO_LINREG = [integer]
Default: ML_IALGO_LINREG = 1

Description: This tag determines the algorithm that is employed to solve the system of linear equations in the ridge regression method for machine learning.

In the ridge regression method for machine learning one needs to solve for the unknown weights $\mathbf{w}$ minimizing

${\displaystyle || \mathbf{Y} - \mathbf{\Phi} \mathbf{w} || \rightarrow \mbox{min}. }$

For the theory of the available methods please see here.

The following options are available to solve for $\mathbf{w}$ :

ML_IALGO_LINREG=1: Bayesian linear regression (see here). Recommended for NSW $\ge$ 1. Usable with ML_MODE = TRAIN, SELECT, and REFITBAYESIAN.
ML_IALGO_LINREG=2: QR factorization. Usable with ML_MODE = REFIT and REFITBAYESIAN.
ML_IALGO_LINREG=3: Singular value decomposition. Usable with ML_MODE = REFIT and REFITBAYESIAN.
ML_IALGO_LINREG=4: Singular value decomposition with Tikhonov regularization. The regularization can be controlled via ML_SIGW0. Usable with ML_MODE = REFIT and REFITBAYESIAN.

For on the fly learning ML_MODE = TRAIN and reselection of local reference configurations ML_MODE = SELECT, it is strictly necessary to use Bayesian regression (ML_IALGO_LINREG=1), since uncertainty estimates are only available for Bayesian regression.

Refitting: Although the above modes result in an ML_FFN file that could be used for production runs, we strongly advise to refit the ML_ABN files. For that copy the ML_ABN file to the ML_AB file and use ML_MODE= REFIT (if Bayesian error estimation is required during production runs ML_MODE= REFITBAYESIAN is an option, but at the cost of significatnly slower calculation time). ML_MODE= REFIT employs ML_IALGO_LINREG=4 by default.

For ML_IALGO_LINREG>1, ML_IALGO_LINREG=3 and 4 are the most tested approaches and we use ML_IALGO_LINREG=4 routinely before employing a machine learned force field. ML_IALGO_LINREG=4 gives more stable force fields and better fitting accuracy than ML_IALGO_LINREG=3, due to the regularization term employed (for details please see here).

ML_IALGO_LINREG=4 dramatically improves the condition number of the fitting compared to ML_IALGO_LINREG=1 since it directly uses the design matrix. In contrast ML_IALGO_LINREG=1 requires to use the covariance matrix (square of the design matrix), which effectively doubles the condition number. However, ML_IALGO_LINREG=4 needs significantly more memory than ML_IALGO_LINREG=1 (at least twice that much). Please always monitor the memory estimates in the ML_LOGFILE! It should be also noted that ML_IALGO_LINREG=4 is computationally somewhat more demanding than ML_IALGO_LINREG=1, but it typically requires between a few minutes and an hour. So usually the extra cost is negligible compared to the original training.

@@ Line 8: / Line 8: @@
 <math>
-  || \mathbf{Y} - \mathbf{\Phi} \mathbf{w} || \rightarrow \mbox{min}
+  || \mathbf{Y} - \mathbf{\Phi} \mathbf{w} || \rightarrow \mbox{min}.
 </math>
-For more details please see [[Machine learning force field: Theory#Matrix vector form of linear equations|here]].
+For the theory of the available methods please see [[Machine learning force field: Theory#Regression|here]].
 The following options are available to solve for <math>\mathbf{w}</math>:
@@ Line 17: / Line 17: @@
 *{{TAG|ML_IALGO_LINREG}}=2: QR factorization. Usable with {{TAG|ML_MODE}} = ''REFIT'' and ''REFITBAYESIAN''.
 *{{TAG|ML_IALGO_LINREG}}=3: Singular value decomposition. Usable with {{TAG|ML_MODE}} = ''REFIT'' and ''REFITBAYESIAN''.
-*{{TAG|ML_IALGO_LINREG}}=4: Singular value decomposition with Tikhonov regularization. Usable with {{TAG|ML_MODE}} = ''REFIT'' and ''REFITBAYESIAN''.
+*{{TAG|ML_IALGO_LINREG}}=4: Singular value decomposition with Tikhonov regularization. The regularization can be controlled via {{TAG|ML_SIGW0}}. Usable with {{TAG|ML_MODE}} = ''REFIT'' and ''REFITBAYESIAN''.
 For on the fly learning {{TAG|ML_MODE}} = ''TRAIN'' and reselection of local reference configurations {{TAG|ML_MODE}} = ''SELECT'', it is strictly necessary to use Bayesian regression ({{TAG|ML_IALGO_LINREG}}=1), since uncertainty estimates are only available for Bayesian regression.
 '''Refitting''':
-Although the above modes result in an {{TAG|ML_FFN}} file that could be used for production runs, we strongly advise to refit the {{TAG|ML_ABN}} files. For that copy the {{TAG|ML_ABN}} file to the {{TAG|ML_AB}} file and use {{TAG|ML_MODE}}= ''REFIT'' (if Bayesian error estimation is required during production runs {{TAG|ML_MODE}}= ''REFITBAYESIAN'' is an option). This mode employs {{TAG|ML_IALGO_LINREG}}=4 by default.
+Although the above modes result in an {{TAG|ML_FFN}} file that could be used for production runs, we strongly advise to refit the {{TAG|ML_ABN}} files. For that copy the {{TAG|ML_ABN}} file to the {{TAG|ML_AB}} file and use {{TAG|ML_MODE}}= ''REFIT'' (if Bayesian error estimation is required during production runs {{TAG|ML_MODE}}= ''REFITBAYESIAN'' is an option, but at the cost of significatnly slower calculation time). {{TAG|ML_MODE}}= ''REFIT'' employs {{TAG|ML_IALGO_LINREG}}=4 by default.
-From {{TAG|ML_IALGO_LINREG}}>1, {{TAG|ML_IALGO_LINREG}}=3 and 4 are the most tested approaches and we use {{TAG|ML_IALGO_LINREG}}=4 routinely before employing a machine learned force field. {{TAG|ML_IALGO_LINREG}}=4 gives more stable force fields and better fitting accuracy than {{TAG|ML_IALGO_LINREG}}=3, due to the regularization term employed.
+For {{TAG|ML_IALGO_LINREG}}>1, {{TAG|ML_IALGO_LINREG}}=3 and 4 are the most tested approaches and we use {{TAG|ML_IALGO_LINREG}}=4 routinely before employing a machine learned force field. {{TAG|ML_IALGO_LINREG}}=4 gives more stable force fields and better fitting accuracy than {{TAG|ML_IALGO_LINREG}}=3, due to the regularization term employed (for details please see [[Machine learning force field: Theory#Solution via regularized SVD|here]]).
 {{TAG|ML_IALGO_LINREG}}=4 dramatically improves the condition number of the fitting compared to {{TAG|ML_IALGO_LINREG}}=1 since it directly uses the design matrix. In contrast {{TAG|ML_IALGO_LINREG}}=1 requires to use the covariance matrix (square of the design matrix), which effectively doubles the condition number.
-However, {{TAG|ML_IALGO_LINREG}}=4 needs significantly more memory than {{TAG|ML_IALGO_LINREG}}=1 (at least twice that much). Please always monitor the memory estimates in {{TAG|ML_LOGFILE}}! It also takes longer to finish than {{TAG|ML_IALGO_LINREG}}=1, but still the time needed for refitting compared to the time for training is negligible.
+However, {{TAG|ML_IALGO_LINREG}}=4 needs significantly more memory than {{TAG|ML_IALGO_LINREG}}=1 (at least twice that much). Please always monitor the memory estimates in the {{TAG|ML_LOGFILE}}! It should be also noted that {{TAG|ML_IALGO_LINREG}}=4 is computationally somewhat more demanding than {{TAG|ML_IALGO_LINREG}}=1, but it typically requires between a few minutes and an hour. So usually the extra cost is negligible compared to the original training.
 == Related tags and articles ==
-{{TAG|ML_LMLFF}}, {{TAG|ML_MODE}}, {{TAG|ML_W1}}, {{TAG|ML_WTOTEN}}, {{TAG|ML_WTIFOR}}, {{TAG|ML_WTSIF}}
+{{TAG|ML_LMLFF}}, {{TAG|ML_MODE}}, {{TAG|ML_W1}}, {{TAG|ML_WTOTEN}}, {{TAG|ML_WTIFOR}}, {{TAG|ML_WTSIF}}, {{TAG|ML_SIGW0}}
 {{sc|ML_IALGO_LINREG|Examples|Examples that use this tag}}