About cross validation procedure in SISSO about sisso HOT 3 CLOSED

Hi,

1. I suggest not to standardize the input primary features in the train.dat file, otherwise the physical meaning of mathematical combination between the primary features may be lost. The SISSO code will do standardization ONLY inside the SIS routine for evaluation of feature importance, but never do it for feature construction and the final model.

2. Usually we use the whole-data descriptor (and corresponding coeff.), instead of the ones from CV. We do CV to check the stability (sensitivity of the descriptor form on samples) of the whole-data descriptor, e.g. how many times the whole-data descriptor is identified in the CV. Of course you can still report the average CV error regardless of the different form of the CV descriptors.

3. Generally, increasing of the prediction error is signaling overfitting, but understanding why is that require closer look at your primary features and your data.

4. "Blank" is not allowed in the code. You could simply remove those features or those materials of missing data, or provide estimated (e.g. interpolation ? ) values to those missing data.

from sisso.

Thank you Dr. Ouyang for your reply.

You answer clarifies me a lot.
I have one more question about multi task learning.
Can I perform cross validation for MT-SISSO?
Since the number of samples included in the each MT-SISSO differs, it's hard to apply the same cross validation scheme in this case.
It seems provided utilities support only cross validation for ST-SISSO.
If there's certain protocol to try for cross validation (e.g. Leave 10% out ) for MT-SISSO, it would be appreciated.

from sisso.

Yes, CV for the MT-SISSO is more complicated than ST-SISSO as in the MT case the samples and coefficients for different tasks could be different. Users can design CV schemes (e.g. the J. Phys.: Mater. 2, 024002 (2019) describes two solutions) suitable for their specific applications and purposes. It is also possible to test on unseen data (even on new tasks beyond training data).

from sisso.