No train test split? about gain HOT 6 CLOSED

Training set itself does not have any labels for the missing components.
The training set is itself an incomplete data. Therefore, it is impossible to overfit because there is nothing to be overfitted.
Usually, people use imputation method without dividing their data into training and testing data. They first impute the missing data and divide them into the training and testing data.

For reporting the results, I did splitting the data into training and testing. However, for better to be used by others, I think it would be better to not splitting the training and testing. You can easily incorporate dividing training and testing set with the following packages (https://scikit-learn.org/stable/modules/cross_validation.html)

from gain.

When you use the imputation method for imputing the dataset with missing data, you usually do not divide them into train and test. You just use the entire dataset to do imputation and get the imputed dataset for the entire data. Therefore, I omit train/test division in this version.

from gain.

@jsyoon0823 Why do you not divide the data into a train and a test set? Couldn't you just overfit to the train set? From my point of view, it would make a lot of sense to test the generalization error, because you will eventually use this method to impute data for which you do not have the ground truth "unobserved" values.

Also, how did you perform cross-validation? (which is mentioned in the paper). Thank you

from gain.

For the cross-validation, which metrics did you use to compare the different hyper-parameters?
Did you use the rmse with the supposedly-unknown values? Or simply the reconstruction loss (L_M in the paper) as a proxy ?

from gain.

For the cross-validation, I used the prediction error as the metric.

• RMSE with the unknown values is not practical and impossible to most datasets (with missing data)
• L_M can be also utilized for the cross-validation; however, this is not what I did.

from gain.

Well, there is overfitting, as you peak into the test features to impute training features. This potentially trains and builds a model on the test data. Clearly overfitting. However, at least it is good that for your experiments you indeed split before. This is actually the main problem of literature concerning imputation. Packages don't allow imputation on new data

from gain.