Which area is used as the testing area for STPLS3D semantic segmentation? about stpls3d HOT 8 OPEN

WMSC is the validation data for the experiments reported in the paper tables 3 and 6. To replicate what we have achieved, I would suggest starting with KpConv (since we have provided the simple instruction to run it) and making sure everything works fine there, then moving on to RandLA and SCF-net.

from stpls3d.

Hi @meidachen

OK. Thanks. I will run KPConv at first, and then move to RandLA-Net and SCF-Net. In addition, I'm not clear that STPLS3D dataset is only splited into training and validation, right? Not training, validation, and testing. During training, the WMSC is used for evaluate to obtain the best model, and then WMSC used as to test the model?

from stpls3d.

That is correct, in the paper we have tested the trained model in another dataset (FDc) which cannot be released. So on the released dataset, you can either validate and test on WMSC, or you could do cross-validation using all four real-world datasets.

from stpls3d.

Hi @meidachen

So on the released dataset, you can either validate and test on WMSC. This operation is similar to the S3DIS with 5 area as testing. But I think it is not reasonable as using the 5 area as validation and testing at the same time. Actually, validation and testing should have no intersection. Or, if not using WMSC as validation, only as testing, 3 or 5 times average is also OK.

Thanks.

from stpls3d.

You are right, it is better to have validation and testing sets without intersection, and yes, I was following S3DIS (testing on aera5) when releasing STPLS3D. One of the main reasons that we can't really do a train, validation, and test split is the lack of real-world data.
Two ways of doing it would be:

1. use say OCCC or RA as the validation, then we lost one scene to train the model.
2. use part of say USC or OCCC or RA as validation, then the model could be overfitted since the validation and training sets do have an intersection in the sense that they came from the same area and may share similar properties.

from stpls3d.

Or, if not using WMSC as validation, only as testing, 3 or 5 times average is also OK.

In this case, I think cross-validation would be a better option.

from stpls3d.

An error occurs when the two sets of data have inconsistent category labels during model fine-tuning using RandLA. Can you post the fine-tuning code? Thanks!

from stpls3d.

Hi @volare1996 ,

Which two sets of data are you using?

from stpls3d.