Benchmarking of all pre-trained weights about torchgeo HOT 4 OPEN

Hi @adamjstewart , I have access to a 1070ti and 1080 on separate desktop computers. They're relatively older machines, but I do have uninterrupted access to them.

from torchgeo.

Thanks @kvenkman. That's not a lot, but it should be possible to work on Sentinel-1, NAIP, and/or fMoW since there is only a single model per satellite that needs to be evaluated.

Here are the steps I think you need to complete for each one of those:

1. Find ~3 benchmark datasets that include images from that satellite and target labels
2. Add those datasets to TorchGeo (if they aren't already in TorchGeo)
3. Add data modules for those datasets (if they aren't already in TorchGeo)
4. Using our CLI, try various hyperparameters to get optimal performance on that dataset

This is obviously a big project, but even if you only manage to finish steps 1 and 2 for a model, that gets us a long way there. Let me know if you have any questions.

from torchgeo.

I'd be interested in contributing to this - could I have this assigned to me? I'd be interested in experiments to help with the benchmarking effort.

from torchgeo.

Hi @kvenkman, thanks for volunteering! How much compute do you have access to? This issue will likely require a lot of GPU time.

from torchgeo.