Client Sampling Strategy about fedml HOT 2 CLOSED

@luke-avionics This design is specialized for client sampling strategy for large client number (total client number >>> client number per round). For example, only 100 users are active among 1 million users in each round.

As for the case that "client number per round = total client number", I think it won't be a problem when we use naive local SGD. If we use local Adam, the optimizer state you mentioned seems also not a problem. In statistics, it doesn't matter which physical worker compute which part of the dataset. I confirmed this with experiments. That's why I reuse the code of client sampling for settings that does not need sampling.

However, I agree that both our arguments are empirical without theoretical guarantee. So let me modify it and fix the client ID, which is more safe without confusion.

Thank you very much for proposing this issue!

from fedml.

@luke-avionics I've updated the code. Please try again. Thanks for using our library for your research.

from fedml.