How use the nccl? about nccl HOT 8 CLOSED

Hi,

NCCL is an inter-GPU communication library ; it is therefore accelerating computation on multiple GPUs.
That means it won't make a difference with only one GPU.

from nccl.

Hello,I begin use one GPU, but then use four GPUs! But the result has no different.

from nccl.

The CIFAR-10 example is not very computationally intensive. The GPU(s) are probably limited by kernel launch latencies. Perhaps try something like AlexNet, or better ResNet-50, on ImageNet 1000.

from nccl.

OK!Thanks,I have a try the AlexNet!

from nccl.

@nluehr @sjeaugey When I use the caffe-0.15.9/build/tools/caffe time --model=./imagenet_winners/alexnet.prototxt --iterations=1000 --gpu all, the result said no use multi-gpu. Why?
I0303 04:52:48.248244 21608 caffe.cpp:334] Not using GPU #3 for single-GPU function
I0303 04:52:48.248602 21608 caffe.cpp:334] Not using GPU #2 for single-GPU function
I0303 04:52:48.248613 21608 caffe.cpp:334] Not using GPU #1 for single-GPU function
I0303 04:52:48.359235 21608 caffe.cpp:341] Use GPU with device ID 0
I0303 04:52:48.359807 21608 caffe.cpp:345] GPU 0: Tesla P100-SXM2-16GB

from nccl.

@getengqing, I'm not that familiar with Caffe. You probably want to direct this question to the caffe project.

from nccl.

@getengqing Please, see int time() function in tools/caffe.cpp. Multi-GPU performance measurement is not supported.

from nccl.

@sergey-serebryakov OK!Thanks!

from nccl.