problem about multi_gpu flag about ferminet HOT 2 CLOSED

Please try patching the tensorflow code.

In tensorflow_core/python/training/slot_creator.py (inside the virtualenv site-packages -- something like /path/to/venv/lib/python3.7/site-packages/tensorflow_core/python/training/slot_creator.py), there are two functions, create_slot and create_slot_with_initializer, which contain the test

 if context.executing_eagerly():
  prefix = primary._shared_name
else:
  prefix = primary.op.name

Change the if statement to:

if isinstance(primary, variables.Variable):
   ...

This matches the upstream code (which didn't get released in TF 1.15) in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/training/slot_creator.py. The commit which fixed this issue is tensorflow/tensorflow@3a22ef3. I'll try to get this backported to the 1.15 release series.

(I believe that this problem only affects using K-FAC -- using ADAM should work fine with multiple GPUs)

from ferminet.

Thank you so much for such a quick and detailed reply! I will try immediately.

from ferminet.