Results are not deterministic in eval mode about performer-pytorch HOT 4 CLOSED

Yep, it helps. I do it after training as you suggest:

for layer in model.performer.performer.net.layers:
    layer[0].fn.fast_attention.redraw_projection = False
    projection_matrix = layer[0].fn.fast_attention.create_projection()
    layer[0].fn.fast_attention.register_buffer('projection_matrix', projection_matrix)

from performer-pytorch.

@arti32lehtonen not without turning off the redrawing of weights, which hurts performance, as indicated in the figures in the paper

from performer-pytorch.

@arti32lehtonen i suppose one could turn off the redrawing of weights only at the end of training though 🤔

from performer-pytorch.

@arti32lehtonen addressed this in https://github.com/lucidrains/performer-pytorch/releases/tag/0.5.1

from performer-pytorch.