error with tf2.4.1 about perceiver HOT 3 CLOSED

Hi @abred ,

1. I was not able to reproduce this issue with the latest TensorFlow 2.4.1, please see this Colab Notebook. I have also tested this model on real images though this is as good as real images for testing purpose. Do you use TensorFlow 2.4.1?
2. Thanks for raising this. In the model, we would be passing the latents, random normal distribution of [num_latents, latent_dim]. On top of this, the Fourier encode could be passed for the Cross Attention layer. This is yet to be implemented.

from perceiver.

thanks

1. Yes, I am using 2.4.1. I am calling it within a @tf.function, maybe that's the difference.
   This is the error message:

ValueError: Weights for model sequential_25 have not yet been created. Weights are created when the Model is first called on inputs or `build()` is called with an `input_shape`.

2. So if I undestand it correctly, in the current state it is not really a cross attention, as the byte array/data is not passed in?
   

   Perceiver/perceiver/attention.py

   Lines 33 to 34 in 4d3b9b0

   	if context is None:
   	context = x

   
   context is None in both cases so far

from perceiver.

1. Ah got it, fixed via #21
2. Yes that is correct

I am closing this issue as of now (after #21 ) however, feel free to reach back or reopen it.

from perceiver.