Question about Parameters about mlp-mixer-pytorch HOT 3 CLOSED

it is taking advantage of the fact that conv module acts on the first dimension, and linear module acts on the last https://github.com/lucidrains/mlp-mixer-pytorch/blob/main/mlp_mixer_pytorch/mlp_mixer_pytorch.py#L30 this trick saves from needing a transpose

from mlp-mixer-pytorch.

I really appreciate the author's explanation here, it makes it much easier for me to understand the code.

from mlp-mixer-pytorch.

Why are the parameters so different from the one in the paper?

Take the S-16 as an example:

* your model : 5.01 M

* Paper : 18 M

I compared your model with official imlementation model (Jax version), maybe you made a mistake.

I can not find any transpose operation in your model, which is necessary for Mixer layer.

If i do not understand your model, please correct me ,thanks a lot!

It appears that the code implementation completely lacks the operation of matrix transposition, and you are correct.

from mlp-mixer-pytorch.