Quantized operation during inference? about qkeras HOT 3 CLOSED

@Duchstf

Maybe I have not fully understand the question. Regarding "operations" between layers, you can put

• quantizer inside one layer like https://github.com/google/qkeras/blob/master/examples/example_qoctave.py#L49
• Or add QActivation layer before or after one layer, like
  https://github.com/google/qkeras/blob/master/examples/example_mnist.py#L82

and make the tensors to be quantized.

Let me know if I answered the wrong question.

Thanks!

from qkeras.

Hi,

what I meant is that, for example, you have two quantized parameters: x and y. However, when you multiply them together x*y the result will still be a float32 and not be quantized anymore.

Would it happen here? and what should I do to prevent it from happening?

Duc.

from qkeras.

Let's assume the mac (multiplication-and-addition) has enough bits to hold x*y (mult=x*y, mac += mult). Then, use that datatype to implement mac in hardware.

FYI, we have been working on tool inside qkeras to derive mac datatype automatically. For example, when x is quantized_bits(bits=4, integer=1, keep_negative=0), y is quantized_bits(bits=2, integer=1, keep_negative=0), mult could be quantized_bits(bits=6, integer=2, keep_negative=0), and set mac type correspondingly. There are different types of quantizers and operations, so we need to map them accordingly. This will make hardware datatype setup easy. So stay tune.

To simulate the numerical behavior, I suggest you apply QActivation layer or SomeLayer(..., activation=quantizer) to ensure the model accuracy.

Hope it makes sense. Feel free to keep discussions in this thread!

from qkeras.