Support for 16 bit quantization about annlite HOT 2 OPEN

AFAIk FP16C allows to convert between fp16 and fp32 should be available on most AVX CPUs. AVX512-fp16 provides the ability to perform maths on fp16 directly, but is only supported by a very small number of the latest CPUs.
For BF16 I am not aware if any architecture supports vectorized math yet . So i would assume we would have to convert back and forth , might be increased search latency (if not bottlenecked on memory bandwidth) but saves quite a bit on embedding space with little degradation on recall if any

from annlite.

That's a good idea. Actually, we are investigating whether bf16 would benefits (and how much) the HNSW search.

from annlite.