Colab on how to use RETVec with TF Lite for on-device use cases

Create a comprehensive README with installation instructions and links to demo colabs, paper, etc.

Hi!

I am trying to use RetVec as an embedding layer for a email spam classification project. When the RetVecTokenizer function is called, I get a Value Error which is described below:

File format not supported: filepath=/root/.keras/retvec-v1. Keras 3 only supports V3 .keras files and legacy H5 format files (.h5 extension). Note that the legacy SavedModel format is not supported by load_model() in Keras 3. In order to reload a TensorFlow SavedModel as an inference-only layer in Keras 3, use keras.layers.TFSMLayer(/root/.keras/retvec-v1, call_endpoint='serving_default') (note that your call_endpoint might have a different name).

Here is the code I wrote:

inputs = layers.Input(shape=(1, ), name="token", dtype=tf.string)

lstm_1 = tf.keras.layers.LSTM(20, dropout=0.2, return_sequences=True)(x)
lstm_2 = tf.keras.layers.LSTM(20, dropout=0.2, return_sequences=True)(lstm_1)
flatten = tf.keras.layers.Flatten()(lstm_2)
dropout = tf.keras.layers.Dropout(0.2, name="dropout")(flatten)
dense = tf.keras.layers.Dense(1, activation="sigmoid")(dropout)

model = tf.keras.Model(inputs=[inputs], outputs=[dense])

How to resolve this?

https://github.com/google-research/retvec/blob/1615625edcff1ae5f517bcfbd14210371ce01c57/setup.py#LL62C14-L62C19

Try to remove unused / non critical dependencies like twine.

• Rename package from tensorflow_retvec back to RetVec
• Clean up retvec layers
• Fix tests

I attempted to save the tflite_retvec model as a tflite model for deployment on an Android device, but I am encountering difficulties in running it. Can you please share an Android demo app to understand how it performs on Android devices.

in your notbooke, why the val_acc just 0.51? it seems not good

I apply the model to Chinese Sentiment classification data. the performance also seems not good, I don't know why

Using the retvec model as embeding methods, but the GPU memory abnormal increase to 40G.
tensorflow 2.15.0
keras 2.15.0
tf-keras 2.15.0
retvec 1.0.1

Using methods:

def get_retvec_tokenizer(model_path):
    with tf.device('/CPU:0'):
        inputs = layers.Input(shape=(1,), dtype=tf.string)
        outputs = RETVecTokenizer(model=model_path)(inputs)
        retvec = tf.keras.Model(inputs=inputs, outputs=outputs)

    return retvec

Init the model state:

Using the model state inference:

In the paper, the authors write

Augmentations Token augmentation consists of randomly inserting up to 4 typos per token up to 25% of the token length. This is consistent with an observed maximum human error frequency of around 20% [11]. We use 22 distinct typo augmentations, which can be grouped into four categories: deletion, insertion, substitution, and transposition. For each token, we randomly select a target augmentation percentage between 0-25%, and for each augmentation step we randomly apply an augmentation from one of the four typo categories. The full list of augmentations used is reported in Appendix D.

Is the code to apply these augmentations available anywhere? I'd like to use & adapt it for my specific use-case.

PyTorch implementation that can be installed by pip install pytorch_retvec

RETVec TFJS package and colab

Write a C python tokenizer to improve efficiency

https://github.com/google-research/retvec/blob/1615625edcff1ae5f517bcfbd14210371ce01c57/retvec/tf/binarizers.py#LL171C26-L171C36

1. Use RetVec on GPU
2. Use RetVec on TPU

Write a Flutter app and demo tutorial to demonstrate how to use RETVec models on-device

Keep only what is need for inference under the tf/
mode training code / layer and utils under training/
create a tokenizer class (we can do this after) that is for when not using layers. Move the code for this from layer to that class.