C++版でも同様のジェスチャー認識ができるものを作ってもらえるとありがたいです。

Jupyter notebook. I have an error in "model.fit(
X_train,
y_train,
epochs=1000,
batch_size=128,
validation_data=(X_test, y_test),
callbacks=[cp_callback, es_callback]
)"
It said "InvalidArgumentError Traceback (most recent call last)
Cell In[32], line 1"
I don't understand why and how to fix this problem, can you help me with this error?
Thank you very much!

Add a requirements file to quickly install the right version of python packages.

https://techvidvan.com/tutorials/hand-gesture-recognition-tensorflow-opencv/comment-page-2/#comment-3660

Hello friend, I have been experimenting with your project to enter a raspberry and try it in the translation but I want to enter more than 9 gestures, I don't know if you can support me, it is from your next project.

Hi,
Taking the footprint of this work , i created my won getsure dataset from media pipe, did the same preprocessing as was written and trained a 3 dense layer classifier with adam and categorical cross entropy , but my accuracy doesnt increase beyond 0.3, I have more than 1k dataset for each class. i am baffled as to why is it not working? any input from your side would be useful

Preprocessing step- #shape(N,21,2)
def preprocess(data):
for i in range(data.shape[0]):
data[i,:,0]=data[i,:,0] - data[i,0,0]
data[i,:,1] =data[i,:,1]- data[i,0,1]
data[i]= data[i]/np.amax(np.abs(data[i]))
data=data.reshape(data.shape[0],42)
return data

model

def get_model():
reg= regularizers.l1(0)
inp= Input(shape=(42,))
x= Dense(32,activation='relu',activity_regularizer=reg, kernel_initializer='he_uniform')(inp)
x= Dropout(0)(x)
x= Dense(16,activation='relu',activity_regularizer=reg, kernel_initializer='he_uniform') (x)
x= Dropout(0)(x)
x= Dense(8,activation='relu',activity_regularizer=reg, kernel_initializer='he_uniform')(x)
out= Dense(5,activation='softmax')(x)
model= Model(inp,out)
opt = tf.keras.optimizers.Adam(learning_rate=0.1)
model.compile(optimizer=opt,loss='categorical_crossentropy',metrics=['accuracy'])
model.summary()
return model

Hi Kazuhito00, thank you very much for you cool job. I have a question about the train dataset of the point_history_classifier. I want to extend your demo with my own gesture, but when I collect my own train dataset and train the new model, I find the new
model can't recognise the new gesture correctly. I collect my data like this, After I Press 'h' and enter the mode to save the history of fingertip coordinates, I press '4' while completing the gesture and save the figure coordinate to the csv for train new model. Total num of the added data is about 500. Can you help me point out the problem？TKS！

I would like to use the models for scientific work.

Please add a CITATION.cff file to indicate how to properly cite this repository.

1. I have finish the training based on the point_history.csv， but I cannot get the correct the results compare with weights you provided， I don‘t known why？
2. I collect the training data about Stop、Clockwise、Anti_Clockwise、Moving、Click、grab，but I also can't get correct the results，just only ’stop‘。

after following: keypoint_classification.ipynb
got error while converting .hdf5 to .tflite

...
Classification Report
              precision    recall  f1-score   support

           0       1.00      1.00      1.00       317
           1       1.00      1.00      1.00       329

    accuracy                           1.00       646
   macro avg       1.00      1.00      1.00       646
weighted avg       1.00      1.00      1.00       646

2022-07-13 00:07:33.159140: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-07-13 00:07:33.159164: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2022-07-13 00:07:33.159600: I tensorflow/cc/saved_model/reader.cc:45] Reading SavedModel from: /tmp/tmpe_iawth6
2022-07-13 00:07:33.160549: I tensorflow/cc/saved_model/reader.cc:89] Reading meta graph with tags { serve }
2022-07-13 00:07:33.160564: I tensorflow/cc/saved_model/reader.cc:130] Reading SavedModel debug info (if present) from: /tmp/tmpe_iawth6
2022-07-13 00:07:33.163551: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2022-07-13 00:07:33.164236: I tensorflow/cc/saved_model/loader.cc:229] Restoring SavedModel bundle.
2022-07-13 00:07:33.190545: I tensorflow/cc/saved_model/loader.cc:213] Running initialization op on SavedModel bundle at path: /tmp/tmpe_iawth6
2022-07-13 00:07:33.196456: I tensorflow/cc/saved_model/loader.cc:305] SavedModel load for tags { serve }; Status: success: OK. Took 36862 microseconds.
2022-07-13 00:07:33.208658: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:268] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
WARNING:absl:Buffer deduplication procedure will be skipped when flatbuffer library is not properly loaded
Traceback (most recent call last):
  File "handsign_train.py", line 94, in <module>
    tflite_quantized_model = converter.convert()
  File "/home/jason/hand-gesture-recognition-using-mediapipe/venv/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 930, in wrapper
    return self._convert_and_export_metrics(convert_func, *args, **kwargs)
  File "/home/jason/hand-gesture-recognition-using-mediapipe/venv/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 922, in _convert_and_export_metrics
    return flatbuffer_utils.convert_object_to_bytearray(model_object)
  File "/home/jason/hand-gesture-recognition-using-mediapipe/venv/lib/python3.8/site-packages/tensorflow/lite/tools/flatbuffer_utils.py", line 84, in convert_object_to_bytearray
    model_offset = model_object.Pack(builder)
  File "/home/jason/hand-gesture-recognition-using-mediapipe/venv/lib/python3.8/site-packages/tensorflow/lite/python/schema_py_generated.py", line 11662, in Pack
    operatorCodes = builder.EndVector()
TypeError: EndVector() missing 1 required positional argument: 'vectorNumElems'

handsign_train.py has the same code as keypoint_classification.ipynb

package:
tensorflow 2.9.1
tf-nightly 2.10.0.dev20220710
flatbuffers 1.12

Dear @Kazuhito00,

First of all thank you so much for sharing this amazing project. Your code is so well written and your tutorials are perfect.

I'm experiencing some issues with the "mode:loggin point history", when pressing "h" key while running the app.

What I'm trying to do, obviously, is to record some new points in order to train the model to recognise a new index gesture. What I find weird is that sometimes it works and I'm able to add new data points to the "point_history.csv" file, and sometimes, for no reason, the recording doesn't work.

As I'm aware the problem could come from many aspects of my environment and the way I'm following each of your tutorial steps, I just wanted to ask a more general question : have you been experiencing problems with the "Mode:logging point history" and would have any idea where this could come from ?

If you find that the issue lacks details about the context of the error, feel free to close and delete it, I'm just asking in case you would have experienced anything similar.

Thank you again, I hope you're fine 🙂

If there is no defined gesture it will also be forced to assign a gesture category, how to solve this problem?

I want to train my new gesture. But I see your dataset has a whopping 5k rows with approx 1k each gesture.

Was your data genuinely taken from 5k press on the app or it was generated by some trick?

How can I give the model a good dataset?

Many thanks

How to change this repo to detect two hands.?

Hi @Kazuhito00 ,

Thank you for your work on this.

I was just trying and playing with the your repo and I have observed that FPS on my Ubuntu 18.04 with NVIDIA GeForce RTX 2080 Ti I am seeing only 4 to 5 FPS, whereas at your side I am seeing 25 to 30 FPS.

I have tried using following:

1. TF lite model which was already in the repo - but same 4 to 5 FPS.
2. Build and compile TF lite model on using the HDF5 model in the repo. Converted HDF5 model from repo to TF lite model on my machine, but same 4 to 5 FPS.
3. Directly using HDF5 models for key point classifier and point history classifier model doing prediction / inference on GPU but still no luck same 4 to 5 FPS.

It will be helpful if you can provide some guidance on why at my side getting only 4 to 5 FPS.