ncbi / aioner Goto Github PK

Hello,
I'm trying to use finetuning, with the example for now, but I get an error. Finetuning didn't create an error, but using the generated model did. I don't get any errors when using a pre-trained model or a newly trained model.

python3 AIONER_Run.py -i ../example/input/ -m bioformer-softmax-es-finetune.h5 -v ../vocab/AnEM_label.vocab -e ALL -o example/output/

`
...
Total params: 42,598,935
Trainable params: 42,598,935
Non-trainable params: 0

Traceback (most recent call last):
File "AIONER_Run.py", line 252, in
NER_main_path(args.inpath, para_set, args.outpath, args.model)
File "AIONER_Run.py", line 194, in NER_main_path
nn_model.load_model(modelfile)
File "/home/amichaud/Documents/main/AIONER/src/model_ner.py", line 128, in load_model
self.model.load_weights(model_file)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 2211, in load_weights
hdf5_format.load_weights_from_hdf5_group(f, self.layers)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 708, in load_weights_from_hdf5_group
K.batch_set_value(weight_value_tuples)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
return target(*args, **kwargs)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/keras/backend.py", line 3576, in batch_set_value
x.assign(np.asarray(value, dtype=dtype(x)))
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 858, in assign
self._shape.assert_is_compatible_with(value_tensor.shape)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py", line 1134, in assert_is_compatible_with
raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (128, 19) and (128, 23) are incompatible

Hi, I followed this set up
conda create -n tf2.3 python=3.7
conda activate tf2.3
conda install -c conda-forge cudatoolkit=10.1 cudnn=7.6
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
then installed the requirements sucessfully but while running AIONER, I am getting this error. Please help resolving this issue as soon as possible. Thank you for this advance tool

AIONER-main$ python src/AIONER_Run.py -i ../example/input/ -m ../pretrained_models/AIONER/Bioformer-softmax-AIONER.h5 -v ../vocab/AIO_label.vocab -e ALL -o ../example/output/
2024-05-31 16:20:56.266464: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2024-05-31 16:20:58.052726: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2024-05-31 16:20:58.093898: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:31:00.0 name: NVIDIA GeForce RTX 4090 computeCapability: 8.9
coreClock: 2.52GHz coreCount: 128 deviceMemorySize: 23.64GiB deviceMemoryBandwidth: 938.86GiB/s
2024-05-31 16:20:58.093962: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2024-05-31 16:20:58.096369: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2024-05-31 16:20:58.098784: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2024-05-31 16:20:58.099218: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2024-05-31 16:20:58.101683: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2024-05-31 16:20:58.102992: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2024-05-31 16:20:58.108054: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2024-05-31 16:20:58.108705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
Num GPUs Available: 1
Traceback (most recent call last):
File "src/AIONER_Run.py", line 27, in
import stanza
File "/home/khyati/anaconda3/envs/tf2.3/lib/python3.7/site-packages/stanza/init.py", line 1, in
from stanza.pipeline.core import Pipeline
File "/home/khyati/anaconda3/envs/tf2.3/lib/python3.7/site-packages/stanza/pipeline/core.py", line 20, in
from stanza.pipeline.langid_processor import LangIDProcessor
File "/home/khyati/anaconda3/envs/tf2.3/lib/python3.7/site-packages/stanza/pipeline/langid_processor.py", line 5, in
import emoji
File "/home/khyati/anaconda3/envs/tf2.3/lib/python3.7/site-packages/emoji/init.py", line 47, in
from emoji.core import *
File "/home/khyati/anaconda3/envs/tf2.3/lib/python3.7/site-packages/emoji/core.py", line 12, in
from typing_extensions import Literal, Match, TypedDict
ImportError: cannot import name 'Match' from 'typing_extensions' (/home/khyati/anaconda3/envs/tf2.3/lib/python3.7/site-packages/typing_extensions.py)