melobio / logo Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
I didn't change any parameters, but the weights and model dimensions don't match
ValueError: Shape mismatch in layer #2 (named Embedding-Token) for weight Embedding-Token/embeddings:0. Weight expects shape (3138, 256). Received saved weight with shape (3138, 128)
type: [1]
label: [1]
(1837498,)
(1837498,)
(1837498,)
Traceback (most recent call last):
File "00_cadd_create_vcf_sequence_data_multi_thread.py", line 548, in
y_label=1
File "00_cadd_create_vcf_sequence_data_multi_thread.py", line 437, in process_raw_text
batch_no=slice_index)
File "00_cadd_create_vcf_sequence_data_multi_thread.py", line 162, in create_tfrecord_with_sequence
'alt_type': tf.train.Feature(int64_list=tf.train.Int64List(value=list(type))),
TypeError: 'numpy.int64' object is not iterable
Hello,
When I ran the model “02_cadd_classification_transformer_tfrecord.py” for CADD data, I used the following data and ran into error messages: can’t parse serialized example. Can you provide help? I would like to apply this process to predict pathogenic effects of somatic variants. Or can you provide revised codes and an example data?
train_slice_files = [
'/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_100000_100000_train.tfrecord','/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_200000_200000_train.tfrecord'
]
valid_slice_files = [
'/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_100000_100000_valid.tfrecord' ]
test_slice_files = [
'/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_100000_100000_test.tfrecord' ]
train_total_size =180000
valid_total_size = 5000
test_total_size = 5000
I ran into the following errors:
Train on 175 steps, validate on 2 steps
Epoch 1/100
2021-10-25 21:11:09.434153: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
2021-10-25 21:11:09.434270: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
Traceback (most recent call last):
File "02_cadd_classification_transformer_tfrecord.py", line 278, in
verbose=1)
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 727, in fit
2021-10-25 21:11:09.434995: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example.
2021-10-25 21:11:09.435078: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example.
2021-10-25 21:11:09.435134: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
2021-10-25 21:11:09.435209: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example.
2021-10-25 21:11:09.435261: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
use_multiprocessing=use_multiprocessing)
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py", line 675, in fit
2021-10-25 21:11:09.435613: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example.
2021-10-25 21:11:09.435697: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example.
steps_name='steps_per_epoch')
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py", line 300, in model_iteration
2021-10-25 21:11:09.436015: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
batch_outs = f(actual_inputs)
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/backend.py", line 3476, in call
2021-10-25 21:11:09.436259: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
2021-10-25 21:11:09.436336: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
run_metadata=self.run_metadata)
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1472, in call
2021-10-25 21:11:09.437045: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example.
2021-10-25 21:11:09.437105: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example.
2021-10-25 21:11:09.437173: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example.
2021-10-25 21:11:09.437225: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example.
run_metadata_ptr)
tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __inference_Dataset_map_single_example_parser_1272}} Key: alt_seq. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext]]
2021-10-25 21:11:09.452867: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example.
2021-10-25 21:11:09.452934: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example.
The following have been reloaded with a version change:
Using TensorFlow backend.
WARNING:tensorflow:There is non-GPU devices in tf.distribute.Strategy
, not using nccl allreduce.
WARNING:tensorflow:From ../bgi/bert4keras/models.py:179: The name tf.keras.initializers.TruncatedNormal is deprecated. Please use tf.compat.v1.keras.initializers.TruncatedNormal instead.
WARNING:tensorflow:From /risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/initializers.py:94: calling TruncatedNormal.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
Traceback (most recent call last):
File "02_train_gene_transformer_lm_hg_bert4keras_tfrecord.py", line 339, in
verbose=1
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 727, in fit
use_multiprocessing=use_multiprocessing)
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_distributed.py", line 685, in fit
steps_name='steps_per_epoch')
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py", line 300, in model_iteration
batch_outs = f(actual_inputs)
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/backend.py", line 3476, in call
run_metadata=self.run_metadata)
File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1472, in call
run_metadata_ptr)
tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __inference_Dataset_map_parse_function_2564}} Key: sequence. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseSingleExample}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]]
[[IteratorGetNext]]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.