auspicious3000 / wavenet-enhancement Goto Github PK

Can you please mention the training time for this model for the given configurations?

Hi,
If you can upload or send me a part your .mat data? I want to look at your data structrue.
Thank you very much!
My e-mail is [email protected]
Thank you.

Hi,
I got a set of files on my Logging directory after the prior trained. which one should be the input to likelihood?
there are files 'checkpoint' & 'bawn_pr_v2.ckpt-349816' and 'losses'

Hi,
You have not mentioned about how do we use the trained model for testing on a noisy speech. Could you please tell me that? Do I need to write separate script myself for this?

Hi,
In what dimension should I give the inputs to the function 'bawn_pr_multi_gpu_train.py' & 'bawn_ll_multi_gpu_train.py'. I know the input data has to be stored in HDF5 format, should I create my data as matrix with each wave files samples as the rows?

Thank you for your sharing!
I am impressed by your work and I have two questions about the details of code.

1. To train the priori model, the input is "train_pr.mat". However, in "bawn_pr_multi_gpu_train.py", the code is "bawn.load_data_prior('train_pr.mat','target_pr.mat')". It seems to input the '"target_pr.mat'", so how to construct the "target_pr.mat'"? Although in "assets", there are "train_pr.mat" and "target_pr.mat', but the type is unmatched. According to the code, the "target_pr.mat" should be "int32".
   Actually, the same questions about "bawn_ll_multi_gpu_train.py".
2. Could you tell me how to inference the noisy signal based on the trained model?
   Looking forward to your reply.

Can you please share a pretrained model? I really don't want to train this from scratch if I can help it?

When creates output data you had mentioned
'Output data is the corresponding prediction shifted by one sample to the right'

which set of samples do I need to look as the prediction?, Does that the 16384 samples being immediately followed the input chunk of 20477 Or Final 16384 samples of the input?

Hi!
Thank you very much for your work.
I would like to improve it with the global and local conditioning in the clean and noisy models. But first I would like to train it and make it work as it is.
I am running it on TF2 and I left it as it is except for some APIs compatibility with TF2 (tf.compat.v1. etc).
I succeded to train the prior model but when I try to train the likelihood model I get these errors:

2021-06-09 15:48:12.013135: W tensorflow/core/framework/op_kernel.cc:1763] OP_REQUIRES failed at save_restore_v2_ops.cc:205 : Not found: Key clean/b0-l0/pre/bias not found in checkpoint
Traceback (most recent call last):
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1375, in _do_call
return fn(*args)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1360, in _run_fn
target_list, run_metadata)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1453, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: Key clean/b0-l0/pre/bias not found in checkpoint
[[{{node save/RestoreV2}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1298, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 968, in run
run_metadata_ptr)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1191, in _run
feed_dict_tensor, options, run_metadata)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1369, in _do_run
run_metadata)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1394, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key clean/b0-l0/pre/bias not found in checkpoint
[[node save/RestoreV2 (defined at models/bawn_ll_multi_gpu_train.py:197) ]]

Original stack trace for 'save/RestoreV2':
File "models/bawn_ll_multi_gpu_train.py", line 290, in
train()
File "models/bawn_ll_multi_gpu_train.py", line 197, in train
saver = tf.compat.v1.train.Saver(tf.compat.v1.global_variables(), max_to_keep=8192) # max_to_keep=4096
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 835, in init
self.build()
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 847, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 885, in _build
build_restore=build_restore)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 515, in _build_internal
restore_sequentially, reshape)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 335, in _AddRestoreOps
restore_sequentially)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 582, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1511, in restore_v2
name=name)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 750, in _apply_op_helper
attrs=attr_protos, op_def=op_def)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3536, in _create_op_internal
op_def=op_def)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1990, in init
self._traceback = tf_stack.extract_stack()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/py_checkpoint_reader.py", line 70, in get_tensor
self, compat.as_bytes(tensor_str))
RuntimeError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1308, in restore
names_to_keys = object_graph_key_mapping(save_path)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1626, in object_graph_key_mapping
object_graph_string = reader.get_tensor(trackable.OBJECT_GRAPH_PROTO_KEY)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/py_checkpoint_reader.py", line 74, in get_tensor
error_translator(e)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/py_checkpoint_reader.py", line 35, in error_translator
raise errors_impl.NotFoundError(None, None, error_message)
tensorflow.python.framework.errors_impl.NotFoundError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "models/bawn_ll_multi_gpu_train.py", line 290, in
train()
File "models/bawn_ll_multi_gpu_train.py", line 236, in train
with sv.managed_session(config=sess_config, start_standard_services=False) as sess:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/supervisor.py", line 1014, in managed_session
self.stop(close_summary_writer=close_summary_writer)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/supervisor.py", line 839, in stop
ignore_live_threads=ignore_live_threads)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/six.py", line 703, in reraise
raise value
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/supervisor.py", line 1003, in managed_session
start_standard_services=start_standard_services)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/supervisor.py", line 734, in prepare_or_wait_for_session
init_fn=self._init_fn)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/session_manager.py", line 295, in prepare_session
config=config)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/session_manager.py", line 225, in _restore_checkpoint
saver.restore(sess, ckpt.model_checkpoint_path)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1314, in restore
err, "a Variable name or other graph key that is missing")
tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key clean/b0-l0/pre/bias not found in checkpoint
[[node save/RestoreV2 (defined at models/bawn_ll_multi_gpu_train.py:197) ]]

Original stack trace for 'save/RestoreV2':
File "models/bawn_ll_multi_gpu_train.py", line 290, in
train()
File "models/bawn_ll_multi_gpu_train.py", line 197, in train
saver = tf.compat.v1.train.Saver(tf.compat.v1.global_variables(), max_to_keep=8192) # max_to_keep=4096
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 835, in init
self.build()
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 847, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 885, in _build
build_restore=build_restore)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 515, in _build_internal
restore_sequentially, reshape)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 335, in _AddRestoreOps
restore_sequentially)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 582, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1511, in restore_v2
name=name)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 750, in _apply_op_helper
attrs=attr_protos, op_def=op_def)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3536, in _create_op_internal
op_def=op_def)
File "/Users/andreagulli/PycharmProjects/Deepbeam_conditioned/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1990, in init
self._traceback = tf_stack.extract_stack()

Hi, do you have a model available? I would like to use your model for speech enhancement, and I don't really want to have to train one.

could you tell me how to create the input data or the format?

I've a RTX 2060. Any idea how long will the training process take?

It is said this model is "speaker dependent", but can anyone give me an intuitive view why this model should be "speaker dependent"? Now that the model has a good generalization on different noises, why it can not generalize well on different speakers? Or should I expect this model generalize well if I add more speaker sources?