xuezhemax / flowseq Goto Github PK

View Code? Open in Web Editor NEW

243.0 243.0 32.0 274 KB

Generative Flow based Sequence-to-Sequence Toolkit written in Python.

License: Apache License 2.0

Python 98.27% Perl 1.73%

flowseq's People

Contributors

Stargazers

Watchers

flowseq's Issues

Training time and distillation

Hi, Thanks for sharing your code. How many steps or training time do it need to train the flowseq model on WMT14 EN-DE? Will you release the distillation dataset? It will be helpful for us to reproduce your results.

trained model

Thanks for your excellent work.
Could you provide us with the trained model parameters and config file so that we can follow your work?

No such file or directory: 'wmt14_en_de/tmp/dev.de.ori'?

I used the script on fairseq to downlaod wmt14 data:
bash prepare-wmt14en2de.sh --icml17

Output file structure:

But when I run the training program, it shows an error:

  File "/content/flowseq/experiments/distributed.py", line 94, in run
    single_process_main(args)
  File "/content/flowseq/experiments/nmt.py", line 763, in main
    train_iter, val_iter, test_iter = init_dataloader(args, dataset)
  File "/content/flowseq/experiments/nmt.py", line 290, in init_dataloader
    bucket_data=args.bucket_batch, multi_scale=args.length_unit)
  File "/content/flowseq/flownmt/data/dataloader.py", line 291, in __init__
    self.data_size, self.data = self.get_dataset()
  File "/content/flowseq/flownmt/data/dataloader.py", line 337, in get_dataset
    ftgt_ori = None if tgt_ori_path is None else codecs.open(tgt_ori_path, "r", encoding="utf-8")
  File "/usr/lib/python3.6/codecs.py", line 897, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: 'wmt14_en_de/tmp/dev.de.ori

Can someone explain why restricting the posterior `z` as diagonal Gaussian?

Maybe I do not understand this paper throughly, but can someone explain this?
The posterior z is modelled as diagonal Gaussian. And in the Zero initialization part, ensures that the posterior distribution as a simple normal distribution.
If it is a simple distribution, why a complex prior flow is needed to learn its distribution?

Question about KL

Hello, I notice that your KL loss is calculated by "KL = (log_probs_posterior - log_probs_prior).mean(dim=1)". But I cannot understand why and cannot find a detailed explanation in your paper either. Could you help explain this to me or provide some references? Thanks so much!

a bug in your code

I think the function ``bwdpass'' in file https://github.com/XuezheMax/flowseq/blob/master/flownmt/flows/flow.py is wrong. The line94 and line99 should be exchanged.
by the way, I have a question. What does bwdpass mean? maping a complex distribution(e.g. latent variables Z in your paper) to a simpler prior distribution (e.g. latent variables V in Eq.5 in your paper)?
finally, thanks for your great work and code release.

question

Hi,
Thanks for sharing your paper,，
Can the methods in the paper be used to name entity recognition tasks?

Code running error

Thanks for the inspiring work!
When running the code I met a problem, maybe due to a version mismatch. I have to run in an environment with newer versions. Anyway, when I ran the python -u distributed.py, I bumped into the error:

Traceback (most recent call last):
  File "distributed.py", line 13, in <module>
    from flownmt.data import NMTDataSet
  File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/__init__.py", line 1, in <module>
    from flownmt.flownmt import FlowNMT
  File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/flownmt.py", line 11, in <module>
    from flownmt.modules import Encoder
  File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/__init__.py", line 1, in <module>
    from flownmt.modules.encoders import *
  File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/encoders/__init__.py", line 1, in <module>
    from flownmt.modules.encoders.encoder import Encoder
  File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/encoders/encoder.py", line 7, in <module>
    class Encoder(nn.Module):
  File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/encoders/encoder.py", line 25, in Encoder
    def forward(self, src_sents, masks=None) -> Tuple[torch.Tensor, torch.Tensor]:
  File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/overrides.py", line 83, in overrides
    return _overrides(method, check_signature, check_at_runtime)
  File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/overrides.py", line 170, in _overrides
    _validate_method(method, super_class, check_signature)
  File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/overrides.py", line 189, in _validate_method
    ensure_signature_is_compatible(super_method, method, is_static)
  File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/signature.py", line 102, in ensure_signature_is_compatible
    ensure_return_type_compatibility(super_type_hints, sub_type_hints, method_name)
  File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/signature.py", line 303, in ensure_return_type_compatibility
    f"{method_name}: return type `{sub_return}` is not a `{super_return}`."
TypeError: Encoder.forward: return type `typing.Tuple[torch.Tensor, torch.Tensor]` is not a `<class 'NoneType'>`.

Seems like it's caused by the behavior of @overrides of package overrides, I have no clue what this is doing, could you clarify this and make some modifications?
I'm runniing with Pytorh==1.10.1, Python==3.7.12, overrides==7.3.1. Thanks in advance.

a question about `tgt_path_ori`

Why we need tgt_path_ori and what is the difference between tgt_path_ori and tgt_path?

xuezhemax / flowseq Goto Github PK

flowseq's People

Contributors

Stargazers

Watchers

Forkers

flowseq's Issues

Training time and distillation

trained model

No such file or directory: 'wmt14_en_de/tmp/dev.de.ori'?

Can someone explain why restricting the posterior `z` as diagonal Gaussian?

Question about KL

a bug in your code

question

Code running error

a question about `tgt_path_ori`

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent