michschli / graphmask Goto Github PK

Thanks for sharing the code!

I have some problem about how to select optimizer when training diffmask. I find that Lookahead RMSprop is used in 'How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking'. But in this work, RMSProp is chosed. Why you change the optimizer? Does the choice of optimizer affect the result a lot?

It will help me a lot if I can get some advice！

Hi, I run your code to train the model on QA dataset using command python train_model.py --configuration configurations/qa.json. However, there is a WARNING bug when it begins as:

Building the train set (or loading from cache if possible): 4%| | 1965/43738 [00:09<04:01, WARNING:allennlp.models.model:Encountered the antecedent_indices key in the model's return dictionary which couldn't be split by the batch size. Key will be ignored.

And then, the Building the train set process have no response all the day.

Why is that? Can you help me with it?

My Environment:

• Python 3
• PyTorch 1.7.1
• PyTorch Geometric 1.7.2
• AllenNLP 0.9.0
• SpaCy 2.1.9

ConfigurationError Traceback (most recent call last)
in ()
9
10 model_trainer = ModelTrainer(configuration, gpu=args.gpu)
---> 11 model_trainer.train()

in train(self)
19
20 def train(self):
---> 21 problem = self.experiment_utils.build_problem()
22 model = self.experiment_utils.build_model()
23

/content/GraphMask/codes/utils/experiment_utils.py in build_problem(self)
31 problem_class = QAProblem
32
---> 33 problem = problem_class(self.configuration)
34
35 return problem

/content/GraphMask/codes/problems/qa/qa_problem.py in init(self, configuration)
44 # self.predictor = pretrained.load_predictor("coref")
45 self.predictor = Predictor.from_path(
---> 46 "https://s3-us-west-2.amazonaws.com/allennlp/models/coref-model-2018.02.05.tar.gz")
47 # self.predictor = pretrained.load_predictor("coref-spanbert")
48 # self.predictor = Predictor.from_path(

/usr/local/lib/python3.7/dist-packages/allennlp/predictors/predictor.py in from_path(cls, archive_path, predictor_name, cuda_device, dataset_reader_to_load, frozen, import_plugins, overrides)

/usr/local/lib/python3.7/dist-packages/allennlp/models/archival.py in load_archive(archive_file, cuda_device, overrides, weights_file)
203 else:
204 logger.warning(f"Archived file {replacement_filename} not found! At train time "
--> 205 f"this file was located at {original_filename}. This may be "
206 "because you are loading a serialization directory. Attempting to "
207 "load the file from its train-time location.")

/usr/local/lib/python3.7/dist-packages/allennlp/models/archival.py in _load_dataset_readers(config, serialization_dir)
229 serialization_dir=serialization_dir,
230 cuda_device=cuda_device)
--> 231
232 return Archive(model=model, config=config)
233

/usr/local/lib/python3.7/dist-packages/allennlp/common/from_params.py in from_params(cls, params, constructor_to_call, constructor_to_inspect, **extras)

/usr/local/lib/python3.7/dist-packages/allennlp/common/params.py in pop_choice(self, key, choices, default_to_first_choice, allow_class_names)
350 This is to allow e.g. specifying a model type as my_library.my_model.MyModel
351 and importing it on the fly. Our check for "looks like" is extremely lenient
--> 352 and consists of checking that the value contains a '.'.
353 """
354 default = choices[0] if default_to_first_choice else self.DEFAULT

ConfigurationError: coref not in acceptable choices for dataset_reader.type: ['babi', 'conll2003', 'interleaving', 'multitask', 'multitask_shim', 'sequence_tagging', 'sharded', 'text_classification_json']. You should either use the --include-package flag to make sure the correct module is loaded, or use a fully qualified class name in your config file like {"model": "my_module.models.MyModel"} to have it imported automatically.