pv-lab / ftcp Goto Github PK

The dummy example (truncated to 100 MP compounds) runs to completion just fine despite the OOM error with the full dataset #8. Thanks for troubleshooting getting this up and running on Google Colab. Feel free to use or modify the following script if desired - might make for a nice badge to include at the top of the README if you decide you want to use/adapt it.

when I run main.py, the data retrieval process seems to be not stable (although it works anyway).

If I use the same configuration of data selection, is the data downloaded from the server each time? (or use the local files?)

Using TensorFlow backend.
0%| | 0/48322 [00:00<?, ?it/s]Unknown server error. Trying again in five seconds (will try at most 5 times)...
2%|▉ | 1000/48322 [00:22<14:44, 53.52it/s]Unknown server error. Trying again in five seconds (will try at most 5 times)...
8%|███▋ | 4000/48322 [01:00<03:44, 197.10it/s]Unknown server error. Trying again in five seconds (will try at most 5 times)...
9%|████▎ | 4500/48322 [01:20<12:15, 59.56it/s]Unknown server error. Trying again in five seconds (will try at most 5 times)...
14%|██████▌ | 7000/48322 [01:33<04:43, 145.64it/s]

When I train the model with your code main_semi.py, after some epochs, I got the error from VAE.fit():

Exception has occurred: InvalidArgumentError
2 root error(s) found.
(0) INVALID_ARGUMENT: slice index 0 of dimension 0 out of bounds.
[[{{node lambda/strided_slice_3}}]]
[[loss/mul/_241]]
(1) INVALID_ARGUMENT: slice index 0 of dimension 0 out of bounds.
[[{{node lambda/strided_slice_3}}]]

It seems that the some dimension of the data or model gets wrong, strangely the training can be excuted for over 100 epochs before raise error. I use:
max_elms = 4
min_elms = 3
max_sites = 8

I am a Linux user and tried to run the data.py file on colab.
I get an error while importing MPDataRetrieval.
So this is what I tried:

!pip install matminer
from matminer.data_retrieval.retrieve_MP import MPDataRetrieval

And this is the error:

ImportError                               Traceback (most recent call last)

[<ipython-input-12-92dab7dbcc0d>](https://localhost:8080/#) in <module>()
----> 1 from matminer.data_retrieval.retrieve_MP import MPDataRetrieval
      2 
      3 def data_query(mp_api_key, max_elms=3, min_elms=3, max_sites=20, include_te=False):
      4     """
      5     The function queries data from Materials Project.

4 frames

[/usr/local/lib/python3.7/dist-packages/scipy/cluster/vq.py](https://localhost:8080/#) in <module>()
     68 import numpy as np
     69 from collections import deque
---> 70 from scipy._lib._util import _asarray_validated, check_random_state,\
     71     rng_integers
     72 from scipy.spatial.distance import cdist

ImportError: cannot import name 'rng_integers' from 'scipy._lib._util' (/usr/local/lib/python3.7/dist-packages/scipy/_lib/_util.py)

Dear the authors,
Can you share the pre-trained models which we can use it to generate materials directly?

Thanks,
Yong

When using parameters:

max_elms = 3
min_elms = 3
max_sites = 3

the input and output dimension are different. Of course, It's only a corner case.

This is great work. The article mentions you generate crystals based on the target property. I would appreciate it if you could share a gist of that code.

Traceback (most recent call last):
File "/Users/mayuxing/Downloads/FTCP-master 2/main.py", line 55, in
VAE.fit([X_train, y_train],
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/var/folders/pr/t7phtfp9055f_v4lkb49g73c0000gn/T/autograph_generated_file32azsqjs.py", line 15, in tf__train_function
retval = ag_.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
TypeError: in user code:

File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/engine/training.py", line 1051, in train_function  *
    return step_function(self, iterator)
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/engine/training.py", line 1040, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/engine/training.py", line 1030, in run_step  **
    outputs = model.train_step(data)
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/engine/training.py", line 890, in train_step
    loss = self.compute_loss(x, y, y_pred, sample_weight)
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/engine/training.py", line 948, in compute_loss
    return self.compiled_loss(
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/engine/compile_utils.py", line 239, in __call__
    self._loss_metric.update_state(
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/utils/metrics_utils.py", line 70, in decorated
    update_op = update_state_fn(*args, **kwargs)
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/metrics/base_metric.py", line 140, in update_state_fn
    return ag_update_state(*args, **kwargs)
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/metrics/base_metric.py", line 449, in update_state  **
    sample_weight = tf.__internal__.ops.broadcast_weights(
File "/Users/mayuxing/opt/anaconda3/lib/python3.9/site-packages/keras/engine/keras_tensor.py", line 254, in __array__
    raise TypeError(

TypeError: You are passing KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='Placeholder:0', description="created by layer 'tf.cast_2'"), an intermediate Keras symbolic input/output, to a TF API that does not allow registering custom dispatchers, such as `tf.cond`, `tf.function`, gradient tapes, or `tf.map_fn`. Keras Functional model construction only supports TF API calls that *do* support dispatching, such as `tf.math.add` or `tf.reshape`. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a custom Keras layer `call` and calling that layer on this symbolic input/output.

I dont know how to debug this. Look forward to you answer. :)

Using all 48322 compounds and the GPU hardware on Colab + the default batch size of 256, I get the following:

Your session crashed after using all available RAM. If you are interested in access to high-RAM runtimes, you may want to check out Colab Pro.

(switch dummy flag to False)

More of an FYI if someone else is trying to use Colab. Wouldn't be surprised if Colab Pro had enough RAM. How much RAM was available on the workstation that was used to run FTCP?