isds-neu / phycrnet Goto Github PK

Hi,
I've been trying to test your code, but while running Burgers_2d_solver_[HighOrder].py there occurs an issue which leads to the values for U and V becoming entirely "nan" after the first iteration and thus all the plots after the first one are blank and the generated file to be used in PhyCRNet_burgers.py does not work correctly.

The warnings which I assume relate to the cause of this problem, when running Burgers_2d_solver_[HighOrder].py, are as follows:
RuntimeWarning: overflow encountered in multiply
u_t = (1.0/R) * laplace_u - U * u_x - V * u_y
RuntimeWarning: invalid value encountered in subtract
u_t = (1.0/R) * laplace_u - U * u_x - V * u_y
RuntimeWarning: overflow encountered in multiply
v_t = (1.0/R) * laplace_v - U * v_x - V * v_y

...and so on, covering what looks like all the operations.

If you have any knowledge on how to solve this issue I'd really appreciate it.
And thank you for making this available for everyone to study.

Thanks for sharing the code.
Your current model input is the initial condition and hidden states for the ConvLSTM,
From a perspective of generalization, have you considered feeding a batch of initial conditions, instead of a single one to train the model? Would that increase the performance?
Also, does a single initial condition in PICRNN equal to a single collocation point in PINN? Not sure how to justify that.

Thanks again for your work.

Hello,

Thanks a lot for this great work that you published transparently

I have a few questions

Firstly, I noticed there are two time-related parameters:

• time_batch_size
• time_steps

But I am not sure how to change them when finetuning. Say, I have a model trained until 100 steps, and I want to finetune to 200 steps (and ultimately until 1000 timesteps). In this case, what would the values of time_steps and time_batch_size be?

Similarly, what would the values be when training from an intialized model to 100 timesteps?

Also, for number of adam iterations, is it 10000 for each finetune, or is it 10000 across the entire sequence of finetunings?

Finally, I noticed that in your paper, you initialized h and c to zero for the LSTM, but in the code they are intialized randomly. Is there a preferred initalization between these?

Best,
Rami

Hi,

Thanks for sharing your work and code. In your paper, I found that as for the Dirichlet BCs, you used the periodic padding. To be honest, I don't understand why this operator can achieve Dirichlet BCs. Besides, if it is Neumann BCs, how can we deal with?

Looking forward to your reply! Thanks!

RTX3060 cannot install torch 1.6.0

Hi Paul,

I hope you are doing well.

I have a question when trying to running the python script. It requires to load a previous trained model, './model/checkpoint500.pt'. Can you please tell me how to obtain this model, or how to define the weights/biases for initializing the network?

many thanks in advance.

Hi,

I was trying to run the model, but I failed. It reminded: 'c' argument has 16641 elements, which is inconsistent with 'x' and 'y' with size 16384, on Line 506. I found that the shape of x_star is [128,128], but the shape of u_pred is [129,129]. Could you please check the code?

Best wishes

When I was training the file PhyCRNet.py, I could find the file burgers_1501x2x128x128.mat in the directory. So I am not able to determine the dimension of uv.

Many thanks!
Johnny

Hi Paul,

I have a question regarding the role of residual connection. In PhyCRNet, the temporal derivative is already incorporated in the loss term, why do you still need the residual connection?

I guess each encoder-convlstm-decoder may be considered a Unet. In this sense, the residual connection may function as skip connections right? In my case, the variable u has two components, say, u1 and u2. The PDE is in the form {\partial u1} / {\partial t} = \alpha {\divergence (\nabla (u2)) +...

If I do u = ut + dt*u, it does not seem to work.

Many Thanks.

Hi,

I have noticed from the random_fields.py that the data is generated by sampling from a Gaussian distribution, and then passed to an inverse Fourier transform, therefore I would assume that values of u and v in the generated data to be in the frequency domain, including the truth values computed using the Runge-Kutta method. However, when comparing the output of the neural network and the truth values, it seems that you computed the error using the Frobenius
norm without converting the values back to the time domain.

May I ask whether what domain the generated data is in and why it never passed through a second Fourier transform?

Thank you in advance!

Hello,

Thanks a lot for this great work that you published transparently.

Pretrain the network is from a small number of time steps (e.g., 100) and then gradually train on a longer dynamical evolution. Taking 2D Burgers' equation as an example, we pretrain the model from 100, then 200 and 500, and finally 1000.

I have a questions anout the initialization condition.

For pretrained networks with different time steps(e.g. 100, 200, 500, 1000), is the initialization condition always 'uv0 = uv[0:1,...]
input = torch.tensor(uv0, dtype=torch.float32).cuda()'?

If I keep training from t=0 for different time steps, it turns out to have extremely large loss for 500 timesteps.
[19998/20000 99%] loss: 0.1154880226
[19999/20000 99%] loss: 0.1154880226
[20000/20000 100%] loss: 0.1154880226
The predicted error is: 0.12718813

Hi,

Very interesting work! I have one small question: taking the 2D Burger's equation as an example, if the viscosity value is changed, we need to retrain the neural network?

Best wishes

Hello author,
I read several issues you responded to in the other question, then I ran the code.
First, I deleted "load_checkpoint" in the train function. then I set time_batch_size=100, time_steps=101, and n_iters_adam=500, I trained the network, I found that there was a huge loss, and also in the post_process function it came an error "'c' argument has 16641 elements, which is inconsistent with 'x' and 'y' with size 16384.
Can you help me see if there are some problems with the code running steps! Thank you very much!

Hi,

one common argument to use surrogate models over numerical solvers is that the latter are more costly to use computational wise.

Did you compare the time to rollout the network against the time to solve e.g. the Burgers equation numerically once the network is fully trained? Unfortunately, I could not find such a comparison in the paper.

Thanks a lot!

Hello! Thanks for sharing the code. I have a small question about the code.

In line 431, you write for time_batch_id in range(num_time_batch):, but the number of num_time_batch is 1, as calculated by num_time_batch = int(time_steps / time_batch_size_load) in line 610. This means the code only calculates the first step. Is there something wrong with the code or my understanding?

I am looking forward to your early reply.

Hi
Thanks for sharing the code. What's the rationale for using this uniform range? What do the numbers 3,3,20 represent?

module.weight.data.uniform_(-cnp.sqrt(1 / (3 * 3 * 320)),
cnp.sqrt(1 / (3 * 3 * 320)))

Hello,

Thanks a lot for this great work that you published transparently.

I have a question that can this method be applied to equations containing elements with a mixture of time and space derivatives? such like Ut+Utt+Uxxt+Uyyt=0.

And in the code, ' output = torch.cat((output[:, :, :, -1:], output, output[:, :, :, 0:2]), dim=3) ', why are the first（output[:, :, :, -2:]） and last（output[:, :, :, 0:3]） elements not symmetrical? ( why not-2: and 0:2 )

Thank you very much for your answer, looking forward to your reply！

Hi,
Very interesting work! We notice the readme file has the comparison with PINN. But we can not find the code. Can you provide the PINN code on this case to appreciate. Thank you so much !

I'm doing the experiment on coupled Burger's, following instructions described in README.md that firstly turn the argument "steps" to a relatively small number. However, it turns out to have extremely large loss:
[1/20000 0%] loss: 18560561113464832.0000000000
[2/20000 0%] loss: 18553907806470144.0000000000
[3/20000 0%] loss: 18551367970848768.0000000000
[4/20000 0%] loss: 18550044584050688.0000000000
[5/20000 0%] loss: 18548749886291968.0000000000
[6/20000 0%] loss: 18547451732426752.0000000000
Does this happen normally?