photon-ai-research / neuralsolvers Goto Github PK

• Creation of the Data
• Running training

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/2a9457257b35265d219db245eb765c6132559748/PINNFramework/InitalCondition.py#L28

/home/maxxxzdn/.local/lib/python3.8/site-packages/torch/nn/modules/loss.py:446: UserWarning: Using a target size (torch.Size([batch_size])) that is different to the input size (torch.Size([batch_size, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
return F.mse_loss(input, target, reduction=self.reduction)

Using prediction = model(x).reshape(-1) solves the problem

Maybe its a good idea to add tqdm

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/acb61830c0f93a286b59caeacaa421291b3a76d3/PINNFramework/PINN.py#L323

I suppose it has to be 'hpm_model' object instead of 'model'

• Split the EC Dataset into three datasets
• Implement the Normalization Condition as a new designed Boundary Condition (https://pytorch.org/docs/stable/generated/torch.trapz.html) could make things easier
• Integrate the new normalization condition into the PINN loss calculation
• Switch from x,y,t representation to a single tensor that represents all cases
• Integrate the implementation of the analytical solution in the IC dataset

PINN.py, line 109: torch.save(self.pinn_loss.model.state_dict(), hpm_path)
self.pinn_loss is a function

The general idea of the framework is not well transported and should be better illustrated

the library is growing, we should write a nice documentation :)

Integrating Horovod Support for faster training

.to(device) implementation does not work properly. Needs to be changed to following:

self.lb = self.lb.to(device)
self.ub = self.ub.to(device)

In the HPMLoss the weighting is not set correctly.

super(HPMLoss, self).__init__(dataset, None, name, norm='L2', weight=1.)

should be

super(HPMLoss, self).__init__(dataset, None, name, norm='L2', weight=weight)

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/acb61830c0f93a286b59caeacaa421291b3a76d3/PINNFramework/PINN.py#L323

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/acb61830c0f93a286b59caeacaa421291b3a76d3/PINNFramework/PINN.py#L461

I suppose it has to be 'self.pde_loss.hpm_model' instead of 'self.pinn_loss.hpm_model' since self.pinn_loss is a method

At the moment there is no measurement of elapsed time per epoch and complete training time. That should be added soon.

Inherit from PDE Loss and implement the baviour of the HPM-loss

Create the markdown files and upload it ro read the docs.

At the moment only the boundary condtion has the attribute name. For the workflow of the framework it would be more consistent if all loss terms have a name.

when choosing extremely small no. of samples , which lead that the len(no._of batches) of the dataset which is fed to the dataloader to be equal to zero, which makes the iteration loop to for i in range( 0,0, step): .. thats why the looping didn't work and the ic_loss wasn't calculated at the first place.
i would suggest creating a warning message when choosing a no. of samples which can make the length of the datasets yields to zero after the using of "//" operator.

the for loops are found in line 503 and 518 in the pinn.fit () function in the pinn.py file

In order to create consistent checkpoints. the checkpoint name should be defined by a prefix.

The jupyter notebooks are using the old API and are not working anymore. The only working code seems to be 1D_Schroedinger.py which is not well documented.

This makes it quite hard to actually use the nice new API :(

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/2a9457257b35265d219db245eb765c6132559748/PINNFramework/PINN.py#L126

Seems like it tries to get the attribute "hpm_model" of the method "pinn_loss" which leads to an error.

Implement benchmark suite with:

• standard setups for various PDEs
• default argument parser
• switch of and on various PDEs by the argument list
• logging
• model saving depending on PDEs

At the moment the logging is not very consistent. The weighted loss is uploaded to the logger one step earlier than the scalars. That should be changed by a consistent set of the epoch in the logger call:

for key, value in self.loss_log.items():
    logger.log_scalar(scalar=value / batch_counter, name=key, epoch=epoch+1)

vs:

logger.log_scalar(scalar=pinn_loss_sum / batch_counter, name=" Weighted PINN Loss", epoch=epoch)

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/acb61830c0f93a286b59caeacaa421291b3a76d3/PINNFramework/PINN.py#L361

Found a bug here

• single flag for storing gradients
• Outsource annealing
• Outsource orthogonalisation

At the moment the FingerNet only support an input size = 4. The Model should be reimplemented for dynamic input sizes to make it available for lower or higher dimensional problems.

According to the convergence of PINN paper this could lead to better convergence behavior

At the moment, you need to set the number of main training epochs to 0, activate the pretaining function, and deactivate lbfgs fine-tuning. Here it would be better to have a separate function that trains initial condition only. This helps to find a suitable architecture.

Adding some callbacks would help to track the model better.
Example callbacks for wandb, tensorboard etc. would be nice.

To keep track of gradient pathologies a tracking of the loss terms gradients is needed.
The logging should be done in the optimization process and can be activated by a separate flag in the fit function.

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/bf9af0a3d23bc156a05a339473a75d0b65d25d5f/PINNFramework/PINN.py#L97

I feel like there should be a line sending the model to cuda if needed.

the function PINNFramewrok.PINN(.., boundary_condition,...) only accepts as input argument for boundary_condition elements of the form boundary_condition= [bc1,bc2,.. ], but even for only one bc it requires the input to be [bc]. Otherwise it raises 'KeyError' because it cannot call the bc from its name without the list form.

Models like the GatedPINN uses a model specific loss in order to balance the utilization of the experts.
An option would be to give the model an attribute called loss.

In the PINN loss we can call the loss function with :

if hasattr(self.model, 'loss'):
    pinn_loss = pinn_loss + model.loss

Inherit from PDE Loss and implement the baviour of the HPM-loss

Here, the interpolation model is sent to cuda():

https://github.com/ComputationalRadiationPhysics/NeuralSolvers/blob/ae913e9a9aaa1b9c5e4c86bbd3bde5b727c26e5a/PINNFramework/PINN.py#L70

Meanwhile, I could not find such a line for the hpm model.

It's not needed. Control structures in the constructor just blow up the code base.

Setting the default batch_size to None leads to a more understandable behaviour (dataloader yields a shape [n, dim])

Currently, the fit function is a very big function that includes tracking, fitting, parallelization, etc.
Here we should modularize the functionalities to get a more maintainable code.

Add the regualization of the PINN loss from this paper to the framework:
https://arxiv.org/abs/2004.01806