Solving Partial Differential Equations with Neural Networks.
Nangs is a Python library built on top of Pytorch to solve Partial Differential Equations.
Our objective is to develop a new tool for simulating nature, using Neural Networks as solution approximation to Partial Differential Equations, increasing accuracy and optimziation speed while reducing computational cost.
Read our paper to know more.
Read the docs.
nangs is on PyPI so you can just run:
pip install nangs
You will also need to insall Pytorch.
Let's assume we want to solve the following PDE:
Different numerical techniques that solve this problem exist, and all of them are based on finding an approximate function that satisfies the PDE. Traditional numerical methods discretize the domain into small elements where a form of the solutions is assumed (for example, a constant) and then the final solution is composed as a piece-wise, discontinuous function.
Nangs uses the property of neural networks (NNs) as universal function approximators to find a continuous and derivable solution to the PDE, that requires significant less computing resources compared with traditional techniques and with the advantage of including the free-parameters as part of the solution.
The independen variables (i.e, x and t) are used as input values for the NN, and the solution (i.e. p) is the output. In order to find the solution, at each step the NN outputs are derived w.r.t the inputs. Then, a loss function that matches the PDE is built and the weights are updated accordingly. If the loss function goes to zero, we can assume that our NN is indeed the solution to our PDE.
import math
import numpy as np
import matplotlib.pyplot as plt
import torch
# import nangs
from nangs.pde import PDE
from nangs.bocos import PeriodicBoco, DirichletBoco
from nangs.solutions import MLP
# define custom PDE
class MyPDE(PDE):
def __init__(self, inputs, outputs, params=None):
super().__init__(inputs, outputs, params)
def computePDELoss(self, grads, inputs, outputs, params):
# here is where the magic happens
dpdt, dpdx = grads['p']['t'], grads['p']['x']
u = params['u']
return [dpdt + u*dpdx]
# instanciate pde
pde = MyPDE(inputs=['x', 't'], outputs=['p'], params=['u'])
# define input values for training
x = np.linspace(0,1,40)
t = np.linspace(0,1,30)
u = np.array([1.0])
pde.setValues({'x': x, 't': t[:-1], 'u': u})
# define input values for testing
x_v = np.linspace(0,1,25)
t_v = np.linspace(0,1,15)
pde.setValues({'x': x_v, 't': t_v[:-1]}, train=False)
# periodic b.c for the space dimension
boco = PeriodicBoco('boco', {'x': x[:1], 't': t[:-1]}, {'x': x[-1:], 't': t[:-1]})
pde.addBoco(boco)
# initial condition (dirichlet for temporal dimension)
p0 = np.sin(2.*math.pi*x)
boco = DirichletBoco('initial_condition', {'x': x, 't': t[:1]}, {'p': p0})
pde.addBoco(boco)
# define solution topology
mlp = MLP(pde.n_inputs, pde.n_outputs, 3, 256)
optimizer = torch.optim.Adam(mlp.parameters(), lr=3e-4)
pde.compile(mlp, optimizer)
# find the solution
hist = pde.solve(epochs=50)Epoch 1/50 Losses 0.40340 PDE [ 0.01088 ] boco 0.02617 initial_condition 0.36635 Val [ 0.02663 ]
Epoch 2/50 Losses 0.30586 PDE [ 0.02973 ] boco 0.05975 initial_condition 0.21638 Val [ 0.03238 ]
Epoch 3/50 Losses 0.20629 PDE [ 0.03367 ] boco 0.04238 initial_condition 0.13024 Val [ 0.02835 ]
Epoch 4/50 Losses 0.09900 PDE [ 0.03062 ] boco 0.02683 initial_condition 0.04155 Val [ 0.01884 ]
Epoch 5/50 Losses 0.05640 PDE [ 0.02106 ] boco 0.01974 initial_condition 0.01560 Val [ 0.01873 ]
Epoch 6/50 Losses 0.04034 PDE [ 0.01548 ] boco 0.01475 initial_condition 0.01011 Val [ 0.01387 ]
Epoch 7/50 Losses 0.03337 PDE [ 0.01387 ] boco 0.01130 initial_condition 0.00821 Val [ 0.01099 ]
Epoch 8/50 Losses 0.02583 PDE [ 0.01125 ] boco 0.00878 initial_condition 0.00581 Val [ 0.00961 ]
Epoch 9/50 Losses 0.02095 PDE [ 0.00977 ] boco 0.00685 initial_condition 0.00432 Val [ 0.00995 ]
Epoch 10/50 Losses 0.01479 PDE [ 0.00736 ] boco 0.00461 initial_condition 0.00283 Val [ 0.00606 ]
Epoch 11/50 Losses 0.01125 PDE [ 0.00604 ] boco 0.00315 initial_condition 0.00206 Val [ 0.00502 ]
Epoch 12/50 Losses 0.00825 PDE [ 0.00508 ] boco 0.00194 initial_condition 0.00123 Val [ 0.00403 ]
Epoch 13/50 Losses 0.00594 PDE [ 0.00393 ] boco 0.00117 initial_condition 0.00084 Val [ 0.00328 ]
Epoch 14/50 Losses 0.00449 PDE [ 0.00311 ] boco 0.00076 initial_condition 0.00062 Val [ 0.00375 ]
Epoch 15/50 Losses 0.00324 PDE [ 0.00236 ] boco 0.00048 initial_condition 0.00039 Val [ 0.00183 ]
Epoch 16/50 Losses 0.00247 PDE [ 0.00182 ] boco 0.00035 initial_condition 0.00030 Val [ 0.00156 ]
Epoch 17/50 Losses 0.00234 PDE [ 0.00175 ] boco 0.00036 initial_condition 0.00023 Val [ 0.00232 ]
Epoch 18/50 Losses 0.00216 PDE [ 0.00169 ] boco 0.00027 initial_condition 0.00020 Val [ 0.00220 ]
Epoch 19/50 Losses 0.00198 PDE [ 0.00165 ] boco 0.00017 initial_condition 0.00016 Val [ 0.00140 ]
Epoch 20/50 Losses 0.00165 PDE [ 0.00121 ] boco 0.00026 initial_condition 0.00018 Val [ 0.00218 ]
Epoch 21/50 Losses 0.00153 PDE [ 0.00119 ] boco 0.00021 initial_condition 0.00013 Val [ 0.00125 ]
Epoch 22/50 Losses 0.00151 PDE [ 0.00113 ] boco 0.00024 initial_condition 0.00014 Val [ 0.00104 ]
Epoch 23/50 Losses 0.00138 PDE [ 0.00107 ] boco 0.00020 initial_condition 0.00011 Val [ 0.00129 ]
Epoch 24/50 Losses 0.00133 PDE [ 0.00101 ] boco 0.00021 initial_condition 0.00012 Val [ 0.00086 ]
Epoch 25/50 Losses 0.00143 PDE [ 0.00093 ] boco 0.00033 initial_condition 0.00017 Val [ 0.00085 ]
Epoch 26/50 Losses 0.00124 PDE [ 0.00086 ] boco 0.00024 initial_condition 0.00014 Val [ 0.00090 ]
Epoch 27/50 Losses 0.00105 PDE [ 0.00087 ] boco 0.00011 initial_condition 0.00007 Val [ 0.00070 ]
Epoch 28/50 Losses 0.00101 PDE [ 0.00072 ] boco 0.00020 initial_condition 0.00010 Val [ 0.00072 ]
Epoch 29/50 Losses 0.00089 PDE [ 0.00068 ] boco 0.00014 initial_condition 0.00008 Val [ 0.00077 ]
Epoch 30/50 Losses 0.00078 PDE [ 0.00065 ] boco 0.00007 initial_condition 0.00005 Val [ 0.00065 ]
Epoch 31/50 Losses 0.00086 PDE [ 0.00064 ] boco 0.00014 initial_condition 0.00008 Val [ 0.00070 ]
Epoch 32/50 Losses 0.00090 PDE [ 0.00059 ] boco 0.00020 initial_condition 0.00011 Val [ 0.00073 ]
Epoch 33/50 Losses 0.00088 PDE [ 0.00054 ] boco 0.00023 initial_condition 0.00011 Val [ 0.00048 ]
Epoch 34/50 Losses 0.00063 PDE [ 0.00044 ] boco 0.00011 initial_condition 0.00007 Val [ 0.00043 ]
Epoch 35/50 Losses 0.00145 PDE [ 0.00049 ] boco 0.00068 initial_condition 0.00028 Val [ 0.00058 ]
Epoch 36/50 Losses 0.00074 PDE [ 0.00052 ] boco 0.00015 initial_condition 0.00007 Val [ 0.00070 ]
Epoch 37/50 Losses 0.00084 PDE [ 0.00065 ] boco 0.00012 initial_condition 0.00007 Val [ 0.00041 ]
Epoch 38/50 Losses 0.00050 PDE [ 0.00041 ] boco 0.00005 initial_condition 0.00004 Val [ 0.00044 ]
Epoch 39/50 Losses 0.00060 PDE [ 0.00041 ] boco 0.00013 initial_condition 0.00006 Val [ 0.00053 ]
Epoch 40/50 Losses 0.00071 PDE [ 0.00043 ] boco 0.00020 initial_condition 0.00008 Val [ 0.00049 ]
Epoch 41/50 Losses 0.00090 PDE [ 0.00068 ] boco 0.00015 initial_condition 0.00007 Val [ 0.00037 ]
Epoch 42/50 Losses 0.00057 PDE [ 0.00043 ] boco 0.00010 initial_condition 0.00005 Val [ 0.00049 ]
Epoch 43/50 Losses 0.00061 PDE [ 0.00038 ] boco 0.00016 initial_condition 0.00007 Val [ 0.00037 ]
Epoch 44/50 Losses 0.00110 PDE [ 0.00041 ] boco 0.00050 initial_condition 0.00019 Val [ 0.00034 ]
Epoch 45/50 Losses 0.00044 PDE [ 0.00037 ] boco 0.00004 initial_condition 0.00003 Val [ 0.00029 ]
Epoch 46/50 Losses 0.00050 PDE [ 0.00032 ] boco 0.00013 initial_condition 0.00006 Val [ 0.00040 ]
Epoch 47/50 Losses 0.00055 PDE [ 0.00034 ] boco 0.00014 initial_condition 0.00007 Val [ 0.00053 ]
Epoch 48/50 Losses 0.00068 PDE [ 0.00035 ] boco 0.00023 initial_condition 0.00009 Val [ 0.00031 ]
Epoch 49/50 Losses 0.00060 PDE [ 0.00041 ] boco 0.00014 initial_condition 0.00006 Val [ 0.00028 ]
Epoch 50/50 Losses 0.00063 PDE [ 0.00043 ] boco 0.00014 initial_condition 0.00006 Val [ 0.00047 ]
/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3257: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/opt/conda/lib/python3.7/site-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5))
ax1.plot(hist['train_loss'], label="train_loss")
ax1.plot(hist['val_loss'], label="val_loss")
ax1.grid(True)
ax1.set_yscale("log")
ax1.legend()
for boco in pde.bocos:
ax2.plot(hist['bocos'][boco.name], label=boco.name)
ax2.legend()
ax2.grid(True)
ax2.set_yscale("log")
plt.show()
# evaluate the solution
x = np.linspace(0,1,50)
t = np.linspace(0,1,100)
p, p0, l2 = [], [], []
for _t in t:
_p0 = np.sin(2.*math.pi*(x-u*_t))
pde.evaluate({'x': x, 't': np.array([_t])})
_p = pde.outputs['p']
_l2 = np.mean((_p - _p0)**2)
p.append(_p)
p0.append(_p0)
l2.append(_l2)
from matplotlib import animation, rc
rc('animation', html='html5')
def plot(x, p, p0, t, l2):
ax.clear()
tit = ax.set_title(f"t = {t:.2f}, l2 = {l2:.5f}", fontsize=14)
ax.plot(x, p0, "-k", label="Exact")
ax.plot(x, p, "g^", label="NN")
ax.set_xlabel("x", fontsize=14)
ax.set_ylabel("p", fontsize=14, rotation=np.pi/2)
ax.legend(loc="upper left")
ax.grid(True)
ax.set_xlim([0, 1])
ax.set_ylim([-1.2, 1.2])
return [tit]
def get_anim(fig, ax, x, p, p0, t, l2):
def anim(i):
return plot(x, p[i], p0[i], t[i], l2[i])
return anim
fig = plt.figure(figsize=(10,5))
ax = fig.add_subplot(111, autoscale_on=False)
animate = get_anim(fig, ax, x, p, p0, t, l2)
anim = animation.FuncAnimation(fig, animate, frames=len(t), interval=100, blit=True)
animCheck the examples to learn more about using nangs to solve PDEs with NNs.
Copyright 2020 onwards, SensioAI. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this project's files except in compliance with the License. A copy of the License is provided in the LICENSE file in this repository.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent