Setting | Documentation | Installation | Main functionality | Cite us | References | Ask a question

Repository for the Deterministic Particle Flow Control framework

• Link to published article: 📰 Phys. Rev. Research 4, 043035
• Link to poster: 📊 poster
• Link to documendation: 📜 readthedocs

[ This is work together with Manfred Opper funded by the SFB 1294. ]

[ PyPI package includes only numpy implementation, the github repository here provides both numpy and pytorch functionalities. ]

Obtaining optimal interventions for stochastic nonlinear systems is a computationally demanding process requiring the solution of nonlinear partial differential equations. Here we build on the Path Integral control formalism to derive a noniterative framework that represents the solutions of the underlying partial differential equations in terms of deterministic particle flows.

We consider a stochastic system described by a stochastic differential equation (SDE)

with drift f(x,t) and diffusion σ(x,t).

We want to constrain the dynamics of this system for a time interval

• either to reach some target state at time T (terminal constraint),
• and/or to visit/avoid specific regions of the state space (path constraints).

We implement the constraints in terms a time- and state-dependent perturbation of the deterministic part of the dynamics, i.e. we apply interventions u(x,t) and the controlled system dynamics become

.

Following the assumptions of the Path Integral control formalism, i.e. assuming that control costs are inversely proportional of noise variance (see paper for more details), we can show that the interventions u(x,t) can be obtained from the logarithmic gradient of the solution of a backward linear partial differential equation rescaled by the noise variance.

Here, instead of solving the backward PDE to obtain the optimal drift adjustment that implements the constraints, we express the optimal interventions as the difference of the logarithmic gradient of two probability flows, and . The probability flow or density satisfies the forward filtering equation, a forward PDE that in the absence of path constraints is the Fokker--Planck equation of the uncontrolled dynamics, while is the marginal constrained density that in turn satisfies the Fokker--Planck equation of the optimally controlled dynamics.

To install the install the software through Pypi run the following commands

# Create and activate a new virtualenvironment
python -m venv venv
source venv/bin/activate
pip install DeterministicParticleFlowControl

or to install from the downloaded repository run

# Create and activate a new virtualenvironment
python -m venv venv
source venv/bin/activate
# install the depndencies
pip install -r requirements.txt
pip install -e ./

To obtain the time dependent control functions create an instance of DPFC

from DeterministicParticleFlowControl import DPFC
control_flows = DPFC(t1,t2,y1,y2,f,g,N,M,reweight, U,dens_est,reject,kern='RBF',f_true=None,brown_bridge=False)

where

• t1, t2 : are define the timeinterval [t1,t2] within which the constraints are imposed onto the system,

• y1, y2 : are the initial and target state,

• f : is the drift function of the uncontrolled system,

• g : is the diffusion coefficient (can be scalar or matrix),

• N : the number of particles that will be employed for sampling the flows,

• M : inducing point number for the sparse kernel estimation of the logarithmic gradients,

• reweight: boolean indicating whether reweighting of forward trajectories is necessary, i.e. when:

  • path constraints are relevant for the problem,
  • the target is a non typical system state and the forward flow will be sampled as a reweighted Brownian bridge,

• U : function handle representing the path constraints, i.e. , for pushing the controlled trajectories towards the sin(t) line. Expects two arguments, so even if path constraint is non timedependent add an extra dummy variable in the argument list,

• dens_est : idicator determining the method to be employed for the logarithmic gradient (score) estimation. Currently supported:

  • nonparametric : for nonparametric sparse kernel estimation,

• reject : boolean variable determining whether rejection of backward trajectories failing to reach the initial condition will take place (this is mostly used as an indicator of numerical instabilities - if everything runs smoothly 1 or 2 trajectories need to be deleted, but often none of them if more trajectories get deleted, then rerun the computation with more particles N)

• kern : the kernel that will be employed for the nonparametric logarithmic gradient estimation of the sampled densities. Currently supported:

  • RBF : radial basis function kernel with lengthscale estimated at every time step from the samples,

• f_true : function handler of system drift function when the reweighted Brownian bridge functionality for forward sampling is used. In that case, f should be supplied with the drift function of the brownian bridge, U should be set to the necessary path constraint (see paper), and reweight and brown_bridge should be turned to True,

• brown_bridge: boolean variable indicating whether the sampling of the forward flow will happen with reweighted Brownian bridge dynamics.

This returns an object that contains the sampled flows and control functions. To compute the controls for the timestep ti when the system is at state x call

u = control_flows.calculate_u(x,ti)

The method is introduced in this paper. To cite it:

@article{maoutsa2022deterministic,
  title = {Deterministic particle flows for constraining stochastic nonlinear systems},
  author = {Maoutsa, Dimitra and Opper, Manfred},
  journal = {Phys. Rev. Research},
  volume = {4},
  issue = {4},
  pages = {043035},
  numpages = {17},
  year = {2022},
  month = {Oct},
  publisher = {American Physical Society},
  doi = {10.1103/PhysRevResearch.4.043035},
  url = {https://link.aps.org/doi/10.1103/PhysRevResearch.4.043035}
}

[1] Dimitra Maoutsa, Manfred Opper. "Deterministic particle flows for constraining stochastic nonlinear systems". 2021. [Physical review Research]

[2] Dimitra Maoutsa, Manfred Opper. "Deterministic particle flows for constraining SDEs". 2021. [Machine Learning and the Physical Sciences, Workshop at the 35th Conference on Neural Information Processing Systems (NeurIPS)]

[3] Dimitra Maoutsa, Sebastian Reich, Manfred Opper. "Interacting particle solutions of Fokker–Planck equations through gradient–log–density estimation". 2021. [Entropy]