sslattery / chimera Goto Github PK

MCLS has implemented the Trilinos Thyra interfaces. Denovo leverages Stratimikos for parameter list driven generation of linear solvers that implement these interfaces at runtime. MCLS needs to be added to the Denovo build and solver classes for use with the SPn equations.

Definition of done: build Denovo with MCLS as an optional dependency and run an SPn problem using an MCLS solver.

An outline of the prelim document must be created to define the scope of the prelim. This will include an outline of content, desired results, and document structure.

An official date and time must be set with the thesis committee for the prelim exam as it will require all participants to be available at the same time and in the same location. I have suggested an initial date of 11/14.

The committee members are:

• Paul Wilson
• Greg Moses
• Carl Sovinec
• Chris Rutland
• Tom Evans

If I'm going to use Phalanx to drive general nonlinear function evaluations for residual and Jacobian generation, much of the boiler plate code to do this exists in Panzer, a Trilinos package that exists in the pre-copyright form. Based on discussions with Roger, this will be in open source Trilinos very soon. I have requested access to the repository, which is doable because I have Sandia repo access.

A latex structure for the prelim document needs to be selected and implemented in the repository. Past CNERG students have likely used various outlines. I'll look around to see what they've used.

This issue is for creating slides covering nonlinear methods and FANM.

I have developed what I'm calling the "Reduced Domain Approximation" as a mechanism for overcoming the dense composite matrices that result from effective preconditioning. This should hopefully speed up calculations and enable domain decomposed parallelism. The basic principles are:

1. Build a composite operator with a quality preconditioner for the problem (i.e. ILUT).
2. The composite operator will generally have many non-zero entries within floating point tolerance of zero - these are filitered with a tolerance.
3. A very dense operator with 1000's of entries per row remains - keep the largest entries by absolute value by using a fill value parameter.
4. Add the specified fraction of the weight lost during filtering from tolerance and fill level.

The net effect here is a reducing the size of the composite operator locally and therefore the size of the Monte Carlo domain while maintaining convergence. As we modify each row of the system this way, we reduce the number of entries and the overall weight of transitions in that row. Reducing this weight causes faster convergence. We can recover this weight to maintain the same weight per transition in that row as before the filtering and decrease the number of iterations required to converge and improve robustness at the cost of longer Markov chains due to the higher weights.

From an acceleration perspective this also makes sense as we are effectively using a reduced order model of the linear system in the Monte Carlo residual solve, giving an approximation for the system that still yields an approximate correction for convergence. Although the above is purely algebraic, it has links to reduced order physics modeling where a simpler representation of the model problem is solved to improve the solution of a more detailed and difficult to solve problem.

The purpose of this task is to assess the performance of the reduced domain approximation with respect to the parameters of filter tolerance, fill level, and weight recovery fraction.

This issue is for creating slides covering the research plan.

A sequence of parallel performance studies are required for testing the domain decomposed Neumann-Ulam method.

The Chimera code base will be the centerpiece of the instrumentation for the thesis. Its design needs to be developed considering the goals of the thesis work and the questions that we will want to explore. A design includes clear definition of all packages, TPLs, and functionality.

FANM analysis of Gartling's flow over a backward-facing step problem.

The convergence behavior of the SPN equations for Reactor problems has been observed to be poor for the block Jacobi preconditioning strategy studied. Given real reactor problems, the goal of this task is to find a preconditioning strategy that yields fewer iterations and therefore shorter Markov chains.

To explore Panzer it would be useful to recreate my previous transient poisson results that used MCSA and a finite difference formulation. This new formulation will choose a weak form for the equations and time integration driven by Rythmos

Okten's expected value estimator for his 2005 paper has been added to MCLS in addition to Gelbard's collision estimator which has been used to this point. Based on the literature and my parallel implementation, I expect the new estimator to significantly outperform the collision estimator. This new estimator does not solve the problem of long stochastic histories, but it should significantly reduce the number of histories need for convergence; with or without MCSA.

The purpose of this task is to determine whether or not the above statements about the new estimator are true within the context of the SPN challenge problems I am working on. This is done when I have run a series of problems that compares iteration and timing performance for MCSA with the two estimators.

This task is complete when a draft of the introduction chapter is complete.

I have read dozens of articles and books regarding the thesis work. This needs to be organized to better manage citations in the thesis work and to improve my background knowledge.

A spectral analysis of the SPn equations for multi-group, higher-order expansions are required. This spectra will give us an idea of convergence, preconditioning requirements, and permit a hypothesis for performance vs. subspace methods.

Monte Carlo methods rely on random number generation for their implementation. Current solver implementations leverage the C++ STL generator. This should be abstracted to a general PRNG concept so that we may swap out implementations for performance comparisons. Initially, we'll also incorporate SPRNG as an implementation of this abstraction.

FANM analysis of Ghia's lid driven cavity problem.

The primary nonlinear problem of study will be the incompressible Navier-Stokes equations. Now that the Poisson equation has been implemented in Panzer, Chimera is ready for a more complicated equation set.

ParaSails has been implemented with MCLS for preconditioning. ParaSails is nice for many reasons:

1. It builds the inverse directly and that inverse has a sparsity pattern controlled by the input operator - giving a sparse composite operator.
2. There are strong controls over the sparsity of the inverse.
3. It has demonstrated good scaling and operating on convection-diffusion problems with hundreds of millions of unknowns.
4. It gives a better preconditioning than Jacobi.
5. Its pretty fast.
6. It is a production implementation from Livermore.

I'm going to do a full analysis of the new preconditioner with the 2 Monte Carlo estimators and compare the performance to my other preconditioners.

This is my last preconditioner. Based on the literature this performed the best for build sparse approximate inverses and I can leverage a production implementation. If this doesn't work out then more research is really required for preconditioning these stochastic problems which is beyond the scope of this work.

FANM analysis of De Vhal Davis' natural convection in a square cavity benchmark.

This issue is for creating the slides for the introduction portion of the preliminary exam talk.

This issue is for creating slides covering Monte Carlo methods for linear problems.

A series of parallel performance experiments are required to explore the domain replicated Neumann-Ulam method.

For parallel performance experiments over O(100) cores, the Titan system at ORNL is the only current means of doing this. We therefore need a build of MCLS/Denovo on Titan to run MCLS-based SPn problems.

This task is complete when a research proposal chapter has been written for the prelim.

The initial Monte Carlo implementations leveraged the Epetra library in Trilinos. At a minimum I would like the new implementation to leverage Tpetra. If possible however, I would prefer an implementation that is agnostic towards either. I'll look to design an operator and vector interface that captures both. This also means that the solver package we generate can be leveraged by those outside of Trilinos (although we will offer Trilinos specific implementations).

This problem will serve as the first benchmark system for MCSA. MCSA will be compared in parallel next to conventional solver and preconditioning methods in terms of iterative performance and scaling.

Regardless of computational platform, Drekar needs to be accessed for easy implementation of the FANM method and easy implementation of the 3 CFD benchmark problems that will be used to test the FANM method. It is not required that this software be available on UW systems but simply that it is accessible somewhere to run problems.

A preliminary FANM calculation with the Navier-Stokes equations needs to be completed to ensure that is actually possible. Any problem will do and I will likely choose the therm convection cavity problem already available.

This task is done when a calculation has been completed with a nonlinear Navier-Stokes solve using a Newton method with automated Jacobian generation and MCSA as the linear solver.

Per #25 block Jacobi preconditioning is the way to go for higher order SPN equations. This will now need to be implemented. First, we must choose either a Denovo or MCLS implementation followed by the actual implementation itself.

This task is done when the implementation is unit tested and utilized for a simple higher order SPN problem.

The current Chimera solvers using Epetra. Based on #14, the decision was made to use Tpetra. The solvers should be rewritten using Tpetra. In addition, based on the parallel Monte Carlo literature review from the prelim, this rewrite should facilitate implementations of those methods.

Problem 3 has been analyzed for the MCSA method with various forms of preconditioning. This problem will serve as the first benchmark system for MCSA. MCSA will be compared in parallel next to conventional solver and preconditioning methods in terms of iterative performance and scaling.

The issue is for creating slides that cover the conventional linear problem solution methods.