High Performance Computing Using Out-of- Core Sparse Direct Solvers
In-core memory requirement is a bottleneck in solving
large three dimensional Navier-Stokes finite element problem
formulations using sparse direct solvers. Out-of-core solution
strategy is a viable alternative to reduce the in-core memory
requirements while solving large scale problems. This study
evaluates the performance of various out-of-core sequential solvers
based on multifrontal or supernodal techniques in the context of
finite element formulations for three dimensional problems on a
Windows platform. Here three different solvers, HSL_MA78,
MUMPS and PARDISO are compared. The performance of these
solvers is evaluated on a 64-bit machine with 16GB RAM for finite
element formulation of flow through a rectangular channel. It is
observed that using out-of-core PARDISO solver, relatively large
problems can be solved. The implementation of Newton and
modified Newton's iteration is also discussed.
[1] T. A. Davis and I.S. Duff, "A combined unifrontal/multifrontal method
for unsymmetric sparse matrices," ACM Trans. Math. Soft., vol. 25, no.
1, 1997, pp. 1-19.
[2] O. Schenk, K. Gartner, and W. Fichtner, "Efficient Sparse LU
Factorization with Left-right Looking Strategy on Shared Memory
Multiprocessors," BIT, vol. 40, no. 1, 2000, pp. 158-176.
[3] B. M. Irons, "A frontal solution scheme for finite element analysis,"
Numer. Meth. Engg., vol. 2, 1970, pp. 5-32.
[4] M. P. Raju, and J. S. T-ien, "Development of Direct Multifrontal Solvers
for Combustion Problems," Numerical Heat Transfer-Part B, vol. 53,
2008, pp. 1-17.
[5] M. P. Raju, and J. S. T-ien, "Modelling of Candle Wick Burning with a
Self-trimmed Wick," Comb. Theory Modell., vol. 12, no. 2, 2008, pp.
367-388.
[6] M. P. Raju, and J. S. T-ien, "Two-phase flow inside an externally heated
axisymmetric porous wick," vol. 11, no. 8, 2008, pp. 701-718.
[7] P. K. Gupta, and K. V. Pagalthivarthi, "Application of Multifrontal and
GMRES Solvers for Multisize Particulate Flow in Rotating Channels,"
Prog. Comput Fluid Dynam., vol. 7, 2007, pp. 323-336.
[8] S. Khaitan, J. McCalley, Q. Chen, "Multifrontal solver for online power
system time-domain simulation," IEEE Transactions on Power Systems,
vol. 23, no. 4, 2008, pp. 1727-1737.
[9] S. Khaitan, C. Fu, J. D. McCalley, "Fast parallelized algorithms for
online extended-term dynamic cascading analysis," PSCE, 2009, pp. 1-
7.
[10] J. McCalley, S. Khaitan, "Risk of Cascading outages", Final Report,
PSrec Report, S-26, August 2007. Available at
http://www.pserc.org/docsa/Executive_Summary_Dobson_McCalley_C
ascading_Outage_ S-2626_PSERC_ Final_Report.pdf
[11] P. R. Amestoy, and I. S. Duff, "Vectorization of a multiprocessor
multifrontal code," International Journal of Supercomputer
Applications, vol. 3, 1989, pp. 41-59.
[12] P. R. Amestoy, I. S. Duff, J. Koster and J. Y. L-Excellent, "A fully
asynchronous multifrontal solver using distributed dynamic scheduling,"
SIAM Journal on Matrix Analysis and Applications, vol. 23, no. 1, 2001,
pp. 15-41.
[13] P. R. Amestoy, I. S. Duff, and J. Y. L-Excellent, "Multifrontal parallel
distributed symmetric and unsymmetric solvers," Comput. Methods
Appl. Mech. Eng., vol. 184, 2000, pp. 501-520.
[14] O. Schenk, "Scalable Parallel Sparse LU Factorization Methods on
Shared Memory Multiprocessors," Ph.D. dissertation, ETH Zurich,
2000.
[15] O. Schenk, and K. Gartner, "Sparse Factorization with Two-Level
Scheduling in PARDISO," in Proc. 10th SIAM conf. Parallel Processing
for Scientific Computing, Portsmouth, Virginia, March 12-14, 2001.
[16] O. Schenk, and K. Gartner, "Two-level scheduling in PARDISO:
Improved Scalability on Shared Memory Multiprocessing Systems,"
Parallel Computing, vol. 28, 2002, pp. 187-197.
[17] O. Schenk, and K. Gartner, "Solving Unsymmetric Sparse Systems of
Linear Equations with PARDISO," Journal Future Generation
Computer Systems, vol. 20, no. 3, 2004, pp. 475-487.
[18] Intel MKL Reference Manual, Intel® Math Kernel Library (MKL), 2007.
Available: http://www.intel.com/software/products/mkl/
[19] J. A. Scott, Numerical Analysis Group Progress Report, RAL-TR-2008-
001, Rutherford Appleton Laboratory, 2008.
[20] P. R. Amestoy, T. A. Davis, and I. S. Duff, "An approximate minimum
degree ordering algorithm," SIAM Journal on Matrix Analysis and
Applications, vol. 17, 1996, pp. 886-905.
[21] P. R. Amestoy, "Recent progress in parallel multifrontal solvers for
unsymmetric sparse matrices," in Proc. 15th World Congress on
Scientific Computation, Modelling and Applied Mathematics, IMACS,
Berlin, 1997.
[22] J. Schulze, "Towards a tighter coupling of bottom-up and top-down
sparse matrix ordering methods," BIT, vol. 41, no. 4, 2001, pp. 800-841.
[23] G. Karypis, and V. Kumar, "METIS - A Software Package for
Partitioning Unstructured Graphs, Partitioning Meshes, and Computing
Fill-Reducing Orderings of Sparse Matrices - Version 4.0," University
of Minnesota, September 1998.
[1] T. A. Davis and I.S. Duff, "A combined unifrontal/multifrontal method
for unsymmetric sparse matrices," ACM Trans. Math. Soft., vol. 25, no.
1, 1997, pp. 1-19.
[2] O. Schenk, K. Gartner, and W. Fichtner, "Efficient Sparse LU
Factorization with Left-right Looking Strategy on Shared Memory
Multiprocessors," BIT, vol. 40, no. 1, 2000, pp. 158-176.
[3] B. M. Irons, "A frontal solution scheme for finite element analysis,"
Numer. Meth. Engg., vol. 2, 1970, pp. 5-32.
[4] M. P. Raju, and J. S. T-ien, "Development of Direct Multifrontal Solvers
for Combustion Problems," Numerical Heat Transfer-Part B, vol. 53,
2008, pp. 1-17.
[5] M. P. Raju, and J. S. T-ien, "Modelling of Candle Wick Burning with a
Self-trimmed Wick," Comb. Theory Modell., vol. 12, no. 2, 2008, pp.
367-388.
[6] M. P. Raju, and J. S. T-ien, "Two-phase flow inside an externally heated
axisymmetric porous wick," vol. 11, no. 8, 2008, pp. 701-718.
[7] P. K. Gupta, and K. V. Pagalthivarthi, "Application of Multifrontal and
GMRES Solvers for Multisize Particulate Flow in Rotating Channels,"
Prog. Comput Fluid Dynam., vol. 7, 2007, pp. 323-336.
[8] S. Khaitan, J. McCalley, Q. Chen, "Multifrontal solver for online power
system time-domain simulation," IEEE Transactions on Power Systems,
vol. 23, no. 4, 2008, pp. 1727-1737.
[9] S. Khaitan, C. Fu, J. D. McCalley, "Fast parallelized algorithms for
online extended-term dynamic cascading analysis," PSCE, 2009, pp. 1-
7.
[10] J. McCalley, S. Khaitan, "Risk of Cascading outages", Final Report,
PSrec Report, S-26, August 2007. Available at
http://www.pserc.org/docsa/Executive_Summary_Dobson_McCalley_C
ascading_Outage_ S-2626_PSERC_ Final_Report.pdf
[11] P. R. Amestoy, and I. S. Duff, "Vectorization of a multiprocessor
multifrontal code," International Journal of Supercomputer
Applications, vol. 3, 1989, pp. 41-59.
[12] P. R. Amestoy, I. S. Duff, J. Koster and J. Y. L-Excellent, "A fully
asynchronous multifrontal solver using distributed dynamic scheduling,"
SIAM Journal on Matrix Analysis and Applications, vol. 23, no. 1, 2001,
pp. 15-41.
[13] P. R. Amestoy, I. S. Duff, and J. Y. L-Excellent, "Multifrontal parallel
distributed symmetric and unsymmetric solvers," Comput. Methods
Appl. Mech. Eng., vol. 184, 2000, pp. 501-520.
[14] O. Schenk, "Scalable Parallel Sparse LU Factorization Methods on
Shared Memory Multiprocessors," Ph.D. dissertation, ETH Zurich,
2000.
[15] O. Schenk, and K. Gartner, "Sparse Factorization with Two-Level
Scheduling in PARDISO," in Proc. 10th SIAM conf. Parallel Processing
for Scientific Computing, Portsmouth, Virginia, March 12-14, 2001.
[16] O. Schenk, and K. Gartner, "Two-level scheduling in PARDISO:
Improved Scalability on Shared Memory Multiprocessing Systems,"
Parallel Computing, vol. 28, 2002, pp. 187-197.
[17] O. Schenk, and K. Gartner, "Solving Unsymmetric Sparse Systems of
Linear Equations with PARDISO," Journal Future Generation
Computer Systems, vol. 20, no. 3, 2004, pp. 475-487.
[18] Intel MKL Reference Manual, Intel® Math Kernel Library (MKL), 2007.
Available: http://www.intel.com/software/products/mkl/
[19] J. A. Scott, Numerical Analysis Group Progress Report, RAL-TR-2008-
001, Rutherford Appleton Laboratory, 2008.
[20] P. R. Amestoy, T. A. Davis, and I. S. Duff, "An approximate minimum
degree ordering algorithm," SIAM Journal on Matrix Analysis and
Applications, vol. 17, 1996, pp. 886-905.
[21] P. R. Amestoy, "Recent progress in parallel multifrontal solvers for
unsymmetric sparse matrices," in Proc. 15th World Congress on
Scientific Computation, Modelling and Applied Mathematics, IMACS,
Berlin, 1997.
[22] J. Schulze, "Towards a tighter coupling of bottom-up and top-down
sparse matrix ordering methods," BIT, vol. 41, no. 4, 2001, pp. 800-841.
[23] G. Karypis, and V. Kumar, "METIS - A Software Package for
Partitioning Unstructured Graphs, Partitioning Meshes, and Computing
Fill-Reducing Orderings of Sparse Matrices - Version 4.0," University
of Minnesota, September 1998.
@article{"International Journal of Engineering, Mathematical and Physical Sciences:54016", author = "Mandhapati P. Raju and Siddhartha Khaitan", title = "High Performance Computing Using Out-of- Core Sparse Direct Solvers", abstract = "In-core memory requirement is a bottleneck in solving
large three dimensional Navier-Stokes finite element problem
formulations using sparse direct solvers. Out-of-core solution
strategy is a viable alternative to reduce the in-core memory
requirements while solving large scale problems. This study
evaluates the performance of various out-of-core sequential solvers
based on multifrontal or supernodal techniques in the context of
finite element formulations for three dimensional problems on a
Windows platform. Here three different solvers, HSL_MA78,
MUMPS and PARDISO are compared. The performance of these
solvers is evaluated on a 64-bit machine with 16GB RAM for finite
element formulation of flow through a rectangular channel. It is
observed that using out-of-core PARDISO solver, relatively large
problems can be solved. The implementation of Newton and
modified Newton's iteration is also discussed.", keywords = "Out-of-core, PARDISO, MUMPS, Newton.", volume = "3", number = "9", pages = "624-7", }
