Performance Analysis of the Subgroup Method for Collective I/O
As many scientific applications require large data processing, the importance of parallel I/O has been increasingly recognized. Collective I/O is one of the considerable features of parallel I/O and enables application programmers to easily handle their large data volume. In this paper we measured and analyzed the performance of original collective I/O and the subgroup method, the way of using collective I/O of MPI effectively. From the experimental results, we found that the subgroup method showed good performance with small data size.
[1] John M. May, "Parallel I/O for High Performance Computing," Morgan
Kaufmann, 2000.
[2] MPICH-A Portable Implementation of MPI, http://www-unix.mcs.anl.
gov/mpi/mpich
[3] LAM/MPI Parallel Computing, http://www.lam-mpi.org
[4] Kwangho Cha, Taeyoung Hong, and Jeongwoo Hong, "The Subgroup
Method for Collective I/O," Proc. The 5th International Conference
on Parallel and Distributed Computing, Applications and Technologies
(PDCAT 2004), LNCS 3320, pp. 301Ôê╝304, Dec. 2004.
[5] Avery Ching, Alok Choudhary, Wei-keng Liao, Rob Ross, and William
Gropp, "Noncontiguous I/O through PVFS," Proc. IEEE International
Conference on Cluster Computing, pp 405Ôê╝414, 2002.
[6] Philip H. Carns, Walter B. Ligon, III, Robert B. Ross, and Rajeev
Thakur, "PVFS: A Parallel File System for Linux Clusters," Proc. 4th
Annual Linux Showcase and Conference, pp 317Ôê╝327, 2000.
[7] David Kotz, "Disk-directed I/O for MIMD multiprocessors," ACM
Transactions on Computer Systems, Vol. 15, No. 1, pp 41Ôê╝74, Feb.
1997.
[8] Rajesh Bordawekar, "Implementation of collective I/O in the Intel
Paragon parallel file system: initial experiences," Proc. 11th international
conference on Supercomputing, pp 20Ôê╝27, 1997.
[9] Rajeev Thakur, William Gropp, and Ewing Lusk, "Data sieving and
collective I/O in ROMIO," Proc. of the 7th Symposium on the Frontiers
of Massively Parallel Computation, pp 182Ôê╝189, 1999.
[10] William Gropp, Ewing Lusk, and Rajeev Thakur, "Using MPI-2: Advanced
Features of the Message Passing Interface," The MIT Press, 1999.
[11] Hakan Taki and Gil Utard, "MPI-IO on a parallel file system for
cluster of workstations," Proc. 1st IEEE Computer Society International
Workshop on Cluster Computing, pp 150Ôê╝157, 1999.
[12] ROMIO: A High-Performance, Portable MPI-IO Implementation,
http://www-unix.mcs.anl.gov/romio
[13] Jean-Pierre Prost, Richard Treumann, Robert Blackmore, Carol Hartan,
Richard Hedges, Bin Jia, Alice Koniges, and Alison White, "Towards a
High-Performance Implementation of MPI-IO on Top of GPFS," Proc.
The 6th International Euro-Par Conference, LNCS 1900, pp 1253Ôê╝1262,
Sep. 2000.
[14] Jean-Pierre Prost, Richard Treumann, Richard Hedges, Bin Jia, and
Alice Koniges, "MPI-IO/GPFS, an optimized implementation of MPI-IO
on top of GPFS," Proc. 2001 ACM/IEEE conference on Supercomputing(
CDROM), pp 17Ôê╝17, Nov. 2001.
[1] John M. May, "Parallel I/O for High Performance Computing," Morgan
Kaufmann, 2000.
[2] MPICH-A Portable Implementation of MPI, http://www-unix.mcs.anl.
gov/mpi/mpich
[3] LAM/MPI Parallel Computing, http://www.lam-mpi.org
[4] Kwangho Cha, Taeyoung Hong, and Jeongwoo Hong, "The Subgroup
Method for Collective I/O," Proc. The 5th International Conference
on Parallel and Distributed Computing, Applications and Technologies
(PDCAT 2004), LNCS 3320, pp. 301Ôê╝304, Dec. 2004.
[5] Avery Ching, Alok Choudhary, Wei-keng Liao, Rob Ross, and William
Gropp, "Noncontiguous I/O through PVFS," Proc. IEEE International
Conference on Cluster Computing, pp 405Ôê╝414, 2002.
[6] Philip H. Carns, Walter B. Ligon, III, Robert B. Ross, and Rajeev
Thakur, "PVFS: A Parallel File System for Linux Clusters," Proc. 4th
Annual Linux Showcase and Conference, pp 317Ôê╝327, 2000.
[7] David Kotz, "Disk-directed I/O for MIMD multiprocessors," ACM
Transactions on Computer Systems, Vol. 15, No. 1, pp 41Ôê╝74, Feb.
1997.
[8] Rajesh Bordawekar, "Implementation of collective I/O in the Intel
Paragon parallel file system: initial experiences," Proc. 11th international
conference on Supercomputing, pp 20Ôê╝27, 1997.
[9] Rajeev Thakur, William Gropp, and Ewing Lusk, "Data sieving and
collective I/O in ROMIO," Proc. of the 7th Symposium on the Frontiers
of Massively Parallel Computation, pp 182Ôê╝189, 1999.
[10] William Gropp, Ewing Lusk, and Rajeev Thakur, "Using MPI-2: Advanced
Features of the Message Passing Interface," The MIT Press, 1999.
[11] Hakan Taki and Gil Utard, "MPI-IO on a parallel file system for
cluster of workstations," Proc. 1st IEEE Computer Society International
Workshop on Cluster Computing, pp 150Ôê╝157, 1999.
[12] ROMIO: A High-Performance, Portable MPI-IO Implementation,
http://www-unix.mcs.anl.gov/romio
[13] Jean-Pierre Prost, Richard Treumann, Robert Blackmore, Carol Hartan,
Richard Hedges, Bin Jia, Alice Koniges, and Alison White, "Towards a
High-Performance Implementation of MPI-IO on Top of GPFS," Proc.
The 6th International Euro-Par Conference, LNCS 1900, pp 1253Ôê╝1262,
Sep. 2000.
[14] Jean-Pierre Prost, Richard Treumann, Richard Hedges, Bin Jia, and
Alice Koniges, "MPI-IO/GPFS, an optimized implementation of MPI-IO
on top of GPFS," Proc. 2001 ACM/IEEE conference on Supercomputing(
CDROM), pp 17Ôê╝17, Nov. 2001.
@article{"International Journal of Information, Control and Computer Sciences:50989", author = "Kwangho Cha and Hyeyoung Cho and Sungho Kim", title = "Performance Analysis of the Subgroup Method for Collective I/O", abstract = "As many scientific applications require large data processing, the importance of parallel I/O has been increasingly recognized. Collective I/O is one of the considerable features of parallel I/O and enables application programmers to easily handle their large data volume. In this paper we measured and analyzed the performance of original collective I/O and the subgroup method, the way of using collective I/O of MPI effectively. From the experimental results, we found that the subgroup method showed good performance with small data size.
", keywords = "Collective I/O, MPI, parallel file system.", volume = "1", number = "8", pages = "2332-4", }
