This paper proposes a new approach to offer a private
cloud service in HPC clusters. In particular, our approach relies on
automatically scheduling users- customized environment request as a
normal job in batch system. After finishing virtualization request jobs,
those guest operating systems will dismiss so that compute nodes will
be released again for computing. We present initial work on the
innovative integration of HPC batch system and virtualization tools
that aims at coexistence such that they suffice for meeting the
minimizing interference required by a traditional HPC cluster. Given
the design of initial infrastructure, the proposed effort has the potential
to positively impact on synergy model. The results from the
experiment concluded that goal for provisioning customized cluster
environment indeed can be fulfilled by using virtual machines, and
efficiency can be improved with proper setup and arrangements.
[1] Amazon Elastic Compute Cloud (EC2),
http://aws.amazon.com/documentation/ec2/.
[2] G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B. P. Berman, P.
Maechling, "Scientfic Workflow Applications on Amazon EC2," in Procs.
5th IEEE International Conference on e-Science. 2009. pp. 59-66.
[3] HPC as a Service,
http://www.penguincomputing.com/POD_old/Benefits/.
[4] Neil Smyth, Red Hat Enterprise Linux 6 Essentials, 2010, ch. 13.
[5] Kernel-based Virtual Machine,
http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine/.
[6] Torque Resource Manager,
http://www.clusterresources.com/pages/products/torque-resource-manag
er.php/.
[7] H. Kim, Y. el-Khamra, S. Jha, M. Parashar, "An Autonomic Approach to
Integrated HPC Grid and Cloud Usage," the 5th IEEE International
Conference on e-Science, Oxford, UK, pp. 366-373, Dec. 2009.
[8] B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster, "Capacity
Leasing in Cloud System using the OpenNebula Engine," Cloud
Computing and Applications, 2008. Chicago, Illinois, USA.
[9] P. Sempolinski and D. Thain, "A Comparison and Critique of Eucalyptus,
OpenNebula and Nimbus," in Proc. CloudCom-2010, pp.417-426.
[10] V. Buge, H. Hessling, Y. Kemp, M. Kunze,O. Oberst, G. Quast, A.
Scheurer, and O. Synge, "Integration of Virtualized Worker Nodes in
Standard Batch Systems," in Journal of Physics: Conference Series,
Volume 219, Issue 5, pp. 052010, 2010.
[11] D. H. van Dok, "Pushing Torque jobs in a chroot environment" Available
at:
http://www.nikhef.nl/pub/projects/grid/gridwiki/images/3/37/Momchroo
t.pdf.
[12] W. Emeneker, D. Jackson, J. Butikofer, and D. Stanzione, "Dynamic
Virtual Clustering with Xen and Moab," in Proc. ISPA Workshops-2006,
pp.440-451.
[13] R. Rose, "Survey of System Virtualization Techniques," Available at:
http://www. robertwrose.com/vita/rose-virtualization.pdf, 2004.
[14] T. Garfinkel and M. Rosenblum, "When Virtual is Harder than Real:
Security Challenges in Virtual Machine Based Computing
Environments," in Proc. 10th Workshop on Hot Topics in Operating
Systems (HOTOS-X).
[15] J. Cope, M. Oberg, H. M. Tufo, and M. Woitaszek, "Shared Parallel
Filesystems in Heterogeneous Linux Multi-Cluster Environments," in
Proc. 6th LCI International Conference on Linux Clusters: The HPC
Revolution.
[16] T. Deshane, Z. Shepherd, J. N. Matthews, M. Ben-Yehuda, A. Shan, and
B. Rao, "Quantitative Comparison of Xen and KVM," Xen Summit,
Boston, MA, USA, pp. 1-2, 2008.
[17] L. Nussbaum, O. Mornard, F. Anhalt, J. P. Gelas, "Linux-based
virtualization for HPC clusters," Available at:
http://www.loria.fr/~lnussbau/files/linux-virtualization-mls09.pdf
[18] O. Khalid, I. Maljevic and R. Anthony, "Dynamic Scheduling of Virtual
Machines Running HPC Workloads in Scientific Grids," in Proc. 3rd
IEEE International Conference of New Technologies, Mobility and
Security, 2009.
[19] Jeffrey Shafer, "I/O Virtualization Bottlenecks in Cloud Computing
Today," Workshop on I/O Virtualization (WIOV 2010), Pittsburgh, PA,
March 2010.
[20] G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B. P. Berman, P.
Maechling, "Data Sharing Options for Scientific Workflows on Amazon
EC2," SC-10 Proceedings of the 2010 ACM/IEEE International
Conference for High Performance Computing, Networking, Storage and
Analysis., submitted for publication.
[1] Amazon Elastic Compute Cloud (EC2),
http://aws.amazon.com/documentation/ec2/.
[2] G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B. P. Berman, P.
Maechling, "Scientfic Workflow Applications on Amazon EC2," in Procs.
5th IEEE International Conference on e-Science. 2009. pp. 59-66.
[3] HPC as a Service,
http://www.penguincomputing.com/POD_old/Benefits/.
[4] Neil Smyth, Red Hat Enterprise Linux 6 Essentials, 2010, ch. 13.
[5] Kernel-based Virtual Machine,
http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine/.
[6] Torque Resource Manager,
http://www.clusterresources.com/pages/products/torque-resource-manag
er.php/.
[7] H. Kim, Y. el-Khamra, S. Jha, M. Parashar, "An Autonomic Approach to
Integrated HPC Grid and Cloud Usage," the 5th IEEE International
Conference on e-Science, Oxford, UK, pp. 366-373, Dec. 2009.
[8] B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster, "Capacity
Leasing in Cloud System using the OpenNebula Engine," Cloud
Computing and Applications, 2008. Chicago, Illinois, USA.
[9] P. Sempolinski and D. Thain, "A Comparison and Critique of Eucalyptus,
OpenNebula and Nimbus," in Proc. CloudCom-2010, pp.417-426.
[10] V. Buge, H. Hessling, Y. Kemp, M. Kunze,O. Oberst, G. Quast, A.
Scheurer, and O. Synge, "Integration of Virtualized Worker Nodes in
Standard Batch Systems," in Journal of Physics: Conference Series,
Volume 219, Issue 5, pp. 052010, 2010.
[11] D. H. van Dok, "Pushing Torque jobs in a chroot environment" Available
at:
http://www.nikhef.nl/pub/projects/grid/gridwiki/images/3/37/Momchroo
t.pdf.
[12] W. Emeneker, D. Jackson, J. Butikofer, and D. Stanzione, "Dynamic
Virtual Clustering with Xen and Moab," in Proc. ISPA Workshops-2006,
pp.440-451.
[13] R. Rose, "Survey of System Virtualization Techniques," Available at:
http://www. robertwrose.com/vita/rose-virtualization.pdf, 2004.
[14] T. Garfinkel and M. Rosenblum, "When Virtual is Harder than Real:
Security Challenges in Virtual Machine Based Computing
Environments," in Proc. 10th Workshop on Hot Topics in Operating
Systems (HOTOS-X).
[15] J. Cope, M. Oberg, H. M. Tufo, and M. Woitaszek, "Shared Parallel
Filesystems in Heterogeneous Linux Multi-Cluster Environments," in
Proc. 6th LCI International Conference on Linux Clusters: The HPC
Revolution.
[16] T. Deshane, Z. Shepherd, J. N. Matthews, M. Ben-Yehuda, A. Shan, and
B. Rao, "Quantitative Comparison of Xen and KVM," Xen Summit,
Boston, MA, USA, pp. 1-2, 2008.
[17] L. Nussbaum, O. Mornard, F. Anhalt, J. P. Gelas, "Linux-based
virtualization for HPC clusters," Available at:
http://www.loria.fr/~lnussbau/files/linux-virtualization-mls09.pdf
[18] O. Khalid, I. Maljevic and R. Anthony, "Dynamic Scheduling of Virtual
Machines Running HPC Workloads in Scientific Grids," in Proc. 3rd
IEEE International Conference of New Technologies, Mobility and
Security, 2009.
[19] Jeffrey Shafer, "I/O Virtualization Bottlenecks in Cloud Computing
Today," Workshop on I/O Virtualization (WIOV 2010), Pittsburgh, PA,
March 2010.
[20] G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B. P. Berman, P.
Maechling, "Data Sharing Options for Scientific Workflows on Amazon
EC2," SC-10 Proceedings of the 2010 ACM/IEEE International
Conference for High Performance Computing, Networking, Storage and
Analysis., submitted for publication.
@article{"International Journal of Information, Control and Computer Sciences:49957", author = "Chin-Hung Li and Te-Ming Chen and Ying-Chuan Chen and Shuen-Tai Wang", title = "Formosa3: A Cloud-Enabled HPC Cluster in NCHC", abstract = "This paper proposes a new approach to offer a private
cloud service in HPC clusters. In particular, our approach relies on
automatically scheduling users- customized environment request as a
normal job in batch system. After finishing virtualization request jobs,
those guest operating systems will dismiss so that compute nodes will
be released again for computing. We present initial work on the
innovative integration of HPC batch system and virtualization tools
that aims at coexistence such that they suffice for meeting the
minimizing interference required by a traditional HPC cluster. Given
the design of initial infrastructure, the proposed effort has the potential
to positively impact on synergy model. The results from the
experiment concluded that goal for provisioning customized cluster
environment indeed can be fulfilled by using virtual machines, and
efficiency can be improved with proper setup and arrangements.", keywords = "Cloud Computing, HPC Cluster, Private Cloud,Virtualization", volume = "5", number = "6", pages = "558-7", }
