Nowadays, HPC, Grid and Cloud systems are evolving
very rapidly. However, the development of infrastructure solutions
related to HPC is lagging behind. While the existing infrastructure is
sufficient for simple cases, many computational problems have more
complex requirements.Such computational experiments use different
resources simultaneously to start a large number of computational
jobs.These resources are heterogeneous. They have different
purposes, architectures, performance and used software.Users need a
convenient tool that allows to describe and to run complex
computational experiments under conditions of HPC environment.
This paper introduces a modularworkflow system called SEGL
which makes it possible to run complex computational experiments
under conditions of a real HPC organization. The system can be used
in a great number of organizations, which provide HPC power.
Significant requirements to this system are high efficiency and
interoperability with the existing HPC infrastructure of the
organization without any changes.
[1] TOP 500, http://www.top500.org
[2] C. Byun, C. Duncan, andS. Burks, "A Comparison of Job Management
Systems in Supporting HPC ClusterTools", Presentation for SUPerG,
Vancouver, Canada, 2000.
[3] OpenSSH, http://www.openssh.com/
[4] I. Taylor, E. Deelman, D. Gannon, andM. Shields, Workflows for e-
Science. Springer Press, 2007.
[5] N. Currle-Linde, F. Boes, P. Adamidis, and M. Resch, "GriCoL: A
Language for Scientific Grids", In Proceedings of the 2nd IEEE
International Conference on e-Science and Grid Computing (ESCIENCE
'06), Amsterdam, Netherlands, 2006.
[6] M. Sonntag, N. Currle-Linde, K. Goerlach, and D. Karastoyanova,
"Towards Simulation Workflows With BPEL: Deriving Missing
Features from GriCoL", In Proceedings of the 21st IASTED
International Conference on Modelling and Simulation,Banff, Alberta,
Canada, 2010.
[7] I. Foster, I. Kesselman, andS. Tuecke, "The Anatomy of the Grid:
Enabling Scalable Virtual Organizations",Int. J. High Perform. Comput.
Appl. 15 3, 2001.
[8] R. Baker, D. Yu, andT. Wlodek, A Model for Grid User Management,
Computing in High Energy and Nuclear Physics. La Jolla, California,
USA, 2003.
[9] T. Antoni, W. B├╝hler, H. Dres, G. Grein, andM. Roth, "Global grid user
support - building a worldwide distributed user support infrastructure",
Journal of Physics: Conference Series, 2008.
[10] E. Deelman, D. Gannon, M. Shields, and I. Taylor, "Workflows and e-
Science: An overview of workflow system features and
capabilities",Future Generation Computer Systems, Volume: 25, Issue:
5 (2009), 528-540.
[11] C. Goble and D. De Roure, "The impact of workflow tools on datacentric
research", In Data Intensive Computing: The Fourth Paradigm
of Scientific Discovery, T. Hey, S. Tansley, and K. Tolle, Ed. Microsoft
Research, 137-145, 2009
[12] S. Callaghan, E. Deelman, D. Guntere, G. Juve, P. Maechling, C.
Brooks, K. Vahi, K. Milner, R. Gravesc, E. Field, D. Okaya, and T.
Jordan,"Scaling up workflow-based applications",Journal of Computer
and System Sciences, 428-446, 2010
[13] V. S. Kumar, P. Sadayappan, G. Mehta, K. Vahi, E. Deelman, V.
Ratnakar, J. Kim, Y. Gil, M. W. Hall, T. M. Kurc, and J. H. Saltz,"An
integrated framework for performance-based optimization of scientific
workflows", In Proceedings of the 18th ACM international symposium
on High performance distributed computing (HPDC '09) (Munich,
Germany, June 11-13, 2009), 177-186.
[14] T. Fahringer, R. Prodan, R. Duan, F. Nerieri, S. Podlipnig, J. Qin, M.
Siddiqui, H.-L. Truong, A. Villazon, and M. Wieczorek, "ASKALON:
A Grid Application Development and Computing Environment", In 6th
International Workshop on Grid Computing (New York 2005), 122-131.
[15] M. Wieczorek, R. Prodan, and T. Fahringer,Scheduling of Scientific
Workflows in the ASKALON Grid Environment.ACM SIGMOD Record,
2005.
[16] Y. Dorozhko, T. Krasikova, Y. Yudin, N. Currle-Linde, andM. Resch,
"An Abstract Language and Environment for the Creation and Execution
of Experiments over Distributed Computers", InProceedings. of the
International Scientific Conference Simulation-2010, Kiev, Ukraine,
2010.
[17] H. Bouziane, N. Currle-Linde, C. Perez, andM. Resch, "Analysis f
Component Model Extensions to support the GriCoL Language", In
Making grids Work, pp 45-59, Springer, 2008.
[18] GLUE Specification v. 2.0, http://www.ogf.org/documents/GFD.147.pdf
[19] Job Submission Description Language (JSDL) Specification, Version
1.0, http://www.gridforum.org/documents/GFD.56.pdf
[20] B. Armstrong, H. Bae, R. Eigenmann, F. Saied, M. Sayeed, andY.
Zheng, "HPC Benchmarking and Performance valuation with Realistic
Applications", 2006 SPEC Benchmark Workshop (spec), 2006.
[21] Java WebStart Overview, http://www.oracle.com/technetwork/java/
javase/overview-137531.html
[22] Lustre Cluster FS, http://www.lustre.org/
[23] sudo command, http://www.gratisoft.us/sudo/sudo.man.html
[1] TOP 500, http://www.top500.org
[2] C. Byun, C. Duncan, andS. Burks, "A Comparison of Job Management
Systems in Supporting HPC ClusterTools", Presentation for SUPerG,
Vancouver, Canada, 2000.
[3] OpenSSH, http://www.openssh.com/
[4] I. Taylor, E. Deelman, D. Gannon, andM. Shields, Workflows for e-
Science. Springer Press, 2007.
[5] N. Currle-Linde, F. Boes, P. Adamidis, and M. Resch, "GriCoL: A
Language for Scientific Grids", In Proceedings of the 2nd IEEE
International Conference on e-Science and Grid Computing (ESCIENCE
'06), Amsterdam, Netherlands, 2006.
[6] M. Sonntag, N. Currle-Linde, K. Goerlach, and D. Karastoyanova,
"Towards Simulation Workflows With BPEL: Deriving Missing
Features from GriCoL", In Proceedings of the 21st IASTED
International Conference on Modelling and Simulation,Banff, Alberta,
Canada, 2010.
[7] I. Foster, I. Kesselman, andS. Tuecke, "The Anatomy of the Grid:
Enabling Scalable Virtual Organizations",Int. J. High Perform. Comput.
Appl. 15 3, 2001.
[8] R. Baker, D. Yu, andT. Wlodek, A Model for Grid User Management,
Computing in High Energy and Nuclear Physics. La Jolla, California,
USA, 2003.
[9] T. Antoni, W. B├╝hler, H. Dres, G. Grein, andM. Roth, "Global grid user
support - building a worldwide distributed user support infrastructure",
Journal of Physics: Conference Series, 2008.
[10] E. Deelman, D. Gannon, M. Shields, and I. Taylor, "Workflows and e-
Science: An overview of workflow system features and
capabilities",Future Generation Computer Systems, Volume: 25, Issue:
5 (2009), 528-540.
[11] C. Goble and D. De Roure, "The impact of workflow tools on datacentric
research", In Data Intensive Computing: The Fourth Paradigm
of Scientific Discovery, T. Hey, S. Tansley, and K. Tolle, Ed. Microsoft
Research, 137-145, 2009
[12] S. Callaghan, E. Deelman, D. Guntere, G. Juve, P. Maechling, C.
Brooks, K. Vahi, K. Milner, R. Gravesc, E. Field, D. Okaya, and T.
Jordan,"Scaling up workflow-based applications",Journal of Computer
and System Sciences, 428-446, 2010
[13] V. S. Kumar, P. Sadayappan, G. Mehta, K. Vahi, E. Deelman, V.
Ratnakar, J. Kim, Y. Gil, M. W. Hall, T. M. Kurc, and J. H. Saltz,"An
integrated framework for performance-based optimization of scientific
workflows", In Proceedings of the 18th ACM international symposium
on High performance distributed computing (HPDC '09) (Munich,
Germany, June 11-13, 2009), 177-186.
[14] T. Fahringer, R. Prodan, R. Duan, F. Nerieri, S. Podlipnig, J. Qin, M.
Siddiqui, H.-L. Truong, A. Villazon, and M. Wieczorek, "ASKALON:
A Grid Application Development and Computing Environment", In 6th
International Workshop on Grid Computing (New York 2005), 122-131.
[15] M. Wieczorek, R. Prodan, and T. Fahringer,Scheduling of Scientific
Workflows in the ASKALON Grid Environment.ACM SIGMOD Record,
2005.
[16] Y. Dorozhko, T. Krasikova, Y. Yudin, N. Currle-Linde, andM. Resch,
"An Abstract Language and Environment for the Creation and Execution
of Experiments over Distributed Computers", InProceedings. of the
International Scientific Conference Simulation-2010, Kiev, Ukraine,
2010.
[17] H. Bouziane, N. Currle-Linde, C. Perez, andM. Resch, "Analysis f
Component Model Extensions to support the GriCoL Language", In
Making grids Work, pp 45-59, Springer, 2008.
[18] GLUE Specification v. 2.0, http://www.ogf.org/documents/GFD.147.pdf
[19] Job Submission Description Language (JSDL) Specification, Version
1.0, http://www.gridforum.org/documents/GFD.56.pdf
[20] B. Armstrong, H. Bae, R. Eigenmann, F. Saied, M. Sayeed, andY.
Zheng, "HPC Benchmarking and Performance valuation with Realistic
Applications", 2006 SPEC Benchmark Workshop (spec), 2006.
[21] Java WebStart Overview, http://www.oracle.com/technetwork/java/
javase/overview-137531.html
[22] Lustre Cluster FS, http://www.lustre.org/
[23] sudo command, http://www.gratisoft.us/sudo/sudo.man.html
@article{"International Journal of Information, Control and Computer Sciences:51513", author = "Y. Yudin and T. Krasikova and Y. Dorozhko and N. Currle-Linde", title = "Modular Workflow System for HPC Applications", abstract = "Nowadays, HPC, Grid and Cloud systems are evolving
very rapidly. However, the development of infrastructure solutions
related to HPC is lagging behind. While the existing infrastructure is
sufficient for simple cases, many computational problems have more
complex requirements.Such computational experiments use different
resources simultaneously to start a large number of computational
jobs.These resources are heterogeneous. They have different
purposes, architectures, performance and used software.Users need a
convenient tool that allows to describe and to run complex
computational experiments under conditions of HPC environment.
This paper introduces a modularworkflow system called SEGL
which makes it possible to run complex computational experiments
under conditions of a real HPC organization. The system can be used
in a great number of organizations, which provide HPC power.
Significant requirements to this system are high efficiency and
interoperability with the existing HPC infrastructure of the
organization without any changes.", keywords = "HPC, Molecular Dynamics, Workflow Languages,
Workflow Management.", volume = "7", number = "2", pages = "182-7", }
