Optimal Maintenance and Improvement Policies in Water Distribution System: Markov Decision Process Approach
The Markov decision process (MDP) based
methodology is implemented in order to establish the optimal
schedule which minimizes the cost. Formulation of MDP problem
is presented using the information about the current state of pipe,
improvement cost, failure cost and pipe deterioration model. The
objective function and detailed algorithm of dynamic programming
(DP) are modified due to the difficulty of implementing the
conventional DP approaches. The optimal schedule derived from
suggested model is compared to several policies via Monte
Carlo simulation. Validity of the solution and improvement in
computational time are proved.
[1] K. Boston and P. Bettinger, “An analysis of monte
carlo integer programming, simulated annealing, and tabu
search heuristics for solving spatial harvest scheduling
problems,” Forest Science, vol. 45, no. 2, pp. 292–301,
1999.
[2] S. A. Andreou, D. H. Marks, and R. M. Clark, “A
new methodology for modelling break failure patterns in
deteriorating water distribution systems: Applications,”
Advances in Water Resources, vol. 10, no. 1, pp. 11–20,
1987.
[3] Y. Le Gat and P. Eisenbeis, “Using maintenance records
to forecast failures in water networks,” Urban Water,
vol. 2, no. 3, pp. 173–181, 2000.
[4] D. Li and Y. Y. Haimes, “Optimal maintenance-related
decision making for deteriorating water distribution
systems: 1. semi-markovian model for a water main,”
Water Resources Research, vol. 28, no. 4, pp. 1053–1061,
1992.
[5] M. A. Cesare, C. Santamarina, C. Turkstra, and E. H.
Vanmarcke, “Modeling bridge deterioration with markov
chains,” Journal of Transportation Engineering, vol. 118,
no. 6, pp. 820–833, 1992.
[6] Y. Kleiner, “Scheduling inspection and renewal of large
infrastructure assets,” Journal of Infrastructure Systems,
vol. 7, no. 4, pp. 136–143, 2001.
[7] S. Madanat, R. Mishalani, and W. H. W. Ibrahim,
“Estimation of infrastructure transition probabilities from
condition rating data,” Journal of infrastructure systems,
vol. 1, no. 2, pp. 120–125, 1995.
[8] H.-S. Baik, H. S. Jeong, and D. M. Abraham,
“Estimating transition probabilities in markov
chain-based deterioration models for management
of wastewater systems,” Journal of water resources
planning and management, vol. 132, no. 1, pp. 15–24,
2006.
[9] F. Guignier and S. Madanat, “Optimization of
infrastructure systems maintenance and improvement
policies,” Journal of Infrastructure Systems, vol. 5,
no. 4, pp. 124–134, 1999.
[10] S. Madanat and W. H. W. Ibrahim, “Poisson regression
models of infrastructure transition probabilities,” Journal
of Transportation Engineering, vol. 121, no. 3,
pp. 267–272, 1995.
[11] R. Wirahadikusumah, D. Abraham, and T. Iseley,
“Challenging issues in modeling deterioration of
combined sewers,” Journal of infrastructure systems,
vol. 7, no. 2, pp. 77–84, 2001.
[12] T. Micevski, G. Kuczera, and P. Coombes, “Markov
model for storm water pipe deterioration,” Journal of
Infrastructure Systems, vol. 8, no. 2, pp. 49–56, 2002.
[13] G. C. Dandy and M. O. Engelhardt, “Multi-objective
trade-offs between cost and reliability in the replacement
of water mains,” Journal of water resources planning and
management, vol. 132, no. 2, pp. 79–88, 2006.
[14] A. Nafi, C. Werey, and P. Llerena, “Water pipe renewal
using a multiobjective optimization approach,” Canadian
Journal of Civil Engineering, vol. 35, no. 1, pp. 87–94,
2008.
[15] G. V. Loganathan, S. Park, and H. Sherali, “Threshold
break rate for pipeline replacement in water distribution
systems,” Journal of water resources planning and
management, vol. 128, no. 4, pp. 271–279, 2002.
[16] Y. Kleiner, B. J. Adams, and J. S. Rogers, “Long-term planning methodology for water distribution system
rehabilitation,” Water resources research, vol. 34, no. 8,
pp. 2039–2051, 1998.
[17] S. Madanat and M. Ben-Akiva, “Optimal inspection
and repair policies for infrastructure facilities,”
Transportation science, vol. 28, no. 1, pp. 55–62,
1994.
[18] G. Buxey, “The vehicle scheduling problem and monte
carlo simulation,” Journal of the Operational Research
Society, pp. 563–573, 1979.
[19] M. L. Puterman, Markov decision processes: discrete
stochastic dynamic programming, vol. 414. John Wiley
& Sons, 2009.
[20] V. Kathuls and R. McKim, “sewer deterioration
prediction,” in Proc., Infra 99 Int. Convention, 1999.
[21] S. A. Andreou, D. H. Marks, and R. M. Clark, “A
new methodology for modelling break failure patterns
in deteriorating water distribution systems: Theory,”
Advances in Water Resources, vol. 10, no. 1, pp. 2–10,
1987.
[1] K. Boston and P. Bettinger, “An analysis of monte
carlo integer programming, simulated annealing, and tabu
search heuristics for solving spatial harvest scheduling
problems,” Forest Science, vol. 45, no. 2, pp. 292–301,
1999.
[2] S. A. Andreou, D. H. Marks, and R. M. Clark, “A
new methodology for modelling break failure patterns in
deteriorating water distribution systems: Applications,”
Advances in Water Resources, vol. 10, no. 1, pp. 11–20,
1987.
[3] Y. Le Gat and P. Eisenbeis, “Using maintenance records
to forecast failures in water networks,” Urban Water,
vol. 2, no. 3, pp. 173–181, 2000.
[4] D. Li and Y. Y. Haimes, “Optimal maintenance-related
decision making for deteriorating water distribution
systems: 1. semi-markovian model for a water main,”
Water Resources Research, vol. 28, no. 4, pp. 1053–1061,
1992.
[5] M. A. Cesare, C. Santamarina, C. Turkstra, and E. H.
Vanmarcke, “Modeling bridge deterioration with markov
chains,” Journal of Transportation Engineering, vol. 118,
no. 6, pp. 820–833, 1992.
[6] Y. Kleiner, “Scheduling inspection and renewal of large
infrastructure assets,” Journal of Infrastructure Systems,
vol. 7, no. 4, pp. 136–143, 2001.
[7] S. Madanat, R. Mishalani, and W. H. W. Ibrahim,
“Estimation of infrastructure transition probabilities from
condition rating data,” Journal of infrastructure systems,
vol. 1, no. 2, pp. 120–125, 1995.
[8] H.-S. Baik, H. S. Jeong, and D. M. Abraham,
“Estimating transition probabilities in markov
chain-based deterioration models for management
of wastewater systems,” Journal of water resources
planning and management, vol. 132, no. 1, pp. 15–24,
2006.
[9] F. Guignier and S. Madanat, “Optimization of
infrastructure systems maintenance and improvement
policies,” Journal of Infrastructure Systems, vol. 5,
no. 4, pp. 124–134, 1999.
[10] S. Madanat and W. H. W. Ibrahim, “Poisson regression
models of infrastructure transition probabilities,” Journal
of Transportation Engineering, vol. 121, no. 3,
pp. 267–272, 1995.
[11] R. Wirahadikusumah, D. Abraham, and T. Iseley,
“Challenging issues in modeling deterioration of
combined sewers,” Journal of infrastructure systems,
vol. 7, no. 2, pp. 77–84, 2001.
[12] T. Micevski, G. Kuczera, and P. Coombes, “Markov
model for storm water pipe deterioration,” Journal of
Infrastructure Systems, vol. 8, no. 2, pp. 49–56, 2002.
[13] G. C. Dandy and M. O. Engelhardt, “Multi-objective
trade-offs between cost and reliability in the replacement
of water mains,” Journal of water resources planning and
management, vol. 132, no. 2, pp. 79–88, 2006.
[14] A. Nafi, C. Werey, and P. Llerena, “Water pipe renewal
using a multiobjective optimization approach,” Canadian
Journal of Civil Engineering, vol. 35, no. 1, pp. 87–94,
2008.
[15] G. V. Loganathan, S. Park, and H. Sherali, “Threshold
break rate for pipeline replacement in water distribution
systems,” Journal of water resources planning and
management, vol. 128, no. 4, pp. 271–279, 2002.
[16] Y. Kleiner, B. J. Adams, and J. S. Rogers, “Long-term planning methodology for water distribution system
rehabilitation,” Water resources research, vol. 34, no. 8,
pp. 2039–2051, 1998.
[17] S. Madanat and M. Ben-Akiva, “Optimal inspection
and repair policies for infrastructure facilities,”
Transportation science, vol. 28, no. 1, pp. 55–62,
1994.
[18] G. Buxey, “The vehicle scheduling problem and monte
carlo simulation,” Journal of the Operational Research
Society, pp. 563–573, 1979.
[19] M. L. Puterman, Markov decision processes: discrete
stochastic dynamic programming, vol. 414. John Wiley
& Sons, 2009.
[20] V. Kathuls and R. McKim, “sewer deterioration
prediction,” in Proc., Infra 99 Int. Convention, 1999.
[21] S. A. Andreou, D. H. Marks, and R. M. Clark, “A
new methodology for modelling break failure patterns
in deteriorating water distribution systems: Theory,”
Advances in Water Resources, vol. 10, no. 1, pp. 2–10,
1987.
@article{"International Journal of Earth, Energy and Environmental Sciences:69517", author = "Jong Woo Kim and Go Bong Choi and Sang Hwan Son and Dae Shik Kim and Jung Chul Suh and Jong Min Lee", title = "Optimal Maintenance and Improvement Policies in Water Distribution System: Markov Decision Process Approach", abstract = "The Markov decision process (MDP) based
methodology is implemented in order to establish the optimal
schedule which minimizes the cost. Formulation of MDP problem
is presented using the information about the current state of pipe,
improvement cost, failure cost and pipe deterioration model. The
objective function and detailed algorithm of dynamic programming
(DP) are modified due to the difficulty of implementing the
conventional DP approaches. The optimal schedule derived from
suggested model is compared to several policies via Monte
Carlo simulation. Validity of the solution and improvement in
computational time are proved.
", keywords = "Markov decision processes, Dynamic Programming, Monte Carlo simulation, Periodic replacement, Weibull distribution.", volume = "9", number = "4", pages = "278-6", }
