Abstract: The Markov decision process (MDP) based
methodology is implemented in order to establish the optimal
schedule which minimizes the cost. Formulation of MDP problem
is presented using the information about the current state of pipe,
improvement cost, failure cost and pipe deterioration model. The
objective function and detailed algorithm of dynamic programming
(DP) are modified due to the difficulty of implementing the
conventional DP approaches. The optimal schedule derived from
suggested model is compared to several policies via Monte
Carlo simulation. Validity of the solution and improvement in
computational time are proved.
Abstract: Many exist studies always use Markov decision
processes (MDPs) in modeling optimal route choice in
stochastic, time-varying networks. However, taking many
variable traffic data and transforming them into optimal route
decision is a computational challenge by employing MDPs in
real transportation networks. In this paper we model finite
horizon MDPs using directed hypergraphs. It is shown that the
problem of route choice in stochastic, time-varying networks
can be formulated as a minimum cost hyperpath problem, and
it also can be solved in linear time. We finally demonstrate the
significant computational advantages of the introduced
methods.