A dairy cow replacement model based on the notion of a hierarchic Markov process is presented. A hierarchic Markov process is a series of Markov decision processes built together in one Markov decision process, called the main process. In the model a cow is described in terms of lactation number, stage of lactation, the level of milk yield during the previous and present lactation, the length of the calving interval, and the genetic class defined from the breeding value of the father. The criterion of optimality is the maximization of the present value under an infinite planning horizon. Revenues from milk, calves, and replaced cows, feed costs and costs of replacement heifers are considered. The future profitability calculated from the optimal solution is used for ranking of the cows in the herd. The genetic class makes it possible to include the heifers available for replacement in the ranking and to let the replacement decision depend on the genetic class of the heifers. It is concluded that the milk yield of previous lactation is not needed as a state variable when the other variables are present in the model.