Markov models are often used in modeling complex out of equilibrium chemical and biochemical systems. However, many times their predictions do not agree with experiments. We need a systematic framework to update Markov models to make them consistent with constraints that are derived from experiments. Here, we present a framework based on the principle of maximum path entropy to update Markov models using stationary state and dynamical trajectory-based constraints. We illustrate the framework using a biochemical model network of growth factors-based signaling. We also show how to find the closest detailed balanced Markov model to a given Markov model. Further applications and generalizations are discussed.