enow.com Web Search

Search results

  1. Results from the WOW.Com Content Network
  2. Markov chain - Wikipedia

    en.wikipedia.org/wiki/Markov_chain

    Usually the term "Markov chain" is reserved for a process with a discrete set of times, that is, a discrete-time Markov chain (DTMC), [11] but a few authors use the term "Markov process" to refer to a continuous-time Markov chain (CTMC) without explicit mention.

  3. Markov model - Wikipedia

    en.wikipedia.org/wiki/Markov_model

    A Markov decision process is a Markov chain in which state transitions depend on the current state and an action vector that is applied to the system. Typically, a Markov decision process is used to compute a policy of actions that will maximize some utility with respect to expected rewards.

  4. Markov property - Wikipedia

    en.wikipedia.org/wiki/Markov_property

    A process with this property is said to be Markov or Markovian and known as a Markov process. Two famous classes of Markov process are the Markov chain and Brownian motion . Note that there is a subtle, often overlooked and very important point that is often missed in the plain English statement of the definition: the statespace of the process ...

  5. Markov decision process - Wikipedia

    en.wikipedia.org/wiki/Markov_decision_process

    The "Markov" in "Markov decision process" refers to the underlying structure of state transitions that still follow the Markov property. The process is called a "decision process" because it involves making decisions that influence these state transitions, extending the concept of a Markov chain into the realm of decision-making under uncertainty.

  6. Gauss–Markov process - Wikipedia

    en.wikipedia.org/wiki/Gauss–Markov_process

    Gauss–Markov stochastic processes (named after Carl Friedrich Gauss and Andrey Markov) are stochastic processes that satisfy the requirements for both Gaussian processes and Markov processes. [1] [2] A stationary Gauss–Markov process is unique [citation needed] up to rescaling; such a process is also known as an Ornstein–Uhlenbeck process.

  7. Balance equation - Wikipedia

    en.wikipedia.org/wiki/Balance_equation

    For a continuous time Markov chain (CTMC) with transition rate matrix, if can be found such that for every pair of states and = holds, then by summing over , the global balance equations are satisfied and is the stationary distribution of the process. [5]

  8. Hidden Markov model - Wikipedia

    en.wikipedia.org/wiki/Hidden_Markov_model

    Figure 1. Probabilistic parameters of a hidden Markov model (example) X — states y — possible observations a — state transition probabilities b — output probabilities. In its discrete form, a hidden Markov process can be visualized as a generalization of the urn problem with replacement (where each item from the urn is returned to the original urn before the next step). [7]

  9. Category:Markov processes - Wikipedia

    en.wikipedia.org/wiki/Category:Markov_processes

    M. Markov additive process; Markov chain approximation method; Markov chain central limit theorem; Markov chain mixing time; Markov chain tree theorem; Markov Chains and Mixing Times