Markov Models
In Our Last Chapter, We have discussed about Stochastic Modeling in Stock Market.
The Markov Model is a type of stochastic model, while the Markov Chain represents a stochastic process.
A Markov Model is a statistical model used to predict a sequence of unknown variables based on the Markov property. It’s an extension of the concept of Markov Chains to more complex scenarios.
States:
State Space (S): The set of all possible states which represent possible conditions or positions a system can be in.
Transition Probabilities:
- Transition Matrix (P): Defines the probability of moving from one state to another.
- Transition probabilities are the chances of moving from one state to another. They are a core component in both Markov Chains and Markov Models, defining the likelihood of transitions between states.
- For each entry, Pij is the probability, also known as the Transition probability of transitioning from state i to state j.
Types of Markov Models:
Markov Models are diverse and adaptable, suitable for various scenarios and data types. Here, we focus on four of the most commonly used models, each distinct in its approach and application.
- Markov Chains: Markov Chains are models that operate in discrete time and have discrete states. For example, a board game where players move based on dice rolls is a real-world analogy. Each roll (a discrete event) determines the next state (position on the board) without regard to previous rolls.
- Markov Jump Processes: Markov Jump Processes function in continuous time but with discrete states. A practical example is a queue system in a bank, where the number of people in line changes at random intervals (continuous time) but the number of people is countable and discrete.
- Markov Diffusion Processes: Markov Diffusion Processes are characterized by continuous time and continuous states, like Brownian Motion. An example is the fluctuation of stock prices in financial markets. Here, the price changes continuously over time, and the range of possible prices is also continuous.
- Hidden Markov Models (HMMs): Hidden Markov Models (HMMs) are used when the state itself is not directly observable, but an output that depends on the state is visible. An example is speech recognition software. The actual spoken words (states) are not known, but the software observes the sound waves (output) to infer the words.
Although these are the msot common variations, Here are some additional types:
- Markov Reward Models:
These models extend Markov chains by associating a reward with each state transition, allowing for the analysis of the total reward over a sequence of transitions. - Continuous-Time Markov Chains (CTMCs):
Unlike discrete-time Markov chains, in CTMCs the time between transitions is also a random variable, which follows an exponential distribution. - Semi-Markov Processes:
These extend Markov chains by allowing for arbitrary distributions of time spent in each state. - Markov Jump Processes:
These are continuous-time processes that jump between discrete states.\ - Markov Regime-Switching Models:
These models allow for the transition probabilities between states to change over time or in response to external factors. - Quasi-Markov Models:
In these models, the next state can depend on both the current state and some aspect of the history of the process. - Higher-order Markov Models:
Unlike standard Markov models which depend only on the current state, higher-order Markov models depend on several previous states. - Factorial Hidden Markov Models (FHMMs):
These are a variation of HMMs where there are multiple interacting hidden Markov chains. - Infinite Hidden Markov Models (iHMMs):
These extend HMMs to allow for an unbounded number of hidden states. - Continuous-Time Markov Chains (CTMCs):
Unlike traditional Markov Chains that transition in discrete time steps, CTMCs transition in continuous time. The time between transitions is exponentially distributed, and the transitions themselves obey the Markov property. - Markov Reward Models:
These are extensions of Markov chains and Markov decision processes that associate a reward rate with each state and transition, allowing for the analysis of the accumulated reward over time. - Semi-Markov Processes:
These extend Markov chains by allowing for arbitrary holding time distributions in each state, unlike the geometrically distributed holding times in traditional Markov chains. - Markov Additive Processes:
These are processes where transitions between states are driven by a Markov chain, but there is an additional, additive component that evolves independently of the Markov chain. - Queueing Models:
Many queueing models, like the M/M/1 queue, are based on Markov processes and utilize Markov chains or continuous-time Markov chains to model the system’s behavior over time. - Multi-Chain Markov Models:
These models consist of multiple interacting Markov chains and are used to model more complex systems with interactions between different processes.
These models and chains are foundational to understanding many processes and systems in fields ranging from computer science to finance to biology. They provide a mathematical framework for dealing with stochastic processes and making predictions in the face of uncertainty
Emission Probabilities (specific to HMMs):
In a Hidden Markov Model (HMM), there’s an additional feature called emission probabilities.
Unlike a basic Markov Model or Markov Chain where states are observable, in HMMs, the states are hidden or unobservable. However, each hidden state emits an observable symbol (or output) with a certain probability.
Emission probabilities define the likelihood of each hidden state producing a particular observable symbol.
For example, consider a simplified weather model where the actual weather (sunny, rainy) is hidden, but you can observe people carrying umbrellas or not.
The emission probabilities might define the likelihood of observing someone with an umbrella given it’s rainy or sunny.
Real-World Applications:
- Finance: In stock market analysis, Markov models can categorize stock price movements into states like “Upside,” “Downside,” and “Consolidation” and analyze the probabilities of transitions between these states.
- Natural Language Processing: HMMs are used in language processing for tasks like part-of-speech tagging, where the words are visible, but their grammatical categories are inferred.
- Weather Forecasting: Markov models assist in predicting weather changes by analyzing transitions between various weather states.
Analytical Approach:
The power of Markov models lies in their simplicity and the analytical approach they offer. By focusing on current states and their probable transitions, they provide a framework for understanding complex systems without the need for extensive historical data analysis.