Computation of the steady-state probability of Markov chain evolving on a mixed state space.
Az-eddine ZakradAbdelaziz NasroallahPublished in: Monte Carlo Methods Appl. (2023)
Keyphrases
- markov chain
- steady state
- state space
- transition probabilities
- queue size
- loss probability
- finite state
- dynamic programming
- monte carlo
- random walk
- reinforcement learning
- markov process
- state transition
- stochastic process
- monte carlo simulation
- confidence intervals
- markov model
- stationary distribution
- product form
- state dependent
- optimal policy
- monte carlo method
- queueing networks
- arrival rate
- state variables
- queue length
- particle filter
- single server queue
- markov processes
- search space
- markov decision process
- dynamical systems
- initial state
- importance sampling
- markov decision processes
- probability distribution
- service times
- infinite horizon
- fluid model
- experimental data
- steady states