Book Chapter10.1007/978-3-642-11492-2_10
Markov Reward Processes
Quan-Lin Li
- 01 Jan 2010
- pp 526-573
5
TL;DR: By using the RG-factorizations, this chapter provides a unified algorithmic framework to derive expressions for conditional distributions and conditional moments of the reward processes of an irreducible continuous-time block-structured Markov chain.
read more
Abstract: In this chapter, we consider reward processes of an irreducible continuous-time block-structured Markov chain. By using the RG-factorizations, we provide a unified algorithmic framework to derive expressions for conditional distributions and conditional moments of the reward processes. As an important example, we study the reward processes for an irreducible continuous-time level-dependent QBD process with either finitely-many levels or infinitely-many levels. At the same time, we provide a simple introduction to the reward processes of an irreducible discrete-time block-structured Markov chain.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
RFID technology-enabled Markov reward process for sequencing care coordination in ambulatory care: A case study
Yi-Chin Kato-Lin,Rema Padman +1 more
TL;DR: A data driven solution for developing prescriptive policies regarding the sequencing of care delivery activities in ambulatory care such that the goal of well-coordinated care may be achieved is proposed.
10
•Posted Content
Visual analytics for team-based invasion sports with significant events and Markov reward process
TL;DR: In this paper, a Markov chain of significant events is used to evaluate the value of any kind of event occurring in a sports match with a continuous parameter space, where the spatiotemporal data used in such analytics is a low-level representation and has a very large size.
3
Resource-Driven Modelling for Managing Model Fidelity
Ashur Rafiev,Andrey Mokhov,Fei Xia,Alexei Iliasov,Rem Gensh,Ali Aalsaud,Alexander Romanovsky,Alex Yakovlev +7 more
- 01 Jan 2017
TL;DR: A resource-driven modelling approach whereby the fidelity of a model can be managed rationally in order to control model complexity, and facilitates parameter-proportional modelling to achieve optimal fidelity vs complexity trade-offs in models.
1
The Gaze Dialogue Model: Nonverbal Communication in HHI and HRI
01 Jan 2022
TL;DR: In this article , a computational model was proposed to encode the interplay of the eye movements during dyadic interaction, and the model also captured the correlation between the different gaze fixation points and the nature of the action.
References
Matrix-geometric solutions in stochastic models : an algorithmic approach
TL;DR: In this paper, a mathematical text suitable for students of engineering and science who are at the third year undergraduate level or beyond is presented, which is a book of applicable mathematics, which avoids the approach of listing only the techniques, followed by a few examples.
•Book
Applied Probability And Queues
Søren Asmussen
- 01 Jan 1987
TL;DR: In this paper, a simple Markovian model for queueing theory at the Markovians level is proposed, which is based on the theory of random walks and single server queueing.
3K
Structured Stochastic Matrices of M/G/1 Type and Their Applications
TL;DR: Structured Stochastic matrices of M/G/1 type and their applications as discussed by the authors, and their application in the field of structured stochastic matrix analysis. But this paper is not an extended version of the paper.
1K