Exact Distributions for Reward Functions on Semi-Markov and Markov Additive Processes
TL;DR: In this article, the Laplace transforms of general reward functions on semi-Markov and Markov additive processes on continuous-time Markov chains have been studied, leading to explicit closed-form expressions for the relevant Laplace transform.
read more
Abstract: The distribution theory for reward functions on semi-Markov processes has been of interest since the early 1960s. The relevant asymptotic distribution theory has been satisfactorily developed. On the other hand, it has been noticed that it is difficult to find exact distribution results which lead to the effective computation of such distributions. Note that there is no satisfactory exact distribution result for rewards accumulated over deterministic time intervals [0, t ], even in the special case of continuous-time Markov chains. The present paper provides neat general results which lead to explicit closed-form expressions for the relevant Laplace transforms of general reward functions on semi-Markov and Markov additive processes.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Limit theorems for stationary Markov processes with L2-spectral gap
TL;DR: In this article, the central limit theorem, the local limit theorem and the first-order Edgeworth expansion of Markov random walks are derived for continuous-time Markov chains.
30
Limit theorems for stationary Markov processes with L2-spectral gap
TL;DR: In this paper, it was shown that a processus additif Markovien satisfies the limite centralale, theoreme local, and theoretique uniforme of Berry-Esseen en dimension un.
Markov Reward Processes
Quan-Lin Li
- 01 Jan 2010
TL;DR: By using the RG-factorizations, this chapter provides a unified algorithmic framework to derive expressions for conditional distributions and conditional moments of the reward processes of an irreducible continuous-time block-structured Markov chain.
5
Occurrence of Patterns and Motifs in Random Strings
Valeri T. Stefanov
- 01 Jan 2009
TL;DR: The exact distribution theory associated with occurrences of patterns (single or compound) and motifs, in random strings of letters, is treated in this chapter.
4
References
•Book
Applied Probability And Queues
Søren Asmussen
- 01 Jan 1987
TL;DR: In this paper, a simple Markovian model for queueing theory at the Markovians level is proposed, which is based on the theory of random walks and single server queueing.
3K
Markov Renewal Processes with Finitely Many States
TL;DR: In this article, the authors studied Markov renewal processes with a finite number of states and derived explicit expressions for the distribution functions of first passage times, as well as for the marginal distribution function of the corresponding Semi-Markov process.
Markov-Renewal Programming. I: Formulation, Finite Return Models
TL;DR: The properties of Markov-renewal processes, the reward structure, and the decision process are described, and algorithms for finite-horizon, and infinite-Horizon models with discounting are presented.
173
Numerical inversion of multidimensional Laplace transforms by the Laguerre method
TL;DR: An efficient FFT-based extension of the previously developed variant of the Fourierseries method to calculate the coefficients of the multidimensional Laguerre generating function, and systematic methods for scaling to accelerate convergence of infinite series.
48