Unbounded-rate Markov decision processes : structural properties via a parametrisation approach

Leiden Repository

Unbounded-rate Markov decision processes : structural properties via a parametrisation approach

Type: Doctoral Thesis
Title: Unbounded-rate Markov decision processes : structural properties via a parametrisation approach
Author: Blok, H.
Issue Date: 2016-06-23
Keywords: Markov decision processes
Unbounded-rate
Structural properties
Truncation
Parametrisation
Optimality equation
Abstract: This research is interested in optimal control of Markov decision processes (MDPs). Herein a key role is played by structural properties. Properties such as monotonicity and convexity help in finding the optimal policy. Value iteration is a tool to derive such properties in discrete time processes. However, in queueing theory there arise problems that can best be modelled as a unbounded-rate continuous time MDP. These processes are not uniformisable and thus value iteration is not available. This thesis builds towards a systemic way for deriving properties, for both disounted and average cost. The procedure that is proposed consist of multiple steps. The first step is to make the MDP uniformisable a truncation needs to be made that keeps the properties intact, we have some recommendations for suitable truncations. In the second step, value iteration can be used to prove the desired structure, we have developed a list of results that can be used for these proofs. As the third step, taking the limit of the truncation to infinity, we have provided conditions that the structures for the truncated processes hold for the unbounded process as well. Applications of this method include the competing queues problem and a server farm problem.
Promotor: Supervisor: W.Th.F. den Hollander Co-Supervisor: F.M. Spieksma
Faculty: Science
University: Leiden
Handle: http://hdl.handle.net/1887/41293
 

Files in this item

Description Size View
application/pdf Cover 5.588Mb View/Open
application/pdf Full text 1.656Mb View/Open
application/pdf Title page_Contents 167.9Kb View/Open
application/pdf Chapter 1 Introduction 165.3Kb View/Open
application/pdf Chapter 2 712.2Kb View/Open
application/pdf Chapter 3 405.0Kb View/Open
application/pdf Chapter 4 462.1Kb View/Open
application/pdf Chapter 5 763.6Kb View/Open
application/pdf Chapter 6 267.7Kb View/Open
application/pdf Chapter 7 334.7Kb View/Open
application/pdf Bibliography 191.7Kb View/Open
application/pdf Summary in Dutch 158.2Kb View/Open
application/pdf Acknowledgements_Curriculum Vitae 100.1Kb View/Open
application/pdf Propositions 105.8Kb View/Open

This item appears in the following Collection(s)