Stochastic dynamic programming for resource allocation

Resource allocation and management is an important part of the future network-based defence. In order to provide an adequate situation picture for commanders in the field, sensor platforms must be guided correctly and the needs of different users must be prioritized correctly. Other important problems which require resource allocation include determining where soldiers should be posted in order to maintain peace in an area and determining where to place ambulances. In order for sensor allocation to take opponent´s future activities into account, detailed analysis of risks and consequences is needed. An important help for doing this is stochastic optimization. One important method for such optimization is stochastic dynamic programming, which can be used to compute the best possible policy when we have a probabilistic model of the opponent´s behaviour. The method can also be used when we have stochastic models for, e.g., reliability of our own communication networks. In this report, we describe the basic methods and algorithms of stochastic dynamic programming and their relation to reinforcement learning. We also give a short overview of some relevant applications from the literature and suggest future work in this area.