Belief based reinforcement learning for data fusion
MetadataShow full item record
Data fusion, learning and uncertainty are concepts that have each been extensively studied, but primarily as distinct areas of investigation. However, data fusion can be used to critically leverage learning, to improve learning performance by increasing the quality of the information provided to decision makers. Equally, in learning the selection of an uncertainty representation profoundly affects the structure of the resultant decision rules and, in the case of sequential decision-making, the decision time selection rule. Belief structures based on Dempster-Schafer evidence theory model is not only aleatory but also epistemic uncertainty. In those cases which are well-modeled by aleatory uncertainty alone it is shown in this dissertation that belief structures can be viable substitutes for probability structures. Moreover, by modeling epistemic uncertainty, it is shown that beliefs provide better treatment of uncertainty in general, and in particular management of the learning exploration vs. exploitation issue. Specifically, the case of reinforcement learning is explored by proposing several belief based reinforcement learning algorithms based on Temporal Difference and Dynamic Programming, and is compared to classical Bayesian results using corresponding Bayesian approaches. Belief structure's computational cost is considered its main drawback in many applications in which uncertainty is a basic element such as decision support. However, close consideration of the nature of the information gathered makes it possible in many cases to reduce these costs significantly. Here we propose adaptive treatment of the resolution of the active frame of discernment. A technique to manage dynamically the size of the frame of discernment is proposed, providing a significant reduction in computational complexity. Numerical results indicate that in at least some applications, a decrease of the scale of orders of magnitude is possible with only marginal reduction in the performance metrics. In data fusion learning, belief based uncertainty represented systems may be improved by using relative reliability among agents and sequential decision making based on prediction from epistemic uncertainty. In experiments presented, relative reliability shows a significant improvement in accuracy for data fusion classifiers when compared to agent-only learning. Prediction from epistemic uncertainty is shown to be a feasible technique that decreases the dependency from arbitrary parameters in sequential decision making. Finally, a testbed for the numerical study of learning in data fusion systems is described and exercised. The treatment of reliable sensor data integrated with less reliable human subjective judgment is investigated, and the use of utility-based data fusion sequential decision making demonstrated.