This report describes the work conducted to further develop and evaluate the tool Lore, which can be used to automate the technical activities of human threat agents during cyber defence exercises. Previously, Lore utilized a combination of rules and specific machine learning models to estimate the hypothesized value of different actions. This system is denoted as the expert system. This report describes the process of replacing the expert system by machine learning models trained to maximize a value function. This new system is denoted as the trained system. A calculation method was created to enable measuring the value of an executed action. Lore was then used to run 90 scenarios to collect observations of the value for 123 000 executed actions. Supervised learning was used to train random forest models based on these observations. The evaluation showed that the trained model on average produced 23 % higher reward and twice as many compromised machines as the expert system. A second evaluation was made by applying Lore during the cyber defence exercise SAFE Cyber 2020. The 17 participants in the exercise thought that it was more fun and educative to be subjected to Lore than a traditional tailor-made scenario.