Analytic Strategies for Collaborative Intelligence Analysis

Intelligence analysis is increasingly important in international crisis management which requires a considerable understanding of socio-cultural dynamics, influential actors' motives and intentions, and geopolitical realities. Due to this complexity, the intelligence analysis is often performed by heterogeneous teams of complementary specialists which may come from different domains and organizations. However, similar to individual analysts, the heterogeneous teams commonly do not use any specific analytic strategy for how to perform the collaborative intelligence analysis. The lack of analytic strategy means that the teams are vulnerable to typical problems in collaborative analysis, such as only considering information that support the current hypothesis, insufficient consideration of alternative interpretations, and incomplete assessment of key assumptions. This report describes an experiment to evaluate how the analytic strategies Critical Thinking and Analysis of Competing Hypotheses (ACH) improve the performance in collaborative intelligence analysis. Critical Thinking encourages team members to use question and answer dialogues to understand the strengths and weaknesses of alternative mental models of the situation that integrate observations, interpretations, possible actions, and expected results. Analysis of Competing Hypotheses, on the other hand, encourages team members to evaluate each type of information, assumption, and argument relative all hypotheses to clearly identify the hypothesis with the least weaknesses. The result of the experiment shows that teams that use Critical Thinking perform better than teams that use Analysis of Competing Hypotheses. Teams that use Analysis of Competing Hypotheses also perform worse than teams that did not use any explicit strategy. The main reason that teams that use Analysis of Competing Hypotheses perform worse is that they have difficulty in fully assessing complex and interrelated information relative all potential hypotheses. The assessment difficulties result in confusing discussions and disagreements. However, although the teams' performance improves when using Critical Thinking, their communication pattern is similar to the teams that did not use any explicit strategy and many of their hypotheses have an unsatisfactory quality. This indicates that there is still considerable potential for improving the collaborative intelligence analysis with better analytic strategies.