Methods for gathering objective and subjective data

Many important public services, as distribution of electricity, are today controlled by computer systems (e.g. SCADA-systems). Serious disturbances with these systems can lead to physical consequences, where people in the worst case will be injured or killed. This report describes work performed in order to produce a method to evaluate a course of handling of incidents of attacks against IT-systems at authorities and owners of important public infra-structure; and to provide feedback to the students at this course. The purpose of the course is to educate advanced users of SCADA systems on defence against IT-attacks. A plan was produced for experimental design and data collection (objective and subjective data) for a three-day course of handling of incidents. This was evaluated during an exercise where two groups attacked one defending group. Although this situation was contrary to the planed course, because the participants played attackers instead of defenders, the experimental plan and the questions worked well. The experiences from the exercise are overall positive, although some modifications are necessary, e.g. some formulations of questions and the observer protocol, and the PDF-questionnaires need to be adapted to work on computers with Linux as operating system. Design of methodology and questionnaires will after corrections of deficiencies, which were commented during the exercise, serve as a foundation for design of courses handling incidents in IT-systems.