An approach to the usability evaluation of the human-computer interaction of a heterogeneous safety-critical complex socio-technical system

Abstract

In this thesis a Usability Evaluation Approach (UEA) is presented. The purpose of UEA is to analyze and evaluate the human-computer interaction (HCI) design for a heterogeneous safety-critical complex socio-technical (CST) system. Heterogeneous safety-critical CST systems play an important part in the operations of socially important infrastructure, such as a mass-transit railway system. CST systems mostly consist of heterogeneous domain specific systems, mainly due to the enormous scale of complexity and other commercial considerations. A CST system typically operates in an interactive environment with safety-critical context. Safety is a property of a system that it will not endanger human life or the environment; safety-critical context assures the safety of equipment within the system is demonstrated. Usability is defined as the extent to which a product can be used by specific users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. In the context of a heterogeneous safety-critical CST system, operability is defined as the ability of human operators to cope with various operational conditions (normal and emergency) without endangering the safety of the system when working together as a unified system; this definition implies the conformance of safety and usability requirements. Compliance to operability is happening to be a common criterion for CST system certification. With few exceptions, the design of individual domain specific systems is aimed to comply with technology-driven functional requirements; HCI of each domain specific system may well satisfy its own design guidelines and usability criteria, but there is no guarantee they can meet the overall operability requirements when working together as a unified CST system. UEA aims to facilitate HCI experts, system operators and safety specialists to analyze HCI requirements and formulate evaluation criteria for heterogeneous HCI design. By discovering interaction problems, UEA seeks to identify design aspects that can be improved, to set priorities, and guidance for how to make changes to a design that confirms the coherence of heterogeneous HCI. UEA extends the usage of scenario concept from the Usability Engineering and further considers human factors and situation awareness perspectives, to create a Unified HCI Requirements Analysis Framework (UHRAF), which generates Problem Scenarios, Network of Scenarios and associated Interaction Models for requirements analysis, and a Safety and Usability Model (SUM) as evaluation criteria, for which the heterogeneous HCI are assessed for compliance to operability. UEA addresses the heterogeneous HCI from three major Building Blocks: (i) Characteristics of Work Environment; (ii) Human Performance and Hazard; and (iii) Cognitive Characteristics of Human Operators. Each Building Block consists of aspects for evaluation criteria from safety and usability perspectives. The benefit of UEA is that it does not prescribe specific analysis tools; instead it enables common analysis tools to be deployed for analysis and evaluation. A usability test is illustrated to analyze HCI requirements and assess the design of heterogeneous HCI for the control room of a mass-transit railway system. The results suggest that UEA is capable of analyzing and evaluating heterogeneous HCI issues in complex environment.