Usability Analysis of a Computer-Based Avionics System

By Christopher J. Hamblin, Shiva Naidu & Cindy Miller

Summary: This study evaluates the usability of computer-based avionics system using a methodology described by Schvanevelt, Berringer & Leard (2004) which calculates the accessibility of information based upon the priorities users place upon the individual information sources. We discuss some of the unique usability issues facing engineers designing hardware and software for technically-advanced avionics systems.

INTRODUCTION

General aviation aircraft, specifically small personal aircraft have finally found their way into the computer age. Beginning in 2004, several general aviation aircraft manufacturers started delivering aircraft equipped with glass cockpits and computer-based avionics. While computer-based avionics are not necessarily new, Technically Advanced Aircraft (TAA) represent a fundamental shift from mechanical instruments to "Glass Cockpits" driven by computers and software that display information on LCD screens. The use of computers provides the opportunity to offer an increased level of functionality previously available only in larger commercial aircraft.

With the advent of computer-based avionics, avionics engineers are faced with many of the same design challenges as software and website designers. However, the consequences of poor usability are potentially far more perilous as pilots are dependent upon the usability of the avionics to safely operate their aircraft.

Perhaps the most important task faced by any avionics engineers is designing GUIs that provide the most relevant information for the task at hand without cluttering the screen. The amount of information the pilot require to safely operate their aircraft is daunting. For example the cockpit of a traditional personal aircraft (i.e., Cessna Skyhawk) typically consists of at least 35 information sources (instruments and radio controls) that are distributed throughout the cockpit. TAA Avionics engineers are challenged with designing GUIs which provide all of this information on two 10" displays along with as many as 124 additional functions (i.e., navigation maps, traffic, weather information.) Like software designers the avionics engineers have distributed the additional functions throughout menu structures. Obviously the size of the menu structures increase as functionality increases and much like software or website design, burying important or frequently used tools deep in the menu structure will decrease the usability of the product. Avionics engineers must be especially cognizant of the level of priority the pilots attribute to various functions and information sources so that they are appropriately accessible.

Avionics engineers must also find novel methods of data entry, menu navigation, and cursor control given the ubiquitous mouse and keyboard are inappropriate for the cockpit environment. The G1000 employs concentric knobs and hot keys to achieve these functions. Thus, in addition to designing GUIs that minimize clutter they must also accommodate atypical input devices to allow accurate and efficient information retrieval. An example of this is the linear menu structure of the G1000. Navigation of the menu structure is achieved using the concentric knobs. A larger knob is used to scroll through 4 menus while a smaller knob is used to scroll through the submenu pages. Linear menu structures are relatively inefficient for navigation; however, they are appropriate given the constraints of the input device.

Considering the unique design challenges faced by avionics engineers, what is the impact on information accessibility and retrieval? One possible effect is the designer's display as much information as possible on the screen. However, this often creates clutter and increases information retrieval times due to prolonged visual searches. Alternatively, designers may distribute various information sources throughout multiple pages and require the user to spend time navigating menu structures.

In this study we explore the accessibility of information provided by the avionics system of a TAA. We employ a methodology described by Schvanevelt, Berringer & Leard (2004) which evaluates information accessibility based upon the priorities of the individual information sources.

METHOD

Participants

The usability evaluation was performed by three members (one licensed and one student pilot) of the Aviation Human Factors lab at the National Institute for Aviation Research at Wichita State University.

Materials

A PC-based simulator of the G1000 NAVIII avionics system was used for the evaluation (see Figure 1).

Figure 1. Sample screen of TAA avionics simulator.

Procedure

The avionics system was evaluated twice, once by each pilot. The pilots worked with the third evaluator who recorded the number of steps each pilot took to access the information. We followed the methodology described by Schvaneveldt et al. (2004) to count the number of actions required to access the information sources. Information that was readily accessible, requiring no interaction with the avionics, received an accessibility score of 1. Information sources that required interaction received a 1 plus the number of actions required to access the information. Navigating the menu structures of the G1000 was accomplished using concentric knobs. Turning the knobs could be considered a single step; however, since pages within the menu structure could only be accessed sequentially, we counted each page as a step. For example, if the desired information was displayed on page three of the second menu it would receive an access score of 4 (3 screens + 1).

Discrepancies between the pilots were noted and later resolved by all three evaluators. Some of the information could be retrieved from multiple sources. In such cases evaluators agreed to use the optimal path for the access score.

The number of actions required to access each information source was entered into an electronic spreadsheet which uses an evaluation matrix (see Table 1) to assign one of three accessibility categories (low access (L), clutter (C), and acceptable(+)) based on the number of actions and relative importance of the information source. For example, an information source that has a high priority but requires more than two actions to access is considered to have low accessibility. On the other hand, low priority information that requires fewer than two actions to access is considered clutter as it impedes access to more relevant information. Priority scores for the information sources were obtained from pilot ratings in a previous study (Schvaneveldt, Beringer & Lamonica, 2001).

Table 1. Evaluation Matrix, L indicates Low Access, C indicates clutter, and + indicates acceptable accessibility (Schvaneveldt et al., 2004).

RESULTS

The format of the results are modeled after those proposed by Schvaneveldt et al. (2004). Information sources were categorized according to its primary purpose resulting in four categories (Aviate, Navigate, Communicate, and Systems Management). Accessibility scores for each system were summed and expressed as percentages of each information category (see Table 2).

Table 2. Percentage of all possible information provided by each avionics system. Values are expressed as percentages.

The results of the evaluation show that the TAA avionics do a relatively good job of minimizing clutter; however, the result is a high percentage of information with low accessibility ratings. In fact, there is a significant negative correlation between Clutter and Low Access scores (r = -0.87, p < .01).

Perhaps the most important finding of this evaluation is the low accessibility of information in the communicate category. An examination of the TAA’s menu structure reveals the source of the Low Access scores as the Communicate functions are located relatively deep within the structure requiring an average of five actions to access information. In addition the Communicate functions are distributed across three different submenus. Thus a reorganization of the menu structure would be one potential source of improvement.

DISCUSSION

The purpose of this study was to evaluate the accessibility of information provided by the avionics system of a technically advanced aircraft. The evaluation employed a tool developed by Schvaneveldt et al. (2004) which considers the importance of the information source when evaluating information accessibility. Results showed that the TAA avionics had relatively little clutter but low accessibility ratings, especially in the area of Communication. Although this study evaluates aircraft avionics, the tool and methodology employed could easily be adapted to evaluate software menu structures or website designs.

REFERENCES

Schvaneveldt, R.W., Beringer, D.B. & Leard, T.M. (2004). Evaluating aviation information systems: The role of information priorities. Federal Aviation Administration. Retrieved 12/01/05 from www.hf.faa.gov/docs/508/docs/gaPriorityReport.pdf

Schvaneveldt, R.W., Beringer, D.B. & Lamonica, J. (2001). Priority and organization of information accessed by pilots in various phases of flight. International Journal of Aviation Psychology, 11, 3, 253-280.