|
Article Excerpt
INTRODUCTION
Clutter can become a serious problem for users monitoring situation displays. For example, in naval air defense, users must monitor airspaces to find threatening aircraft. These airspaces are frequently in busy environments near land and contain multiple commercial air lanes and other air traffic. Clutter increases search times by increasing the number of objects that must be sifted through or searched to find objects of interest (e.g., Treisman & Gelade, 1980). Clutter also increases the chance for "change blindness," the chronic human inability to detect changes occurring in a scene when attention is focused elsewhere (Rensink, 2002). These problems can result in reduced situation awareness and delayed response times to critical events.
A common method for reducing clutter and promoting situation awareness is to identify important objects and then mark or highlight them in some manner. Highlighting, when the identification process is reliable, allows users to focus on a subset of objects and thereby effectively reduces the number of objects that must be sifted through or monitored. For example, in a search through a matrix of words, Fisher, Coury, Tengs, and Duffy (1989) found that highlighting a subset of words improved response time, even when the highlighting was less than completely reliable. In a visual search task for symbols on a tactical map display, Van Orden, DiVita, and Shim (1993) found that highlighting a category of symbols improved response time. In an augmented reality search task, Yeh and Wickens (2001b) found that highlighting targets improved response time. However, one downside of highlighting is that because it is such an effective form of cuing, it can impede the detection of important objects that are mistakenly left unhighlighted (and hence uncued) when the automation is imperfect or the situation is uncertain (e.g., Baddeley, 1972; Posner, 1980; Yeh & Wickens, 2001b).
A related method for reducing clutter is to identify less important objects and then declutter them from the display by making them less visually salient in some manner. This method also reduces the effective search space by eliminating some objects from the search set. In several studies of visual search for targets in tactical map displays, researchers have shown that users appreciate and benefit from the decluttering of irrelevant categories of symbols (Johnson, Liao, & Granada, 2002; Nugent, 1996; Osga & Keating, 1994; Schultz, Nichols, & Curran, 1985; Yeh & Wickens, 2001a).
A number of methods have been used to declutter objects by reducing their visual salience, including size reduction, dimming, turning symbols into dots, and even complete removal. Ideally, a good declutter method should visually segregate important from less important objects but with minimal disruption to the information content of the symbols. For example, in a visual search task for target symbols on a cluttered display, St. John, Feher, and Morrison (2002) found that simply dimming irrelevant symbols to one third of their initial luminance (thereby reducing their contrast against a dark background) supported easy segregation but without removing any identifying information.
An often overlooked issue, which we address here, is how the highlighted or decluttered objects are identified in the first place. In most experimental studies, the identification function is simply assumed to exist, but it is left unspecified. In applied tactical domains such as air defense, the identification functions are typically simple classification rules, such as all friendly aircraft or all aircraft with altitudes over 25,000 feet (standard U.S. Navy practice). Although attractive because of their simplicity, these rules often fail to meet the needs of sophisticated users because they do not align with the categories of most interest to these users.
A more sophisticated approach is to define meaningful categories of objects and then use these categories as the basis for decluttering. For example, in air defense, rules can be defined to identify commercial versus military aircraft, and then the commercial aircraft can be decluttered. Of course, such rules are necessarily heuristic and are bound to miscategorize aircraft on occasion. Moreover, the identification function of most interest to tactical users is not the type of aircraft, per se, but its level of threat to own ship or other assets. Navy users monitor tactical situations in order to assess threats and then execute responses in order to minimize them. Threat, however, is an ill-defined and complex function of many aircraft attributes and requires years of experience to train (Kaempf, Wolf, & Miller, 1993; Liebhaber, Kobus, & Feher, 2002; Marshall, Christensen, & McAllister, 1996; Morrison, Kelly, & Hutchins, 1996).
Development of reliable automated threat assessment algorithms has long been a goal for aiding situation awareness generally, and air defense in particular. Unfortunately there are several challenges to producing reliable threat evaluation automation. First, the problem can grow extremely complex in attempting to account for all possible variables, including aircraft kinematics, coordinated aircraft behaviors (the big picture), intelligence information, and situational factors such as the geopolitical context. Second, the problem can suffer from ambiguity because important data may be unknown or unknowable. For example, aircraft identity is often based on electronic emissions that may not be detectable or that may have multiple interpretations; ultimately, the intent of an aircraft can never be established with certainty.
Third, expert decision makers frequently disagree about the threat of individual aircraft. For example, Marshall et al. (1996) found that all six of the teams they studied agreed on the interest level of only 41% of the aircraft. Consequently, an automated algorithm can never perfectly match the threat ratings of every user. Fourth, well-known problems of automation trust, complacency, and confirmation bias (e.g., Parasuraman & Riley, 1997) can undermine the effective use of automation and lead to disastrous consequences. On one hand, for example, a user might monitor only those aircraft indicated as threats by the automation, or if the automation missed a threat, the user might be significantly delayed in noticing it. If the automation mistakenly overrated the threat of an aircraft, a user might treat it more aggressively than necessary. On the other hand, distrust of automation might actually increase workload by driving users to increase their monitoring of lower threat aircraft.
Our approach is to treat the automation and the user as a "mixed initiative" system that combines "heuristic automation" that is known to be imperfect with engaged, knowledgeable users who use the automation as a guide but ultimately rely on their own best judgment. According to this design strategy (e.g., Parasuraman & Riley, 1997, pp. 244, 249; St. John & Manes, 2002; St. John, Oonk, & Osga, 2000), users are taught how and where the automation is likely to be trustworthy or make errors, and they verify the automation accordingly. This design strategy fits well with what are termed "low levels of automation" (e.g., Kaber & Endsley, 2004; Parasuraman, Sheridan, & Wickens, 2000), which might involve merely identifying alternative solutions rather than recommending a single best solution or executing a solution unless countermanded by the user. For example, in a visual search task, St. John and Manes (2002) used heuristic automation in the form of an imperfect target detection tool to make a rough first cut at identifying the likely locations of hidden targets. Users then exploited this information to guide their own searches. This approach led to a 23% improvement in search times, even when the automation was only 70% reliable. In a dual-task paradigm, Sorkin, Kantowitz, and Kantowitz (1988) used a "likelihood alarm display" to indicate the likelihood of a signal occurring in the secondary task. Users exploited the likelihood information to decide how carefully to attend to the secondary task. In both studies, knowledgeable users exploited the information provided by imperfect, heuristic automation to guide their attention.
We applied this heuristic automation design strategy to air defense. First, a heuristic threat assessment algorithm evaluated all aircraft every second as they moved about the display by weighing several aircraft attributes and computing a "threat score." Then, lower scoring, less threatening aircraft were decluttered by reducing the salience of their symbols on the display. In this way, the decluttered aircraft would not distract from the higher threat aircraft, yet they would remain available for inspection. We predicted that users would be able to exploit the information provided by the automation to focus the majority of their attention on the fully visible threatening aircraft while periodically scanning the entire display to verify the automation's assessments of the decluttered aircraft. Situation awareness would be enhanced and responses speeded because significant threats would be clearly visible. Decluttering might be especially useful for facilitating the early detection of significant threats at longer ranges from own ship.
Time freed up from searching the cluttered display could be used to verify decluttered aircraft opportunistically on the chance that the heuristic algorithm decluttered an aircraft in error. Thus the potential costs of automation-induced misses would be minimized.
The current experiment tests these predictions in a scenario-based, quasi-realistic air defense task with experienced naval users. Our goal was to assess whether heuristic automation in combination with decluttering could facilitate performance and garner user acceptance within the naturalistic constraints of a real task with experienced users in realistic scenarios. Accordingly, participants performed the normal tasks involved in air defense--namely, monitoring...
|
