|
Article Excerpt
Abstract
Managers in modern organizations are confronted with ever-increasing volumes of information that they must evaluate when making a decision. Data warehousing and data mining technologies have given managers a number of valuable tools that can help them store, retrieve, and analyze information contained in large databases; however, maximizing user performance with these tools remains a challenge for information systems professionals. One important and under-explored aspect of the effectiveness of these tools is the design of the query interface. In this study, we compared the use of visual and text-based interfaces on both low and high complexity tasks. Results demonstrated that decision maker performance was more accurate using the text-based interface when task complexity was low; however, decision makers using the visual interface performed better when task complexity was high. In addition, decision makers' subjective mental workload was significantly lower when using the visual interface, regardless of task complexity. In contrast to expectations, less time was needed to make a decision on low complexity tasks when using the visual interface, but those results were reversed under conditions of high task complexity. These results have important implications for the design of managerial decision-making systems, particularly in complex decision-making environments.
Keywords: Database, computer interface, decision-making
Introduction
Effectively manipulating and interpreting data are critical organizational capabilities in today's hypercompetitive business environment. Decision making often must occur more quickly and with finer granularity than in the past--for example, allocating product mix for a specific customer rather than an entire market (Swink 1995). As a result, many firms have invested in information technologies such as data warehousing and data mining to help managers make sense of data previously scattered throughout the enterprise (e.g., Cooper et al. 2000). In addition, these investments have led to an increase in the number of decision makers using these new data management technologies throughout the organization (Wixom and Watson 2001).
Despite the impressive technical advances associated with these new data management tools, there are a number of behavioral concerns associated with how these tools are used on an everyday basis. For example, while new data management technologies empower users to evaluate vast information repositories, they also have the potential to overwhelm decision makers, leading to information overload (Dillon 2000). Decision makers may now feel obligated to exhaustively scan large databases (Keim and Kriegel 1994), which can be time consuming and result in diminishing returns (Lohse 1997a).
As the information environment becomes increasingly saturated, getting managers' attention and helping them find and focus on the most relevant data becomes increasingly difficult (Davenport and Beck 2001). Therefore, identifying querying techniques that can support efficient information retrieval and decision making to overcome the problem of overload has become critical. One potential mechanism for improving decision-making performance is information visualization (Card et al. 1999; Chen 1999; Tegarden 1999; Tufte 2001 ; Ware 2000) because it allows decision makers to leverage individual perceptual processes more effectively (Tegarden 1999).
Information visualization techniques have been widely applied in science and geography but have been integrated into business applications only recently (Mirel 1998; Roth et al. 1997). Existing research examining information visualization largely focuses on the construction of visualization techniques and is surprisingly silent on the evaluation of these techniques (Au et al. 2000). Therefore, empirical studies that assess the effectiveness of visualization techniques are sorely needed (Mirel 1998). To address this gap in the literature, the research question guiding this research is" "How do information visualization techniques such as visual query interfaces influence decision-making performance?"
While the design of query tools is an important aspect to understanding decision-making performance with technology, it is also important to examine contextual influences that might enable a decision maker to most effectively exploit these techniques. Building on the research question above, we extend our investigation into how query interfaces (which guide query construction and information representation) interact with task complexity and individual spatial ability to influence key decision-making outcomes.
Theoretical Background and Hypothesis Development
Historically, there have been two major alternatives to query interface design: traditional text-based approaches and more modern visual approaches that take advantage of information visualization techniques (Shneiderman 1998). Central to our understanding of how a given query interface might support (or limit) decision-making effectiveness is the concept of restrictiveness (Silver 1991). System restrictiveness refers to the degree and manner in which a system limits its users' decision-making processes to a subset of all possible processes. Implied by this definition of restrictiveness is the notion that the type of system (here, the query interface) employed could enforce a particular type of decision strategy. In the context of this research, both the text-based interface and the visual query interface enforced an "elimination by aspects (EBA)" strategy (Todd and Benbasat 1999), thus the decision strategy enforced by the interface was controlled. However, while enforcing the same decision strategy, the visual query interface differed from the text-based design across four different design dimensions.
First, the visual query interface employed information visualization techniques. Modern query interfaces go beyond simple text data presentation and include interactive, graphical representation and manipulation of data that purport to promote better user performance (Card et al. 1999; McCormick et al. 1987). The visual interface used here, the Homefinder (Ahlberg and Shneiderman 1994), presented data points corresponding to addresses on a city map as compared to a tabular display common of many popular query interfaces.
Second, the visual query interface included elements of "direct manipulation" (Shneiderman 1998). Here, the Homefinder employed dynamic visual query filters that were used to adjust query criteria, allowing more flexible (i.e., less restrictive), iterative querying (Keim and Kriegel 1994; Kumar et al. 1997). By adjusting each of these query filters (using slider bars and buttons) as needed, the user was able to enter an initial query and then dynamically refine it.
Third, as part of the direct manipulation element, users of the visual query interface received immediate feedback about the nature and scope of the data as they manipulated the various query filters (Ahlberg and Wistrand 1995; Keim and Kriegel 1994). This allowed them to immediately "see" the results of their queries vs. having to wait (albeit sometimes less than a second) for a text-based query to be processed by the computer and the results displayed on screen.
Finally, while both interfaces supported an EBA decision strategy, the manner in which it was implemented across the two interfaces varied. In the text-based interface, the user formulated a query, entered it, and received text output listing all of the attributes of each data point responsive to the query. However, in the visual query interface, users adjusted the aforementioned slider bars and received an overview of all the data that met the specified criteria, but received detailed data attributes only "on demand." The visual query interface here supported this capability through the use of a starfield display (see Appendix A). Consistent with the design goals applied in this research, starfield displays allow a decision maker to gain a quick understanding of the nature and scope of the data without overwhelming them with unnecessary detail (Ahlberg and Wistrand 1995; Shneiderman 1998).
Given these contrasts in design, the research model developed in this study suggests that interface-based differences (e.g., how data are displayed, filtered, and manipulated) will affect the decision maker's perception of effort required to use the system as well as their decision accuracy and speed in reaching a solution. In addition, it is reasonable to assume that the amount of data examined and one's ability to understand and interpret graphical displays will moderate the relationship between query interface design and decision outcomes. Thus, this study extends the cognitive fit perspective from that of information presentation (i.e., match between information presentation and task--see, for examples, Vessey 1991; Vessey and Galletta 1991) to an interactive querying environment (i.e., match between querying interface and task).
The research model used in this study is illustrated in Figure 1. Decision accuracy and decision time were the primary outcomes of interest in this research; however, in order to gain some additional insight into the cognitive effort required when using visual and text-based interfaces, subjective mental workload (SMW) was also measured. Subjective mental workload represents the subjective experience of a decision maker and is influenced by a decision maker's processing strategies, perceptions, and skill level (Hart and Staveland 1988; Scerbo and Mouloua 1999; Shiffrin and Schneider 1977).
[FIGURE OMITTED]
Task Complexity
Prior research suggests that task complexity increases when there are more information cues to process, more acts to execute, or increased interdependence between the cues and acts--for example, when one uses iterative querying in order to narrow down a large range of feasible solutions to a set of "best" answers (Wood 1986). In an information retrieval context, complexity increases as the number of potential solutions increases because the user must examine and compare each feasible solution to other feasible solutions if they wish to determine the most effective or accurate response (Campbell 1988; Card et al. 1983; Newell and Simon 1972).
In the context of this research, when querying a data set, there may be a small or very large number of feasible solutions that meet specific criteria. A decision maker should find it easier to evaluate a small number of feasible solutions to find the best outcome as opposed to evaluating a large range of possible solutions. A small number of feasible solutions can be presented simultaneously in a manageable display space (e.g., a single computer screen), allowing decision makers to focus their attention and more easily compare alternatives. Decision makers can typically process text-based information effectively while maintaining low levels of subjective mental workload when feasible solution sets are small (Payne et al. 1988).
Conversely, visual query interfaces typically do not present all of the detailed data simultaneously on the screen and instead provide these details only on demand (Shneiderman 1998). Thus, the use of visual querying may actually increase SMW when task complexity is low, since acquiring details of the feasible solutions involves assessing each feasible solution independently from all other possible solutions, making the decision-making process unnecessarily cumbersome. Therefore, we hypothesize:
H1a: Subjective mental workload will be lower with text-based querying than with visual querying when task complexity is low.
Under conditions of high task complexity, subjective mental workload increases as more data elements are evaluated and retained in working memory (Rossano and Moak 1998). Decision makers retain as much data as their working memory allows until the amount of data increases to a point where their ability is constrained (March and Simon 1958; Miller 1956). Once this point...
|
