|
Article Excerpt
Pedagogical agents are gaining acceptance as effective learning tools (Baylor & Ryu, 2003; Moreno, Mayer, Spires & Lester 2001; Moreno, 2004). The increase in the use of agents highlights the need for standardized measurements for evaluating user performance in these environments. While learning gains are a primary variable of interest in such environments, the role of affective variables may be at least as important as learning gains (Anderson, 1995; Bardwell, 1984; Cognition and Technology Group at Vanderbilt, 1992; Kort, Reilly, & Picard, 2001). The purpose of this research was to design and validate an instrument, the Attitude Toward Tutoring Agent Scale (ATTAS), to measure users' perception of pedagogical agents who use conversational dialog to teach (i.e., as tutors). Items were developed from existing higher education teacher rating scales. Scale items were administered to 129 participants from three large urban universities in the south and northwest after interactions with AutoTutor, an animated pedagogical agent designed to teach conceptual physics. Results of factor analysis indicate a scale with three constructs: (a) conversation/pedagogy, (b) attitude toward student, and (c) student interest/attention. Reliability analyses showed strong reliability coefficients for each construct (alphas of .84, .87 and .89, respectively). Scales may be used independently or together in pedagogical agent tutoring environments.
**********
STATEMENT OF THE PROBLEM
Research has shown that individualized instruction and one-to-one tutoring that encourages students to provide in-depth explanations of their answers promotes learning gains (Chi, de Leeuw, Chiu, & LaVancher, 1994; Bloom, 1984). One of the goals of computer-based instruction is to recreate these types of tutoring environments. One of the more promising ways to do this is through the use of pedagogical agent environments. Pedagogical agents allow designers to create an environment in which learners can interact with a computer-based conversational partner to get advice, feedback, or instruction. These agents can take the form of humans, animals, and/or inanimate objects (e.g., the Microsoft Paperclip), or fantastic creatures (e.g., genies or space aliens). With the advent of more powerful computer technology, these agents are being designed to interact with the learner in much the same way as in a human teacher-student relationship, which allows designers to address the social aspects of human-computer interaction (HCI) while tapping the benefits of individualized instruction.
Reeves and Nass (1996) have presented evidence that people apply human social interaction rules to computer characters. This research has particular relevance for the design and evaluation of pedagogical agent environments; the extent to which a pedagogical agent can be effective will be largely influenced by its ability to mimic and support the application of those social rules in the learning environment. Because pedagogical agents are becoming more prevalent (Baylor, 2000; Baylor & Ryu, 2003; Graesser, Van Lehn, Jordan, Rose, & Harter, 2001; Johnson, 2004; Lester et al., 1997; Moreno, 2004; Moreno, Mayer, Spires, & Lester, 2001), it is of increasing importance to researchers to be able to evaluate the user perceptions of the pedagogical agents' ability to effectively reproduce a teacher-student environment.
While learning gains are an important variable of interest in the study of any pedagogical tool, the role that affective variables such as mood, motivation, attitude toward instruction, and attitude toward content can play in the learning process has been well documented, and many believe that these factors are at least as important as direct measures of learning gains (Anderson, 1995; Bardwell, 1984; Cognition and Technology Group at Vanderbilt, 1992; Kort, Reilly, & Picard, 2001; Lent, Brown, & Larkin, 1984; Lepper & Chabay, 1987; Marsh, Cairns, Relich, Barnes, & Debus, 1984; Picard, 1997; Sedighian & Sedighian, 1996; Shaw & Costanza, 1970; Smead & Chase, 1981). This is largely the rationale behind a current practice in the evaluation of college teachers: having students rate instructors. While learners may not necessarily be the best judges of teaching efficacy, we recognize that their perceptions of efficacy reflect their attitudes, and that these attitudes are important components of the learning process. In particular, if we expect our pedagogical agent environments to accurately represent the human-tutor/instructor environment, then we must be able to assess the degree to which our agents are able to mimic and support the same social/affective processes and characteristics we expect of human-tutors/instructors. The importance of these characteristics is reflected in our routine process of having students rate their instructors on openness, willingness to answer questions, encouragement of discussion, and solicitation of multiple points of view.
Some researchers have begun to examine these kinds of affective variables with pedagogical agents (Baylor & Ryu, 2003; 2004; Lester et al., 1997; Lester & Stone, 1997). When addressing the affective impact of these agents, researchers ask a variety of questions covering issues such as the perceived intelligence and likeability of the agent, its ability to respond in natural language, and learner engagement in the task. Because there is no established research tool for assessing the affective impact of pedagogical agents, these questions are often the result of "common sense" approaches, with each researcher re-inventing the wheel. This article will describe the initial development, validation, and reliability assessment of an instrument to measure the perceived efficacy of pedagogical agents as tutors, the Attitude Toward Tutoring Agent Scale (ATTAS).
REVIEW OF THE LITERATURE
Pedagogical Agent Environments
Early history of agents. The roots of the modern pedagogical agent lie in the early development of software agents. Initially, an agent was defined as software programmed with complex algorithms designed for effective communication of statements back and forth from agent to program (Genesereth, 1994). As such, these agents did not communicate with human users, but with other software in the program. In this context, agents were best used as "software interoperators" (p. 53), for example, as communication tools for the integration of databases. No thought was given to communication with human users.
Other researchers at this time were discussing the benefits of designing an environment containing animated characters, which interacted with users in a believable fashion through the expression of emotional responses (Bates, 1994). These emotional responses were thought to create "an illusion of life" (p. 124), allowing for a more engaging experience for the user. The initial attempts at creating believable characters indicated that emotional responses, even ones that were exaggerated and not considered human-like, could increase the likeability of the characters and thereby increase the motivation to interact with the agents.
At this initial stage of development, intelligent agent programmers had created software agents that were able to aid in complex mechanical tasks or prepare itineraries through interactions with human users. Even then, some researchers were beginning to consider what role these agents might play in future human-computer interaction. In his 1994 essay, Norman speculated on the psychology of interaction with these agents and proposed that many issues in the affective domain could come into play when humans were interacting with intelligent agents.
Norman's (1994) essay focused on the expectations users place on intelligent agents. He contended that users will anthropomorphize any agent that appears even the least bit intelligent and that this may cause exaggerated expectations of the agent's capabilities. It is important to emphasize that this projection of human-like qualities applies not just to agent characters, but to any system which, through its actions and responses, appears to respond even somewhat intelligently. He also pointed out that users may also have increased expectations when agents can engage in conversation. "Develop a system that recognizes words, and people assume it has full language understanding, which is not at all the same thing" (p. 69). Such expectations can obviously have an impact on learner attitude, and being able to measure such attitudes accurately and consistently is important, especially with the increased use of pedagogical agents in educational settings.
Agents in educational settings. More recently in the field of education, the focus of pedagogical agent research has been on the effectiveness of animated pedagogical agents as a method...
|
