|
Article Excerpt
In the last years, Instructional Design has seen the development of visual notation systems for supporting and enhancing the design process. Some of them concern the definition of learning goals, while others address the definition of learning activities or learning materials. These tools supposedly reduce the cognitive load and enhance design communication. Few contributions try to assess the impact that such models have on the practice of Instructional Design. This article introduces a general framework for evaluation, indicating key issues and providing guidelines for the design of an evaluation program. As an example, an implementation of the framework along with the data collected about [E.sup.2]ML is presented.
Introduction: Instructional Design and Visual Languages
The design of instruction is a peculiar type of design, as its outcome is like the script of a play: it is an important element of the show, but not the only one. Your enjoyment of a theatre event also depends on the skills of the actors, on the performance of the orchestra, and on several other contextual elements such as the functioning of the heating system or the silence of the audience. Transposed to education, the quality of instruction does not only depend on the lesson plan and on the learning materials, but also on the ability of the instructor, on the mood of the students, etc. As Morrison, Kemp & Ross (2003, p.2) put it, "learning is haphazard; instruction is planned."
Design in complex situations requires conceptual tools for organizing the work both mentally and physically, and that is what happens with the aid of visual notation systems in Architecture, Mechanical Design, and recently in Software Engineering with UML (UML, 2001) or in Hypermedia Design with, e.g., W2000 (Garzotto, Paolini, Bolchini & Valenti, 1999). Visuals indeed allow a synthetic representation of complex objects and reduce the cognitive load (Blackwell, 1997; Lewalter, 2003).
Instructional Design (ID) models have always been visually supported (cf. the use of visuals in Dick, & Carey, 1996; Morrison Kemp & Ross, 2003; Greer, 1992). The difference with other disciplines is that such models represent the design process as a sequence of steps or a set of elements, and not the object being designed, as it happens in architectural blueprints. We could label these models instructional design process models--they will not be a topic in this article.
Instructional Design Languages
In more recent years some authors have claimed that, as other design-oriented disciplines, ID would benefit from a visual notation system that represents the instruction being designed (cf., Waters & Gibbons, 2004). Actually, the ID research tradition offers a very limited number of contribution in this direction. A part of them concerns the visualization of learning goals. To this class belong, for example, Merrill's Content-Performance Matrix (1983), the revised Bloom's taxonomy (Anderson & Krathwohl, 2001), and the Quail model (Botturi, 2004). The goal of such models is to provide a mental classification framework for learning goals, useful for discussing them and creating a common understanding within the design team.
Some novel works (Botturi, 2003; Belfer & Botturi, 2003; Belfer & Botturi, 2004) focused on the development of a blueprint language called [E.sup.2]ML--Educational Environment Modeling Language for representing the educational activity as such, thus producing a documentation of the design process. A similar approach was followed by other European researchers, who developed eduWeaver (Lischka & Karagiannis, 2004; Bajnai & Lischka, 2004) and the Person-Centered e-Learning patterns (Derntl & Mangler, 2004; Derntl & Motschnig-Pitrik, 2004). Finally, CADMOS-D (Psaromiligkos & Retalis, 2002; Retalis, Papasalouros & Skordalakis, 2002) is another language specifically developed for the design of web-based educational software applications.
Another interesting project under this respect is LAMS--Learning Activity Management System (LAMS, 2005). According to its authors, "LAMS is a revolutionary new tool for designing, managing and delivering online collaborative learning activities. It provides teachers with a highly intuitive visual authoring environment for creating sequences of learning activities." Its novelty is in fact the coupling of a learning management system with a visual design tool based on the IMS Learning Design standard (IMS, 2005). The beta testing phase, currently in progress, will reveal much of the potential of visual tools for ID.
Finally, another indication that the time is mature for a leap forward in the field of visual tools for instructional design is Reload (RELOAD, 2005), a recently released tool that provides a visual interface for creating IMS Learning Design instructional units.
In order to distinguish them from design process models I will call these contributions instructional design languages--understanding that they support the representation of the object being designed (e.g., learning goals, instructional activities or learning materials).
Goals and Structure of the Article
Despite the call of several authors for a continuous evaluation of design practices and tools (e.g., Osguthorpe & Zhou, 1989), few if any scientific contributions try to assess the impact of instructional design languages in the actual design practice. No validated answer is available to questions as: Does this model enhance the quality of instruction? Does it make the design process more efficient? Does it allow the implementation of more challenging solutions? This is probably due to the intrinsic complexity of ID, and to the complex relationships among the elements analyzed below.
The main goal of this article is therefore to propose a conceptual framework for supporting the development of evaluation studies about visual instructional design languages. In order to achieve this goal, the next section introduces some basic insights from communication theory and design theory, which will be reprised later, in the third section, for creating the evaluation framework. After the conceptual presentation of the evaluation framework, the article reports its first implementation for the set-up of an evaluation study of [E.sup.2]ML. The fourth section is therefore devoted to a short introduction of the language, and then reports the evaluation method and presents and discusses the data. The conclusions, concerning both the evaluation framework and [E.sup.2]ML, are presented in the last part of the article.
Notice that the article will not cover in detail any of the aforementioned languages, nor try to compare and evaluate them against each other (1). The development of a general evaluation framework is proposed here as a step toward a more widespread use of ID languages. In the first place, evaluation can promote a comparison of different languages; it can also provide evidence for a sensible choice of the language to be deployed in a specific design context.
Insights from Communication and Design Theory
Evaluating the impact of a design language means evaluating the impact of a new medium within a communication system. Medium is a word with several meanings, two of which are related to this case. In its most straightforward meaning related to communication, it can be defined as "a means or instrumentality for storing or communicating information" (WORDNET, 2005), such as the phone, or a CD. A language surely shares this nature. Yet a language is a medium for communication also as "the surrounding environment" (WORDNET, 2005), as in the sentence "fish require an aqueous medium"--communication requires a language in order to happen. Language is therefore a medium both as a tool that we use for communication, and as an environment in which communication events can exist and acquire meaning.
Language makes communication possible, or more affordable (Clark & Brennan, 1991), by creating a favorable environment. As fish have acquired a hydrodynamic shape through the evolution, so our communication will acquire a shape that makes it the most effective in our communication environment.
A language provides not only the words, but also the conceptual categories and metaphors through which we can express our experience and collaborate with others (Lakoff & Johnson, 1980). The introduction of a new...
|
