Learner and User Experience Research

Abstracts

Introduction to the Edited Volume

, , , , &
Researchers have been engaged in productive scholarly endeavors at the intersection of Learning Design, User Experience (UX), Human-Computer Interaction (HCI), and associated disciplines for some time. Our work as editors has sought to capture and disseminate the collective voices of authors working in this area within a single volume. This book focuses on explicating the ontological and epistemological underpinnings of user-centered design (UCD) and UX as applied in the field of Learning/Instructional Design & Technologies (LIDT) with the goal of foregrounding the importance of learner experience (LX) as an emerging design paradigm for the field of LIDT. This chapter introduces the 15 chapters of our open access edited volume. The book is clustered in three parts of (a) Methods and Paradigms (5 chapters), (b) Models and Design Frameworks (6 chapters), and (c) LX Design-in-Use (4 chapters). This volume serves as a contribution to an emerging, transdisciplinary, and complex phenomenon that requires multiple literacies. LX is not only concerned with the effectiveness of designed learning interventions, but also with the interconnected and interdependent relationship between the learner- (or the teacher-/instructor-) as-user, the designed technology, novel pedagogical techniques or instructional strategies, and the learning context. The diversity and breadth of perspectives presented herein serve as a topographical sketch of the emerging focus area of LX and represent an opportunity to build upon this work in the future.

Methods of User Centered Design and Evaluation for Learning Designers

, , , &
Various theories and models have been published that guide the design and development of learning technologies. While these approaches can be useful for promoting cognitive or affective learning outcomes, user-centered design methods and processes from the field of human-computer interaction (HCI) can also be of value to those in the learning design and instructional design and technology (LIDT) community. In this chapter, we present user-centered design, development, evaluation methods, and processes derived from HCI that lead to highly usable technologies promoting positive user experiences. We begin by aligning these methods and processes with theories commonly referenced in the field of LIDT. We then outline specific methodologies that can be applied during the design and development of digital learning environments. The detailed descriptions outline the goals of the various methods and the ideal stage in which to apply the methods; theory and practice are also discussed. Multiple case examples illustrating how the methods can be used in practice are provided.

Paradigms of Knowledge Production in Human-Computer Interaction: Towards a Framing for Learner Experience (LX) Design

In this chapter, I contextualize the knowledge production of the human-computer interaction (HCI) community within broader epistemological, historical, and disciplinary framings of this scholarship. I describe the historical landscape of HCI as a discipline, including the significant subcommunities that have formed over time as the discipline has become more inclusive of disciplines and forms of knowledge. This description will map across cognitivist, social constructivist, and humanist/design threads of the community, all of which are still active participants in the creation of HCI knowledge. These threads are contextualized for a learning, design, and technology (LDT) audience, including historical and theoretical connections to scientific and humanist modes of instructional design scholarship. I conclude with a preliminary grounding for learner experience (LX) design and a conceptual roadmap that draws from strengths in the LDT and HCI communities.

Theories of Change in Learning Experience (LX) Design

, , &
Designers’ theories about how people learn are the keys to impactful educational design. While much effort and attention is appropriately paid to usability in the development of educational tools and materials, the centrality of learning theories is often underappreciated. Learning theories, in combination with considerations of usability, form coherent theories of change. Theories of Change frame how designers expect to shift learners’ knowledge, skills, and abilities. They play out in the features included in digital learning tools and in the activities chosen for learners in technology-enhanced experiences. They are critical to recognizing whether a design is effective. The clearer the theory, the more specific and measurable the indicators; and reliable, focused measures are key to ensuring that a design is working as planned. Additionally, good measures can transform data into launchpads for design iteration. Aligning learning theory, design, and measures, however, is easier said than done. Through illustrative cases of two learning projects, this chapter gives readers useful frameworks and intuitions to approach this process. Designers will be better prepared not only to create effective experiences, but also to communicate their impact to a range of stakeholders including learners, teachers, buyers, and funders.

Flow Theory and Learning Experience Design in Gamified Learning Environments

&
Learning technologies should be designed in a way that facilitates student knowledge construction. While much of the learning technology literature often focuses on retention of content, less is known about the theories and models that influence how learners perceive and react to the interface. In this chapter, we describe the theoretical tenets of flow theory and its application for learning experience design, especially for gamified learning environments. We then present a design case that details how flow theory was applied towards the redesign of an online professional development course. Implications for theory and practice are also discussed.

Color Theory in Experience Design

Choosing colors is an essential component of UX and LX design that is often either overlooked in design coursework and research or that is approached in a non-scientific manner. Yet, colors elicit various emotional and physiological reactions from users that are important for designers to understand, and these reactions are determined by various factors associated with the colors themselves (e.g., hue, saturation, brightness) as well as the cultural and experiential backgrounds of users (e.g., this color reminds me of X). This chapter explains essential knowledge about how color technically works, summarizes what research has shown about how color choice relates to emotions and learning, and provides guidelines for designers to follow in order to more effectively incorporate color into their designs. I then conclude by providing guidance on how to intentionally develop and use different types of color schemes to better achieve design goals and to improve user experiences.

Sociotechnical-Pedagogical Usability for Designing and Evaluating Learner Experience in Technology-Enhanced Environments

, , , &
Learner experience in technology-enhanced learning environments is often evaluated or analyzed with traditional usability heuristics, as in Nielsen (1994a, 1994b), in order to understand if a certain tool is usable or user-friendly. However, Nokelainen (2006) has established that pedagogical usability is often neglected, an approach which takes into account issues of pedagogical design such as instructions and learning tasks. In addition, the social dimension for evaluating online or hybrid learning environments is largely absent in existing usability heuristics. We analyzed relevant literature in order to develop a conceptual framework that includes the three dimensions of social (S), technological (T) and pedagogical (P) usability. In this chapter, we present this as sociotechnical-pedagogical usability. We assert this framework can serve as the basis for future researchers to advance a new set of STP heuristics for learning design. Design recommendations are provided that address social, technical and pedagogical usability for evaluating online or other formats of learning with technologies.

Learning Experience Design: Challenges for Novice Designers

&
Learning Experience Design (LXD), defined as the practice of designing learning as a human-centered experience leading to a desired goal, poses many challenges to novice designers. This chapter presents common challenges experienced by novice learning designers through the lens of design problem solving as well as expert suggestions on how to address the challenges. Without expert knowledge and schema, novice designers experience difficulty conceptualizing and analyzing complex learning problems. An insufficient or erroneous definition of the learners and contextual needs poses further challenges in drawing effective design solutions grounded in learning theory and design principles that flexibly accommodate multiple learning experiences. It is important that novice designers develop their identity as a designer as they learn to think and problem-solve as one.

The Role of Needs Assessment to Validate Contextual Factors Related to User Experience Design Practices

&
This chapter will address how instructional designers can validate user needs and contextual factors influencing performance within their user experience design to ensure the transfer of learning to real-world contexts. It will also demonstrate how information gathered from needs assessments can be leveraged to identify and develop the necessary scaffolds to optimize the user experience. While contextual analysis aims at understanding the user’s work practice, needs assessment delves into identifying, classifying, and validating the needs of users as they pertain to their work practice. It is imperative that an instructional designer fully understands the intricacies and nuances of the application setting/environment to design a prototype that addresses specific contextual factors that may support or inhibit the transfer of learning into that environment. Instructional designers trained to engage in needs assessment that incorporates context into the design of the user experience will be better positioned to facilitate transfer from the learning space to the work practice space. The chapter proposes a framework to assist designers with leveraging outputs of the needs assessment to the user experience design so that contextual factors can inform the entire experience from project conception to transfer of knowledge to real-world applications.

The Design Implementation Framework: Guiding Principles for the Redesign of a Reading Comprehension Intelligent Tutoring System

, , &
The Design Implementation Framework, or DIF, is a design approach that evaluates learner and user experience at multiple points in the development of intelligent tutoring systems. In this chapter, we explore how DIF was used to make system modifications to iSTART, a game-based intelligent tutoring system for reading comprehension. Using DIF as a guide, we conducted internal testing, focus groups, and usability walk-throughs to develop iSTART-3, the latest iteration of iSTART. In addition to these evaluations, DIF highlights the need for experimental evaluation. With this in mind, we describe an experimental evaluation of iSTART-3 as compared to its predecessor, iSTART-ME2. Analyses revealed an interesting tension between system usability and user preference that has important implications for instructional designers.

From Engagement to User Experience: A Theoretical Perspective Towards Immersive Learning

&
As we see immersive technology becoming ever more prominent in education, from our mobile devices to highly immersive virtual reality headsets, there is a growing need to look beyond existing, pragmatic conceptualizations of the technology. We need to explore immersive technology from a theoretical perspective focused on the user and learner experience for it to make meaningful contributions to education. The way we experience technology often frames how we engage with it and determines whether we continue to use it. Immersive technology is often conceptualized based on the affordances provided to the user without any associations made back to theoretical underpinnings drawn from user experience and learning experience research. The authors take known conceptualizations of user experience and immersive technology and link them to the learning experience to form a working theoretical framework that can help develop an understanding of learner engagement with this emerging technology. We begin with concepts related to immersive technology, then build a theoretical framework and conclude with implications for learning approaches through examples from the spatial disciplines.

Intentional Learning Design for Educational Games: A Workflow Supporting Novices and Experts

This chapter proposes that learning experience design (LXD) and game-based learning (GBL) are mutually beneficial conceptual frameworks for increasing the effectiveness, appropriateness, and user experience of educational games. Drawing on a range of theories, five core LXD principles are defined. LXD is: human-focused, enjoyable and/or playful, goal-oriented, situated and relevant to learner’s desires, and placed in supported environments and/or platforms. These principles are mapped to aspects of GBL, proposing considerable overlap between these disciplines, and supported by a wide range of literature. LXD principles are then suggested as solutions to some current challenges in GBL. A workflow is presented which synthesizes interdisciplinary GBL and LXD design processes and offers guidance suitable for GBL designers at a novice to expert level. The workflow categorizes GBL activities into related disciplines (Instructional Design, Empathy/Emotional Design, Interaction Design, and Game Design) to assist readers in analyzing where their own skills or skill gaps lie. A worked example illustrates every activity within the workflow, including practical methods of mapping learning mechanics to game mechanics and of performing gameplay loop analysis. The workflow aims to increase the rigour of GBL design and ensure it benefits from LXD principles, addressing a prevalent challenge in GBL design by focusing on the importance of both an appropriate pedagogical foundation and the needs and desires of learners.

Integrating Learner and User Experience Design: A Bidirectional Approach

, , , &
As online learning experiences become more common in higher education, professional development, and lifelong learning contexts, the importance of understanding how learning experience design and user experience design intersect increases. In this chapter, we offer a vision for a bidirectional approach to learning design through the presentation of two design cases where both learning experience design and user experience design approaches inform the design of tools and designs for learning. We explicate two forms of bidirectional learning: (a) learning that occurs through academic partnership, which includes experiential learning and learning through groups and teams, and (b) learning through inquiry and through mental models. In this chapter, we present a working context that embodies these two forms of bidirectional learning, one that emphasizes shared activity and common projects, a joint working space, and prioritization of teamwork. This context fosters bidirectional learning across two disciplines, allowing team members in both disciplines to learn from each other and to develop shared practices and understanding of disciplinary approaches to design. We conclude with a list of guiding principles for learning experience design and user experience design collaborations.

Participatory Design and Co-Design—The Case of a MOOC on Public Innovation

&
Today, a large body of knowledge exists on Instructional Design, for practitioners as well as for researchers working in Adult Education and Training. However, historical models used for the design of training do not seem to be effective enough in the context of digitalization or multi- and transdisciplinarity. Other fields of research can provide more comprehensive approaches in analyzing the key factor success of training design. In this chapter, we focus on a design case (Howard, 2011) in the field of adult vocational training in France, showing how mix methodologies using participatory design and co-design processes were used to engineer a connectivist Massive Open Online Course (MOOC) (Siemens, 2005). Its purpose is to show how participatory design can be used to design a MOOC and to expose its limits and underlying dimensions.

Supporting Equity in Schools: Using Visual Learning Analytics to Understand Learners’ Classroom Experiences

, , , &
This chapter discusses the affordances of applying user experience (UX) methods to designing technology for understanding learners’ classroom experiences. The chapter highlights a student survey designed to support teachers in building equitable classrooms. The surveys, called “Student Electronic Exit Tickets,” are based on three constructs related to equity: coherence, relevance, and contribution. Survey responses are provided to teachers through a visual analytics dashboard intended to help them readily interpret their learners’ experiences. We explain how UX methods, such as think alouds and cognitive interviews, guided design choices in the dashboard that help teachers attend to equity concerns by disaggregating student data by gender and race. UX methods were also used to validate our argument that visual analytics can be a supportive agent for teachers, prompting them to notice and attend to inequity in their classes. We propose an agenda for future designers of educational technology that incorporates UX methods in the creation of tools that aim to make learning in classrooms more equitable.

Think-Aloud Observations to Improve Online Course Design: A Case Example and “How-to” Guide

, , , , , &
User-experience (UX) problems in course design can be challenging for students as the web interface mediates most online learning. Yet, UX is often underemphasized in e-learning, and instructional designers rarely receive training on UX methods. In this chapter, the authors first establish the importance of UX in the contexts of both learner experience and pedagogical usability. Next the authors describe their study using think-aloud observations (TAOs) with 19 participants. This TAO study resulted in the identification of the following design principles for maximizing the UX of online courses offered within an LMS: (a) avoid naming ambiguities, (b) minimize multiple interfaces, (c) design within the conventions of the LMS, (d) group related information together, and (e) consider consistent design standards throughout the University. Finally, in this chapter the authors offer specific guidelines and directions to enable others to conduct similar TAO testing within their university context.