Our research explores coherence between diversity, equity, and inclusion goals that faculty articulate in advance of the design process and their enactment within massive open online courses (MOOCs). The purpose of the study is to gain an understanding of the types of goals identified by faculty within course design proposals and how those goals are instantiated in corresponding course designs when working with design teams. Our team analyzed 11 single MOOC and MOOC series proposals to characterize the design goals stated. Following the proposal analysis, we analyzed 32 corresponding courses to identify instances in which stated goals related to diversity, equity, and inclusion were realized. Our analysis revealed patterns between proposed goal types and the ways in which goals manifest in courses related to the way in which content or learning processes were central to the design. We intend to use the results to inform the development of processes to engage in a systematic and purposeful approach for the realization of equitable and inclusive design goals in MOOCs.
Words like disruptive and revolutionary have often been used to describe the potential impact of massive, open, online courses (MOOCs) on access to elite universities and their faculty (Carver & Harrison, 2013; Toven et al., 2014). Yet the promise of equitable and inclusive access for a global audience of learners has not always been realized, and some scholars have argued that MOOCs may even perpetuate educational disparities as they do not spread benefits equitably. For example, learners from developing countries may be particularly disadvantaged and even experience social identity threat when courses do not adequately attend to their learning traditions, contexts, and needs (Kizilcec et al., 2017). Inequitable course designs include a narrow focus on Western epistemological perspectives and the prevalence of unidirectional, or pedagogies of transmission, utilized in courses (Rhoads et al., 2013). Designing for diversity, equity, and inclusion (DEI) in MOOCs, therefore, presents a variety of challenges, but because of the potential reach of MOOCs opportunities exist for advancing innovative pedagogies and content that take into account learners from a variety of global contexts (Ebben & Murphy, 2014). Knowing this, MOOC faculty may have good intentions for advancing DEI goals in their course designs. Yet more research is needed to understand their specific aspirations and the extent to which they are successful in realizing these goals with the help of design teams.
Our research explores the coherence between DEI goals that faculty articulate in advance of the design process and their enactment within the resulting MOOCs. Our study details work within an instructional services unit specializing in the design, development, and production of open online courses at a Research I university in the midwest of the United States. The purpose of the study is to gain an understanding of the types of DEI goals identified by faculty and how those goals are instantiated in corresponding course designs. Additionally, we intend to use the results to inform the development of processes (e.g., pre-proposal consultations assisting faculty in the development of actionable DEI design goals, embedding DEI reflection points during each phase of design, seeking qualitative feedback from colleagues or potential learners pre-launch, and sourcing qualitative learner feedback related equity and inclusion post-launch) to engage in a systematic and purposeful approach for the realization of equitable and inclusive design goals in MOOCs.
We use the following research questions to guide our study:
- What goals for DEI do faculty identify in their MOOC project proposals?
- In what ways are stated faculty DEI goals manifest in the final design of their MOOC?
As designers of learning experiences, we see design as a potential avenue to mitigate issues related to equity and inclusion in open education. We draw on theories of intercultural and transformative learning and a framework for diversity scholarship as a conceptual framework to guide data analysis and the synthesis of results.
Intercultural Education and Transformative Learning Theory
When designing online learning experiences, faculty and design teams should consider the social identities of learners, the situatedness of learners and the curriculum, and pedagogical strategies that promote respectful, intercultural dialogue between learners. Theories of intercultural education and transformative learning have the potential to serve as frameworks for faculty and design teams to create more equitable and inclusive online courses that center the needs of diverse learners and advance the democratization of online education in a global society.
Intercultural education emphasizes “dialogue, social inclusion, interaction, and exchange through . . . empathy, flexibility, and curiosity” (Portera, 2008, p. 399). Intercultural education creates the possibility for personal and social growth through interacting with “individual[s]of different cultural origin[s]” (Portera, 2008, p. 485). Intercultural education lies between two epistemological poles in which “differences and similarities are taken into consideration, brought into contact, and bring about interaction” (Camilleri, 1985 as cited in Portera, 2008, p. 486). Technology has increased opportunities for intercultural communication within online learning spaces and as such, intercultural education may “represent the most appropriate response to the challenges of globalization and complexity,” as educators and institutions continue to strive to demonstrate their commitment to diversity, equity, and inclusion within online learning experiences (Portera, 2008, p. 488). Online learning experiences that draw on existing knowledge of learning in different socio-cultural contexts may foster intercultural competence and communication and create space for curricular and pedagogical flexibility. In turn, these designs could increase equity and inclusion within learning environments and augment learning for those across the globe (Gunawardena, 2014). Online intercultural education requires learners to engage in discursive communication with others, which requires critical introspection and the capacity to simultaneously embrace the similarities and differences of others.
The tenets of transformative learning theory offer a framework for carrying out online intercultural education as the central premise of the theory is the transformation of frames of reference through “critical reflection on the assumptions upon which our interpretations, beliefs, and . . . points of view are based” (Mezirow, 1997, p. 7). Learning experiences that foster transformative learning require:
- Autonomous thinking
- Imaginative problem-solving
- Group deliberation
- Learner-centered discourse
- Content that reflects the lives and experiences of learners (Mezirow, 1997, p. 10)
We use intercultural education and transformative learning theories as lenses in the subsequent literature review focusing on the design and implementation of MOOCs. We explore three main themes in the review: 1) the democratization of online education, 2) the motivations of faculty engaging in equitable and inclusive design, and 3) the impact of equitable and inclusive design on learning.
Democratizing Online Education
The democratization of online education extends beyond open enrollment in MOOCs. According to Rhoads et al. (2013), democratizing open online education requires thinking beyond expanded access to educational resources and must include divergent epistemological perspectives and engage non-dominant “social actors, institutions, and nations” (p. 106). Relatedly, Chen et al. (2020) investigated what they refer to as a “third way” of course design that reimagines the epistemological perspectives of a large Western-centered MOOC for a local, non-Western context through a process the authors refer to as an “ecological circle for MOOC development” (p. 20). More specifically, the Learning How to Learn MOOC was collaboratively redeveloped by a cross-national team for a Chinese context and renamed The Tao of Learning. The authors indicated that during the redesign of the course, culturally-specific content and analogies within the course were updated to be relevant to learners in China.
Rhoads et al. (2013) call for the use of liberating and democratic pedagogies in open online education, and may also include critical pedagogical perspectives drawing on Freire’s (2014) work. Critical pedagogies are liberating in the sense that learners are encouraged to critically reflect on content and the context in which they are learning as “there is no such thing as a neutral education” (Shaull, 2014, p. 34). Morris et al. (2017) expand the notion of critical pedagogy to the digital space—specifically to MOOCs—explaining that critical digital pedagogy revolves around collaborative communities that are open to multiple voices and international perspectives that communicate beyond social, cultural, and political boundaries outside of traditional higher education institutions.
In their study, Dennen and Bong (2018) found that courses encouraging cross-cultural dialogue between learners with divergent national cultures (e.g., individualistic and collectivist) tend to make those in the non-dominant culture susceptible to feelings of otherness. To reduce feelings of otherness, flexible and dynamic course design and facilitation recognize and respect the cultures of learners, support learners so they feel safe to experiment with those who are different from them, and promote dialogue as a way to help learners understand each others’ lived experiences (Dennen & Bong, 2018; Shahini et al., 2019). Relatedly, Mittlemier et al. (2018) reported that learners’ tended to engage in collaboration when content was directly related to their personal backgrounds. Mittlemier et al. suggest that when incorporating collaboration in online courses, educators should consider flexible grouping strategies and encourage learners to share their experiences as sources of content for other learners.
Faculty Considerations When Designing for DEI in Online Courses
It is important to consider faculty motivations for creating MOOCs in general to understand the reasons they may choose (or not choose) to pursue DEI goals. Scholars also present a view of MOOC instructors who desire to reach a wide audience of global learners. Freitas and Paredes (2018) explored faculty motivations driving MOOC development and found that faculty valued the opportunity to widely share specific knowledge to a general audience. Kolowich (2013) asserted that MOOC instructors most frequently cite the desire to reach a worldwide audience of learners and a goal of increasing access to instructional materials. Similarly, Hew and Chung (2014) noted that faculty may be interested in engaging in MOOC design and production processes because they are motivated by a sense of altruism, with the goal of reaching learners who might otherwise not have access to educational experiences. Kleinman (2018) described faculty who were interested in utilizing MOOCs as a means of sharing widely on a topic they are passionate about and feel is of great societal importance, thus increasing learners’ awareness on critical topics. Although these scholars do not focus on faculty intentions concerning DEI, these studies do shed light on faculty motivations for expanding reach and access, which is a related goal.
Beyond reaching global audiences of learners, one MOOC instructor demonstrated an ambition to incorporate the voice of the learner into the instructional materials through the creation of locationally-specific data science problems (Quintana et al., 2018). In another study, a group of instructors indicated a desire to meet the unique needs of MOOC learners during course design, and some even described efforts towards personalization (Bonk et al., 2018).
Impact of Designing for Equity and Inclusion on Learning
Learners who enroll in MOOCs enter with a variety of expectations and prior experiences. Intercultural differences in the way learners perceive expectations and communicative norms could lead to tensions during interactions and discussions, which may inhibit learner engagement and motivation reinforcing feelings of difference (Andersen et al., 2018; Lawrence, 2013). Intercultural competence can be defined as the “cognitive, affective, and behavioral skills and characteristics that support appropriate and effective interaction in a variety of cultural contexts” (Bennett, 2014, p. 157). Croft & Brown (2020) suggested that a lack of racial and cultural diversity amongst higher education faculty could perpetuate implicit biases within their online courses, because they base their assumptions about online learners on personal experience, believing them to hold the same kinds of privileges that they have themselves. Within the MOOC environment, faculty are designing for undefined audiences and may view them as a homogenous group (Macleod et al., 2016).
Some of these tensions can be addressed through the design of the learning environment itself. In terms of bringing learners into an experience in the first place, Kizilcec et al. (2019) theorized that diversity statements within MOOCs could influence enrollment patterns, but suggested that more work is needed to develop diversity statements that are effectively reliable in advancing these goals. Kizilcec et al. (2019) also investigated the impact of psychological cues (e.g., written content, visual design, and interaction design) on enrollment and participation in a statistics MOOC and found that changes to a course image and description appeared to lead to increased enrollment for women (Kizilcec et al., 2017). Thus, the learning environment should promote a sense of safety and trust for sharing ideas and critical reflections related to content and tasks that welcome diverse perspectives and multiple literacies (Blayone et al., 2017; Loizzo & Ertmer, 2016; Marshall, 2014; Stewart, 2013). In order to foster intercultural competence and respect learners’ prior experiences and beliefs regarding communicative norms, Loizzo and Ertmer (2016) suggest acknowledging the notion of “lurking as learning” by removing discussion forum posting requirements and encouraging collaboration beyond platforms (p. 1022). Moreover, the peer-review process for assignments can be utilized as a feature to build intercultural competencies for learners to have space to reinforce content and expand their worldviews through collaborative interactions (Bali, 2014; Loizzo & Ertmer, 2016).
As researchers, our interpretations are filtered through our social identities, and “all research is “positioned” within a stance” (Creswell, 2013, p. 215). We provide a description of our research setting and our positionality as researchers for the sake of transparency (Malterud, 2001). Our research team consists of four learning experience designers who work within a provost-funded instructional services unit to support faculty interested in advancing online learning experiences. As a team, we shared responsibilities for research design, data collection, and analysis. We examined course content for every course and distributed the data analysis work as evenly as possible. Additionally, we took care to analyze courses for which we were not the assigned learning experience designers. During our investigation, we worked together in person and also asynchronously using collaborative word processing tools. We frequently engaged in reflexive dialogue during our research meetings when we discussed our values, beliefs, and interpretations of data to develop a shared understanding.
In concert with a university-wide initiative, the instructional services unit developed a DEI strategic plan, which focused on their specific context—open online courses. At this time, the MOOC proposal form was updated to reflect the unit’s commitment to DEI asking faculty to outline goals and describe how their course design would contribute to the university becoming more diverse, equitable, and inclusive. The question is open-ended, broad, and yielded a variety of faculty responses and goals. Faculty were not explicitly asked to relate their DEI goals with the university or instructional unit’s broader strategic DEI plan.
When we began the investigation, this updated DEI question within the proposal had been in use for three years, allowing for the development of a sufficient number of courses to provide insight into the manifestation of faculty course design goals. We were specifically interested in understanding the range of goals that were articulated in proposals and the extent to which these goals were instantiated in various MOOCs.
Data collected for this study included faculty responses to the DEI question in the MOOC proposal and textual elements from corresponding courses. We conducted the study in three phases (see Figure 1):
- Phase 1: Select MOOC proposals. We established a set of inclusion criteria to select course proposals and corresponding course designs:
- the proposal must respond to the DEI question about how the proposed MOOC addresses DEI goals set forth by the university;
- the proposed course must fall into the open-content category (i.e., with no restrictions on enrollment);
- the proposed course must be live (i.e., not archived).
Based on these criteria, we identified 11 MOOC proposals in total. Six of the selected proposals were for single MOOCs, or stand-alone courses that are typically estimated to take four to six weeks to complete. The other five proposals were for a MOOC series or a set of three to six related courses meant to be taken together.
- Phase 2: Code MOOC proposals. We modified the National Center for Institutional Diversity’s Framework for Diversity Scholarship (NCID, 2021) (see Appendix A) for use in the MOOC context and used it to deductively code 11 MOOC proposals (see Appendix B).
- Phase 3: Review live MOOCs. We examined the MOOCs (n=32) that were developed from the 11 proposals identified through our selection process, looking for evidence of ways in which DEI goals were realized in the design of each course through deductive and inductive coding.
Overview of Three Phases of the Study
We engaged in qualitative content analysis (Elo & Kyngäs, 2008), using an iterative and flexible approach that included inductive and deductive coding to identify themes (Deterding & Waters, 2018). Using a deductive approach, we analyzed stated course design goals in response to the DEI question from the MOOC proposal. To begin our analysis, we explored potential thematic categories and discovered NCID’s Diversity Scholarship framework. NCID’s mission is to bring together interdisciplinary scholars to pursue research to create a more equitable and just society. The framework presents categories of diversity scholarship, which we modified for use in the online course design context and used as a coding scheme for the first phase of analysis (see Appendix A).
While coding the equity goals, we first identified discrete excerpts from MOOC proposals related to one or more dimensions of the modified NCID framework. Second, we coded these excerpts, applying dimensions from the modified NCID framework. Third, we reviewed the coded goals taking into consideration the affordances of the MOOC environment and what would reasonably be instantiated in a course for global learners. For example, some proposals included additional DEI goals that went beyond the course design and were related to the impact on faculty’s pedagogy in general or the dissemination of course materials to university personnel for professional development. Goals such as these were excluded from our analysis.
After the initial analysis of the proposed DEI goals, we performed an artifact analysis of course elements found within each MOOC. We examined a variety of textual elements found within the design of each individual MOOC including course syllabi, video transcripts, discussion prompts, course readings, and assessments. Visual and graphic elements of the course were not included in this stage of the analysis. Each researcher was assigned two single MOOCs and 5-7 individual courses within a MOOC series. Our individual analyses of assigned courses consisted of an initial close reading (line-by-line) of textual course elements in which relevant excerpts or descriptions were recorded, coded, and memoed in relation to the associated NCID dimensions identified in the initial proposal analysis.
Following the initial close reading, the research team convened to discuss and come to a consensus about the extent to which the coded artifacts related to the DEI goals from the proposal would be considered in the analysis. Coded instances were counted in the final tally if they met the following criteria (see Tables 2-3):
- Elements were deemed by the team to be related to the stated DEI goals from the proposal (e.g., stated goal of featuring rural library examples to increase diversity in the field of librarianship coupled with rural library case studies throughout the course)
- Elements were sustained across the course OR where they were deemed to be impactful in relation to learner experience and other course elements (e.g., sustained assessment techniques drawing on learner lived experiences/contexts or a single culminating assessment drawing on learner lived experiences/contexts).
- Elements that appeared more than once throughout the MOOC series (e.g., introductory videos) were counted as one instance.
- Textbooks used throughout a single course were counted one time.
Our analysis was limited in the sense that faculty and design team intentionality was not addressed and the analysis was from the perspective of outside designers attempting to draw connections between proposed goals and course elements after the course had already been designed and launched.
Proposed DEI Goals
Faculty proposed a range of DEI goal types in their MOOC proposals. An overview of the findings for the initial DEI goal analysis in relation to the NCID Framework themes for both single MOOCs and MOOC series is presented in Table 1. Our findings show that on average, single MOOC proposals tended to outline more goals related to DEI than MOOC series. Both single MOOCs and MOOC series proposed goals most commonly aligned with the Addressing Social Inequality theme.
The Climate Change Action single MOOC was an outlier in that our team coded seven DEI-related goals in the proposal across four of the five themes: Addressing Social Inequality, Recognizing the Impact of Power and Privilege, Including Multiple Perspectives, and Valuing Individuals. (see Table 1).
Prevalence and Description of DEI Goals by NCID Framework Theme
|Theme||MOOC Type||Instances||Description of Goals|
|Addressing Social Inequality||Single||6|
- Expanding educational access within US & globally
- Increasing learner capacity to address inequalities
- Expanding educational access within US & globally
|Including Multiple Perspectives||Single||5|
- Inviting guests & outside experts
- Incorporating content featuring non-US & underrepresented perspectives
- Incorporating content a variety of representing social identities
- Interdisciplinary perspectives
- Learner sourced content
- Utilizing non-US examples that are applicable across the globe
- Using inclusive pedagogies & UDL
- Exceeding accessibility standards
- Valuing learner lived experience & context
- Including personalized & self-directed learning
- Including opportunities for learner choice
|Recognizing the Impact of Power and Privilege||Single||4|
- Including content explicitly outlining disparities caused by imbalances of power & privilege
|Finding Common Ground||Single||1|
- Facilitating difficult conversations when discussing the complexity of diversity, equity, & inclusion
- Facilitating social learning with learners who have different lived experiences
Manifestations of DEI Goals Within Courses
In the following section, Tables 2 and 3 delineate tallies of coded instances in each course or series. Notable findings and outlier cases are organized by the themes used to code DEI goals in proposals, which will allow us to represent relationships between goals stated and their instantiations (or lack thereof in some cases).
Instantiations of DEI Goals by NCID Framework Theme: Single MOOCs
| ||Addressing Social Inequality (n)||Including Multiple Perspectives (n)||Valuing Individuals (n)||Recognizing the Impact of Power & Privilege (n)||Finding Common Ground (n)|
|Searching in the Health Sciences||2||7||2||n/a||n/a|
|Climate Change Action||1||23||5||2||n/a|
|Introduction to Dentistry||9||n/a||n/a||n/a||n/a|
|Enacting Social Change||3||5||n/a||n/a||n/a|
|Higher Education Leadership||n/a||7||n/a||2||1|
|Implications of Decision Making in Accounting||0||n/a||0||n/a||n/a|
Instantiations of DEI Goals by NCID Framework Theme: MOOC Series
| ||Addressing Social Inequality (n)||Including Multiple Perspectives (n)||Valuing Individuals (n)||Recognizing the Impact of Power & Privilege (n)||Finding Common Ground (n)|
|How to Manage Public Libraries on a Budget||46||n/a||n/a||n/a||n/a|
|Advanced Applications of Python||n/a||n/a||8||n/a||2|
|Utilizing Python for Statistical Calculations||n/a||7||9||n/a||n/a|
|User Experience Research and Design||5||n/a||n/a||n/a||n/a|
|Basics of Web Applications||59||n/a||n/a||n/a||n/a|
The following section is organized thematically by goal type. We describe findings from courses and series in which we identified the highest number of instantiations of proposed DEI goals. We also describe notable or unique instances as well as outliers. A summary of types of DEI goal instantiations for MOOCs and series organized by theme is included in Table 4.
Summary and Descriptions of Common Instantiations of DEI Goals in MOOCs
|Theme||MOOC Type||Summary of Instantiations|
|Addressing Social Inequality||Single|
- Faculty lecture videos discussing social inequalities
- Guest interviews discussing social inequalities
- Readings created by faculty
- External readings such as website or news articles
- Culminating action-oriented assignments
- Quizzes assessing content related to addressing social inequality
- Course syllabi statements explicitly stating faculty commitments to expanding educational access for the content area/profession
- Faculty lecture videos discussing social inequalities
- Guest interviews discussing social inequalities
- Tutorial videos in which faculty discuss expanding access to guided walkthroughs of content
- Use of OER textbook
|Including Multiple Perspectives||Single|
- Faculty interviews of other academics and experts
- Textual vignettes of experts’ stories
- External videos (e.g., YouTube, TEDx)
- External readings (e.g., news articles, websites)
- Lecture videos defining US-centric jargon and offering alternative ways of thinking about concepts
- Optional practice resources based on datasets from around the world
- Accessible captions for videos
- Video transcripts
- Accessible lecture slide files
- Activities in which learners share their stories and local contexts
- Assignments focused on learner lived experiences and their impact on their local contexts
- Curated list of external coding tutorials for learners with less coding experience
- Curated lists of definitions and common notations for learners with less coding experience
- Specially developed interactive OER text with auto-graded exercises and elaborative feedback
- Goal-setting assignment to help learners identify their goals for the course
- Open-ended code drawing assignment posted in an interactive gallery for viewing (similar to a gallery walk)
|Recognizing the Impact of Power and Privilege||Single|
- Guest interview with climate activist discussing the impact of climate change on under-resourced communities around the world
- Campus map labeling activity for which learners label physical spaces on campus that represent power
- Discussion prompt asking learners to reflect on-campus structures that perpetuate inequality
|Finding Common Ground||Single|
- Discussion prompt asking learners to reflect on the multiple student perspectives of contested campus discourse
- Discussion prompt asking learners to reflect on the multiple student perspectives of contested campus discourse
- Code drawing assignment in which learners are asked to respond to each others’ drawings and provide constructive feedback
- Peer-review assignment asking learners to provide constructive feedback
Addressing Social Inequality
Addressing Social Inequality was the most common theme applied to goals in single MOOC proposals with five out of six proposing this goal type. Of the proposals coded with this theme, all but one, Implications for Decision Making in Accounting, included manifestations of the goals proposed. Introduction to Dentistry exhibited the highest number of manifestations of the theme (n=9). Within the introductory module, five lecture videos, three readings, and one quiz focused on addressing the lack of access to dentistry education and resulting lack of diversity in the profession in the US. While not the highest number of instances, Climate Change Action includes specific examples of actions to take each week related to inequality and climate injustice at the individual, community, political, and adaptation levels (e.g., individual sustainability, community organizing, and writing to politicians). Each week, learners engage in a self-reflection quiz in which they are prompted to check off the actions they took toward climate justice. Additionally, the culminating assessment within the course is an individual climate action plan related to each of the levels of action outlined each week.
Of those three series, Basics of Web Applications (n=59), a four-course series, exhibited the highest number of manifestations of proposed goals. Goals were made visible through tutorial videos the instructor referred to as “code walkthroughs” in which coding mistakes by experts were normalized.
Although How to Manage Public Libraries on a Budget, an eight-course series, did not exhibit the highest number of instantiations of this goal (n=46), an interesting pattern emerged in the data. The series was developed by a large instructional team of outside experts led by one faculty member. Courses taught by the lead faculty member had the highest number of goal manifestations.
Including Multiple Perspectives
Four out of the six single MOOC proposals included goals related to the theme of Including Multiple Perspectives and manifestations of goals were found in each of the four courses that proposed this type of goal (see Table 4). Climate Change Action included the highest number of instantiations of the theme (n=23). The proposed goals related to this theme included perspectives that would not typically be found in a MOOC and seeking guests representing diverse experiences and social identities. This course was unique in that it was co-developed by students working with the faculty and design team, which was outlined in the syllabus. Each of the seven weeks featured seven students interviewing seven guest experts in the field and additional readings outlining the stories of community organizers outside of academia.
Using Python for Statistical Calculations, a three-course series, was the only MOOC series proposal to include a goal related to the theme of Including Multiple Perspectives, which focused on including examples and data from outside the US and offering explanations and where possible, alternatives to US-centric jargon. In total, seven manifestations of the stated goal were identified. Within the series, two videos spent time explicitly breaking down some of the jargon used that was US-centric (e.g., describing the US census process and various acronyms related to datasets) and three videos touched on topics applicable to settings outside of the US such as population health and people who have taken swim lessons. Furthermore, extension activities within the resources section provided exercises using global current issues datasets.
Three of the six single MOOC proposals included goals related to the theme of Valuing Individuals. Of those three, only two included manifestations of the goals (Strategies for Searching in Health Sciences and Climate Change Action). Implications for Decision Making in Accounting stated the goal of creating course content and assessments that “reflect the diversity of the country and the globe”. While there were many case studies outlined detailing both fictional and real companies, the diversity was mainly reflected in the function of the company. Within examples provided, one reference to a Canadian company was identified and company types included restaurants, vineyards, bakeries, and jewelers. Many examples included specific references to the city in which the university is located.
Two out of five MOOC series, Utilizing Python for Statistical Calculations and Advanced Applications of Python, included goals related to the theme of Valuing Individuals. Within Utilizing Python for Statistical Calculations, the proposed goals of meeting learners where they are and providing opportunities for learner choice were made visible through the use of a variety of additional resources such as YouTube Python tutorials, a supplemental resource with frequently used notations and definitions, and a learner-sourced frequently asked questions page to provide additional support for those who may not have as much experience with Python.
Advanced Applications of Python proposed the goal of providing a personalized learning experience through the use of automated feedback on low-stakes exercises. All five courses in the series utilized an interactive OER textbook with auto-graded exercises and elaborative feedback. The textbook was specifically developed for a residential course related to this series. Additionally, this course linked out to two tools geared toward personalized learning. One tool allowed learners to set and track learning goals for the course and the other was used to allow learners to create their own drawings via code to share with others.
Recognizing the Impact of Power and Privilege
Two out of the six single MOOC proposals included goals related to the theme of Recognizing the Impact of Power and Privilege—Climate Change Action and Higher Education Leadership. Both courses exhibited instantiations of the proposed goals. Climate Change Action included a question and answer session with a climate activist outlining the disproportionate impacts of climate change on under-resourced communities around the world. Additionally, a link to a website exploring this topic in relation to Ecuador was included as a specific example.
The instantiations identified in Higher Education Leadership involved an activity in which learners were prompted to examine campus maps and label physical spaces on campus that represent power and importance. Additionally, a discussion prompt asking learners to reflect on the ways in which structures on campuses perpetuate inequality.
Finding Common Ground
Higher Education Leadership was the only single MOOC proposal to relate to the theme of finding common ground. One module focusing on the topic of contested discourse (discussed above in the Including Multiple Perspectives section) ended with a discussion prompt asking learners to reflect on the multiple student perspectives with which they just engaged and outline how they would work toward resolving similar disputes on their campuses.
Advanced Applications of Python was the only MOOC series proposal to outline a goal related to the theme of Finding Common Ground. The proposal emphasized the importance of learners engaging in social learning with others with different lived experiences. Two assessments with social learning components were identified. One instance in the first course involved a linked tool that allowed learners to post drawings created with code to a gallery for peer comments and feedback (similar to a digital gallery walk activity). The second instance was identified in the fourth course and was a culminating peer-review assignment. Both instances involved the social aspect of learners communicating with one another about their work; however, no specific communication guidelines or reminders for students to practice providing constructive feedback were included.
What Types of DEI Goals Were Proposed?
A range of goals were proposed with respect to the NCID themes and also their level of specificity. A clear pattern emerged in relation to the high frequency with which broad, less specific goals focused on expanding educational access were proposed. Notably, the goal for Climate Change Action and Enacting Social Change Through Narrative Experience moved beyond expanding access and incorporated more specificity and cited the need for learner agency and activism (via climate justice and local activism). It is perhaps reasonable to posit that the subject matter of these courses and their relation to social justice issues necessitated the invitation to action on the part of the learner.
The proposal of broad goals related to expanding access to education aligns with current research on faculty motivations for designing MOOCs (Kolowich, 2013; Hew & Cheung, 2014; Freitas & Paredes, 2018). Numerous studies critique some MOOC advocates for being overly ambitious about their potential for expanding educational access and educational equity across the globe (Ebben & Murphy, 2014; Toven et al., 2014; Portmess, 2013). As learning experience designers, it may be helpful to provide faculty with research that critically examines such claims and provides further insights into the complexity of designing MOOCs for global audiences (Portmess, 2013; Rhoads et al., 2013; Kizilcec et al., 2017). Furthermore, our findings suggest that learning experience designers may be able to assist faculty to develop shared, specific DEI goals at the onset of the project in order to promote their enactment in the final course design (Cvitanovic et al., 2020).
How Were Goals Enacted in Course Designs?
Our findings revealed a relationship between goal type and the way in which goals were enacted in courses. Didactic video and text content within the MOOCs and series was the most prominent way in which proposed DEI goals were realized in relation to the most common goal type, Addressing Social Inequality (see Table 1). Conversely, Valuing Individuals and Finding Common Ground related goals were more frequently enacted through activities in which learners share and produce content via learning activities and peer-review type assessments. Bali et al. (2020) delineate typologies of open education practices on a continuum from content-centric to process-centric. Within content-centric practices, the content is the main focus whereas process-centric practices focus on the process of learners sharing and producing their own content contributions. In Figure 2, we use the content and process-centric typologies described by Bali et al. to represent the ways in which DEI goals were realized in courses.
Instantiations of DEI Goals within Courses
Based on our findings, we draw two conclusions related to the content and process typologies described by Bali et al. First, video and text content—especially when it does not have to be created by faculty or design teams—may be an efficient way to address DEI goals when barriers to other means may be present (i.e., time and resources). Second, MOOC platform affordances do not always allow for the realization of process-centric goals. For example, some platforms provide little in the way of dialogue and sharing and tend to rely on asynchronous threaded discussions and peer-feedback communication tools. Studies have shown, however, that learner participation in MOOC discussion forums involves a relatively low number of learners and those that do participate are generally a homogenous group of professionals from the Western world who are well-educated (Gillani & Enyon, 2014; Ayer et al., 2018). Alternatively, Ito et al. (2020) suggest connected learning environments provide opportunities for learners to share “work, skills, and knowledge across networks, groups, and communities . . . [through] [b]logging, publishing work, or streaming” (p. 61). These forms of engagement and communication are typically not available MOOC learning platforms but may be possible by integrating or linking out to other tools as was the case in Climate Change Action and Advance Applications of Python.
Our findings also suggest differences between DEI goal development and course design for single MOOCs and MOOC series. Single MOOCs tend to exhibit a greater number of enacted DEI goals whereas MOOC series were less consistent in terms of enacting DEI goals. We postulate this difference may be attributable to three potential features of designing MOOC series. First, the scope of work involved in designing a series of MOOCs is substantially more than that of designing a single MOOC. Second, MOOC series require coordinating design efforts across multiple courses increasing the complexity of the design work. Third, MOOC series are often led by teams of faculty members, with individual faculty responsible for parts of the design. The individual motivations of faculty on teams may present challenges for cohesively enacting DEI goals across courses, which may have been the case for the How to Manage Public Libraries on a Budget series team. MOOC series design projects will likely require higher levels of support and attention in order to realize DEI goals. Learning experience designers and faculty leads should attempt to ensure DEI goals are collaboratively developed, that plans for implementation are put into place for each course, and that time is taken to holistically examine the series for the enactment of goals.
Limitations and Future Work
Our study is limited in the sense that it is situated within a single instructional services unit at a single university. Our inclusion criteria resulted in a fairly small number of courses and series to analyze, namely courses that were included within the unit’s new proposal template and were “live” at the time of analysis. Since the time of writing, several more courses and series have been completed that meet our inclusion criteria. Furthermore, our chosen methodology, document analysis, may be insufficient to adequately address the nuances of faculty stated and realized goals (Bowen, 2009). Future work could include interviews with faculty to flesh out course goals with respect to DEI more specifically. Future data sources could include interviews with MOOC learners to better understand their experience of taking a course or series, to provide further data concerning whether or not course DEI goals were realized.
Our team is committed to examining our design processes and conducting further design-based research to explore opportunities to enact equitable and inclusive design principles. Since the time of writing, we have created resources to support faculty and design teams. First, we developed a guide (with feedback from our colleagues) for using the modified NCID framework as a set of lenses for critical reflection and a catalyst for action at various points during the design process. Second, a number of our team members, along with colleagues from the instructional services unit formed a group tasked with developing a repository of equitable and inclusive design examples to share with faculty during course design. Third, the instructional services unit is developing processes to assist faculty in the development of actionable DEI goals within the proposal that will be communicated to design teams. Beyond our institution, resources such as design guides and repositories will be important as universities are forced by COVID-19 to enter online spaces with a renewed attention to issues of racial injustice and inequity worldwide. Providing additional research-informed guidance for faculty and course design teams will be essential for continued efforts to design equitable and inclusive online learning experiences open to global audiences.
Andersen, B.L., Na-songkhla, J., Hasse, C., Nordin, N., & Norman, H. (2018). Perceptions of authority in a massive open online course: An intercultural study. International Review Education, 64, 221–239. https://doi.org/10.1007/s11159-018-9708-z
Ayer, N., Sukhathankar, H., Deshmukh, U., & Sahasrabudhe, S. (2018, December). Impact of learner-centric discussion forums on learner engagement in skill development MOOC. In Proceedings of the 2018 IEEE Tenth International Conference on Technology for Education. 69-72. https://doi.org/10.1109/T4E.2018.00021
Bali, M., Cronin, C., Jhangiani, R.S. (2020). Framing Open Educational Practices from a Social Justice Perspective. Journal of Interactive Media in Education, 10, 1–12. https://doi.org/10.5334/jime.565
Bennett, J.M. (2014). Intercultural competence: Vital perspectives for diversity and inclusion. In B.M. Ferdman & B. Deane (Eds.), Diversity at Work: The Practice of Inclusion (155-176). John Wiley & Sons. https://doi.org/10.1002/9781118764282.ch510.1002/9781118764282.ch5
Blayone, T. J., Barber, W., DiGiuseppe, M., & Childs, E. (2017). Democratizing digital learning: theorizing the fully online learning community model. International Journal of Educational Technology in Higher Education, 14(1), 1-16. https://doi.org/10.1186/s41239-017-0051-4
Bonk, C. J., Zhu, M., Kim, M., Xu, S., Sabir, N., & Sari, A. R. (2018). Pushing toward a more personalized MOOC: Exploring instructor selected activities, resources, and technologies for MOOC design and implementation. International Review of Research in Open and Distributed Learning, 19(4). 92-115. https://doi.org/10.19173/irrodl.v19i4.3439
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative research journal, 9(2), 27. https://doi.org/10.3316/QRJ0902027
Carver, L. & Harrison, L. (2013). MOOCs and democratic education. Liberal Education, 99(4), 20-25. https://www.aacu.org/publications-research/periodicals/moocs-and-democratic-education
Creswell, J. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.) [eBook edition]. Sage. https://read.amazon.com/?asin=B00AFFRONG&language=en-US
Croft, B., & Brown, M. (2020). Inclusive open education: presumptions, principles, and practices. Distance Education, 41(2) 1-15. http://doi.org/10.1080/01587919.2020.1757410
Cvitanovic, C., Colvin, R.M., Reynolds, K.J., & Platow, M.J. (2020). Applying an organizational psychology model for developing shared goals in interdisciplinary research teams. One Earth, 2(1), 75-83, https://doi.org/10.1016/j.oneear.2019.12.010
Deterding, N. M., & Waters, M. C. (2018). Flexible coding of in-depth interviews: A twenty-first-century approach. Sociological methods & research, 50(2), 1-32. https://doi.org/10.1177/0049124118799377
Ebben, M. & Murphy, J.S. (2014). Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328-345. https://doi.org/10.1080/17439884.2013.87835210.1080/17439884.2013.878352
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of advanced nursing, 62(1), 107-115. https://doi.org/10.1111/j.1365-2648.2007.04569.x
Freire, P. (2014). Pedagogy of the oppressed: 30th anniversary edition [eBook edition]. Bloomsbury Academic.
Freitas, A. & Paredes, J. (2018). Understanding the faculty perspectives influencing their innovative practices in MOOCs/SPOCs: a case study. International Journal of Educational Technology in Higher Education, 15(5), (1-13). https://doi.org/10.1186/s41239-017-0086-610.1186/s41239-017-0086-6
Gillani, N. & Enyon, R. (2014). Communication patterns in massively open online courses. The Internet and Higher Education, 23, (18-26). https://doi-org.proxy.lib.umich.edu/10.1016/j.iheduc.2014.05.004
Gunawardena, C. N. (2014). Globalization, culture, and online distance. In O. Zawacki-Richter & T. Anderson (Eds.), Online distance education: Towards a research agenda, pp. 75-107. https://doi.10.15215/aupress/9781927356623.01
Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45-58. https://doi.org/10.1016/j.edurev.2014.05.001
Ito, M., Arum, R., Conley, D., Gutiérrrez, K., Kirshner, B., Livingstone, S., Michalchik, V., Penuel, W., Peppler, K., Pinkard, N., Rhodes, J., Tekinbaş, K., Schor, J., Sefton-Green, J., & Watkins, S. (2020). The connected learning research network: Reflections on a decade of engaged scholarship. Connected Learning Alliance: Irvine, CA. https://clalliance.org/publications/the-connected-learning-research-network-reflections-on-a-decade-of-engaged-scholarship/
Kizilcec, R. F., & Saltarelli, A. J. (2019). Psychologically Inclusive Design. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM (Vol. 10, No. 3290605.3300704). https://doi.org/10.1145/3290605.3300704
Kizilcec, R. F., Saltarelli, A. J., Reich, J., & Cohen, G. L. (2017). Closing global achievement gaps in MOOCs. Science, 355(6322), 251-252. https://doi.org/10.1126/science.aag2063
Kleinman, M. A. (2018). Teaching at scale: Instructor experiences with massive open online courses (MOOCs) (Publication No. 11006724) [Doctoral dissertation, University of Michigan]. ProQuest Dissertations Publishing. https://www.proquest.com/docview/2124411811
Kolowich, S. (2013). The professors behind the MOOC hype. The Chronicle of Higher Education, 18. https://www.chronicle.com/article/the-professors-behind-the-mooc-hype/?cid=gen_sign_in
Lawrence, G. (2013). A working model for intercultural learning and engagement in collaborative online language learning environments. Intercultural Education, 24(4), 303-314. https://doi.org/10.1080/14675986.2013.80924710.1080/14675986.2013.809247
Loizzo, J., & Ertmer, P. A. (2016). MOOCocracy: the learning culture of massive open online courses. Educational Technology Research and Development, 64(6), 1013-1032. https://doi.org/10.1007/s11423-016-9444-7
Macleod, H., Sinclair, C., Haywood, J., & Woodgate, A. (2016). Massive Open Online Courses: Designing for the unknown learner. Teaching in Higher Education, 21(1), 13-24. https://doi.org/10.1080/13562517.2015.1101680
Maha B. (2014). Why doesn't this feel empowering? The challenges of web-based intercultural dialogue. Teaching in Higher Education, 19(2), 208-215. https://doi.org/10.1080/13562517.2014.867620
Malterud, K. (2001). Qualitative research: Standards, challenges and guidelines. The Lancet, 358(9280), 483-488. https://doi.org/10.1016/S0140-6736(01)05627-6
Marshall, S. (2014). Exploring the ethical implications of MOOCs. Distance Education, 35(2), 250-262. https://doi-org.proxy.lib.umich.edu/10.1080/01587919.2014.917706
Mezirow, J. (1997). Transformative learning: Theory to practice. New directions for adult and continuing education, 1997(74), 5-12. https://doi.org/10.1002/ace.7401
Mittelmeier, J., Rienties, B., Tempelaar, D., & Whitelock, D. (2018). Overcoming cross-cultural group work tensions: mixed student perspectives on the role of social relationships. Higher Education, 75, 149–166. https://doi.org/10.1007/s10734-017-0131-3
Morris, S. & Stommel, J. (2017). 11. Open Education as Resistance: MOOCs and Critical Digital Pedagogy. In E. Losh (Ed.), MOOCs and Their Afterlives: Experiments in Scale and Access in Higher Education (pp. 177-197). Chicago: University of Chicago Press. https://doi.org/10.7208/9780226469591-013
National Center for Institutional Diversity. (2021, October 26). NCID’s statement on diversity research & scholarship. Framework for diversity scholarship. https://lsa.umich.edu/ncid/research-scholarship/statement-on-diversity-research-and-scholarship.html
Portera, A. (2008). Intercultural education in Europe: Epistemological and semantic aspects. Intercultural Education, 19(6), 481-491. https://doi.org/10.1080/14675980802568277
Portmess, L. (2013). Mobile Knowledge, Karma Points and Digital Peers: The Tacit Epistemology and Linguistic Representation of MOOCs/Savoir mobile, points de karma et pairs numériques: l’épistémologie tacite et la représentation linguistique des MOOC. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 39(2), 2-8. https://doi.org/10.21432/T23S30
Quintana, R. M., Brooks, C., Smothers, C., Tan, Y., Yao, Z., & Kulkarni, C. (2018, June). Mentor Academy: Engaging Learners in the Creation of Data Science Problems for MOOCs. Proceedings of the 13th International Conference of the Learning Sciences (ICLS). (pp. 1415-1416). June 24-27. London, England. https://repository.isls.org/handle/1/654
Rhoads, R.A., Berdan, J., & Toven-Lindsay, B. (2013). The open courseware movement in higher education: Unmasking power and raising questions about the movement’s democratic potential. Educational Theory, 63(1), 87-109. https://doi.org/10.1111/edth.1201110.1111/edth.12011
Shahini, R., Davis, H. C., & Borthwick, K. (2019). Design recommendations to address cultural issues in multicultural MOOCs: a systematic literature review in A. Plutino, K. Borthwick, & E. Corradini (Eds.), New educational landscapes: Innovative perspectives in language learning and technology, (pp. 55-62). Research-publishing.net. https://doi.org/10.14705/rpnet.2019.36.9782490057481
Shaull, R. (2014). Forward in P. Freire, Pedagogy of the oppressed: 30th anniversary edition (pp. 29-34) [eBook edition]. Bloomsbury Academic.
Stewart, B. (2013). Massiveness + openness = new literacies of participation? MERLOT Journal of Online Learning and Teaching, 9(2), 228-238. http://jolt.merlot.org/vol9no2/stewart_%0bbonnie_0613.htm
Stommel, J., Friend, C., Morris, S.M. (2020). Introduction. In J. Stommel, C. Friend, S.M. Morris (Eds.), Critical digital pedagogy: A collection (p. 1). Hybrid Pedagogy. https://cdpcollection.pressbooks.com/front-matter/introduction/
Toven-Lindsey, B., Rhoads, R., & Berdan, J. (2015). Virtually unlimited classrooms: Pedagogical practices in massive open online courses. Internet and Higher Education, 24, 1-12. https://doi.org/10.1016/j.iheduc.2014.07.001
Modified NCID Framework
Below are two sample MOOC proposals that were de-identified and coded using the modified NCID framework outlined in Appendix A.
How does your Initiative help the University of Michigan to become more diverse, equitable, and inclusive?
|Single MOOC Title|
|Climate Change Action|
|Enacting Social Change Through Narrative Experience|
|Higher Education Leadership: How to Incorporate Diversity, Equity, and Inclusion|
|Implications of Decision Making in Accounting|
|Introduction to Dentistry|
|Strategies for Searching in Health Sciences|
|Series Title||Number of Courses|
|How to Manage Public Libraries on a Budget||8|
|Advanced Applications of Python||5|
|Utilizing Python for Statistical Calculations||3|
|User Experience Research and Design||6|
|Basics of Web Applications||4|