The micro level looks at the teaching and learning level; in other words, at the faculty members. The same aspects as in the previous levels were analyzed, with different focuses: infrastructure (I, local environment), quality (Q, quality of (O)ER), policy (P, local policies) and change (C, incentives, support).
The research questions posed at the micro level are as follows:
- I: How do teachers know about and use the existing local infrastructures? Which infrastructures / working environments (e.g., tools, platforms) do teachers prefer to use to create and edit (O)ER? Which types of (O)ER do teachers prefer to use in their teaching? Which functionalities would be helpful for teachers to edit their own or others' (O)ER and/or for collaborative work?
- Q: Which aspects do teachers use to define the quality of (O)ER and their infrastructures? Are teachers involved in defining quality of (O)ER and their infrastructures? Are teachers aware of how institutional quality procedures related to (O)ER work and who is in charge of them?
- P: Are there policies specific to certain study programs or departments or schools? Are teachers involved in policy making? Are teachers aware of institutional policies related to (O)ER?
- C: How are teachers involved in the technical and informational aspect of creating (O)ER and advancing the infrastructures? Are teachers being supported in the technical-informational aspects of (O)ER material creation? How? (e.g. incentives, support) How do teachers integrate external (O)ER into their own (O)ER? What changes do teachers make to their own and external (O)ER? With whom, where and how do teachers share (O)ER?
Although most of the work in which the micro level report is, as well as in the previous levels, based on desk research, some country reports included other methods: survey (Spain, Germany, Australia), personal interviews (Canada, China, Turkey).
Personal interviews were designed in a semi-structured form, following the research questions for the micro level that were provided in the protocol of the project. The number of interviews varied from one country to another (China, n=3; Canada, n=8; Turkey, n=5).
In the case of the surveys conducted within the framework of this microlevel study, we include the main characteristics as follows (Table 1):
Characteristics of the surveys conducted by the COER experts.
Number of participants
Characteristics of the sample (survey)
1.Email for university vice-chancellors (28.10.2019)
2. Survey (13.11.2019-31.01.2020)
41 full responses, 29 partial responses
- Library professionals (n = 24, 34.29%), educators (n = 10, 14.29%), researchers and senior managers (n = 6 each, 8.57%)
- Working at their institution for eight or more years (n = 30, 42.86%), 20% (n = 14) working there for less than two years.
- Faculty most represented was Library Services (38.57%, n = 27), followed by Education (n = 8, 11.43%) and Health & Welfare (n = 6, 8.57%)
- From 22 different HE institutions
76 full responses, 49 partial responses
- Research Associate (36%), Full-time Lecturer (regular and term contracts) (27%) and Professor (25%)
- Years of teaching experience: > 20 (27.6%), 12-15 (22.4%), 4-7 (22.4%)
- Most represented disciplines were Humanities (34.2%), Mathematics/Natural Sciences (26.3%) and Law, Economics and Social Sciences (15.8%)
- 46% female and 35% male (3% did not answer)
- HE institutions in the federal state of Lower Saxony
2. Survey COVID-19 (24.08.2020-16.09.2020)
400 full answers, 176 partial answers (1)
46 full answers (2, reflect the proportions of 1 in terms of characteristics of the sample, participants that voluntarily gave their contact details in 1)
- Adjunct Professor (nontenure, part-time) (20.4%), Associate Professor (civil servant, tenured, full-time) (25.3%), Associate Professor (not civil servant, tenured, full-time) (15.6%).
- > 20 years of teaching experience (46%), > 20 years of teaching seniority at their current institutions (36%). 17.1% participants between 1 and 3 years of teaching tenure at their current institution
- All the disciplines were represented, Social and Law Sciences (42%), Health Sciences (17.5%) and Engineering and Architecture (15.5%).
- 49.9% female and 47.5% male (2.6% did not answer)
- 23- 34 years old (9.7%), 35-40 years old (13.6%), 41-46 years old (20.4%), 47-52 years old (22.2%), 53-58 years old (20%) and more than 59 years old (14.1%)
- From 64 universities (out of 84)
- Representative sample
As in the previous levels, a comparative multicase study was conducted based on the COER experts’ reports (Yin, 2009). Quantitative and qualitative data from the reports were then analysed through thematic coding with MAXQDA2020 in several iterations (Miles, Huberman & Saldaña, 2014).
In a first iteration, the data were categorized into main codes based on the four elements of the reports described in the research questions (Infrastructure, Policy, Quality and Change). In a second iteration, the OER Adoption Pyramid (Cox & Trotter, 2017b) was integrated as a way of understanding some of the elements, especially concerning awareness, capacity, availability and volition. In a third phase of coding, codes and subcodes were added based on inductive coding and using the above mentioned two frameworks. A last phase of coding involved the revision of some codes and subcodes according to the literature that has explored faculty’s perceptions about OER (e.g., Baas et al., 2019; Belikov & Bodily, 2016; Cox & Trotter, 2017b), as well as renaming codes for more concretion and deleting redundant codes.
The number of passages that correspond to each code are marked with “n”. The same report could have more than one passage related to a code.
The OER adoption pyramid (Cox & Trotter, 2017b, p. 155).
This process resulted in an enhanced model that combines the OER Adoption Pyramid and the four elements of the EduArc research at the micro level based on data from the international reports. Therefore, the enhanced model provides a broader view to the previous literature focusing exclusively on one institution or country (see Figure 19). In the figure the different codes and subcodes are depicted.
Enhanced OER adoption pyramid combined with infrastructure, policy, quality and change (differentiation in parenthesis and with colours).
We present the results according to the four elements of the project and to the concrete research questions, including information that connects to this new model.
At this level, we considered the perceptions and use of lecturers with regard the (O)ER (institutional) infrastructure and (O)ER.
Academics’ use and perceptions about (O)ER infrastructures
(O)ER tools (n=12) that were commonly used by academics in most of the countries studied included, to different degrees, video creation tools and presentation tools (slides). For instance, in South Korea, video production and editing tools such as xinics’ Everlec and iMaxSoft’ Lecture Space were most widely used to create (O)ER. In Japan, Kaltura and Powerpoint were often used to create video lectures, and tools such as Kahoot and mentimeter to promote interactive online teaching. In China, PowerPoint was found as the most popular (O)ER tool by academics at the Northwest Normal University (71%), and only a few used other advanced technologies as Authorware or Flash (Li, 2015). In Spain, MSTeams and the institutional repository were commonly mentioned in the survey conducted by the COER expert, several faculty members also referred to Google Drive, and just a few referred to tools such as Kaltura, their own webpage, Kahoot, Padlet, Socrative, Edmodo, Google Classroom, infographics tools and mind map tools.
In Turkey, a large number of educators used Youtube and Vimeo to share their videos; and in addition, Anadolu educators had Anadolu University’s AKADEMA as the only Turkish platform actively allowing educators to share their materials (MOOC), which is going to open up to other educators outside Anadolu and to publish other OER in video, text or audio formats. With the COVID-19, Turkish educators used usually either web conferencing tools, Powerpoint presentations with narrations or screen recording to create their videos. In Spain, a participant in the survey mentioned Youtube and Vimeo to upload their own videos with restricted access for students, but the most common used tool was the institutional virtual learning environment. In Canada, respondent D stated that her department was negotiating with a publisher to create open, digital texts.
The use of (O)ER repositories (n=20) varied in each country. In Canada, one of the interviewees (respondent B) highlighted eCampus Ontario as a fairly good source of resources. Also, some interviewees used social media for evidence of new and relevant materials (respondent B) or searched for materials on SlideShare (respondent D). In China, in a survey study of 246 faculty members from Chongqing, Hebei, Beijing, Jiangsu and other cities, most (O)ER we reported to be localised on the Internet by the academics, instead of using specific repositories for that purpose (Xu, 2011). In Japan, in the study by Jung, Ho and Suzuki (2013), faculty used Youtube as educational content (53.3%) but none of the Japanese faculty members had created video lectures and uploaded them to Youtube. In Spain, faculty members reported on the survey to use institutional repositories in different ways but being the common ones as a place to store (and share) (O)ER – referring, in most of the cases, although not explicitly, to the institutional virtual learning platform; also, several mentioned using them for searching (O)ER. However, many participants were not aware of the existence of this kind of institutional repository or they did not visit them. Similarly, a high percentage of the German faculty participants in the survey did not know about the existence of (O)ER repositories in their institutions (36.8%) and if they aware of them, many did not use (O)ER from them (54%), searched for (O)ER in them (55.3%), neither published in them (43.4%) or in non-institutional repositories (86.8%). Along these lines, a general lack of knowledge about tools and repositories was identified in Australia, but some survey participants mentioned FutureLearn as the non-institutional platform that they were most using (12.86%, n=9) to house their OER and other short courses, followed by OpenLearn and OpenDOAR, MIT Open Education Consortium, OER Foundation, OERu, the OER World Map and WikiEducator. If an educator wanted to house their OER in a repository outside of the institution, the repository must enable the content to be open, reusable and shareable, as well as preferably to be licensed under Creative Commons (Stevens, Bradbury, & Hutley, 2017). In South Africa, all teaching materials (outside of prescribed textbooks) were available on the institutional LMS. In Turkey, Udemy, Khan Academy Turkey and HEC’s YOK Dersleri Platform were the most frequently used OERs by academics, especially with the COVID-19.
On the other hand, several challenges were identified in some countries in this sense. Thus, in South Africa, the lack of adequate infrastructure to assist and support the use and creation of OER was the barrier with the highest response (de Hart, Chetty & Archer, 2015). In Turkey, many (so-called) OER repositories do not function fully, which means that the content that can be uploaded are usually restricted to publication types (e.g., conference proceedings and pre-print versions of the publications) and confines them only to the own institutional members, which prevents the dissemination of OER in a broader sense.
Types of (O)ER preferred by academics
Concerning types of (O)ER (n=14), concrete kind of resources were common across the countries, especially videos and presentations. For example, in Turkey, videos, presentations, PDF or Word versions of lecture notes were among the most preferred (O)ER. In the case of Korea, many instructors develop and use visual materials, such as PPT, PPT-based audio/video lectures, and other freely available videos on Youtube, Ted Talks, MOOCs, etc. In Canada, most OER reported to use in the study by Hayman (2018a) in Ontario were Youtube videos (79%), web links (83%) and OER articles (55%); and one of the interviewees in the study by the COER expert (respondent D) mentioned using open materials in the form of presentation slides and videos. In Spain, most popular (O)ER formats reported by participants in the survey were slide presentations (87.7%), (O)ER in text format (74.5%) and pictures (65.9%); but videos (48.4%) and assessment tests (43.3%) received a high degree of use too. In Australia, learning objects were the most preferred type of resource used in teaching and learning (72%), followed by module sections (28%) (Bossu, Brown, & Bull, 2014); in another study, websites were the most utilised from of OER for both teaching and learning (Bossu, 2015a). In addition, educators preferred to use OER that requires little modification, for example freely available videos such as TedX talks and YouTube clips (Kandlbinder & Chelliah, 2015). As specific institutional cases, more than half the lecturers of large classes at the University of Technology Sydney “do not create any additional materials beyond the lecture recordings” (Kandlbinder, 2015, p. 1) and educators at Swinburne University reported creating videos and MOOCs as OER (Swinburne Commons, 2015). In Japan, OCW was often mentioned as one of the most popular OER among faculty members (Shigeta et al., 2017). In China, the top three frequently used digital resources were multimedia, e-learning materials and instructional resources (Xu, 2011). In a later study with Chinese academics from the Northwest Normal University (Li, 2015), most interviewees used text, images, audio, animations and videos in their daily teaching. Image processing technology was used by 92%, 69% used audio processing technology, and only a few faculty members used video or animation technology. In a study in which 603 courses from 14 Chinese MOOC platforms were analyzed, 266 (44.11%) were identified as recorded lectures given in classrooms. Animation, Khan’s style, discussion and interviews were rarely used in MOOCs, accounting for less than 4% across 603 courses (Zheng, Li, & Chen, 2015).
Some of the reports specify some differences in the types of (O)ER used by academics with the COVID-19 (n=6). For instance, Japanese instructors most frequently used (O)ER that were created by them as written materials, figures and tables followed by printed/e-textbooks or books. A few instructors used online materials offered by Open University UK and Creative Commons. Among the video materials used, 48% were instructor-created, 47% YouTube, and 20% were from other sources (e.g., Ted Talk, MIT OCW, OUUK, NHK, edX, IU library’s Academic Video Online, etc.), according to a survey with 82 faculty members by ICU’s CTL. In South Korea, over 32% of the academics used self-created video lectures, and over 22% offered task-based online lectures (multiple answers allowed) in a survey with 716 faculty members at SNU (Park, 2020). In Turkey, the majority OER created were in videos (usually created by web conferencing tools, some as PowerPoint Presentations with narrations or as screen recordings), presentations and lecture notes of the professors. In Spain, online questionnaires (68.9%) and videos (62.2%) were the ones with a higher increase of use/creation with the COVID-19 compared to the previous answers; to a lesser extent, the use of teaching text-based materials (instructions, class notes, activity guides) also experienced an increase (51.1%). In Germany, resources based on text (64.5%), presentation slides (84.2%), images (72.4%) and infographics (51.3%) were most strongly reported to be used often and always, followed by videos (43.4%). In China, and in particular at Peking University, academics mostly adopted live streaming, accounting for 50% of the total number of courses, whereas most of the graduate courses were more diverse, including seminars and live streaming and other formats (Gong, 2020). About 25% of these graduate courses adopted two or more instructional forms to improve the teaching quality. Faculty members used diverse platforms, such as the Peking University teaching platform (Blackboard), ClassIn, and Canvas, and Zoom. Most faculty members chose ClassIn. In addition, faculty members who needed to organize seminars and record course videos primarily chose the Peking University teaching platform (Blackboard).
Challenges about (O)ER infrastructure according to academics
Interoperability issues (n=3) was among the challenges of (O)ER infrastructure mentioned in the Turkey, Canada and Spain’s reports. For instance, in Turkey, the library services and OER repositories were incapable of operating between and among similar services. One of the interviewees from Canada (respondent C) said to have experienced technical issues in making adaptations from various platforms and suggested that some standardization of platforms would make the process much easier. In Spain, when academics were asked if the (O)ER repositories were connected to other institution systems, such as the LMS, the intranet, etc., 45.6% of them stated that this integration exists, but a high number of academic staff were unsure (34%).
If we look at the OER adoption pyramid framework, the main category involved in (O)ER Infrastructure is Availability as an additional challenge to interoperability. Subcodes that we integrated in this category are accessibility, lack of applicability, license issues and lack of discoverability.
- Accessibility (n=2). In this sense, university teachers in Japan used YouTube and OCW as OER platforms more often than other ones because their OER were perceived to be simple to use and easily accessible without making any changes. In Australia, the ability for a repository to include information about licensing and metadata options, available alongside where to input the information, was mentioned by a survey participant as something that would possibly also improve educator understanding and ease of use.
- Lack of applicability (n=11). A common theme in the interviewees from Canada were the difficulties to find OER fit well to a niche course/content/field (respondents A and B). Some of them also remarked the insufficient supply of OER for their disciplines or content (respondents C and D). There was an exception among the interviewees, who felt that being an experienced educator helped him to “know where stuff is” and that it was “so much more difficult for new faculty who are not as familiar with what’s “out there”” (respondent G). In the study by Hayman (2018b) in the context of Ontario, participants were persistent in their attitudes that there was an insufficient supply of OER for their disciplines. In the case of Spain, a high percentage of academic members did not find (O)ER useful for their teaching (43.7%). In Germany, in a study with 662 HE instructors across the country (Schmid et al., 2017), participants rather agreed with their field of teaching lacking adequate OER. Similarly, academics at the Australian universities of Technology Sydney, Tasmania and Southern Queensland highlighted that they had found it difficult to find suitable OER for their subject, outside of TedX talks and Youtube videos (Bossu, 2015b; Kandlbinder & Chelliah, 2015; Stagg & Partridge, 2019) particularly in regard to licensing and university policy environments. In addition, academics at the University of Technology Sydney remarked that although students could easily access Khan Academy and iTunes U, they did not necessarily match course aims as closely as their own content would (Kandlbinder, 2015). On the other hand, the lack of availability of context relevant (O)ER was a general concern in South Africa, since most works are from the Global North and particularly Eurocentric.
- License issues (n=3). Two academic interviewees in Canada highlighted this specific concern. One of them stated that she was often frustrated due to the lack of the appropriate licensing in the materials she wanted (respondent A), and the other one remarked the conflicting interests dependent on funding sources that determine the licensing and restricts the use of some resources (respondent B). In Australia, the need to include metadata to improve OER searchability was identified as a barrier for educators (Open Education Licensing Project, 2016a). Therefore, an advanced function to support the input of metadata in a repository including a place to input licensing details were considered useful. One Australian survey participant stated that “unfortunately the repository does not have a visible licence field which undermines our ability to support content in terms of infrastructure”.
- Lack of discoverability (n=6). Being able to find suitable, high quality OER had been found a particular issue by Australian academics (Kandlbinder & Chelliah, 2015), with pressure placed by academics on libraries to purchase more resources, which is often encouraged by publishers. Several respondents from Australia to a survey suggested that the establishment of standardized metadata for OER and/or a national/institutional repository would help with the major challenge of discoverability of (O)ER (Bossu et al., 2014, p. 21). An interviewee in Canada (respondent H) stated to advocate for open resources but found them difficult to access, and another one (respondent B) remarked the difficulty to find OER that meet her standards. In the study by Hayman (2018b), participants also highlighted the difficulty to find OER. Participants in the German survey indicated as a third reason for not using the institutional repositories not being able to find the appropriate resources (7.9%, the question was only shown to the participants that had answer negatively to the use of institutional repositories).
Concerning quality at this micro level, we looked at the how academics defined quality of (O)ER by examining their perceptions about quality of (O)ER, and at how aware academics were in terms of quality institutional measures and procedures, as well as of institutional agents for quality assurance.
Academics’ defining quality of (O)ER
Perceptions about the quality of (O)ER (n=10) appeared in six reports (Australia, Canada, South Africa, South Korea, Spain and Turkey). In most of the countries, these perceptions referred to a common prejudice against OER being of low quality. For instance, in Turkey openness and OER related concepts were related to free sources with low quality. Such a perception by academic circles at individual and institutional levels inherit the developments of OER related issues in Turkey. In South Africa, the scepticism about the quality of (O)ER remains a challenge in the adoption, use, production and dissemination of (O)ER; in particular, Madiba (2018) referred to lecturers’ misunderstanding and feeling frustrated “about how to strike a balance between determining the quality of educational resources on the open platforms and the maintenance standards that their respective departments or faculties demand from them” (p. 73) and fearing use of (O)ER by authors whose reputation is in doubt, or not yet established. Similarly, the quality of OER is a major issue and barrier to adoption for Australian academics (Kandlbinder & Chelliah, 2015). In the study by Hayman (2018a, 2018b) in Ontario (Canada), participants considered OER as reduced cost to learner and quality and an increased supply of high-quality OER developed was identified as one of the needs from academics involved. Furthermore, according to respondent E, OER supporters felt in the need to challenge the “myths” about the use of OER and their quality.
On the other hand, some perceptions referred to the elements that academics considered in (O)ER for being of quality. For Spanish academics, the participants in the survey valued the most the type of resource (text, video, audio, etc.) (55.1%), followed by the reputation of the author of the resource (50.8%) and its availability in the institutional repository (50.3%). In addition, some of them mentioned as an important element the presence of an evaluation or comments about the quality of the resource and aspects that relate to quality of the resource in terms of content and design as well as its pedagogical potential. The survey participants in Germany valued the reusability of the resource in their course first (72.3%), and then the resource type (65.8%), availability in the institutional repository (47.4%) and the reputation of the authors (42.1%). In Australia, the study by Wilson, Myatt and Purdy (2018, p. 6) identified as markers of good quality OER “currency, clarity, relevance and brevity”. In the case of South Korea, faculty members often utilized the evaluation criteria offered by their institution in defining the quality of (O)ER and infrastructures.
Academics’ awareness of institutional quality procedures
In most of the countries studied, a low awareness along with a lack of frameworks regarding quality of (O)ER and their infrastructures (n=8) was highlighted. For example, in the case of UCT (South Africa), while there was evidence of individual lecturers being committed to producing high quality (O)ER, there was no uniform understanding of quality in (O)ER, nor frameworks and processes to ensure quality in (O)ER. In Canada, respondent B remarked that in disciplines where there were regulatory bodies governing professional licences, curriculum developers were always wary of stepping out of/over the line set by the governing body; she suggests, however, that a process ensuring quality, a process of review and assessment, could and should nullify these concerns and restrictions. The study by Hayman (2018b) also found out the necessity for quality assurance for OER as an important theme among the participants. In Australia, Stagg et al. (2018) found that no Australian university had an open licensing policy, open assessment, or a quality assurance framework to support OEP; nor a quality assurance framework for OER. In German survey, focusing only on answers that explicitly sating that they do not know how QA works, it is 26 participants answering with a “I do not know” type of statement; out of 45 participants who provided an answer to this question. In China, participants from eight universities in the survey study of Xu (2018) stated that there was a lack of teaching quality supervision to ensure that faculty members implemented online teaching and (O)ER of a high quality. On the other hand, the report remarked that Chinese scholars had actively proposed different approaches and ways to evaluate the quality of (O)ER, but few documents addressed how the quality of educational resources was evaluated at the learning and teaching level in practice. In South Korea, a challenge pointed out by Lee and Kim (2015) for the active adoption of OCW was the lack of mechanism to ensure the quality of OCW.
Almost all the reports (Australia, China, Japan, Spain, Turkey and South Africa) mentioned institutional / general guidelines for (O)ER quality (n=9), although most of them did not refer to the academics’ awareness. For example, in Japan there were no QA criteria for OCW, MOOCs and other OER at the organizational and national levels; however, CTL or a similar unit within some universities provided general guidelines for OER selection that individual faculty members could refer to when using (O)ER. Takahashi (2018) reported QA guidelines for e-learning design which have been applied in a collaborative online learning project linking five universities in Shikoku region in Japan. These guidelines were developed based on agreement between experts in online learning and instructional design from the five universities and have been applied in QA for e-learning and (O)ER. In Spain, not being the norm, a survey participant mentioned that their university had included as new elements for (O)ER quality in their guidelines: short, clear, multiplatform, use of simple formats (inclusion).
In China, an interview participant who previously held a managerial role at Beijing Open University mentioned that the development of (O)ER by academics had been in accordance with national policies, and therefore the focus of quality evaluation had changed accordingly. According to this interviewee, the online course quality depended mainly on the evaluation carried out before it was implemented, and the feedback received after it was developed. Similarly, in an interview study with the Director of the Academic Affairs Office of a provincial normal university (Hou & Wang, 2012), the Director stated that the National Top-quality Courses program had improved the quality of the faculty members’ offline courses and deepened their understanding of instructional methods and standards. Some faculty members interviewed also mentioned that all teaching staff knew the national standards for each course, and these standardized instructional methods set guidelines for faculty members, to improve the quality of their teaching.
In Turkey, quality assurance of OER practices at micro level functions according to traditional measures in which the individual university and the HEC play the prime function. However, in the particular case of Anadolu University’s AKADEMA platform, academics used OpenupEd Quality Label assessment tool, developed based-on EADTU’s (European Association for Distance Teaching Universities) E-xcellence Label, learner satisfaction survey, and a few other criteria, in order to secure the quality of these courses (AKADEMA, 2020). In South Africa, (O)ER Africa recommended the quality criteria on evaluating an (O)ER developed by British Columbia OER Librarians to be used by lecturers as a checklist. However, little evidence existed to suggest that South African lecturers were indeed using this checklist. In the case of the University of Tasmania in Australia, staff were encouraged to use the Quality Matters framework to review their (O)ER and the ones from their peers (Brown et al., 2013).
Institutional and individual QA agents
In this section, we address the awareness of faculty members concerning the institutional QA agents involved in (O)ER and the academics’ involvement as QA agents in (O)ER at the teaching and learning level (institutional and individual QA agents, n=15).
For instance, in South Korea, the CTL or a similar unit within each university was responsible for defining the quality of (O)ER and deciding platforms and tools for online education and its decision-making follows its own QA process and involved a representative faculty member from each school or discipline. In China, Xu’s survey (2018) showed that a low percentage of the faculty members (30.2%) agreed with the statement that “the university has a teaching team for developing online educational resources”, whereas 25% responded with “completely disagree”. Another survey investigated staff working in MOOC offices, information centers, educational resource offices, and multimedia offices from 50 universities in China (Liu, 2016). The majority of participants (96.15%) believed their offices played a role in overall regulation and planning, and only 23.08% considered that their offices managed or examined formative evaluations of the quality of (O)ER. With COVID-19, different college and universities in China acted in their own ways to ensure quality of (O)ER. For example, Peking University organized a teaching research group, composed of experienced professors, to investigate online courses to ensure their quality. In the first two weeks of the teaching research group, this group attended more than 60 classes.
In other countries institutional agents were part of other services. This is the case of Australia, where the library played a key role in OER development at the Queensland University of Technology (QUT), through an optional stage of quality assurance (Stevens, Bradbury, & Hutley, 2017). The University Copyright Officer or a Learning Designer could provide guidance to educators on the quality or suitability of OER, as well as appropriate repositories.
In Spain, the awareness of institutional QA agents reported by participants in the survey was low, since only a few academic staff mentioned some institutional services; for example, the vice rectorate of digital campus, the teaching department, the course manager/coordinator or the author (teacher/s), the technological or informatics unit, the unit of educational advising, or the library. Another participant mentioned the work towards (O)ER quality of the pedagogical and technological support unit. In Germany, between 30-45% of participants were unsure or did not provide an answer to the question. According to the participants in both surveys, the actors with more influence in their universities to define the quality of (O)ER, their metadata and their repositories are mostly the same academic staff that use them (Spain: 41.2%; Germany: 42.1%). This involvement and responsibility of (O)ER quality was present in other countries too. For example, in Turkey, individual instructors and professors were responsible for the quality of the (O)ER they provided to their students in their courses. In Japan, most universities did not have its own QA guidelines for (O)ER and infrastructures, and therefore, individual faculty members were the ones who make their own QA decisions during the (O)ER creation and selection. In South Africa, it was remarked that while many educators emphasized ensuring the quality of (O)ER, there was no evidence of quality assurance and feedback activities as “personal practices” (Hodgkinson-Williams, Arinto, Cartmill & King, 2017, p. 50).
In addition, in Australia and Spain’s reports, collaboration among faculty members to ensure quality of (O)ER was mentioned. In particular, at the University of Tasmania in Australia there was an institutional expectation that educators were involved in peer review procedures, both for their own work, but also to review the work of colleagues (Brown et al., 2013). In Spain, some participants in the survey mentioned that a collaborative work between faculty members with this respect took place.
In this section we looked at the existence of policies specific to study programs, departments or schools, the academics’ involvement in policy making and the academics’ awareness of institutional policies related to (O)ER.
Institutional specific policies
A lack of institutional policies for (O)ER (n=8) was acknowledged for most of the countries (Australia, Canada, Turkey, South Africa, Spain). No specific policies related to certain study programs or departments or schools, were acknowledged in any report.
For instance, in Turkey the shortage of clear legislation about copyrights for educational use of different resources created a hesitation among individuals to share their resources openly. In addition, the pandemic made the HEC to start developing tactics rather than entire policies to institutionalise OER in the country.
In the case of South Africa, and concretely at Unisa, staff were used to working within strict policy frameworks and the lack of an OER policy could be a contributing factor to the perceived barriers to staff engagement with OER (Cox & Trotter, 2017).
Among the interviewed educators in Canada, respondent E and H stated that there was at that point no guiding institution policy or direction in (O)ER. Respondent F was positive that policy and guidelines were coming to her institution and that, “in their absence, meanwhile, practice is not consistent. Integration is not consistent”. In Australia, a dearth of institutional policies was also noted in the Open Education Licensing Project case studies (Open Education Licensing Project, 2016b), finding that is reflected in the expert survey study: when asked about explicit institutional OER policies or frameworks in the present study, 25 respondents out of 39 (64%) indicated that these are non-existent in their institution. Five people (7.14%) indicated that OER practices have been incorporated in their institution’s current strategic plan, six people (9.48%) indicated that their institution has no plans to consider OER practice in future strategic plans, and fourteen people (20%) indicated that their institution will incorporate OER practice into their future strategic plans.
In the cases where some kind of (O)ER policy was in place (a few institutional cases in Canada and Spain, and in the centralised cases: China, South Korea), some remarks were done regarding the need for policy improvement (n=9).
A clear case of this is South Korea where, despite the emphasis on (O)ER creation and utilization, there was still acknowledged a need to improve policy to further promote (O)ER adoption by individual faculty members. Similarly, in the study survey by Wang and Wu (2013) at Peking University, 153 faculty members over 20 departments argued that policies and mechanisms for motivating faculties to develop (O)ER by protecting their intellectual properties were key to promoting (O)ER, and more policies were needed in these aspects. Along the same lines, Xu’s (2018) study showed that the majority of faculty members in the eight Chinese universities surveyed were in disagreement with the statement that the university had operated well in terms of mechanisms in place for providing support and incentives.
In Turkey, there was a need to develop some policies and walkthroughs to catch quality standards, widen their scopes, and make them compatible with international counterparts. Academics at Unisa (South Africa) must comply with tight policies, “but doing so yields productive results because academics see how they contribute to the broader institutional strategy” (Cox & Trotter, 2016, p. 152). Therefore, the authors pointed out that a strong policy imperative would be crucial for faculty in the context of Unisa to actively embrace (O)ER.
In Canada, three of the interviewees referred to a need for improvement in (O)ER policy at their institutions. For example, respondent B stated that “policy changes must occur so that teachers have easy access to free material”. Respondent C hoped that a new strategic plan at her institution would include a designated person to head up the OER initiative, and respondent G found that “while institutional policy could be both a challenge and a solution, it did not contribute to increased OER use”.
One of the participants in the survey study in Spain elaborated further on this topic regarding the situation in their institution: “There is a policy, but it will have to be improved and more widely disseminated. I do not believe that there is a lack of interest, on the contrary, but there is a lack of time and more measures in the direction taken so that it becomes part of the culture of the institution. Among these measures are all those that facilitate and make it possible to use it among that part of the academic staff interested: time, space, incentives, recognition...”.
Academics’ involvement in policy making
Academics’ involvement in policy making (n=8) was present in some institutions but rather as anecdotical cases, except from South Korea, where individual faculty members were involved in policy making via various committees and internal/external reviews. Major committees for (O)ER included: planning and steering committees of CTL, Cyber education committee and IT committee.
For instance, in China, according to a field study at a university in Nanjing (Meng, 2018), faculty members were invited to attend seminars to give feedback on the policy for calculating their workloads related to using online courses to develop flipped classes, with face-to-face tutorials. The university administrators took their advice and feedback into consideration policy formulation.
In Australia, and as particular institutional case, the OER policy of the QUT approved in 2016 was developed with the input of the University Copyright Officer, Learning and Teaching Unit, Technology, Information and Learning Support, eLearning Services, and various individual academics interested in OER and OEP (Open Education Licensing Project, 2016c). When asked in the expert survey which actors were involved in OER policy making at their institutions, only 30% (n=21) of survey participants provided some level of response. Four participants stated that it was not applicable to them, two participants were unsure, four participants mentioned the Pro Vice-Chancellor’s office or Associate Deans of Education for faculties, one mentioned Learning Design and one mentioned IT. The most involved actors of OER policy mentioned were the libraries and, whilst seven respondents mentioned educators in some respect, it seemed that only “individual academics” were involved, or “individual/small group of educators who are OER champions”. The role of librarians is similarly emphasized in Canada. One of the interviewees (respondent E), a university librarian and point person for OER at her institution, belonged to a provincial working group on OER and was involved in pushing her institution for change and for policy development.
Along the lines remarked in Australia, most of the surveyed academics in Spain and Germany were either not involved in the preparation of institutional (O)ER policies (Spain: 36.3%; Germany: 19.7%) or uncertain about it (Spain: 54.6%; Germany: 57.9%). Regarding the possibilities to influence explicit policies, both Spanish and Germany academics were mostly unsure (Spain: 57.8%; Germany: 64.5%), and without them was more commonly perceived in Spain (26.2%) than in Germany (9.2% against 13.2% that perceived this possibility).
Finally, in the case of South Africa, there was no published evidence of how individuals’ actions informed policy development for (O)ER. However, some referent, individual voices in (O)ER in South Africa were known to have impacted on policy development at their respective institutions (two individuals were mentioned for UCT and two more for the early days of the formulation of a strategy for the implementation of (O)ER in Unisa).
Academics’ awareness of institutional policies
Overall, academics’ awareness of institutional policies (n=7) was low across the countries, with the same exception as before (South Korea). Korean faculty members were well aware of institutional policies related to (O)ER as they were provided with announcements regularly via their department and CTL and received frequent emails and social networking messages promoting the development and utilization of (O)ER.
In Spain and Germany, most of the academics surveyed were unsure about the existence of institutional policies for specific study programs or to department/faculties (Spain: 67.4%; Germany: 56.6%) and only 14.8% in Spain and 17.1% in Germany were positive about this existence. In addition, only 21.7% of the Spanish participants and 25% of the German surveyed stated that there was an institutional explicit policy or regulations concerning the use and/or creation of (O)ER in their universities. Most of the participants were uncertain about this (Spain: 61.7%; Germany: 54%). Slightly some more participants in Spain and some less in Germany stated that there was an institutional implicit policy (Spain: 23.7%; Germany: 22.4%) – against a majority that did not know (Spain: 60.5%; Germany: 57.9%).
In a survey research with 172 faculty members from 8 representative Chinese universities (Xu, 2018), only 33% of the participants were aware of relevant national policies, and 37.2% knew about relevant university policies. The degree to which faculty members were familiar with policies was ranked by participants from high to low, with the following results for different policies (from high to low rankings, from most to least familiarity): (1) course management; (2) changes in teaching methods; (3) course content; (4) course resources; (5) course structure; (6) the training system; (7) the training system for technical personnel; and (8) support for relevant resources. Regarding faculty members’ awareness of how courses or educational resources were managed, Xu’s (2018) survey found that only 10.6% knew that their university had an office that specialized in online courses, and 45.9% reported that that they were not aware of such an office, whereas 43.5% stated that there was no such office in their university. This shows that most faculty members were not aware that policies regarding online courses and educational resources were managed from a central and specialized office.
On the other hand. in a study by de Oliveira Neto, Pete, Daryono and Cartmill (2017, p. 101), most South African (63%) said their institutional policies supported OER, but only a minority had used them (37%). In the study with Unisa staff by Cox and Trotter (2016), the findings showed that although staff could petition the relevant tuition committees to make the works available as (O)ER, most of the staff interviewed were not aware of the possibility and/or the processes.
Promotion of change at the micro level was related directly to different parts of the OER Pyramid Adoption model. In particular, with permission, awareness, capacity and volition. In addition, individual volition had a clear and relevant extrinsic motivator factor: the presence or absence of incentives (see Figure 19).
Academics’ involvement in creating (O)ER and advancing the infrastructures
To describe academics’ involvement in (O)ER, we need to acknowledge different elements that directly affect this involvement.
The first of them is the factor permission (n=8), which refers to the institutional dispositions to which the academics are tied, particularly related to copyright issues and who owns (O)ER developed by academics within the frame of the institution.
In this sense, in the case of South Korea, (O)ER that are copyrighted cannot be used unless permission is obtained and copyright issues are cleared, and NILE’s guidelines on copyrights are to be followed in developing a MOOC. However, in Turkey, copyright policy is one of the biggest challenges of OER related library services and repositories, but also in OERs and resources in general. The current Law of Intellectual and Artistic Property Rights (1951) had revised and included two articles (33th) in 2001 and (34th) that were related to use of resources created by others. According to these articles, as long as the creators were cited the resources could be used for not-for-profit educational processes. However, especially in the 33th article, it was clearly indicated that these could be used in face-to-face educational processes but nothing about open and distance learning.
On the other hand, the ownership of (O)ER is a crucial aspect within the factor permission. For example, in South Africa, and particularly in Unisa, the institution owned all the intellectual property of work by staff members created “within the normal courses and scope of their employment” (Cox & Trotter, 2016, p. 154). However, unlike other South African universities (like, e.g., Unisa), “UCT academics are allowed to possess the copyright of their teaching materials and thus turn them into OER” (Cox & Trotter, 2016, p. 152). In Canada, respondent A explained that part of the challenge to adopt OER is that at her institution, created material belonged to the institution, thus inhibiting some instructors from creating their own OER. Their contracts prevented them from seeking a Creative Commons license for their products. Similarly, in many Australian higher education institutions, teaching and learning resources “are traditionally closed to those outside of a course or unit, and ownership is retained by the university – the lecturer must seek policy approval to release course materials outside of the institution” (Stagg & Partridge, 2019, p. 479). For example, the QUT had an institutional Intellectual Property Policy, which incorporated the use of OER. This policy stated that before staff were able to release teaching and learning materials as OER, they must first receive approval from their Head of School and then seek approval from the Deputy Vice-Chancellor (Technology, Information and Learning Support).
The second factor that affects teachers’ involvement in (O)ER actions is awareness (n=18), which refers to the degree of knowledge that faculty members have concerning (O)ER, OER and the philosophy behind (openness). Most of the countries have related statements that refer to either high or low awareness, being the latter more common.
For instance, in Turkey there was low awareness regarding the philosophy of openness and the way scholars perceive openness. Shortage of awareness about Creative Commons like licensing methods is also related to this barrier. Similarly, in South Africa, the research carried out by de Hart, Chetty and Archer (2015) found that participants had a limited understanding pertaining to IP and open licencing formats and processes. A further interesting finding in the study by de Hart, Chetty and Archer (2015) was that the respondents indicated generally that they were prepared to share their own work only if they could make money from this. This showed that many of them did not understand or comprehend the ethos of OER. A lack of knowledge about OER was also remarked by the study of Chikuni et al. (2019) in four South African universities. In addition, there was also evidence that in the context of South African universities, individuals’ understanding, awareness of, and engagement with (O)ER were also shaped, and in many ways determined by their institutional location (e.g. role, department, discipline) and institutional environment (the character, values and mission and vision of the institution) (e.g., Cox & Trotter, 2016).
In Canada, the low response rate to the eCampus Ontario study on OER (Hayman, 2018a) was interpreted as a correspondingly level of interest or knowledge of the OER topic. Some concrete answers from participants in the survey in Spain brought light into this topic and referred concretely to the OER infrastructures. For instance, one instructor stated that “my university's repository has only been functioning for a few years, it still lacks dissemination, awareness, human resources and time to consolidate itself as a tool for everyday use”. A second one stated that faculty members’ awareness of the importance of uploading the used/created (O)ER in the institutional websites had been increased. The fact that the (O)ER transparency, sharing (O)ER (and use of internal repositories), had been improved was mentioned by several faculty members in Spain.
In China, the study by Zhang, Zhao and Li (2015) pointed out that faculty members’ lack of understanding about the standards of (O)ER and open courses was problematic, because their perceptions of the universities’ policies and standards of (O)ER could influence their (O)ER practices. Along the same lines, the most relevant challenge for faculty members highlighted in the study by Li and Li (2012) was lack of awareness (41.5%).
Australian educators’ limited knowledge about OER, as well as licensing issues, was mentioned as a barrier at QUT and Swinburne University (Open Education Licensing Project, 2016a; 2016c). Similarly, concerns around staff knowledge of copyright and intellectual property policies were still abound among OEP grant holders (Stagg & Partridge, 2019). As Bossu and Meier (2018, p. 5) highlighted, there was still “an overall lack of understanding about the use of open licences and instititional practices in terms of copyright permissions”. The small-scale study by Bossu (2015a) at the University of Tasmania in medical education showed that a significant number of students and staff had a limited awareness of OER and Free Open Access Medical Education (FOAM); and therefore, reinforces the previous statements about academics’ OER awareness. However, results from the study by Bossu, Brown and Bull (2014) showed high levels of OER awareness and knowledge by participants. Similar findings were obtained in the expert survey, with 83% (n = 48) of the participants having previously heard of OER.
High levels of awareness of OER among a majority of the respondents from the four-year institutions were also found out in a large-scale survey with educators from Japanese higher education institutions between 2015 and 2016 (Shigeta et al, 2017, p. 199).
Capacity (n=11) is the third relevant factor in teachers’ involvement with OER. Most of the countries highlighted some shortage of academics’ digital skills and emphasised the importance of institutional professional development support. This latter will be described in the next section.
Concerning academics’ digital skills, for instance, in a survey of teaching and research staff at Unisa (South Africa), Roberts (2016) found that the respondents’ perception of their own ability to be technically sound, was very low and that training in this area was required. Similarly, a shortage of digital skills among Turkish educators was also noted as a barrier for (O)ER in Turkey. In Canada, according to respondent E, the lack of technical skills was an element that impeded some educators. This was also an important theme in the study by Hayman (2018b), which highlighted the need for educators to be better educated in OER-related skills, such as finding appropriate materials.
Similarly, in Australia, different studies have shown findings related to academics’ capacity. For example, some of the lecturers at the UTS cited a lack of knowledge about video creation as a reason behind their lack of resource creation (Kandlbinder, 2015). Ward (2015) pointed out that the move of academics at Charles Sturt University to offer two courses on OERu involved a lot of technical and learning design capabilities, which not all educators had. These examples suggested the need for developing educator digital capabilities, such as in a case study by the Higher Education Standards Panel (Ewan, 2016), which raised “the need to develop the abilities of academics to select and curate content from multiple sources” (p. 2). Different authors had further acknowledged that educator capacity for OER creation and adaptation remained an area requiring improvement in Australian higher education (Stagg, 2014; Udas et al., 2016).
In China, the lack of skills was considered a challenge for faculty members to use OER in 24.6% in the study by Li and Li (2012). Interestingly to remark in the Chinese context was a comment from an interviewee from the BNU Centre of Information & Network Technology. He stated that the most considered important factor that impacted on the implementation was IT literacy among leaders and administrative staff who were involved directly in digitalization work at the institutional level. Similarly, a challenge pointed out by Lee and Kim (2015) for the active adoption of OCW in South Korea was the lack of technology competence at both faculty and institutional levels.
The last factor included here is volition and concerns the desire of academics to create, use, adapt, remix and share (O)ER when referred to individual volition (n=44), but there could be social and institutional volition too. Different elements that were identified within individual volition are: costs benefit, precarious employment situations, lack of time, concerns related to copyright, perceptions of (O)ER, and attitudes related to sharing (O)ER. We also identified a few elements within institutional and social volition, as will be described later.
The first element described here related to individual volition was identified in some countries where teaching materials (and particularly, textbooks) were expensive for students: costs benefit (n=4) as a motivator for academics to publish their materials as OER, in most cases referring to open textbooks. For instance, academics at UCT (South Africa) were aware of the challenges related to the cost and utility of traditional textbooks and were therefore experimenting with new approaches towards resource creation through open practice (Cox, Masuku, & Willmers, 2020, p. 5). In Canada, one of the most common themes mentioned by the interviewees was costs benefit as a motivator for OER. For example, respondent A (an OER leader at her institution) especially liked to avoid textbooks for her students due to their high cost and, therefore, used OER. Another interviewee (a professor from a technology-focused institution) moved away from “licensed, subscription-based homework/assessment” (#OERThankU) systems to open source and one of his reasons was the cost of resources. The importance of costly textbooks as a driver for OER was also an important theme among the findings of the study by Hayman (2018b).
A first barrier in individual volition for (O)ER was identified in Canada and Spain with the employment conditions (n=3) specific types of academic ranks / institutions involved. Concretely, in Canada, respondent E stated that faculty in temporal positions did not want to lose what they saw as “leverage” as regards their own materials; existing a sense of possessiveness and insecurity. In addition, a finding regarding this barrier by Hayman (2018b) referred to a situation of less autonomy in the case of college educators with respect to course decisions than university educators. Similar remarks regarding faculty in temporal positions were mentioned by different participants in the expert survey in Spain, who pointed out the need to improve the working conditions of adjunct professors that already suffered from an excess of unpaid work, and therefore, the creation of (O)ER would be added to this work.
Lack of time (n=14) was considered a second barrier in individual volition for (O)ER, usually related to the already heavy workloads that academics experienced. For example, in Turkey, shortage of time for creating (O)ER along with a heavy workload were highlighted as barriers for the educators. In South Africa, the research by Cox, Masuku and Willmers (2020) remarked that “there is still a cost involved in the production and ongoing delivery of open textbooks, particularly in terms of the time required on the part of the academic to author, format and publish these resources”. In Canada, diverse interviewees mentioned lack of time as a barrier for (O)ER. For instance, respondent B did not have the time to focus on creating materials and it neither helped that there were staff shortages at her institution at the point of the interview, since this increased her workload. A second interviewee, a communications professor at a technology-focused institution, admitted that creating OER was labor-intensive. Not only OER creation but also locating OER was related to lack of time. For example, respondent C said that it was a lot of work, and time-intensive, to find what she needs. Similarly, respondent H identified time to locate the right resources also as a challenge. In the study by Hayman (2018b), the lack of time to use OER (revise, adapt) were one of the reasons for general confusion about OER in potential users. In addition, the study found that f2f faculty members had less time to devote to learning how to use OER than online faculty members. Similar remarks were found in the reports of Spain and Australia. Several Spanish faculty members in the expert survey mentioned the lack of time as a barrier for (O)ER; for instance, one stated that it was very time consuming to create (O)ER, and another highlighted that “there is a big problem today with the availability of time for the work of the academic staff”. In the German study by Schmid et al. (2017), instructors indicated to (rather) agree with lacking the time to search for OERs and judge their quality. In the study by Kandlbinder (2015) at the UTS (Australia), lack of time was one of the reasons to explain why lecturers of large classes did not create additional materials beyond lecture recordings. Lim, Kim and Choi (2017) identified as barrier to South Korean faculty involvement in the creation of (O)ER the heavy workload put onto individual faculty members in creating all course materials. On the other hand, although one of the challenges for faculty members to use OER in the study by Li and Li (2012) with Chinese academics was lack of time, this was not a major reason (9.2%).
In some countries, faculty members expressed concerns related to copyright (n=6) that could be considered an additional barrier to (O)ER. For example, in the study by de Hart, Chetty and Archer (2015) in South Africa, concerns about the copyright and legal considerations were manifested by Unisa staff. In Spain, the concern about the management of the (O)ER author rights was present in 21.1% of the participants in the expert survey (multiple selection was possible), and in Germany it was the most ranked option regarding the challenges of (O)ER and their repositories (9.2%). In Australia, the study by Bossu et al. (2014) exposed as main concerns behind people at their institution not developing and/or re-purposing OER the potential loss of intellectual property and the fear of exposure. In Canada, interviewee E stated that faculty member at her institution were hesitant to create their open content because they do not own it. On the other hand, interviewee H stated to be sensitive to legality and copyright and wanted to “stay within the rules”, but wished the rules featured more openness and access.
An element that may act either as a driver or barrier for (O)ER were academics’ perceptions of OER (n=10). As drivers we could identify the following:
- Participation in (O)ER is important (South Korea). It was a general perception of many faculty members that their participation in (O)ER was important for the university as well as for their own teaching.
- OER as a form to enhance the own reputation and for personal fulfilment (South Africa). Evidence suggested that “personal motivation, especially the desire to enhance one’s reputation, underpins some educators’ practice of creating and sharing teaching materials as OER”, as well as feelings of “personal fulfilment and confidence” and educators seeing their participation in (O)ER as “a way of asserting an epistemic stance, or one’s own unique (individual or collective) perspective of knowledge” (Hodgkinson-Williams, Arinto, Cartmill, & King, 2017, p. 586).
- OER as a way to improve teaching (Australia, Canada, Germany, South Korea). In the survey by Bossu et al. (2014) with respondents from Australian universities, remarked benefits for OER were their possibility to improve the quality of educational materials and their helpfulness to enhance quality of teaching and learning in HE. An interviewee in the Canada study (a professor from a technology-focused institution) explained as one of the reasons to move to OER was the understanding that there was a “better way” to engage students in their learning. The study by Jung and Lee (2020) with Korean faculty members revealed that academics were using (O)ER rather habitually with an expectation of their teaching performance improvement. German instructors in the study by Schmid et al. (2017) (rather) agreed that OER help them to prepare for their teaching and that OER enrich their courses.
- OER as a time-saver (Australia). Among the benefits of OER in the survey by Bossu et al. (2014), respondents highlighted that educators could save time and avoid duplication of effort.
- OER leads to institutional innovation (Australia). The majority of participants who answered questions to rate various statements about the use of OER in HE in the Australian expert survey agreed that using OER leads to institutional innovation (34.29%) and that the adoption of OER promotes the sharing of knowledge and the university service mission (36.71%).
Among the barriers for (O)ER in terms of academics’ perceptions, we could highlight the following:
- Openness and OER related concepts as equivalent to free resources with low quality (Turkey). Such a perception by academic circles at individual and institutional levels inherited the developments of OER related issues in Turkey.
- (O)ER are incompatible with traditional instruction (China). The survey by Xu (2018) found out that 69.8% of the participants considered online educational resource or courses incompatible with traditional instruction. In online resources and courses, the methods of organizing content, scheduling time and making the course plan are different from those required for face-to-face teaching, and ineffective coordination significantly impacts the effectiveness of online teaching. Additionally, the lack of a scientific approach to resolving the tension between fragmented learning and integrated disciplinary knowledge was an important factor that accounted for the current problems in developing (O)ER online.
Another element related to individual volition that was present in some of the reports and that we wanted to analyse separately were the attitudes related to sharing (O)ER (n=11). Some of them relate to the concerns regarding copyright issues. For example, Educators at Swinburne University (Australia) reported creating videos and MOOCs as OER, with some expressing the desire to share their videos, but also to license them correctly, so as to not to allow their being “cut up” and redistributed (Swinburne Commons, 2015). In Turkey and Canada, similar concerns were reflected in the reports. In Turkey, educators hesitated to open up their materials either because they thought that others might use them for their own use (either for profit or not) and would not cite them, or because of fear of getting evaluated or criticized by their peers/colleagues. In Canada, two interviewees showed attitudes related to sharing OER in their institution. Respondent E indicated that faculty members at her institution would share OER within limited parameters – e.g., within their own department or with close colleagues. Similarly, respondent F found that sharing resources was dependent on personalities and collegiality. However, findings from the study by Hayman (2018b) showed different results: she found that her study participants were willing to take risks, were excellent curators of open content and happy to share their intellectual property, including via social media. In Spain, a survey participant indicated that sharing (O)ER was more a particular initiative of academic staff, rather than institutional encouragement. In Germany, a second reason provided by survey participants not to use the institutional repositories was not wanting to publish (O)ER (7.9%, the question was only shown to the participants that had answer negatively to the use of institutional repositories) and, as in Canada, sharing (O)ER happens commonly in confined spaces (email, LMS) (Schmid et al. 2017). In South Africa, there were contradictory findings related to this topic: on the one hand, the study by Hodgkinson-Williams, Arinto, Cartmill and King (2017) found that there were also concerns by educators about the quality of their own work and to open their work for peer assessment, and evidence that educators would use existing (O)ER to benchmark the quality of their own work. On the other hand, the study by de Hart, Chetty and Archer (2015) at Unisa found that “respondents were apparently confident about the quality of their [(O)ER] offerings and were not concerned about their work being subject to scrutiny by others” (p. 33).
Concerning social volition (n=11), we refer to the social environment of the faculty members (department, other faculty members, colleagues) and how interested they are to be involved in (O)ER processes, but also the influence this operates on individual volition through modelling or social desirability. An element that stood up in the reports from Canada and China was the presence OER forerunners as advocators of OER and inspiration for colleagues at their institutions or even in a broader level. Many of the interviewees in Canada were identified as OER pioneers, champions or forerunners in their institutions (respondents A and F) and, some of them, even in the province and/or in the whole country (respondent B). For example, respondent A championed OER’s foothold at her institution, and organized an “Open Day” in 2018 which served as a catalyst for raising interest among her colleagues. Two years later, there were 12 courses offered with open texts. Respondent F was a strong believer in OER and an advocate within her large university. In response to student demand, she had developed online programs in her field and various OER resources that respected the diversity in voice. She was committed to keeping available resources up to date and had created mobile OER resources that were being widely used in her field. Another respondent, a communications professor at an Ontario HE institution and creative creator of OER, credited librarians with being “rockstars” in the OER world. The importance of OER champions was constated by respondent E: “strong champions are needed to push OER acceptance forward”. In Japan, social influence from peers was highlighted as more important than performance improving in adopting OER (Jung & Lee, 2020). In China, most interviewees in the study by Hou and Wang (2012) thought that top-quality courses were used mainly by peer colleagues from the same university or other universities. As an example, one faculty member from University A, whose course was a National Top-Quality course, mentioned that many universities used the course materials they produced, and that this was very helpful in order to broaden the influence of their course in the country. Faculty members from other universities had contacted them about how they produced their resources and some faculty members even visited their university in person to learn more about the course. In addition, another interviewee’s course was very influential. He believed that his National Top-quality course had played an important role in improving the standards of instructional design and had had an impact on similar courses in many domestic universities. His course had an extremely high usage frequency, because it was also offered by other universities, and many faculty members browsed his course resources or contacted him when developing their own courses (Hou & Wang, 2012). Another element that should be considered as part of social volition is the resistance of faculty members to (O)ERs; this was remarked also in the report of Canada. Respondent D (a university teacher) did not know anyone who used OER but knew that some of her colleagues balked at the idea and were suspicious of open resources. Similarly, respondent E reported that faculty members were reluctant to use or trust repositories and respondent B (OER champion) had found faculty at her institution to be resistant to OER. A further interesting remark for social volition is the one made by respondent G, who stated that his experience at his institution had shown him that younger graduate students who assisted him in teaching were more receptive to creating and sharing materials, although they could be shared more widely, than older professors, who were not as open to sharing.
Institutional volition (n=22) is another factor related to development and adoption of (O)ER, which refers to the interest of the institution to push (O)ER forward. A main element are incentives, which are described in the next section. In some countries, institutional encouragement, commitment or requirement were noted also as important factors.
With regards to institutional encouragement (n=12), we found both presence and lack of it in the different reports. It was present in the reports of Australia, China, Japan and South Korea. The second most frequently mentioned activity in Australia concerning OER projects or initiatives in the expert survey was the promotion of using OER (n=3), next to the institutional library (n=6). A concrete measure of encouragement was highlighted by a participant in the same survey: “unofficially, slight pressure is being applied to lecturers at a very low level to encourage them to consider open textbooks as a cost-reduction measure for students”. An example of an effective OER information dissemination strategy was an email from OER project coordinators to all staff in the faculty, which prompted increased engagement (Lambert, 2015). However, as pointed out by Kandlbinder and Chelliah (2015, p. 3), “the encouragement to use OERs is only likely to succeed if there is an institution-wide approach to that makes adoption and modification of OERs more attractive to subject coordinators”. Furthermore, support from senior administrators and at the national level, would also assist institutions in making changes towards further OEP development (Bossu & Meier, 2018; Bossu & Stagg, 2018; Open Education Licensing Project, 2016a, 2016b). In Canada, there were a couple of examples if this institutional encouragement explained by the interviewees. For example, respondent F stated that her institution was set up for cross-collaboration among areas where quality assurance experts, technical experts, and the teaching and learning center all united to create OER and encouraged its development. Another interviewee (a professor of a technology-focused institution) said that his university strongly encouraged the OER movement and had in place designated OER website as well as a “steward” program. In 2019-2020, the steward program had 19 members. This supports the finding from Hayman (2018b) that when OER advocates and educators from a variety of institutional roles were given encouragement and opportunities to share their knowledge, use of OER increased. In South Korea, this encouragement is even clearer, since individual faculty members were strongly and continuously asked to engage in the creation and utilization of (O)ER by their university. In the Japanese context, where (O)ER had been developed mainly at the institutional and individual faculty levels, member universities had encouraged individual faculty members to create OCW and MOOCs and linked them to JOCW and JMOOC sites. Finally, the case of a university in Nanjing (China) stood out: no matter how expensive the online course was, the universities provided sufficient financial support and encouraged the development of more such quality materials (Meng, 2018). Lack of institutional engagement was noted in the reports of Turkey and Spain. Turkish educators were not encouraged to produce and share OER at the individual level, but in contrast, they were encouraged to use institutional subscribed services. Similarly, in Spain, some participants noted that (O)ER repositories had very little visibility in general, and especially, that its reuse was not encouraged.
The institutional requirement or invitation (n=4) to create/use/share (O)ER and/or use the institutional repositories was another element identified within the institutional volition. For instance, an interviewee in the expert study that held a managerial role at BOU (China) stated that all the faculty members were required to curate one course online which would be integrated into their university learning portal and that would be reviewed by a university management group. On the other hand, the use of OER in Canada was best framed as an invitation for educators to explore and not as a requirement of practice (Hayman, 2018b); however, in exploring what the future of HE in Canada looked like, recommendations for policymakers highlighted the value of openness, and suggested “open” becoming a requirement. In Turkey, no requirement to integrate technology (lack of demand from students and administrators) into the courses was considered as a barrier related to institutional volition.
Finally, we would like to remark institutional support (n=6) as a third element within institutional volition. Although there was a signal of this support in the report of Australia, the reports of the other two countries (South Africa and Turkey) referred to the lack of it. So, in the expert survey from Australia, some academics indicated that their university was “overtly committed to transforming 2 degree programs each year into fully OEP over the next 3 years”. However, the study by Bossu et al. (2014) showed that despite the recognition that OER could assist improve the quality of teaching and learning, participants identified the lack of interest in creating (good quality) OER and the difficulties involved in changing academic culture. Additionally, only six participants (8.57%) in the expert survey either agreed or strongly agreed that ‘teaching strategies promoting the use of OER are supported in [their] institution’, and only seven (10%) agreed or strongly agreed that the ‘adoption of OER is supported in [their] institution’. In a study at four South African universities (Chikuni, Cox & Czerniewicz, 2019) a noted challenge for OER was the lack of institutional support. In the case of Unisa (South Africa), it was the institution , who at the end, “is in fact the unit of agential analysis regarding the ‘creation’ side of OER adoption” - “the institution would need to decide whether it wanted to openly license and share the teaching materials that it holds copyright over” (Cox & Trotter, 2017). In Turkey, administrators’ hesitation for use of OERs created by others was considered as a barrier for dissemination of OERs and OEPs related to institutional volition and, in particular, to institutional support.
Support for academics in (O)ER creation
Academics were supported in creating, sharing, using and remixing (O)ER and using their repositories by two main elements at this level: institutional professional development support (addressed at improving capacity) and the use of incentives (targeted at increasing individual volition).
In terms of institutional professional development support (n=28), most of the reports mentioned different forms of this support and training. We incorporated institutional training and support as two different professional development offers, as well as the academics’ awareness of professional development institutional agents.
Concerning institutional training (n=13), there was a wide range of possibilities, including workshop, information and training sessions or orientation programs, among others. For instance, at the UCT, in South Africa, support for faculty members included regular OER workshops and training sessions held by CILT and getting legal advice when licensing their materials as OER (Cox & Trotter, 2016, p. 153). The case of South Korea was special, since most universities had been implementing a policy which required all newly hired faculty members to receive an orientation program. During this orientation program, the new faculty members were provided with information about the (O)ER and their infrastructures available for their use, and offered opportunities to develop skills to use the institution’s LMS, educational software, MOOCs, and other technologies, and learn about flipped learning, blended learning, copyright issues and more. Campus-wide organisations or teams concerned with teaching and learning matters in Japan provided faculty members with information and training sessions regarding the creation and utilization of (O)ER. For example, the Center for Promotion of Excellence in HE at Kyoto University offered frequent (optional) faculty development workshops on how to utilize (O)ER for effective teaching and learning (Fujioka et al., 2019). In addition, it ran an online training program called the Mutual Online System for Teaching & Learning to help faculty members develop effective teaching and learning strategies including the use of various kinds of (O)ER available for HE (Fujioka et al., 2019; Kubo, 2017). In Spain, participants in the survey mentioned that they used more often the faculty professional development offer as results of the COVID-19 remote teaching, and that to create (O)ER they needed pedagogical and technical support. The increase of (O)ER use and creation’s competence by faculty members and the relevance of online training was mentioned by others too (not about sharing or the evaluation of (O)ER quality). Another participant mentioned the institutional use of peer mentoring. In Germany, 39.5% of the participants stated that support for professional development for (O)ER was a measure applied at their institutions.
In Canada, respondent A said that one day each semester there was an OER information session at her institution’s teaching and learning centre. The 2017-2019 Canadian Digital Learning Research Association survey data show that institutions offered an extensive variety of practices to provide instructors with professional development and support; for example, workshops, one-to-one interactions and group professional development opportunities. Participants in the study by Hayman (2018b) sought and valued professional development for OER use and among its recommendations we could find that “advocates and interested educators may benefit from social professional development experiences where they can communicate and support each other”. In Australia, a free, open and online professional development course about curriculum design as a micro Open Online Course was developed within a joint project between the University of Tasmania and the USQ (Bossu, Fountain, Smyth, & Brown, 2016).
In China, the study by Xu (2018) found that faculty members mainly lacked an understanding of training programmes and policies regarding evaluation that might act as incentives for them to develop more resources. Regarding information technology training, only 14% of faculty members in the study by Xu (2018) responded “completely agree” and 16.9% responded “agree” to the statement that “the university conducted special training and assessment for faculty members”, indicating that IT support for faculty members to develop online resources and courses was not sufficient. During the first moments of the emergency teaching due to COVID-19, as an exemplary case, the Centre for Excellent Teaching and Learning at the PKU held a training programme for faculty members very quickly in two stages with the idea that the first priority was to solve technical problems to ensure the development of online teaching and then to increase the instructional design and strategy gradually to improve the quality of courses (Gong, 2020). In addition to these, regular training for young faculty members and teaching assistants was still continuing, also focusing on online learning and teaching.
With respect to institutional support (n=7), most of the references in the reports were related to technical support and, in some cases, also pedagogical support. In the institutional case of the University of Tasmania (Australia), the Strategic Plan for Teaching and Learning 2016-2020 outlined a focus on supporting staff in planning and development of OER. In the Australian expert survey, participants recognised the need to provide specific skill support for the development and use of OER. In the example case of PKU in China, at the beginning of remote teaching and to promptly solve the faculty members’ problems in online teaching, the teaching centre opened a hotline, a consultation mailbox and a WeChat group to provide 24/7 support (Gong, 2020). Technical support from CTL or a related unit was among the most common incentives, along teaching load and assignment of teaching assistants in Japanese universities. In, Spain, support measures most commonly applied in institutions according to participants were technical support (42.2%) and training support (39.3%). However, this support was variable depending on institutions; for example, a participant stated that only technical support was offered and was extremely limited in its conditions (to produce short videos with a standard format). Technical support (52.6%) was the most common institutional measure highlighted by participants in the German survey.
We could also identify the academics’ awareness of professional development institutional agents (n=4). For example, in the study by Bossu et al. (2014), Australian interviewees expressed a lack of knowledge of exactly which staff members at their institution could help them to learn about licensing and the development of OER. In the expert survey, a number of participants mentioned being supported by their libraries (6 participants mentioned them as OER initiative), including workshops and sessions conducted by them or by OER working groups. Another participant reported that each college in their university had a full-time learning technologist assigned to it for support and advice that takes place at an informal level. In Canada, respondent F stated that she had support from the teaching and learning centre and from their librarians at her institution. In Spain, a participant in the survey mentioned a specific unit in their university for the creation of (O)ER that worked pretty well and made annual calls, and offered technologies for their production, as well as a series of annual courses.
On the other hand, some reports highlighted a lack of professional development support (n=5). In particular, a shortage of modern and structured professional development opportunities as well as faculty support was identified as a barrier for the educators in Turkey in terms of promoting capacity. In China, faculty members stated that they had yet to be given the type of administrative and technical support that should be provided by professional teaching teams in the study by Xu (2018). In South Korea, a challenge pointed out by Lee and Kim (2015) for the active adoption of OCW was the lack of support for the development and location of quality OCW. In Spain, the lack of support mechanisms to use (O)ER repositories was one of the reasons not to use them (25.6%) given by participants in the survey. Similarly, the study by Hayman (2018b) in Canada exposed the lack of support resources (staff) to help academics with OER as a reason for general confusion about OER in potential users.
Incentives (n=29) were important measures to support change for (O)ER at the individual level. Diverse kind of incentives were addressed in the reports: assignment of teaching assistants, measures for recognition and faculty evaluation, reduction of teaching load and monetary incentives.
Concerning recognition and faculty evaluation (n=8), Australia’s report cited it in three occasions. For instance, it was noted that of the three approaches used in the Learning2014 strategy – recognition, reward and monitoring progress – it was recognition that had the greatest impact on early adopters’ decisions to make a change to their teaching practice” (Kandlbinder & Chelliah, 2015, p. 3). Similarly, in a study at USQ, those responsible for OEP had admitted that they “have learned that open practice by academic staff needs to be an individual decision but the University can reward and recognise open behaviour and support experimentation” (Udas et al., 2016, p. 338). Also as an institutional case in Australia, the Strategic Plan for Teaching and Learning 2016-2020 of the University of Tasmania outlined a focus on integration of recognising contribution to OEP through the Teaching Performance Expectations. In China, an interview participant (former manager at BOU) stated that BOU had policies to award ‘high-quality courses’ and ‘teaching excellence’ to faculty members who proved themselves able to create high-quality educational resources or who demonstrated excellent instructional designs in their courses every year. Some Turkish HE institutions, like Anadolu University, would give at least 50 percent more points to each article or book or any scientific work published in open access journals or resources. Along with monetary support, additional points for faculty evaluation were the second most common incentive measure in South Korea. Two participants in the survey in Spain highlighted specific recognition measures as incentives in their specific institutions: a higher job stability, and faculty evaluation and accreditation.
The measure of reduction of teaching load (n=2) was one of the concreted in the case of a university in Nanjing (China), where faculty members who participated in developing of online open courses received also a certain workload subsidy (Meng, 2018). Faculty members were entitled to three times the workload of traditional face-to-face teaching in the first round of the flipped class, 2.5 times in the second round, and twice in the third round and beyond. In Japan, monetary incentives were not common in Japanese universities, but there might be a reduction in teaching load and assignment of a teaching assistant (TA) (n=1).
Some forms of monetary incentives (n=13) were present in some of the reports, often as part of institutional cases. For example, in the Learning2014 project at the UTS (Australia) (Kandlbinder & Chelliah, 2015), learning and teaching grants were offered to participating academics, which were primarily used to pay learning technologists to develop resources, rather than for academics to develop resources themselves. A special category was also created in the annual Vice Chancellor awards. Another institutional case in Australia is USQ, which had annual Open Educational Practice Grants, with one granted in 2019 for Open Assessment and two for open textbooks. Only one participant in the expert survey mentioned the existence of an annual grant for OEP that academics at their institution can apply for, in order “to transform a subject they teach”. In Japan, monetary incentives were provided to faculty members who created and implemented the courses in the particular case of JMOOC. In Spain, a participant in the survey mentioned that in their institution there were at the point of the study incentives for shared creation and use of (O)ER without further concretion. In Canada, according to respondent F, the province-wide eCampusOntario had been useful in providing funding for (O)ER. Furthermore, the pandemic – in addition to causing a rapid shift to digital education - had led to increased calls for the use of OER in Canada. In Spain, a respondent referred to the Award of teaching innovation of the social council and the Call or program for the production of (O)ER at their institution. In South Africa, the UCT “provision of OER grants by the Centre for Innovation in Learning and Teaching (CILT)” was a support measure for academics (Cox & Trotter, 2016, p. 153). Where monetary incentives were rather common was in China and, especially, in South Korea. In China, the institutional case of a university in Nanjing (Meng, 2018) as an example described how the university provided financial support for open courses, which included all course-related expenses. In South Korea, monetary support for course/material development for faculty members was one of the major incentives related to (O)ER: as (O)ER creation and utilization was important at the institutional level, faculty members who decided to develop (O)ER were well supported with (O)ER development grant and technical and administrative support. However, insufficient financial incentives for faculty members to continuously share their copyrighted work such as textbooks and other materials from their lesson plans was noted by Lim, Kim and Choi (2017) as a barrier to faculty involvement in the creation of (O)ER.
In some reports (Canada, South Africa and Spain) it was referred to the lack of incentives too (n=7). For example, in South Africa, the lack of formal institution recognition for (O)ER was emphasized at UCT (Cox, Masuku & Willmers, 2020) by reporting that “the lack of institutional reward for open textbook development was compounded by a lack of support for the textbook development process, a lack of established quality assurance mechanisms and a lack of funds to buy out academics from their teaching commitments” (p. 6). In Canada, some of the interviewees pointed out the lack of incentives in their institutions; for instance, respondent B mentioned that there was no compensation for her “extra” work and she must complete it as volunteer labour. Respondent C stated that she received no consideration for creating or searching for OER in terms of funding. In the survey with Spanish faculty members, the 21.1% of the participants indicated as a reason not to use the institutional (O)ER repository the lack of any compensation for its use. In the German survey, 43.4% of the participants stated not knowing any measure for promoting (O)ER in terms of incentives at their institutions, and 6.6% stated a lack of incentives for using institutional repositories (the question was only shown to the participants that had answer negatively to the use of institutional repositories).
(O)ER sharing, integration and remixing by academics
(O)ER practices (n=24) by faculty members in the reports show that (O)ER uptake is overall in its infancy, which could also be explained by factors identified in the previous sections. Although many of the reports mentioned (O)ER use and development, much less space is devoted to describing practices beyond these activities (e.g. sharing, remixing, integrating). However, some of these could be identified within the above-analyzed factors.
For instance, it is remarkable the case of South Korea, where despite consistent professional development and support for both new and experienced faculty members, only a few faculty members were engaged in (O)ER development at the individual level. Among the barriers to faculty involvement in the creation of (O)ER, the continuous use of the created (O)ER without an opportunity for revisions or updates was identified by Lim, Kim and Choi (2017). Similarly, in the study on OER by Shigeta et al. (2017) in Japanese HE institutions, 13.6% of the four-year institutions, 2.3% of the two-year colleges and 3.7% technical colleges developed OER, and 13.4% of the four-year institutions, 8.6% of the two-year colleges and 14.8% technical colleges reported to utilize OER. Regarding this, it should be noted that in both Japan and South Korea, OER were often used as supportive technology to provide quality content and offer learner-centered materials rather than disruptive technology to offer expanded and extended learning opportunities and improve sharing and networking. In China, 75% of academics in an interview study at the Northwest Normal University chose to design courseware either by themselves or by integrating teaching materials by themselves, whereas the others adopted multimedia resources either from CDs, websites, QQ sharing or WeChat (similar to WhatsApp) (Li, 2015).
The study from de Oliveira Neto, Pete, Daryono and Cartmill (2017) found that “South Africa – the most economically developed country by GDP per capita [...] had the lowest rate of instructor OER use compared to Ghana and Kenya” (p. 83) and that gender, age, digital proficiency, or qualifications of instructors played a significant role in instructor’s use of (O)ER in South Africa. Furthermore, de Hart, Chetty and Archer (2015) in their study with Unisa staff found that “although there is knowledge and understanding of OER, this has not been converted into active participation” (p. 18) and that “activities relating to the use of OER (accessing, redistributing and re-using) are far more frequent than activities relating to contributing to OER (revision, remixing, developing)” (p. 32).
In Canada, concrete OER practices were shown by respondents H, F and C. For instance, respondent H used search engines when looking for fast access to resources -libraries were thorough but often the search took too long. He revised material to suit his needs, looking for ways to effectively present the “found” resources and also created his own repositories of found resources for future use. Respondent F was committed to providing easy-to-access resources, which were also revised, adapted, and translated for international use as both an app and in web-based format. Respondent C had been an avid OER creator for several years and co-created together with her students a textbook with eCampus Ontario, then published via PressBooks. Furthermore, she would rather invest the time in adapting materials to her own needs than re-invent the wheel. In terms of OER sharing, she and her colleagues did it for relevant material, “the good stuff,” in its original format, often by email. So far, their networking was casual and unofficial, led by “unofficial” figureheads. Respondent E gave her vision from her perspective as university librarian and stated that faculty at her institution were looking for open materials to replace textbooks so they could build a course around the open resource. While initially not keep on adapting OER for their own use, faculty became more comfortable with this idea as they got used to it. However, these cases seemed to be rather anecdotical OER practices, since the study by Hayman (2018b) noted that “the use of OER is not widespread among Ontario college and university educators” (p. 150), apart from advanced OER practitioners (often also OER advocates). Similarly, in Germany, in HE is a field reliant on - quite frequently – the drive and perceived intention of individuals who have created a connected community.
Along the same lines, despite high levels of OER awareness and knowledge by participants, the majority of the faculty members in Australian HE involved in the study by Bossu et al. (2014) had rarely or never used, developed and/or re-purposed OER. Also, participants in the same study identified the lack of interest in creating (good quality) OER. However, in the expert survey, a participant mentioned that there was a push to “tag everything as being an OER upon completion” but that there “is very small uptake”, as well as storing OER (not just open access publications) in institutional repositories (n=3). Similarly, despite the creation of policies and frameworks, and increasing awareness of OER, OER use and creation remains low, as noted in the Open Education Licensing Project (2016c) for QUT. Some exceptions exist though, for instance, academics at USQ that were successful in receiving an OEP grant added value to curated OER by providing an explicit learning design that sequenced and aided students in “sense-making” (Stagg & Partridge, 2019, p. 479) and educators at the UTS were happy to use OER, as long as they were deemed to be as good as material they could produce themselves and that they required minimal modification (Kandlbinder & Chelliah, 2015).
 A reduced version of this chapter has been published in the following reference: Marín, V. I., Zawacki-Richter, O., Aydin, C.H., Bedenlier, S., Bond, M., Bozkurt, A., Conrad, D., Jung, I., Kondakci, Y., Prinsloo, P., Roberts, J., Veletsianos, G., Xiao, J., & Zhang, J. (2022b). Faculty Perceptions, Awareness and Use of Open Educational Resources for Teaching and Learning in Higher Education: A Cross-Comparative Analysis. Research and Practice in Technology Enhanced Learning, 17, 11. https://edtechbooks.org/-BSYU
 41.43% of the participants in the survey did not answered that question.