Motivation and objective of the study

Higher education institutions are more and more often encouraged by employers to find ways to add to their graduates’ technical knowledge and competencies a set of soft skills such as team skills, communication and interpersonal skills, emotional intelligence, or leadership qualities (Deepa & Seth, 2013; Thompson et al., 2021). These skills became especially important for STEM graduates that traditionally were seen as more individualistic and focused on technical skills (e.g., Ahmed et al., 2013; Lumague, 2017). 

Researchers have found that, similar to the development of cognitive skills, soft skills can be enhanced through the implementation of metacognitive strategies (Mitsea et al., 2021). Previous research indicated that students who were exposed to prompting strategies to become metacognitively aware were more likely to (a) have better critical thinking skills, (b) be more strategic in their learning process, (c) be better at self-regulating their learning, and (d) perform better than unaware learners (Kim, 2018; Saks & Leijen, 2019; Saraff et al., 2020; Schraw & Dennison, 1994). Enhanced metacognitive skills proved to be beneficial across multiple areas such as architectural design (Kurt & Kurt, 2017), programming performance (Çakiroglu & ER, 2020), or nursing (Hsu & Hsieh, 2014). 

The objective of this pilot study was to understand to what degree making undergraduate students aware of their main metacognitive strategies (strengths and weaknesses) impacst their performance, both as part of a team and individually.

Research intervention

Undergraduate students at a Midwestern university enrolled in two sections of a science course (Genetics) were randomly designated as a control and respectively an intervention group, following a quasi-experimental research design. During the first week of the semester, all students enrolled in the Genetics course completed a metacognitive awareness survey. 

During the second week of the class, the intervention group was exposed to an activity that included a presentation and short interactive exercises conducted by the instructor and an instructional designer. The presentation focused on the impact of collaboration, active learning, and metacognitive awareness on learning. For each of two metacognitive strategies, students scored themselves on the questions associated with that strategy, then the presenters explained the strengths of that strategy on the learning process and asked them to discuss in small, ad-hoc groups, what strengths each of them is bringing to the team. Following these exercises, students were provided with their survey data in the form of a metacognitive score card (Figure 1). The instructor then recommended that students use these cards to enhance the effectiveness of their work during the weekly in-class small-group projects. 

In addition to the presentation for the intervention group, the instructor added a brief required reflective essay to two in-class projects for both groups; these essays were guided by prompting questions focusing on metacognitive aspects of learning related to the topic of the project and were intended to serve as metacognitive awareness refreshers for students. Therefore, besides the in-class activity and the individual metacognitive score cards handed to the students in the intervention group, the instructional process was identical for the control and intervention sections.

Figure 1

Sample Metacognitive Score Card

During the last week of the semester, all students completed the same metacognitive awareness survey administered at the beginning of the semester and, once the grades were officially uploaded, the research team retained for analysis students’ performance scores for the assignments and assessments associated with this study. Figure 1 summarizes the timeline and the structure of the research intervention associated with this study.

Figure 2

Timeline and Structure of Research Intervention

Research methodology

All students enrolled in the course, 52 in the control group and 50 in the intervention group, were invited to participate in the study. Of these, 38 (73%) students in the control group, and 36 (72%) students in the intervention groups completed a matching pre- and post-intervention metacognition awareness survey. Participation was voluntary and there was no bonus points or other type of reward for their participation in the study. The study was approved by the university institutional review board. 

The metacognitive awareness survey was adapted from the Metacognitive Awareness Inventory (MAI) instrument proposed by Schraw and Dennison (1994). Five strategies from the original MAI instrument were used in this study: The Information Management strategies (10 items), Debugging strategies (5 items), Planning strategies (7 items), Monitoring strategies (7 items) and Evaluation strategies (6 items). The evaluation used a 7-point scale with 1 (never) and 7 (always). The results were presented to the students in the intervention group as a percentage of total score from the maximum possible. 

SPSS v28 ® was used to analyze the data. Internal reliability for the five metacognitive constructs used in this study was strong, with Cronbach’s Alpha values ranging between 0.73 and 0.81. 

Summary of findings

An independent-samples t-test found no statistically significant differences between the mean prior-knowledge scores of the control and respectively intervention groups. This result indicated that the two cohorts involved in the study were homogenous at the entry point in the study. 

Metacognitive constructs

An independent-samples t-test showed no statistically significant differences between the mean values of all metacognitive constructs for the control and respectively intervention group (Figure 3). The debugging and information management metacognitive strategies were the top two strategies self-reported by the participants in this study.

Figure 3

 Comparison on Mean Values for the Metacognitive Constructs Post Intervention

However, an independent-sample t-test using as test value 4, the neutral point of the scoring scale indicated a statistically significantly higher mean scores than the neutral point of the scale for planning, monitoring, information management and debugging strategies for both control and intervention group and the evaluation strategies for the control group (p<0.05). The evaluation metacognitive strategy for the intervention group was the only one that had a mean score that was not statistically significantly higher than the mean of the scoring scale. 

When the mean values for top two metacognitive constructs, debugging and information management, were compared between the pre and the post intervention for each group it could be observed that the direction of mean score change from pre to post for the control group was different from intervention group (Figure 4).

A repeated measures ANOVA indicated that the control group had a pre-intervention lower information management mean score than the control group and increased post intervention, and the control group lost from the pre to post intervention for both information management (Figure 5a) and debugging (Figure 5b) strategies. While the interactions were not statistically significant, the higher exit mean values for the intervention group (see Figure 5) suggested that it might be worthwhile to explore if a stronger intervention focused on the metacognitive processes would generate statistically significant interactions. 

Figure 4 

Change in Mean Metacognitive Scores for the Information Management and Debugging Constructs

Figure 5

Interaction of Mean Metacognitive Scores from Pre to Post Intervention a) Information Management Metacognitive Strategy; b) Debugging Metacognitive Strategy

Analysis of student performance

When we analyzed the impact of the intervention on the three weekly projects closely linked to the focus of the intervention due to their complexity and link to the additional reflective essays, an independent-samples t-test indicated that intervention group had statistically significant higher mean weekly projects scores (90.7%) than the control group (85.4%), t (72) = -2.14, p < 0.05. In addition, a regression analysis indicated that the weekly project scores that were selected because they required the greatest metacognitive investment, were a statistically significant predictor for the total exams score, F (1, 72) = 21.8, p < 0.001, and explained 23% (R2=0.23) of the variance in the total exam scores. 

In addition to the analysis of the quantitative data, a quick comparison of the reflective essays introduced to targeted weekly projects indicated that in the control group, most answers were either a reiteration of the question, or a statement indicating that students had looked through their notes as shown below.

“I looked at Mendel's postulates and mitochondrial genes and how these were different.” 

 “Recalling information we talked about in lecture today and using these step by step to explain my answer”

In the intervention group, multiple students mentioned peers as a resource (compared to none in the control group). There was also a little more diversity and depth (on average) among the student responses in the intervention section, as shown in the next examples.  

“I pictured how one would find genetics using a Mendelian route and contrasted it with the idea of mitochondrial DNA and how they differ from one another in transmission from parent to progeny.”

“I first thought about Mendel's postulates and how mitochondrial genes are different. Then, I also thought about the cell cycle, mitosis and meiosis.”

“We looked over Mendel's postulates and compared with mitochondrial genes. As well as collaborating with other groups.”

Conclusion

Despite the fact that the metacognitive awareness intervention in this study was relatively reduced in amplitude, the analysis of data showed the potential of this strategy in: 

• Increasing students’ overall performance on more open-ended instructional tasks such as the weekly projects in the focal course of this study;
• Potentially increasing students’ overall performance in the course.

The overall score increase for the metacognitive awareness intervention was also supported by a higher quality of reflective essays associated with the weekly projects in the intervention group. Together, our findings suggest that making students aware of their metacognitive strengths and weaknesses prompts them to capitalize on their strengths when working with peers and pushes them to become better learners and contributors, enabling them to transform their own learning.

References

Ahmed, F., Capretz, L.F., Bouktif, S., & Campbell, P.R. (2013). Soft skills and software development: A reflection from the software industry, International Journal of Information Processing and Management, 4(3), 171-191, http://dx.doi.org/10.4156/ijipm.vol4.issue3.17

Çakiroglu, Ü., & ER, B. (2020). Effect of using metacognitive strategies to enhance programming performances, Informatics in Education, 19(2), 181-200, http://dx.doi.org/10.15388/infedu.2020.09.

Deepa, S. & Seth, M. (2013). Do soft skills matter? - Implications for educators based on recruiters’ perspective, The IUP Journal of Soft Skills, 7(1), 7-20.

Hsu L-L & Hsieh S-I. (2014). Factors affecting metacognition of undergraduate nursing students in a blended learning environment, International Journal of Nursing Practice, 20, 233–241, http://dx.doi.org/10.1111/ijn.12131.

Kim, J.H. (2018). The effect of metacognitive monitoring feedback on performance in a computer-based training simulation, Applied Ergonomics, 67, 193-202.

Kurt, M. & Kurt, S. (2017). Improving design understandings and skills through enhanced metacognition: Reflective design journals. The International Journal of Art & Design Education, 36(2), 226-238, http://dx.doi.org/10.1111/jade.12094.

Lumague, E. (2017). Relative value of hard skills and soft skills for hiring employees in manufacturing sector. Journal of Business & Management Studies, 3(1), 1-5.

Mitsea, E., Drigas, A., & Mantas, P. (2021). Soft skills & metacognition as inclusion amplifiers in the 21st Century, International Journal of Online Engineering, 17(4), 121-131, https://doi.org/10.3991/ijoe.v17i04.20567

Saks, K., & Leijen, A. (2019). The efficiency of prompts when supporting learner use of cognitive and metacognitive strategies, Computer Assisted Language Learning, 32(1-2), 1-16, https://doi.org/10.1080/09588221.2018.1459729

Saraff, S., Rishipal, Tripathi, M., Biswal, R.K., & Saxena, A.S. (2020). Impact of metacognitive strategies on self-regulated learning and intrinsic motivation, Journal of Psychosocial Research, 15(1), 37-48, https://doi.org/10.32381/JPR.2020.15.01.3

Schraw, G. & Dennison, R.S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460-475. http://dx.doi.org/10.1006/ceps.1994.1033

Thompson, K., Conde, R., Gade, M., & Mims, T. (2021). An immersion approach to client-sponsored projects: Preparing students with softs skills required for hiring - face to face & virtual methods. International Journal of Higher Education, 10(2), 42-61, https://doi.org/10.5430/ijhe.v10n2p42