All researchers hope to find significant results in their research, and journals prefer to publish significant studies. But what should you do if you obtain non-significant results, especially for your first study in your doctoral program? Do you give up or move forward?

I graduated from the Learning Design and Technology (LDT) program at Purdue University in May 2022 with three first-authored publications before embarking on my career as an assistant professor in the Educational Technology and Literacy department at Towson University in August 2022. This seemed like a perfect beginning, but I almost gave up on my doctoral studies and decided to earn a master's degree instead when I encountered failure in my first study.

My road to academia was not typical. My husband, two kids, and I had lived in Beijing, China, for 19 years when I decided to pursue my doctorate at 38 years old. Almost everyone around me thought I was crazy for giving up all my family and I had in Beijing, including my general manager position and my husband's career as an air controller. It would also be a significant change for my whole family moving from China to the United States, especially for the kids. But, after 11 years of working experience in corporate, I was eager to reach the frontier of human knowledge and expand my understanding through the process of earning a doctorate.

Understanding my background, you can see how painful it was when I almost decided to give up on my educational goals after failing my first study.

I was in the second year of my doctoral program. After completing some foundational courses, I initiated my first study during a writing proposal course. I was interested in motivation, especially intrinsic motivation. So, I was excited when I found a list of positive psychology interventions reported to improve participants' well-being, optimism, and motivation. Since I was in the instructional design area, it was natural to apply theory to a real-world class to see if it worked. I decided to study one of the most widely used positive psychology interventions, the Best Possible Self (BPS)—a writing activity created to guide participants to envision their ideal future and write down their best possible self (King, 2001). In an introductory course focused on technology integration, 105 preservice teachers were assigned to a control and treatment group through a quasi-experimental method. Those students were then provided three writing activities and four testing points. The four testing points included a pretest before the first writing activity, a posttest after the third writing activity, a one-month survey, and a two-month survey. In the treatment group, the participants were guided to write their BPS in general, BPS as a teacher, and BPS of technology integration in their future teaching. In the control group, the participants were asked to write a list of things they did the day before, what they would do on a typical weekday, and what they would do on a Saturday, which was reported to be a neutral writing activity. The four test points measured the participants' well-being and attitudes toward technology integration. The study aimed to examine if the three BPS writing activities promoted participants' well-being and attitudes toward technology integration compared with the control group.

Everything went well until COVID-19 was declared a global pandemic by the World Health Organization (WHO) on March 11, 2020. Five days later, on March 16, 2020, Purdue University announced that the campus was closed, and all the courses were moved online for the remainder of Spring 2022. This happened between my third and fourth testing points. So, all the participants completed the last test point from home online. When I analyzed the data, the last test point was, unsurprisingly, influenced by the pandemic, showing a sharp drop in the participants' well-being. But unexpectedly, in the first three test points, BPS did not significantly improve participants' well-being and was less effective than the control group. This was worse than a non-significant finding! I was disappointed initially and struggled to explore any potential significance in different directions and among different subsets of the data, such as comparing only pre- and post-tests, only female or male respondents, only first- or second-year students, and so on. I also tried all the analysis methods I learned from the methods courses and kept exploring. Despite my hard work, I did not find any significance in the data! I thought it was a failed study, and that I might need to give up. At the same time, I was drowning in self-doubt. I felt like I wasn’t good enough as a researcher and that I should not complete my doctorate.

Fortunately, I talked to my advisors, Timothy Newby and Marisa Exter. They encouraged me to try to publish my research because might provide information for other researchers, even though the results were non-significant. So, I returned to work on it and published two papers from that study and one more from the subsequent research, which also yielded non-significant results (Duan et al., 2021a; Duan et al., 2021b; Duan et al., 2022). From other research, we knew that interventions might not work well during the pandemic and in unusual contexts. As Zacher and Rudolph (2020) indicated in their study, “individual differences in life satisfaction were…unexpectedly negatively related to planning… the high levels of insecurity associated with the pandemic may have turned future planning into a dissatisfying experience” (pp. 59–60). Results from my study perfectly echoed the findings from other studies that were also conducted during the early stage of the pandemic (Zacher & Rudolph, 2020; Satici et al., 2020): under the higher level of uncertainty, guiding participants to think about their future resulted in more stress. Instead, the “neutral” writing activity that asked participants to focus on their daily chores was more beneficial under the unnormal contexts (Duan et al., 2021a; Duan et al., 2021b; Duan et al., 2022). So, despite not showing significance, my three published papers helped refine our understanding of pandemic effects (Duan et al., 2021a; Duan et al., 2021b; Duan et al., 2022). They were included in my dissertation and contributed to my career.

Lessons Learned

I reflected on this experience and felt it was significant for me to be a researcher in the following three aspects:

First, the purpose of research is not only to find significant results. We start with a question and hope to find an answer. We can hypothesize based on the literature but cannot control what kinds of results we will get from it. Whatever we discover, we need to face it, try to explain it, and report it. This helps other researchers know what happened in your study and how that could help them improve their design.

Second, I believe building off of stupidity is one of the critical factors that makes a good researcher, as Schwartz (2008) highlighted in his article. Schwartz discusses that what makes doctoral students feel difficult is “research is immersion in the unknown” (Schwartz, 2008, p. 1771). He proposed to teach doctoral students to be productively stupid, which means to keep trying and learning from each failure. As I read this article and reflected on my own experience of failure, I realized how beneficial that mindset was to my academic career. It is something I will continue to keep in mind throughout my future research projects and my mentorship experiences with doctoral students.

Last, it may seem that my experience with failure ended happily simply because I was able to publish my research. However, the happy ending resulted from a lot more than that. Not only was my research published, but my mindset changed. I started to enjoy my research once I realized the true purpose of research and what makes a good researcher. With this attitude, I became more open-minded about any results I received. Moreover, I continued to learn and improve with each of the studies that followed, which was my original goal!

Now I feel more at peace with these initial “failures” and have learned there can be positive results even from non-significant findings in a study.

References

Duan, S., Exter, M., & Newby, T. (2022). Effect of Best Possible Self writing activities on preservice teachers' attitudes towards technology integration. TechTrends. https://doi.org/10.1007/s11528-022-00696-y

Duan, S., Exter, M., Newby, T., & Fa, B. (2021a). No impact? Long-term effects of applying the best possible self intervention in a real-world undergraduate classroom setting. International Journal of Community Well-Being. https://doi.org/10.1007/s42413-021-00120-y

Duan, S., Watson, S.L., & Newby, T. (2021b). Novice instructional designers' attitudes towards the Best Possible Self activity and the correlation with personality and subjective well-being. TechTrends. https://doi.org/10.1007/s11528-021-00653-1

King, L. A. (2001). The health benefits of writing about life goals. Personality and Social Psychology Bulletin, 27(7), 798–807. https://doi.org/10.1177/0146167201277003

Satici, B., Saricali, M., Satici, S. A., & Griffiths, M. D. (2020). Intolerance of uncertainty and mental wellbeing: Serial mediation by rumination and fear of COVID-19. International Journal of Mental Health and Addiction. https://doi.org/10.1007/s11469-020-00305-0

Schwartz, M. A. (2008). The importance of stupidity in scientific research. Journal of Cell Science, 121(11), 1771–1771.

Zacher, H., & Rudolph, C. W. (2020). Individual differences and changes in subjective wellbeing during the early stages of the COVID-19 pandemic. American Psychologist, 76(1), 50–62. https://doi.org/10.1037/amp0000702