Conceptualization Phase

Basic Processes and Principles
Survey Research

Prior to creating survey items, you need to determine the precise topic and purpose for the study as well as the research questions you wish to answer. Careful consideration of the overall study’s purpose and questions will help identify whether a survey is the best way to collect data, what data are needed, and who should take the survey.

Research Topics

A good study starts by clearly and succinctly identifying the specific topic that will be addressed. A good research topic will be interesting, important, and researchable. The topic may be of interest simply because you are an expert in the field and this is your chosen field of study. However, a topic may also be of interest to you and others because current events have identified the topic as something we need to better understand. Another characteristic of a good research topic is that of importance. Importance can be a somewhat subjective factor; however, a research topic has significance if it has the potential to add to our general understanding of theory and/or improve practice. The issue of whether a research topic is researchable has to do with the scope of the topic and the degree to which the researcher can manage the study. Narrowing a topic makes it more manageable, which then makes the topic researchable. In addition, making sure the researcher can gain access to or gather relevant data will also determine whether the topic can be researched properly.

There are many ways to find a good topic. However, a review of past research is generally a good idea. Conducting a review of published research will identify potential areas for new research or the need to replicate previous findings. For example, studies may have explored a specific topic in the past but, over time and given changing circumstances, the research may need to be replicated to determine what (if anything) has changed or to verify the results. A good topic might also come from personal experience. A researcher may find that they need to better understand some aspect of their work in order to improve their practice in a certain area.

A research topic might address attitudes, perceptions, or some personal characteristic of individuals within a specific population. The topic to be explored might involve some social condition or circumstance. A well-written topic statement will identify the variables the research plans to study, possibly including the hypothesized relationship between variables. The topic statement should also delineate the scope of the research by describing important characteristics of the participants. Depending on the complexity and familiarity of the topic, precisely stating the topic in a single sentence may be a challenge. Remember, specific terms and definitions are not always understood universally; you may need to take additional time to precisely define the topic, both theoretically and operationally, as part of the introduction or through a more in-depth review of previous research.

Topic Statement Example

Consider the following topic statement. “This study investigated counseling services available to first-year students.” In this example the general topic involves counseling services, specifically the availability of such services. The participant population for the study is narrowed to include first-year students. However, it is unclear what is meant by first-year students and what type of counseling services will be considered. A better version of the statement would specifically state the type of counseling services (e.g., academic, emotional, or psychological). It would also more clearly define the scope in terms of which students would be involved (e.g., first grade elementary, high school, or university students). A better version of this topic statement might be, “This study investigated counseling services for first-year university students who suffer from depression and anxiety.” Should the researcher desire, the scope might be more narrowly defined by specifying a specific university, type of university, program within the university, or subgroup of the student population. It could be expanded to include students other than first-year students.

Research Purpose

Once the specific topic has been defined, a purpose for the study must be established. The purpose of the study is different from the purpose of the survey. Simply put, the purpose of a survey is to gather information. The study’s purpose will be to address a specific problem or resolve a particular issue. The purpose of the study is closely associated with the research questions in that the answers to the research questions (obtained from the survey) will help solve the problem or address the issues.

A clear purpose statement will help focus the survey so the survey will concentrate on what is needed rather than tangential and irrelevant data collection which might distract from or sidetrack the actual purpose of the study. The topic statement might include the purpose; in other words, the statement might indicate the problem the research attempts to solve. However, depending on the situation, a separate purpose statement is most often provided.

A complete purpose statement should include an explanation as to why the problem is important (i.e., providing a justification without simply stating the research is important) and a description of how a better understanding of the topic might contribute to generalizable theory or specific practice (i.e., the potential benefit for conducting the study). Basically, you need to determine what problem is being addressed and how the research you plan to conduct will help solve the problem.

The purpose for a research study might be to

Proving vs. Providing Evidence

Rarely will anyone use the word “prove” when describing the purpose of a study. Even when attempting to establish a potential causal link or attribute some outcome to a specific intervention, the general purpose of any inquiry is to gather evidence that will help those investigating answer the questions posed in a study. The evidence obtained will either support or refute hypothesized conjectures, never prove them.

Research Purpose Example

You will recall that in the counseling services topic statement example, the precise purpose for the study is not fully specified. In this example, the purpose could have been simply to determine students' awareness of the services available to them; however, a more interesting purpose might have included the need for or adequacy of such services. A potential topic and purpose statement for the counseling services study might have been stated as follows.

This study investigated counseling services available to first-year university students who suffer from depression and anxiety. Given the increased number of students who report suffering from stress in their transition from high school to university, the purpose of this study was to determine students' awareness of counseling services, as well as the adequacy and accessibility of those services intended to alleviate excessive anxiety and depression. A better understanding of the current situation will be used to set university policy and address any issues that need to be resolved.

Research Questions

Good research questions align directly with the purpose of the study. Answering the research questions helps resolve the problem being addressed by the study. A good research question allows for rich, in-depth answers. Multiple questions could be poised, but the questions should address the overall purpose of the study, not specific survey questions. Listing too many research questions can be a problem when reporting because reporting survey research will already be a lengthy endeavor.

Impossible Questions

There are some research questions that should be avoided. Questions that involve values, morals, or ethics can be a challenge—especially when attempting to determine which value-based moral or ethical stances everyone should take. These questions cannot be answered through empirical research because, like opinions and feelings, there is no universally accepted correct way to think or feel. Differences in the way we think or feel are an essential part of being human and having agency. This does not mean that topics involving values, morals, and ethics cannot be studied. Research questions might address the prevalence of specific value-based moral or ethical positions being held by individuals in distinct societal groups. One might ask about the rationale or reasons for why people hold such positions. One might even explore the outcomes that result from holding specific value-laden positions, both intended and unintended. Answering questions in this unbiased way can provide information that has the potential to enhance dialogue regarding important social, emotional, and religious issues. However, when choosing the research questions for a study, one should avoid asking which value-based moral or ethical stance people should take or opinion people should espouse. These are impossible questions.

Research Questions Example

Continuing with the counseling services example, a researcher could ask “Do participants value the counseling services provided?”  However, the answer to this question is a yes/no response (i.e., a binary or dichotomous response). Values are rarely dichotomous in nature, rather they fall on a continuum. A better question might be “To what extend and in what ways do participants value the counseling service programs?” The researcher will also need to decide whether values should be the intended focus. Rather than use the term "value," researchers might consider determining participants’ perceptions of effectiveness. Participants may value having access to counseling services but not feel they are particularly effective or accessible. Depending on the purpose for the study, the study may attempt to answer multiple research questions. A second research question might involve gaining a better understanding of which services are of greatest need. However, it is generally good practice to keep the research question at a broader, more general level rather than the specific—unless it is determined that doing so would be needed in order to narrow the scope of the study or directly resolve the issue being addressed. For example, providing research questions that targeted a lengthy list of specific counseling services may not be advisable; a more general research question would suffice. However, if specific services are truly the focus of the study, it could and should be specified in the purpose statement and methods section of any planned research.

Possible research questions for the counseling services study example might include.

  • To what degree are students aware of the counseling services currently being offered?
  • What additional services do students feel they need?
  • How willing are students to avail themselves of specific counseling services?
  • To what degree do students feel needed counseling services are accessible to them?
  • To what degree do students feel the available services adequately meet their needs?

Chapter Summary

  • Before attempting to write survey items, the topic, purpose, and research questions should be clearly defined and stated.
  • A good research topic will be interesting, important, and researchable.
  • A study’s purpose will address a specific problem or issue.
  • It is never a good idea to presume the purpose of research is to prove anything. Research can only provide evidence that either supports or refutes a hypothesis. 
  • Good research questions align directly with the purpose of the study.
  • Research questions involving morals, values, and ethics that are unlikely to have a universally accepted correct answer should be avoided. This does not mean these topics should not be studied, rather it is recommended that any impossible "should" questions be reframed in ways that they can provide information that has the potential to enhance dialogue and understanding. 

Discussion Questions

  1. Why is it important to clearly define the topic, purpose, and research questions prior to writing items for the survey?  
  2. How are the research purpose and research questions related?

Practice Tasks

  1. Consider the following topic statement: “In this study we will explore the relationship between high school students’ feelings of safety and their scholastic effort.” 

    What is the topic? How does the statement narrow and delineate the scope? What specific variables will be addressed? What aspect of the variables will be explored? How could it be improved?

  2. Choose a potential topic. Write a sentence that clearly and succinctly states the topic. Evaluate the topic using the criteria of interest, importance, and research viability. Revise as needed.

  3. For the topic you choose in Practice Task 2, create a purpose statement and potential research questions. Evaluate their potential adequacy, importance, and completeness. Revise as needed.

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/designing_surveys/survey_concept_Design.