Designing a Scale
Unlike a questionnaire, a scale is designed to estimate the strength of a personal characteristic or disposition held by an individual. The characteristics measured by a scale are called constructs. Scales are most often needed when measuring constructs in the affective domain. Measurement in the cognitive domain attempts to determine what people know, understand, or are able to do intellectually (e.g., recall, understanding, analysis, evaluation, synthesis, problem solving, or critical thinking). Measurement in the psychomotor domain describes the quality of an individual’s ability to perform physical tasks (e.g., athletic or musical ability). We do not need a self-report instrument to measure these attributes; other more direct measures are available and are more appropriate. However, measurement in the affective domain targets an individual’s interests, perspective, beliefs, attitudes, or personality traits. Scales are needed to measure affect because the best (and possibly only) way to obtain data is to ask individuals to self-report. In addition, because affective constructs targeted by a scale are complex abstractions, multiple items are required to obtain an accurate measurement (each item on a scale designed to capture a nuanced aspect of the construct). The related items on a scale are combined to provide a numerical indication of the strength of the construct for individual respondents.
Anderson and Bourke (2000) explain that affective characteristics being measured must (a) involve feeling, (b) describe a somewhat permanent trait rather than a temporary state, (c) be targeted toward something (e.g., feelings about specific people, situations, or entities), (d) have a positive or negative direction (e.g., like or dislike), and (e) vary in degree or magnitude for different individuals. For example, physical characteristics (e.g., height, hair color, or even appearance) do not involve feeling and can be measured without the need of a self-report instrument. Likewise, anything that can be simply categorized does not require measurement (e.g., age, gender, or nationality). These characteristics can be determined through simply reporting. Attitudes and perceptions on the other hand are personal attributes that often can only be determined through self-report. And while general feeling may be useful, specific feeling regarding specific targets are more helpful especially when they represent typical or deeply held perceptions rather than temporary feelings (e.g., you may have an opinion about politics in general, you would likely have contrasting and distinct opinions about various aspects of the political process). Finally, in order for affective characteristics to be measured they must have both direction and intensity (i.e., the potential for individuals to have more or less positive or negative feelings about the topic).
The Affective Domain
Affective constructs (and dispositions) represent inherent qualities of mind and character. A person’s innate disposition can refer to one’s predominant or typical mental and emotional temperament (e.g., a positive or happy disposition); however, a disposition can also refer to a person’s typical state of mind, tendencies, and inclinations. These are attitudinal dispositions rather than innate personality traits. Attitudinal dispositions are often the target of scales used in survey research. Measuring these constructs is important because attitudes affect an individual’s behavior (see Fishbein and Ajzen’s (1975) theory of reasoned action). However, attitudinal dispositions, although in the affective domain, also have a conative influence in that while our behaviors are influenced by our feeling, beliefs, and opinions, they are moderated by free will and agency (Davies, 2009). According to Ajzen’s (1991) theory of planned behavior, people typically align their behavior with their personal belief systems after taking into account how they believe others would perceive their actions. They may also experience conflicting intentions, values, and beliefs (Davies, 2009). For example, an individual may be capable of performing specific skills, they may understand important concepts, they may recognize the value of the behavior, they may have a personal desire to behave a certain way but still choose to act in ways we might not expect due to other deeply held beliefs or compulsions.
Some scales target unidimensional constructs, others are multidimensional. For example, when measuring students’ attitude towards school, a scale would need to be multidimensional because one’s interests and attitudes are based on a variety of unidimensional aspects. A student’s attitude towards school might be influenced by the individual’s perception of school being a safe place (physically and emotionally), a welcoming/fun place (socially), or a good place to learn (academically). The attitude may be also be influence by views of individual teachers or their personal interest in learning (career goals). It is very possible that a student may, for example, have a positive perception of school with regards to social aspects, but no interest or a negative attitude towards the academic opportunities provided. Each dimension would be measured separately (using a variety of items) then conceivably combined to get an overall measure of one’s attitude towards school.
Creating Scale Items
Anderson & Bourke (2000) describe two methods for writing items for a scale – the mapping sentence and domain reference approaches.
Mapping Sentence Approach. When using this approach, the item developer pairs a situation with a behavior (or list of distinct actions). The respondent will either be asked to select the behavior that best describes the way they would react given the situation, or indicate the likelihood they would behave in the manner specified given the situation.
Before writing items, the item developer must carefully identify the construct (defining its theoretical nature) and describe in detail how the construct might manifest itself as a behavior (operationalizing the various aspects or facets). When identifying the core of the construct, think about the kinds of beliefs and attitudes someone with this characteristic might have. When operationalizing the construct, consider what actions or behavior would be exhibited by someone who embodied this characteristic. If providing a list of possible behaviors, each potential response should represent an action likely to be taken by individuals possessing the characteristic at different intensities (e.g., high, moderate, low). If the target is to be more specific (rather than a general measure of the construct) the situation can be contextualized.
Mapping Sentence Example
Suppose you were attempting to measure the construct of self-confidence (in general). At its core, confidence requires a high level of self-efficacy, a belief in one’s ability or competency. In operational terms, someone with self-confidence may try new things, attempt difficult endeavors, or be willing to take risks. They might also express feelings of success. This understanding could help you develop items to place in the scale. Pairing situations with behaviors we believed would be taken by someone with self-confidence allows us to get an indication of the strength and direction of the characteristic for individuals. While we might be interested in obtaining a general measure of confidence, contextualizing the situations allows us to get targeted measure of an individual’s confidence in specific situations (i.e., ability to compete in a specific sport or succeed in a specific endeavor).
- starting a new task
- trying a difficult maneuver
- giving an opinion
- learning a new skill
- begin without hesitation
- wait until encouraged
- think carefully whether to proceed
- let someone else try first
- believe they will be successful
- When learning a difficult maneuver for the first time, I immediately begin without hesitation.
- When learning a new skill I am confident I will succeed.
Domain-Reference Approach. This approach also begins with an examination of the construct’s critical features. However, particular consideration is given to the target, direction and intensity of the characteristic. Statement are created by specifying a target, using a directional form of the “to be” verb (positive or negative), then adding an adjective denoting intensity. Once the basic structure of the statement is determined, it can be transformed to fix grammatical issues. The statements produced should have a direct alignment between conceptual and operational definitions of the construct.
Suppose you were attempting use this approach to write items that could be used to measure the construct of self-confidence. Possible targets are listed. A verb is selected followed by an adjective that describes a possible reaction. The statement should describe a precise behavior and describe a strong emotion or position.
Target: learning a new skill, participating in a debate, expressing my opinion
Verb: I am, is,
Adjective: exciting, thrilling, eager, happy,
Possible Revised statements:
- I am eager to try difficult maneuvers for the first time.
- I am happy to express my opinion to others.
- I usually express my opinions quite well when debating.
- I am very good at learning new skills.
Testing and Revision
As with all survey questions, items must be carefully tested to make sure the they are meaningful and clear to potential participants. Pilot testing the items using think aloud cognitive interviews can be quite useful for this purpose. Multiple revisions may be needed. Some additional guidelines include:
- Write multiple items that cover all aspects of the topic and likely targets. Test each item and use those that work best.
- Keep the language clear, simple, direct, and concise (see writing clear items).
- Each statement should contain only one thought or situation. (see double barreled items)
- Avoid statements of fact.
- Avoid general statements everyone or no one would likely endorse. Write statements that represent strong clear position rather than a middle of the road (ambiguous or noncommittal) declaration.
- Write in the present tense using an active voice.
- Avoid the use of vague modifiers (e.g., usually, sometimes)
- A precise agreed-on theoretical definition for the construct being measured is required before writing items. This includes both conceptual and operational definitions.
- Conceptual definitions describe the abstract meaning of the construct and the core beliefs or motivations foundational to the construct.
- Operational definitions describe how those with the characteristic would likely behave. Manifestation of anticipated behaviors provide inferential evidence that the individual posesses the characteritic.
- A close alignment between conceptual and theoretical definitions should exist and be evident in each item.
- Scales are most often needed when measuring personal characteristics in the Affective domain.
- Characteristic in the Affective domain must (a) involve feeling, (b) target stable traits rather than temporary states, (c) be specifically targeted (d) have direction (positive or negative, absence to increasing positive), (d) vary in intensity by degrees.
- Two approaches to writing items include the sentence-mapping and domain-reference approaches.
- Multiple items for each construct should be created and tested prior to pilot testing and administering the instrument.
- Statistical procedures can be used to validate the instrument including exploratory and confirmatory factor analysis, or structural equation modeling.
- Constructs may be unidimentional or multidimentional.
End-of-Chapter Survey: How would you rate the overall quality of this chapter?
- Very Low Quality
- Low Quality
- Moderate Quality
- High Quality
- Very High Quality