Introduction

Characterized by their simplicity and the requirement for learners to select either a "Yes" or "No" response or determine whether a statement is "True" or "False," binary test items (also called alternative response items) are widely used in assessments to measure learners' understanding of specific concepts or factual knowledge. These items offer a quick and efficient way to assess learners' understanding of basic facts, concepts, or relationships. In this subchapter, we provide an overview of these items' pros and cons, and guide you in creating effective binary test items.

Affordances and Constraints

Before diving into the process of creating effective binary test items, it is important to understand the advantages and limitations of these types of questions. By considering the pros and cons of using binary test items, you can make informed decisions about when and how to incorporate them into your assessments. Let's explore the affordances and constraints of using binary test items to measure learning.

Affordances

The following represent some of the major affordances of using binary test items:

1. Quick: Binary test items are easy to create and score. Instructors can make them out of only a few words. This makes them time-efficient for both instructors to create and learners to complete. On average, you can expect learners to complete 3-4 of these test items per minute.
2. Objective: Since there are only two possible responses, scoring is typically less subjective, reducing the potential for bias.
3. Efficient: Binary items allow for the assessment of a large amount of content in a short period, making them suitable for summative assessments that cover many topics or a lot of material. 

Constraints

Despite their affordances, binary test items are not without the following constraints.

1. Simple: Binary test items focus on factual knowledge rather than higher-order thinking skills or complex problem-solving abilities. These means they are limited in their ability to measure many learning objectives.
2. 50/50: The limited number of response options may result in learners guessing the correct answer rather than truly understanding the material. 
3. Unreliable: learners who answer correctly based on guessing may have an inflated perception of their understanding and provide you with a false sense of their knowledge or ability. In order to overcome the chances of guessing correctly, a test that solely uses binary test items would need to have at least 75 questions to provide reliability.

Writing Clear Instructions

Before you read the guidelines for creating binary test items, we want to introduce an important tip for getting the most out of your binary items. As part of your instructions, ask learners to write out their answers as either "TRUE," "FALSE," "YES," or "NO." Asking learners to circle "T" or "F" and then listing the letters in columns begs for some wily learner to circle both, or worse, place a circle somewhere in the middle. Additionally, requiring learners to just write either "T" or "F" can result in some pretty artistic expressions that somehow combine those letters into a non-recognizable "TF" hybrid letter. The same is true if you ask learners to write "Y" or "N." Both letters require three lines, and learners will find some creative ways to make you guess whether they intended to write a "Y" or an "N." The best way to always clearly know a learner's answer is for them to write out the whole word. This can be somewhat mitigated through the use of an electronic test, but with a 50/50 chance of guessing correctly, some learners might claim to have mis-clicked when choosing a response.

Guidelines for Creating Effective Binary Test Items

When creating binary test items, it is crucial to follow certain guidelines to ensure their effectiveness and validity. By adhering to the following six guidelines (as well as the general guidelines for Item Creation), you can craft well-constructed items that accurately assess learners' understanding of concepts or factual knowledge. Our exploration of these guidelines for effective binary test items are accompanied by good and bad examples from various contexts. These examples illustrate how to apply the guidelines. The examples include both true/false and yes/no questions. The biggest difference between these formats is that yes/no items are questions - ending in a question mark - and true/false items are statements - ending in a period or, if you're feeling spicy, an exclamation mark.

1. Avoid Blatant Constructs: Ensure that the correct answer is not too obvious or predictable, challenging learners to think critically.

Bad Example:

     True/False (English): Books are primarily used for reading.

Good Examples:

     True/False (Science): Photosynthesis is the process by which plants convert sunlight into energy.

     Yes/No (Theatre): Was scenery important to Elizabethan theatre?

2. Avoid Absolutes: Be cautious of using absolute terms like "always" or "never" in the statements, as they can make the item more difficult to assess accurately.

Bad Examples:

     True/False (Social Studies): The American Revolution only impacted the United States.

     True/False (Physical Education/Health): Everyone should exercise daily.

Good Example:

     Yes/No (Math): Are all prime numbers, greater than 2, odd?

3. Avoid Negative Constructions: As with our general guidelines for test items, minimize the use of negative phrasing, as it can confuse learners and lead to misinterpretation.

Bad Examples:

     Yes/No (Physical Education): Is lack of exercise not associated with an increased risk of heart disease?

     True/False (Music): Syncopation is not a rhythmic technique used in jazz music.

Good Example:

     True/False (Social Studies): Canada is north of the United States.

4. Test Single Concepts: Each test item should focus on assessing understanding or knowledge related to a single concept to ensure clarity and avoid confusion.

Bad Examples:

     True/False (Science): Gravity is a force that pulls objects toward the center of the Earth, though each planet has its own gravity.

     Yes/No (Social Studies): Did Jean Piaget make some revolutionary discoveries about child behavior during the nineteenth century?

Good Examples:

     Yes/No (Science): Does a solid change its shape when subjected to external forces?

     True/False (Math): Subtraction the inverse operation of addition.

5. Balance Number of True/False or Yes/No Responses: Aim for a relatively equal distribution of true and false responses across the test items to avoid unintentional patterns or biases. You have probably seen or experienced instructors who like to make all the answers "True" or all but one answer "False." These are trick questions that play psychological games with learners and can cause them to second guess themselves. The defense of, "Well if they really know it, they won't second guess themselves," is a shameful attempt to hoard power over learners and validate causing them to suffer psychogical distress.

Bad Example:

     True/False (Social Studies):

1. The Magna Carta was signed in the 20th Century. (False)
2. The Great Wall of China was built as a defense against oceanic invasions. (False)
3. The Cold War as a conflict between North and South Korea. (False)
4. Christopher Columbus landed in North America in the 15th Century. (True)
5. The Berlin Wall was constructed to separate East and West Germany during World War II. (False)

Good Example:

     True/False (Math):

1. The sum of two prime numbers is always an even number. (False)
2. The square root of any positive number is always a positive number. (True)
3. All right angles measure 90 degrees. (True)
4. Zero is an even number. (True)
5. The product of any two negative numbers will be negative. (False)

6. Provide Test Items of a Similar Length: Strive for consistency in the length of the statements to prevent learners from relying on statement length as a clue for the correct response. It is often the case that longer statements are true because they contain qualifying clauses or phrases. Though this is not always the case as is seen in the bad examples below.

Bad Examples:

     True/False (Physical Education): Cardiovascular endurance is the ability of the heart, lungs, and blood vessels to deliver oxygen to the muscles. (False)

     Yes/No (Physical Education): Are the lungs the primary respiratory organ? (Yes)

Good Examples:

     True/False (Math): The area of a rectangle can be calculated by multiplying its length and width. (True)

     True/False (Math): The sum of a rectangle's sides is equal to double its area. (False)

Alternative Binary Test Item Formats

The examples above represent the most common form and uses of binary test questions, but there are some alternative forms that can be really helpful. For example, the "If ... Then ..." binary test item presents two true clauses, but asks learners to analyze the causal relationship between the clauses. Consider these examples from science, English, and physical education:

     If the chemical equation for a compound is HCl, then Chlorine is present in the compound.

OR

     If you use a singular noun, then you must use a plural verb.

OR

     In baseball, if the bases are loaded, a run can only be scored off of a hit.

In each of these examples, both parts of the statements contain facts - HCl is a compound, and Chlorine can be present in compounds; nouns can be singular, and verbs can be plural; the bases can be loaded in baseball, and runs can be scored off of a hit. In these kinds of binary test items, we ask learners to focus on the relationship between the two statements - letting them know in the instructions that each statement is true on its own. This allows us to use binary test items to measure some higher order thinking skills.

Similarly, we can also use more advanced forms of binary test items to get at higher order thinking skills. One example of these forms is the True/False Correction. This item type combines binary test items and constructed response test items. For True/False Correction items, learners must answer whether the statement is true or false, and then correct the false statements to become true. If you choose to use this form of binary test items, you should be careful to identify the part of the statement you want learners to analyze and rewrite if the state is false. Otherwise, there are many ways in which false statements can be rewritten to be true. Consider the following social studies examples.

Instructions: Answer True or False by writing your answer on the line provided. If the answer if false, rewrite the statement to be true.

Question:

_______ 1. Abraham Lincoln was the 12th President of the United States.

Answer:

False 1. Abraham Lincoln was the 16th President of the Unied States.

OR

False 1. Zachary Taylor was the 12th President of the United States.

This is a standard example of how there are two elements that could be changed to make the statement true. It would be better to determine which element is the most important to test, and underline that element. If we wanted learners to know that Abraham Lincoln was the 16th President, we would want to underline "12th President" in the question. If we were trying to determine whether learners knew who the 12th President of the United States was, then we would want to underline "Abraham Lincoln" in the question. Here is a more extreme example of what can go wrong with "True/False Correction" items if you do not specify which element to correct.

Bad Example:

Instructions: Answer True or False by writing your answer on the line provided. If the answer is false, rewrite the statement to be true.

Question:

_______ 1. As part of the Apollo 11 mission, Michael Collins landed on the moon.

Answer:

False 1. Even though he was part of the Apollo 11 mission and went to space, Michael Collins did not land on the moon.

OR

False 1. As part of the Apollo 11 mission, Neil Armstrong landed on the moon.

OR

False 1. As part of the Apollo 11 mission, Edwin "Buzz" Aldrin landed on the moon.

OR

False 1. Not part of the Apollo 11 mission, I have never landed on the moon.

OR

False 1. Even though he was not part of the Apollo 11 mission, Superman can easily land on the moon.

Good Example:

Instructions: Answer True or False by writing your answer on the line provided. If the answer if false, rewrite the underlined portion of the statement to make it true.

Question:

_______ 1. As part of the Apollo 11 mission, Michael Collins landed on the moon.

Answer:

False 1. As part of the Apollo 11 mission, Michael Collins did not land on the moon.

OR

False 1. As part of the Apollo 11 mission, Michael Collins orbited the moon.

Another type of advanced binary test item are context-dependent binary choice test items. These items include a cluster or group of binary choice test items that have a common theme and/or rely on a common reference. Context-dependent test items can be used for any kind of traditional assessment item and provide some kind of content for questions to be based on. This form of question is common on standardized tests because it allows all learners to have access to the same content in answering the questions. For example, the science portion of the ACT is filled with questions about reading tables, understanding charts, or analyzing images - these are context-dependent test items. More about these kinds of test items is explored in the Context Dependent subchapter of Item Creation. The following provides examples of context-dependent binary choice test items for science, math, and English.

Science Example:

Instructions: Answer True or False by writing your answer on the line provided.

     GUA CAG UCG UUC CCG

_____ 1. GUA is the starting point for this translation.

_____ 2. This is an example of DNA.

_____ 3. In this example, five amino acids are being translated.

Math Example:

     learners earned the following scores on a 10-point math assessment. Use the following data to answer the statements below. Write either TRUE or FALSE in the line provided.

     5, 6, 7, 7, 7, 7, 8, 8, 8, 8, 9, 10

_____ 1. The median for the learners' scores is 7.5.

_____ 2. The mode for the set of scores is 8.0.

_____ 3. The range of the learners' scores is 5.0.

_____ 4. The median value is different than the mean value.

English Example:

     Read the following passage and answer the questions. Write either TRUE or FALSE in the space provided to answer the questions.

Once upon a time in the bustling city of Chicago, there lived a passionate hockey fan named Sam. With a heart that beat for the Detroit Red Wings, Sam could often be found sporting their jersey and cheering for their beloved team. Little did they know that their loyalty would soon thrust them into an adventure. It all began when a group of die-hard fans from a rival team, the Edmonton Oilers, conjured an intergalactic dragon to wreak havoc on the Detroit Red Wings. Upon learning of the imminent danger, Sam knew they had to step up and save their team. Embarking on a three-day journey, they faced numerous trials, meeting allies along the way, and ultimately discovering the inner strength needed to defeat the dragon. With bravery in their heart and unwavering determination, Sam confronted the fearsome dragon, unleashing their love for the Red Wings as a beacon of hope. In a thrilling climax, Sam outwitted the monster and saved the Detroit Red Wings from certain destruction, forever etching their name in the annals of hockey history.

_____ 1. This story starts in Detroit.

_____ 2. Sam is the main character in the passage.

_____ 3. The events of the passage occur over at least three days.

_____ 4. The events of the passage follow the Hero's Journey.

The final questions of the science and English example are examples of application questions that can be asked as a result of multiple binary choice test items. In the science example, learners have to apply their knowledge of amino acids to the context presented, and in the English example, learners have to compare the passage to the elements of the Hero's Journey. In this way, these alternative binary test items can allow you to measure some higher order thinking skills, though that is not their strength.

Final Thoughts on Binary Test Items