Introduction

Context-dependent test items are specially designed assessment items that require learners to utilize or analyze content included within the test itself to answer questions effectively. The content used for these items can take on many forms such as reading passages, primary documents, graphs, tables, maps, images, audio, or even video. For an item to be a context-dependent test item, learners should be unable to respond to the item correctly without the provided content. Context-dependent items are pivotal for evaluating higher order thinking skills because they allow learners to answer questions aimed at higher-order thinking skills focused on application, analysis, synthesis, or evaluation. By creating test items that focus on content or examples that are unfamiliar to learners, you can infer how well learners are able to interpret, manipulate, and analyze content related to various learning objectives.

In standardized testing environments, context-dependent items are commonly employed to assess comprehension and analytical abilities across a spectrum of subjects. For instance, science tests often incorporate charts, graphs, and maps to gauge a student's ability to interpret scientific data. Similarly, literacy and language arts exams frequently use reading passages as a basis for questions that test reading comprehension and textual analysis skills.

The creation of context-dependent test items builds on the foundational knowledge presented in the previous item creation chapters of this book. Whether adapting the straightforward nature of binary test items to require deeper analysis, or transforming multiple choice formats to challenge students to apply learned concepts in novel situations, each previously discussed item type can be enhanced to assess more complex cognitive skills by transforming these item types into context-dependent items. It should be noted that matching items and the various forms of constructed response items, including fill-in-the-blank, short answer, and essays items, can also be tailored to draw upon the power of context-dependent test items; however, such items should be adapted with care. The nature of matching items can already create a high cognitive load for learners. Adding secondary content to these items furthers their complexity and can cause cognitive overload for learners - invalidating their answers and your assessment data. Concerning constructive response test items, short answer and essay questions already allow you to measure higher order thinking skills, but adding novel content to constructed response test items can more easily allow you to enact these measurements. Additionally, context-dependent test items can amplify constructed response items by providing rich media for the questions through the inclusion of images, audio, or video.

This chapter explores how you can design and implement these powerful items to assess and foster a deeper understanding and application of knowledge by leveraging real-world scenarios and examples.

Higher Order Thinking Skills

To effectively utilize context-dependent test items, it is essential to understand the cognitive levels they aim to assess, particularly the higher-order thinking skills described in Bloom's Taxonomy. These skills—Analyzing, Applying, and Evaluating—are pivotal for fostering deep learning and critical thinking among students. Note that this section will not cover the “create” level of Bloom’s Taxonomy. This level focuses on learners being able to create novel demonstrations of course content. While this level could be assessed using context-dependent test items, it would be better suited to a performance assessment, which we cover in a later chapter.

Applying: Application requires students to use learned information in new and concrete situations. In the case of context-dependent test items, this might involve using mathematical formulas to solve real-world problems presented in a scenario, employing historical knowledge to predict outcomes in hypothetical situations, or referring to best practices for exercise in physical education to provide the next step of a workout routine focused on building long lean muscle.

Analyzing: This level of Bloom’s Taxonomy involves breaking down information into its constituent parts to understand its structure and underlying ideas. Context-dependent test items that assess analysis might ask students to dissect a complex graph, image, or diagram; identify relationships among various components of provided content; and/or interpret what these relationships between various components of provided content mean for the broader topic at hand.

Evaluating: At this level, students make value-based judgments based on criteria and standards established through instruction to check or to critique provided content. Context-dependent items designed for evaluation might present learners with experimental results or differing viewpoints on a subject, and ask learners to assess the validity of these results or arguments. Questions might also go a step further by asking learners to justify their conclusions by identifying relevant evidence.

Context-dependent test items are particularly well-suited to assess these cognitive skills because they place standard questions within a context-dependent framework that requires the application of knowledge. For example:

• In science, a context-dependent item might present learners with a scenario involving an ecological system and provide them with two different data sets - before a new species was introduced to the system and after. Questions could ask them to apply their understanding of ecosystems to predict potential changes for a third data set, to analyze the impact of an introduced species, and evaluate conservation strategies based on their analysis.
• In mathematics, learners might be given a real-world financial scenario requiring them to apply algebraic knowledge to devise a budget, analyze financial outcomes based on different variables, and evaluate the most economically efficient solution based on various desired outcomes
• In language arts, a reading comprehension passage could serve as a basis to apply close reading strategies, to analyze figurative language, and evaluate the author’s message in light of historical or contemporary social issues

These examples illustrate that context-dependent test items not only challenge learners to think deeply but can also integrate knowledge from various domains, such as combining mathematical and science knowledge to predict changes in data, mathematical knowledge with business principles to select business plans, or relying on historical understandings in language arts for literary analysis. Such cross-curricular approaches to assessment can enhance learning and retention of material. By framing questions within real-world contexts, you can better assess learners’ abilities to synthesize information and apply it in meaningful ways.

Designing Context-Dependent Text Items

To design your own context-dependent test items you will want to follow six steps. Remember that you only need to use context-dependent test items if you want to use a traditional assessment to measure higher order thinking skills. If your learning outcomes only focus on learners’ ability to recall or understand course content, creating context-dependent test items might create more work for you than necessary. In such cases, however, context-dependent test items could be used to measure learners’ ability to go beyond mastery of the learning objectives.

1. Determine if your assessment’s learning outcomes require measurement of higher order thinking skills.
2. Find or create content that would be appropriate for learners to (a) apply their knowledge to, (b) analyze the constituent parts of, or (c) or evaluate the quality of, based on your learning outcomes.
3. Insert this content into your assessment.
4. Create test items that require learners to interact with the content to respond to the items.
5. Follow best practices for the kind of items you create, referring to the guidelines provided in previous chapters.
6. Double check your context-dependent test items to make sure that learners cannot reasonably complete each item without engaging with the content.

Below we provide some examples of context-dependent test items for each of the item types covered in previous chapters

Binary Items

Instructions: For the following items, write either TRUE or FALSE in the space provided.

_____ 1. This is a painting by Michelangelo.

_____ 2. This painting is an example of a fresco.

_____ 3. This painting uses the technique of colore.

_____ 4. The subject of this painting is portrayed as royalty.

_____ 5. The road in the painting symbolizes the growth of the merchant class.

_____ 6. The shades of blue in this painting suggest that it is a “retouched” version of the original.

Multiple Choice Items

Instructions: For the following items, write the CAPITAL LETTER of the correct response in the space provided.

_____ 1. According to the graphic, which two animals can fall into more than one food chain category?

A. Cows and Owls

B. Snakes and Lions

C. Grass and Insects

D. Alligators and Sharks

_____ 2. According to the graphic, the yellow arrows most likely suggest that _____

A. all consumers need sunlight.

B. green plants need sunlight.

C. primary consumers live in the sun.

D. sunlight is essential for green plants and primary consumers.

_____ 3. Adding dotted lines from Sun to levels above Primary Consumers would suggest higher levels of consumers _____

A. indirectly need the sun.

B. receive some nutrients from the sun.

C. can only live in sunny ecosystems.

D. are not impacted by the sun.

_____ 4. Looking at the organisms placed at each level, most of the organisms are ____

A. predators.

B. carnivores.

C. omnivores.

D. herbivores.

_____ 5. If we added fungi to this food chain we would place them ____

A. with green plants.

B. on level with the cow.

C. parallel to each level.

D. on level with the shark.

Matching Items

Instructions: For the following items, use the family tree to match the correct response(s) from the second column with the relationship in the first column by writing the CAPITAL LETTER of the response(s) next to the relationship. Some responses will be used more than once.

Constructed Response Items

While any of the previous examples could easily be altered to become constructed response items. Please consider the following example of an essay question that operates at the evaluation level of Bloom’s Taxonomy.

Instructions: You have just been hired to work with Modern Medical Facility’s physical therapy team. Evaluate the proposed physical therapy plan for the following patient considering his specific circumstances and symptoms.

Rad, a 42-year-old male with moderate obesity, has been experiencing persistent sciatic nerve pain down his right leg following a recent accident. In this accident, Rad was pinned against a wall by a tractor that rolled off a trailer, resulting in moderate back pain that has mostly subsided. He now reports numbness in his right foot and leg, particularly below the calf muscle.

In response to his condition, his general practitioner has proposed cardio exercise and core strengthening to see if it alleviates the nerve pain. The general practitioner has also prescribed him with muscle relaxers to use as necessary.

Your essay should address the following points in no more than 5 paragraphs. Each requirement below is worth 10 points.

1. Appropriateness of the Focus on Cardio and Core Strengthening
2. Risks and Benefits of the Proposed Treatment
3. Alternative Approaches to Treatment
4. Conclusions About the Patient’s Treatment Plan

 Affordance and Constraints

Context-dependent test items provide unique opportunities and present distinct challenges in creating your assessment items. This section explores both the affordances and constraints of employing such items in testing environments.

Affordances

One significant advantage of context-dependent items is their ability to measure higher-order thinking skills using test item formats that can be assessed quickly, such as binary choice and multiple choice questions. This benefit cannot be overstated. Being able to measure learning using case studies and real-world scenarios without requiring a performance assessment allows for a streamlined approach to using assessment FOR learning. This level of authenticity not only makes the testing process more engaging but also allows learners to demonstrate their ability to apply theoretical knowledge in practical situations. When you employ context-dependent test items, you can assess higher-order thinking skills—like analysis, synthesis, and evaluation—without the intensive grading typically associated with performance assessment.

Additionally, these items excel at measuring how effectively learners can utilize resources specific to a discipline as well as provide opportunities to reach beyond content-specific silos in assessments. For instance, in a history exam, learners might be asked to analyze primary source documents to support their arguments, thereby demonstrating their skills in handling subject-specific materials. But these test items can also be effective in measuring cross-curricular outcomes, where students must apply knowledge and skills from multiple disciplines. For example, as seen above, a test item might require students to use both mathematical calculations and scientific knowledge to solve an environmental problem.

Constraints

Developing context-dependent items can be time-consuming as it requires the creation or curation of fresh, engaging content that provides a fair assessment of your learners. Additionally, you cannot simply recycle examples used during instruction because familiarity might skew the assessment's effectiveness. Each piece of content used on an exam will need to be new to your learners or your assessment will suffer from familiarity bias. The content for context-dependent items can also easily suffer from other forms of assessment bias. Based on their unique backgrounds, learners might find that the content provided for context-dependent items suffers from ambiguity or cultural bias. Lastly, content may also quickly become outdated if it draws from current events or culturally specific scenarios. This means content requires frequent updates and careful consideration to ensure that it is relevant to learners. These risks means that you may need to spend more time creating and evaluating these test items than you do for other test items.

Because of their complexity, the content chosen for context-dependent items should be meticulously evaluated for its potential to generate meaningful test items. Not all content is equally useful in assessment, and poor content selection can lead to fewer or lower-quality test items, affecting the overall utility of the test. It is best to make sure any content provided for these test items can maintain relevance for an amount of time that is acceptable to you based on your desire to revise and edit future implementations of the assessment. You will also want to make sure that the content selected can provide multiple test items for measuring learners’ abilities. If content can only produce one or two questions, you may want to reconsider its use.

Conclusion

Context-dependent test items are powerful tools in the arsenal of educational assessment, bridging the gap between theoretical knowledge and practical application. These items not only challenge learners to apply, analyze, and evaluate information within a structured context but also enhance the authenticity and relevance of assessments. Through the integration of real-world scenarios, these items enable educators to assess complex cognitive abilities in a manner that is efficient for the educator and engaging and rigorous for the learner.

However, the implementation of context-dependent items is not without its challenges. The time and effort required to develop unique, relevant content that avoids bias and remains current should not be ignored. Moreover, the necessity to frequently refine and adapt these items to ensure clarity and fairness demands a persistent commitment from educators. Despite these hurdles, the potential of context-dependent test items to provide positive impacts on learning and assessment justifies the investment. By carefully balancing the affordances and constraints of context-dependent test items, you can effectively harness them to foster deeper understanding and critical thinking among your learners. As a result, you can better prepare them for both academic success and real-world challenges.

Resources

Mona Lisa by Leonardo da Vinci is in the Public Domain and accessed via C2RMF at Wikimedia Commons.

Food Chain is by Suwachanee Maneeratanachot and shared with a Creative Commons Attribution Share-Alike 4.0 International License.

Greek God Family Tree is by deathchronicler and shared with a Creative Commons Attribution Non-Commerical No-Derivatives 3.0 Internaional License.