Designing effective audience response questions is not always an easy, or intuitive task, particularly when trying to align questions with the stated course outcomes. This document serves to introduce several approaches that you may consider when creating your questions or revising an old one.

Question Driven Instruction

Question Driven Instruction is a pedagogical approach that focuses specifically on teaching that is supported by an Audience Response System. Here the core of the instruction is the “question cycle” which replaces traditional “transmit and test” instruction by suggesting a paradigm with an iterative process of question-posing, deliberation, commitment to an answer, and discussion. (Banks, 2006). In other words, the classroom instruction begins with posing a question and the rest of the lesson is constructed around this question.

The authors of this approach suggest strategies and tactics for designing effective Audience Response System questions. According to them “a general framework for thinking about question design must address the role that the questions will play within a course, the specific goals a question can be designed to attain, and the various mechanisms by which it can attain them” (Beatty, Gerace, Leonar, and Dufresne, 2006, p.32)
a) Role: What part do questions play?
b) Goals: What should the question accomplish?

● Content goal: What piece(s) of the subject material do we want to illuminate?
● Process goal: What cognitive skills do we want students to exercise?
● Metacognitive goal: What beliefs about learning and doing do we wish to reinforce?

c) Mechanisms: How can a question accomplish its goal?

1. Directing attention and raising awareness

● remove nonessentials
● compare and contrast
● extend the context
● reuse familiar question situations
● oops-go-back

2. Stimulating cognitive process

● interpret representations
● compare and contrast
● extend the context
● identify a set or subset
● rank variants
● reveal a better way
● strategize only
● include extraneous information
● omit necessary information

3. Formative use of response data

● answer choices reveal likely difficulties
● use “none of the above”

4. Promoting articulation discussion

● qualitative questions
● analysis and reasoning questions
● multiple defensible answers
● require unstated assumptions
● trap unjustified assumptions
● deliberate ambiguity
● trolling for misconceptions

See our companion document that further describes the aforementioned tactics, here.

Note: Descriptive note. Adapted from “Designing effective questions for classroom response system teaching” by Beatty, Gerace, Leonar, and Dufresne, (2006), American Association of Physic Teachers, 74, 1.

Critical Incident Questionnaire (CIQ) – Student Reflection

This is a pedagogical method of finding out how students are experiencing their learning and the instructor’s teaching. The CIQ is a questionnaire that can be delivered online or via an audience response system at the end of the last class of the week. It comprises five questions, each of which asks students to address events that happened in the class that week. The aim of the CIQ is to help students focus on specific, concrete happenings that were significant to them (Brookfield, 1995).

1. At what moment in the class this week did you feel most engaged with what was happening?
2. At what moment in the class this week did you feel most distanced from what was happening?
3. What action that anyone (teacher or student) took in class this week did you find most affirming and helpful?
4. What action by anyone (teacher or student) in class this week did you find most puzzling or confusing?
5. What about the class this week surprised you the most? (This could be something about your own reactions to what went on, or something that someone did, or anything else that occurs to you.)

Academic Technology Services would also like to suggest that the CIQ’s purpose can be adapted to fit the format and structure of your own course, be it online, or face-to-face.  The questionnaire can be abstracted from the weekly structure and adjusted to fit the architecture of your course module, or chapter outlines.

Note: Descriptive note. Adapted from ‘Becoming a Critically Reflective Teacher’ by Brookfield S. D. (1995), San Francisco: Jossey-Bass.

Socratic Questioning

Socratic questioning is one of the most popular and powerful teaching approaches that can be used to foster learners’ critical thinking skills. Instead of providing direct answers, the Socratic questioning approach stimulates students’ minds by continually probing into the subject with thought-stimulating questions. As a result, through active interactions between instructors and students and among students, Socratic questioning can facilitate students’ critical thinking skills by the exchange of ideas and viewpoints, giving new meaning to content, exploring applications to problems, and providing implications for real-life situations (Ya-Ting, Timothy & Robert, 2005).

According to literature, there are 6 types of Socratic questions that help instructors to probe critical thinking among learners (Salam & Khe Foon, 2010)

1. Questions of clarification: These questions ask for verification or additional information of one point or main idea.
2. Questions that probe assumptions: These ask the student for an explanation or reliability of an assumption.
3. Questions that probe reasons and evidence: This category of questions ask for additional examples, reasons for making statements or process that lead the student to his or her belief.
4. Questions about viewpoints: These questions ask the student whether there are alternatives to his viewpoint or a comparison of similarities and differences between viewpoints.
5. Questions that probe implications and consequences: This category of questions helps the student to describe the implication of what is being done, or the cause-and-effect of an action.
6. Questions about the question: These ask the student to identify or interpret the question, the main point, or the issue at hand.

Practical suggestions for designing ARS questions

• Limit the number of answers to five or less, so that the question is easy to read and consider (Caldwell, 2007, p.18)
• Create wrong answers (distractors) that seem logical or plausible to prevent “strategizing” students from easily eliminating wrong answers (Caldwell, 2007, p.18
• Include “I don’t know” as an answer choice to prevent guessing (Caldwell, 2007, p.18)
• Add relevant diagrams, clip art, or pictures (Banks, 2006, p. 62)
• Survey for opinions and feelings. Not every question should be a test with a right or wrong answer (Banks, 2006, p. 62)
• Design graded questions to encourage attendance, homework and preparation (Thalheimer, 2007, p. 19)
• Use pre-questions to focus attention, for example words in pre-questions should be similar to the learning material you want to focus attention on (Thalheimer, 2007, p. 34).\

References

Banks, D. A. (2006). Audience response systems in higher education: Applications and cases. Hershey, PA: Information Science Pub.

Beatty, I. D., Gerace, W. J., Leonar, W. J., and Dufresne, R. J. (2006). Designing effective questions for classroom response system teaching. American Journal of Physics. 74(1), 31–39.

Brookfield, S. (1995). Becoming a critically reflective teacher. San Francisco: Jossey-Bass.

Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best practice tips. CBE Life Sciences Education, 6(1), 9-20

Salam, S., & Khe Foon, H. (2010). Enhancing Social Studies Students’ Critical Thinking Through Blogcast and Socratic Questioning: A Singapore Case Study. International Journal of Instructional Media, 37(4), 391-401.

Thalheimer, W. (2007). Questioning Strategies for Audience Response Systems: How to Use Questions to Maximize Learning, Engagement, and Satisfaction. Retrieved November 31, 2009, from http://www.work-learning.com/catalog/

Ya-Ting C. Y., Timothy J.N. & Robert L.B. (2005) Using Socratic Questioning to Promote Critical Thinking Skills Through Asynchronous Discussion Forums in Distance Learning Environments. American Journal of Distance Education, 19(3), 163-181, DOI: 10.1207/s15389286ajde1903_4