Teaching Reasoning: Our research-based approach to teaching critical thinking skills and a Guide for Teachers

Teaching reasoning is my way of building a better world, in the most enduring and transformative way I can imagine. I founded this school to help our students think more deeply and effectively about any problem or decision they encounter.

Part 1: An Introduction to, and the Research Behind, Our Instructional Approach

Teachers, you can skip directly to "Part 2: A Guide for Teachers" below if you want to create your own reasoning rubrics, but if you want to know more about the empirical basis for our approach to teaching reasoning, you should start here.

Because I wanted to figure out how to address the persistent flaws in human reasoning, I entered graduate school to explore what was known about why people think as they do, and about how to educate to improve long-term human outcomes. I wanted to know why we think as we do and how we can “think better.” This is why my research in graduate school focused on critical thinking and on judgment and decision making processes, culminating with my dissertation research on how preconscious processes influence complex political decisions. That’s another way of saying: what cognitive processes get in the way of making important decisions well?

I chose to study how elected officials make complex educational policy decisions because these are the sorts of decisions I am most interested in and because this allowed me to test my theory that most people in most cases reason after they make a decision, even when faced with an important decision that should only be made after completing research, analysis, and reflection.

What Cognitive Science Reveals: Use Rubrics to Improve Reasoning

In graduate school, I spent six years trying to learn why we think as we do and how we can improve the thinking process. Here are the most relevant findings for the purposes of this introductory document:

  1. Definitions of higher order, “critical,” or normative thinking or reasoning are manifold, with little agreement on the process we should follow when we think (beyond cognitively infeasible models based on maximizing subjective expected utility) or on what teachers should do to teach students how to reason about important decisions.
  2. Complex policy decisions are not necessarily the product of normative reasoning. Decision makers do not make important policy decisions in a thoughtful, deliberate way, but rather rely on preconscious processes to decide very quickly and with very little decision-specific information or conscious awareness of their reasoning process.
  3. The best way to reason critically or normatively before making a decision, and to resist the influence of preconscious processes, poor decision-specific information, or the failure to consider long-term consequences, is to use a decision rubric.

Based on my review of existing literature and my original research on how elected officials make educational policy decisions, I concluded that prescribing a specific decision making process for important decisions was the best way to help people make important decisions well. Without a reliable process, decision makers could fall back on, and fall prey to, intuitive, affective decision making routines that are quicker and cognitively less burdensome, but that lead to decisions that are not the product of a reasoned analysis of long-term costs and benefits. So, I created a decision rubric for making important decisions.

The rubric consists of a set of questions designed to help decision makers think critically about the factors that should influence their final decision before they make a final decision. Cognitive science shows the decision making process too often actually works in the opposite direction: you first make a decision and then you look for reasons to justify it. This rubric is designed only for decisions that are important enough to warrant the investment of the time and effort required to make the decision well. The questions and the order of the questions are designed to counter the most common and most serious defects in human reasoning. For responses to be satisfactory, they would have to be precise, clear and complete.

Here is the rubric I created for policymakers to use when they encounter complex questions:

  1. What is the question to be answered or the decision to be made (the “Question”)?
  2. What criteria will be used to measure success or failure on this Question?
  3. What are your goals; i.e. what are you trying to accomplish?
  4. How much time do you have?
  5. What is your first impression: what is the right answer to the Question; what is the best decision or course of action?
  6. List other possible answers to the Question.
  7. What are the costs or consequences of these alternatives?
  8. Which answer or decision to the Question is best? List your evidence.
  9. What would someone who disagreed with you say the correct answer to the Question is? List the reasons why.
  10. What evidence would you offer to convince those who disagree with you?
  11. Has anyone faced this question or decision before you? If so, what can you learn from their experience?
  12. Are there alternatives you have not considered?

This rubric, like the subject-specific instructional rubric I will outline below, could be written differently. This is just one example of how to turn the otherwise preconscious, haphazard and inadequate process of thinking into something that will produce better outcomes. The purpose of showing you this rubric is to make the point that any effective reasoning rubric is going to be composed of the questions that are most likely to require the reasoning process you want to elicit.

Having completed my academic research on reasoning, I decided to begin the more important work of practical research in a school setting. Working on academic research alone seemed to be an imperfect way to go about saving the world because there is no way to know whether your ideas will work until you try them. And our only real hope is to show students how to make important decisions well, because in my experience adults are rarely as open-minded about new ideas or possibilities as students are.

 Part 2: A Guide for Teachers: Creating Instructional Rubrics to Teach Reasoning

As I thought about how we could revise our academic program to improve long-term student academic, social, and civic outcomes, I realized something for the first time: I had opened this school to improve student reasoning, yet I and my colleagues had pursued the goal of teaching better reasoning only indirectly.

We assigned materials, asked questions, and introduced experiences designed to promote thoughtfulness, skepticism, and open-mindedness. We worked to show students why learning more deeply about a subject or story is important, and to help them identify weak or misleading arguments and evidence so they are not misled. This is all wonderful and, as we know, it is a great improvement on what is being done in all but a few secondary schools. At the same time, directly is more likely to work than indirectly.

 Accordingly, we now develop and deploy subject-specific instructional reasoning rubrics that make visible the steps students should take in thinking about the material they encounter in each subject area. With this approach, we work directly to improve student reasoning and student learning outcomes. And, once we measure student learning and reasoning outcomes and improve our rubrics, we will be able to share our model for improving student reasoning with educators everywhere, which moves us towards the larger objective of helping as many students as possible. I will turn now to guidance on designing a reasoning rubric for academic courses.

The best way to explain what a rubric should include is to show you a model rubric. Table 1 is an example of a reasoning rubric in history classes to make visible to students a process for thinking about cause and effect when studying historical periods, trends, or specific events. This rubric also serves to:

  1. Highlight for students what is most important in the texts they read for class;
  2. Cue students on what to listen for in classroom discussions;
  3. Provide a framework to help students organize written notes; and,
  4. Guide students as they try to assess the relative importance of course content and concepts when preparing for assessments.

This rubric could be handed out when the course begins to introduce the preferred analytical approach, a blank rubric could be handed out at the beginning of each class as a template for note-taking, and partially completed rubrics could be offered as study guides at certain points during the term. Teachers could also review student-completed rubrics periodically as a formative assessment.

In Table 1, the first five rows (after the header row) are things students should consider whenever they try to understand why something happened in the past. For younger students, I might have them focus only on one row (e.g., the role of individuals or of economic factors). The rows with brackets are additional elements I consider important but that I would not give students on a rubric in advance, as I want them to take responsibility for thinking on their own (even though I am making some key considerations visible in advance). The columns allow for comparative analysis of different geographic areas, events, time periods, and so on. It need not have five areas. It could have fewer or more depending on the learning goals, prior content knowledge, and students’ developmental readiness.

Table 1: A reasoning rubric for analyzing causation in historical events between 1492 and 1789 in several geographic areas

Loudoun_School_History_Rubric_image_1.jpg

Humanities rubrics will likely be fundamentally different than rubrics in math and science. At the same time, I believe all rubrics should have the following characteristics:

  1. MODEL EXPERT ANALYSIS: The rubric should be based on how a subject matter expert would analyze a new complex problem in the subject area, in the form of, for example, the steps the expert would follow, the types of information they would look for or focus on, and the questions they would ask.
  2. CENTRAL TO COURSE CONTENT: The rubric should be part of your core course content, and the rubric should be specific enough to be a useful tool that students actually use regularly, in the ways I described above (highlight, cue, etc.). Also, the rubric should work across courses in a single subject area (e.g., history, algebra, or biology), so that students understand that the rubric applies whenever they encounter a new problem in this subject area, for example.
  3. INITENTIONALLY INCOMPLETE: The rubric should be incomplete and should not be presented or treated as the definitive and final way to think about the subject. A teaching rubric should only be a reliable framework that allows students to practice taking the most important reasoning steps in the subject while giving them room to add their own “steps,” so long as they can defend logically what they add.
  4. IMPROVE LEARNING: The rubric should help students better understand the material that is assigned in your classes by showing them the types of information or processes that are most important for them to learn and how to reason about the information and processes they are learning.

 This list is only a starting point. It will change as we work on developing rubrics for each subject area.

 In conclusion, we teach students how to reason well by teaching reasoning directly.  We do so by making good reasoning visible to students with rubrics, and other tools, we create for each subject area. Students will learn the steps of sound reasoning in each subject area through regular use, and through discussion and debate about these steps and rubrics, with the opportunity to make justified additions to the rubrics based on their own reasoning about the subject. This process will improve long-term student learning outcomes as well, because (1) the rubric embodies what they will ultimately need to learn in each subject and (2) they will use the same rubric and steps over and over in each course, and then again across courses. This framing and repeated exposure will improve learning in a single course and transfer of learning to new courses (the latter of which is the “holy grail” of formal education).

An Aside: Teaching Reasoning, Not Ideology

 It is important to note that nothing herein should be interpreted to mean that we know in advance what our students should decide is best for them over the long-term. This entire educational effort is about the decision making process, not about specific decision making outcomes. We are not working to persuade students to follow an existing ideology, but rather to give them a way to reason about important questions so they make decisions with a higher probability of producing desirable long-term outcomes. I submit that having the training and discipline to follow a sound process for making important decisions will produce better outcomes, personally and socially. Better world, here we come.