Judgment in Practice Who decides?

Lead the discussion.

A 25-minute guide for Jack and other facilitators. Start with the opening reading on the shared screen, then move to the five claims. Use this page for your notes.

What people should leave with

A concrete example of where judgment can disappear, a distinction between useful help and lost practice, and a way to make ownership visible.

Listen for someone to name a choice and explain what evidence would change their view. Invite people to challenge the claims using their own experience.

Set up before people arrive
  • Open the discussion page on the shared screen. Keep this guide on your own device or print it.
  • Share judgment-in-practice.web.app for optional reading. No account, advance reading, or participant device is needed.
  • Choose two claims as a fallback: 1 — evidence of learning and 4 — the meaning of better work well for an education audience. Follow a different choice if the room has energy for it.
  • Invite examples without student names or identifying details. Use the fictional examples if needed.
  • For each claim, invite a brief response from two panelists, then a contrasting experience from the room. Keep individual responses to about a minute.

The 25-minute run of show

Times are elapsed from the start. Choose two claims and protect the last eight minutes for implications and the close.

0–3 minutes · Open and choose

Read together, then choose a claim.

First minute: Show the opening reading. Read it aloud or give the room about 45 seconds to read.

A student submits an excellent paper with help from an AI agent. The agent helped define the problem and develop the argument. To understand what the student learned, we need to hear which choices they made and see how they respond when an assumption changes.

AI can help people do more demanding work. It can also take over the decisions through which they learn. As educators, we have to decide where learners need to exercise judgment and what would show it has developed.

Adapted from The Irreducible Officer. Fictional education example.

Second minute — ask: “Where have you seen the quality of the work become harder to separate from the person’s understanding?” Take one brief example; return to it during the claims.

Third minute: Show the five claims. Give people 20 seconds to read. Ask which they would challenge, take two nominations, and select two claims. If no one starts, use claims 1 and 4.

Clarify if needed: By an agent, we mean AI that can carry out several steps toward a goal. An agent might find evidence, draft a plan, and send it to someone when authorized. Judgment can be delegated at any of those steps.

3–10 minutes · First claim

Find the judgment inside a real example.

Ask: Read the selected claim and its main question. Invite one concrete example before offering the example on the site.

  • About two minutes: hear the example. “What did the person ask AI to do?”
  • About three minutes: expose the choice. “Who chose the goal or standard? What did the output make harder to notice?”
  • About two minutes: invite an objection. “When would this assistance strengthen judgment? What would distinguish those situations?”

Transition: “We disagree about ___. What happens when we try a different example?”

10–17 minutes · Second claim

Compare a second example.

Do: Open the second claim. Invite a different panelist or participant to offer an example. Ask which judgment changed hands, then invite a different view.

Push: “Does our first explanation still work here? What might AI notice that a person would miss? Who can challenge or redirect the work before it has consequences?”

At minute 17: Summarize the disagreement in one sentence. Move on even if it is unresolved.

17–22 minutes · Educational implications

Apply it to something you teach or lead.

“Choose something you teach, assess, or lead. Which judgment needs to stay visible? How would you design the work so someone practices that judgment while using AI?”

Do: Allow one minute to think or talk with a neighbor. Use the remaining four minutes for two contrasting examples and follow-up.

Listen for: Ask participants to name a capability and describe where they would practice it. They might explain a choice, then adapt it when the conditions change.

Test: “Could someone meet that requirement without understanding the decision? How would you make the demonstration accessible?”

22–25 minutes · Close

Leave with one judgment to make visible.

Show: The closing question. Give everyone 30 seconds to form an answer. Invite two or three short responses.

“Which choice will you pay closer attention to after today? What would you ask the person responsible to explain?”

Finish: Reflect back one useful distinction and one unresolved disagreement. Point to the reading room and sources for continued exploration.

Adapt for 20 or 30 minutes

20 minutes: 0–3 opening; 3–13 discuss one claim; 13–17 implications; 17–20 close.

30 minutes: Keep the 25-minute opening and two-claim discussion through minute 17. Add five minutes at 17–22 to test the school example. Discuss implications at 22–27 and close at 27–30.

If the start runs late, reduce the number of claims. Preserve the implications and closing question.

Prompts for each claim

Choose a follow-up that helps the group examine its example.

1 · Better work, less evidence of learning

Ask: What changed in the artifact? What changed in the person’s capability?

Follow up: How might a learner demonstrate skill in directing AI? What would happen if one important assumption changed?

Listen for: Ask about both what the learner can do independently and how they direct AI. Look for evidence of development in each.

Open this claim for the room

2 · Approval after the judgment

Ask: Which decisions were made before the recommendation reached a person?

Follow up: Who could change the objective or restore an excluded option? What happens if an agent can act before that challenge occurs?

Listen for: Identify a point when a person has enough information and authority to intervene.

Open this claim for the room

3 · The struggle that teaches

Ask: Which exact capability does this effort develop?

Follow up: Would the same assistance help a novice and an expert differently? Could we preserve the reasoning while removing a language or access barrier?

Listen for: Ask what the learner would practice through this effort and how you would tell whether it helped.

Open this claim for the room

4 · Who defines better?

Ask: Did the revision change what earns credit?

Follow up: Whose interests does each version serve? What if the AI’s suggested standard is better than the original?

Listen for: Ask the person to explain why they accepted the AI’s contribution and how they checked it.

Open this claim for the room

5 · Different answers, same frame

Ask: What do all these answers take for granted?

Follow up: Whose perspective could change the question itself? What evidence would distinguish a shared blind spot from reasonable agreement?

Listen for: Ask which assumptions the group shares and whose perspective could challenge them.

Open this claim for the room

When the conversation stalls

Everyone agrees too quickly
“When might this claim be wrong?” Open the objection beneath the claim.
The discussion becomes “AI is good / AI is bad”
“Take one task. Which judgment is changing hands? When would that help?”
Someone asks whether research proves the claim
Open its evidence link. Describe what the researchers studied and found. Ask what further evidence we would need to apply that finding here.
Only panelists or confident voices are speaking
Give 30 seconds for quiet thought or a neighbor exchange. Invite an experience we have not heard; allow people to pass.
“Just keep a human in the loop” ends the discussion
“What information, time, and authority does that human have? What could they change?”
The group needs something concrete
Use the fictional attendance example. Ask who decided what problem the agent should solve.