How AI Learns: Building a Classifier From Your Data
7–10 · Science / Math · 90 minutes
Kit: micro:bit + sample lesson plan — "How AI Learns: Building a Classifier From Your Own Data" (grades 7-10, science/math). Students build a simple machine-learning classifier using micro:bit sensor data, then evaluate its accuracy. Embedded AI literacy: students see AI from the engineering side — how systems learn, where they break, and what the humans building them need to know. Designed for Silicon Valley — where understanding AI from the inside is a community expectation.
Without the PD, building an ML classifier is a project. After: students see AI from the engineering side — how systems learn, where they break, what the humans building them need to know — exactly the depth a Silicon Valley community has every right to expect.
Show full lesson plan objectives · procedure · materials · assessment · teacher pack ▾
Content Objectives
- Design a binary classification problem
- Collect labeled training data
- Train, test, and analyze a small classifier
AI Literacy Objectives
- Understand AI from the engineering side
- Identify how training data shape classifier behavior
- Apply structured verification practice to a system the team built
What Students Do
Teams of 3+ design a binary classification problem (e.g., "indoor vs. outdoor", "morning vs. afternoon"), collect labeled training data with micro:bit sensors, and train a small classifier in the browser framework.
Don't standardize sensor placement across teams. Different microclimates make Phase 2 richer. Move between teams every 5 minutes; check that students are recording observations *and* numerical readings. The qualitative notes are the wedge they'll use to challenge AI in Phase 3.
Watch forTeams logging only numbers. Push them to write at least one observation per reading ("breeze picked up", "cloud passed over"). If the campus has visibly varied environments — shade vs. sun, paved vs. planted — push teams to spread out.
Teams test the classifier on new data. Document accuracy, identify systematic failures, and trace each failure to a training-data limitation.
Frame the AI tool as a teammate, not an authority. When the AI prediction is wrong, students often default to "we'll fix our data." Interrupt that — the goal is to surface where AI and ground-truth diverge, not to reconcile.
Watch forTeams that find zero divergence. Either they're smoothing data unconsciously, or the AI is generic enough to match anything. Have them pick a single 5-minute window and compare in extreme detail.
Apply structured verification practice to a system the team built. Class produces engineering-side insights about how AI systems learn and where they break — directly relevant to a Silicon Valley community.
The class trust guidelines are the deliverable. Push for specificity: not "AI is bad at humidity" but "AI underestimates humidity in conditions like ours when [specific local condition]." Local knowledge + data = the trust criteria.
Watch forGeneric statements ("AI is sometimes wrong"). Reject these gently — every guideline must reference a specific divergence the team observed.
A four-step verification protocol your teachers will build with us
A practice students learn once and apply to any AI output, in any subject, for the rest of their lives.
Where did the AI get its data? Is it the same data we used or generated?
How did the AI reach its conclusion? Can we follow the logic?
Does the output match what we observed with our own senses, instruments, or knowledge?
What might we have missed? What would we want a second opinion on?
More on the thinking behind this — the framework we built it from.
Materials
- micro:bit with multiple sensor types (included in kit)
- USB cables and student devices
- Browser-based ML classifier framework (link at landing page)
- Training-data design worksheet
- Chart paper and markers
Assessment
Each team produces a one-page artifact: their findings, the AI output they evaluated, and a written verdict on when this kind of AI work is worth trusting.
Final analysis names the classification problem, accuracy results, and at least two systematic failures traced to training data.
For the Facilitator
Prior Knowledge Required
- Read and create simple data tables and bar/line graphs
- Distinguish between an observation (what we measured) and an inference (what we conclude)
- Familiarity with one-step variable assignment in block-based or text-based code
Exit Ticket
"Describe one moment today when your direct measurement told you something the AI missed. What did you measure, and what should the AI have done differently?"
Look for- Specific reference to a measurement (number + unit + location)
- Specific reference to what the AI output said
- A concrete claim about what the AI should have changed (input, comparison, caveat)
Anticipated Misconceptions
"If the AI says it, it must be right — it has access to all the data."
→ Show the AI a deliberately wrong dataset and have students predict the (wrong) output. Reinforce: AI confidence ≠ AI correctness. The AI processes whatever input it receives, including noise and bias.
"Our sensor data is wrong because it doesn't match the AI."
→ Have students re-measure with a second device or different location. Direct measurement is the ground truth — divergence with AI is a signal worth investigating, not an error to "fix."
"The AI is broken if it gives a different answer to the same question twice."
→ This is a feature, not a bug. Use it to discuss probabilistic vs. deterministic systems. Two valid outputs can describe the same data — students should learn to ask "what stayed the same?"
Differentiation
Pre-load the data table with column headers and one example row. Pair with a peer for the AI comparison phase. Provide a sentence frame for the evaluate phase: "We trust the AI when ___ because ___. We don't trust it when ___ because ___."
Standard procedure as written.
Add a fourth phase: students design a follow-up experiment that would resolve a specific AI-vs-observation disagreement they found. They write the protocol; they don't need to execute it in class.
Provide vocabulary cards in advance: sensor, prediction, observation, evidence, divergence. Allow team discussion in students' home language; final class artifacts can be authored bilingually. The verification protocol works in any language — emphasize that direct measurement and lived knowledge are the most authoritative inputs.
Slide Cues — 6 slides
- Collect real data on our campus
- Compare it with what an AI predicts
- Decide together when AI is worth trusting
- Source · Reasoning · Reality · Yourself
- You don't need to know how the AI works to check its work
- A protocol, not a checklist — adapt to the situation
- Teams of 3 deploy sensors at chosen spots
- Record at intervals; vary your locations
- Log a written observation with each reading
- Input your data into the AI tool
- Find at least 3 places it agrees, 3 places it diverges
- Don't correct your data to match the AI
- Apply structured verification practice to your divergences
- Present one trust guideline based on what you found
- Class builds the shared trust framework together
- One moment when your measurement beat the AI
- Be specific — number, location, what AI missed
Standards Alignment — 9 frameworks
Apply scientific principles to design monitoring method for human impact
Analyze data from tests to determine best design solutions
Statistics & Probability — Grade 6
Model with mathematics — Math Practice
Collect data using computational tools; transform for reliability
Refine computational models based on generated data
Discuss bias and accessibility in technology design
Evaluate accuracy, credibility, relevance of information
Collect data and identify data sets; use tools to analyze data
Dear families, This week your student is learning a skill that will matter for the rest of their lives: how to decide when to trust an AI system. In this lesson, students used real sensors to measure conditions around our school and compared what they measured with what an AI predicted. The point is not that AI is bad — the point is that AI works best when paired with someone who knows the real situation. Your student is learning to be that someone. We call the protocol the verification protocol. It has four steps: check the source the AI used, check the reasoning, check the result against reality, and check yourself for what you might have missed. You can use this with your student at home — every time an AI assistant gives you an answer, ask: "How would we check this?" Questions? hello@freyjalabs.com — Freyja Labs (working with San Jose USD)