Researchers are focusing on professional noticing—the ability to observe, interpret, and make instructional decisions based on students' mathematical thinking—as they look for ways to improve classroom teaching. Molly Fisher, professor at Rowan University, and Jonathan N. Thomas, chair of the Department of Teaching and Learning at the University of Kentucky, are leading a study to see if generative AI tools can reliably evaluate this teaching skill and provide feedback to teachers more quickly than traditional methods. The project has a two-year grant of $67,875, with work set to finish in July 2028.
Why Professional Noticing Matters in Math Education
Teachers who notice effectively can spot how students approach arithmetic, build units, and combine sets as they move through math concepts. Jonathan N. Thomas says this skill shows both students' strategies and their confidence or development. The study's framework treats noticing as three connected actions: attending to student thinking, interpreting its meaning, and deciding on instructional responses. This draws from the professional-noticing tradition linked to Elizabeth van Es and Miriam Sherin. The project describes professional noticing as "the ability to attend to, interpret and make instructional decisions based on students’ thinking." Recent research points to professional noticing as a key part of teacher expertise, especially in math instruction.
The Challenge of Measuring Noticing Skills
Traditional ways to evaluate professional noticing take time and resources. Molly Fisher notes her team has found it difficult to get reliable, timely assessments of how teachers notice and respond to student thinking. The usual process involves classroom observation, note-taking, and post-lesson discussion, but feedback rarely arrives fast enough for teachers to use it right away. Research on professional noticing stresses that teacher judgment comes from how teachers reason about knowledge, learners, and instructional contexts, not just from what happens in the classroom, as discussed in Frontiers in Psychology.
AI as a Tool for Faster Assessment
The project will test whether generative AI tools—specifically Claude, ChatGPT, and a third system yet to be chosen—can match expert human raters in evaluating teachers' noticing skills. About 150 elementary teachers will take part, including roughly 100 in-service teachers and 50 preservice teachers. Participants will complete three established noticing assessments: two using classroom video and one using an in-class writing assignment. All teacher responses will be de-identified before AI evaluation. No student videos will be entered into the AI systems.
Expert human raters will score the responses with validated protocols, so researchers can compare AI-generated evaluations with human ratings for validity and reliability. The team is creating standardized scoring materials, such as rubrics, decision trees, and background information for each noticing measure, bundled into packages that other educators might use later. The project does not share rubric dimensions, scoring scales, inter-rater reliability coefficients, model versions, prompt wording, error rates, or statistical thresholds for equivalence between AI and human evaluation. The sample size of 150 participants sets the planned scope for the study.
Evaluating AI Feedback and Teacher Reactions
By comparing AI-generated scores with those from expert raters, the team wants to see how well generative AI tools can measure teachers’ professional noticing of children’s mathematical thinking and generate feedback that helps teacher learning. Teachers will also be surveyed and interviewed about how they interpret and use the feedback from AI systems.
Robin Jocius, a co-investigator at the University of Kentucky, says the project is still in its early stages. Initial testing will use written responses that mimic real teacher answers before moving to actual teacher data, with data collection planned for spring 2027. Feedback reports for teachers are still being developed, but the aim is to deliver actionable insights—such as pointing out specific student strategies or suggesting areas for closer observation—within 24 hours, a speed not currently possible with human-only evaluation.
Perspectives on AI's Role in Teacher Development
Some educators are cautious about using AI to assess nuanced teaching skills. Court Shuller, a middle school English teacher, questions whether AI can truly help teachers improve their noticing abilities and suggests these tools might be more useful for administrators tracking broader trends. Shuller notes that current feedback cycles usually take two to three days and wonders if AI can really speed up or improve this process.
Others see possible advantages. Riya Nathrani, an instructional technology coach, thinks AI could help teachers spot learning patterns faster and respond to student thinking in a timely way. She imagines AI flagging inconsistencies in student explanations, prompting teachers to follow up and address misunderstandings directly with students.
Balancing Automation and Human Judgment
Molly Fisher points out that some educators worry about losing the human element—like reading body language or emotional cues—when using AI for assessment. She adds that professional noticing is qualitative by nature, and researchers already use AI to code qualitative data. The project aims to find out if AI can support, rather than replace, the reflective and interpretive work teachers do in the classroom.
Professional noticing research stresses that AI output should be treated as provisional and checked against professional, mathematical, and contextual evidence. As highlighted in Frontiers in Psychology, "AI-assisted planning was most educative when participants used mathematical structure, student thinking, practicum memory, and feasibility to transform provisional output." The transfer of epistemic authority from teachers to AI systems is a central risk, especially if AI acts as both the source and validator of recommendations without clear pedagogical reasoning.
The research team states that all teacher responses evaluated by AI will be de-identified, and no student videos will be used in the AI tools. Available material shows that any future use of teacher responses or classroom-linked data would require careful separation between research evaluation and operational teacher assessment. The project does not specify the governing institutional-review-board protocol, consent language, retention period, vendor data-use terms, or whether prompts and outputs will be excluded from commercial model training. The study's findings could shape future AI-powered feedback tools for educators, but the technology's effectiveness and acceptance among teachers remain open as the research continues.
No regulator has approved Claude, ChatGPT, or any other commercial generative-AI system as a validated instrument for evaluating teacher professional noticing. The project is set up to investigate whether these systems can match expert human raters, not as a regulatory certification or deployment decision. As noted in Frontiers in Psychology, "They have not been tested and are offered for evaluation in future intervention studies, not as established evidence-based practices." The reported timeline puts the grant’s end in July 2028, so claims about effectiveness, teacher improvement, or student-learning impact are unverified at this stage.
