A new equation for teacher + student success

The need for better training of math teachers has been a widely discussed topic in education for decades, especially as the policy focus has shifted to STEM (science, technology, engineering and math) education and STEM-related careers as drivers of economic prosperity. The U.S. has lagged behind China and India in producing STEM graduates, and one of the challenges for the education system is finding good, qualified math teachers to inspire students to pursue STEM subjects. Creating effective professional development (PD) opportunities for K–12 teachers is difficult, and claims about the effectiveness of PD can be murky because of a reliance on teacher self-assessments. For instance, teachers may self-report that they learn more after participating in a training, but the outcome in the classroom could show a lack of improvement in teacher performance.

To find a solution to these problems, Associate Professor Yasemin Copur-Gencturk has been researching the impact on student performance of an AI-driven professional development program for middle school math teachers. Over the course of three years, Copur-Gencturk and a team of researchers—Chandra Orrill, a former professor in the University of Massachusetts, Dartmouth’s Department of STEM Education and Teacher Development; Jingxian Li, a USC Rossier PhD student; Ben Nye, director of learning sciences at the USC Institute for Creative Technologies; and Allan Cohen, a professor emeritus at the University of Georgia—developed a program that uses a virtual facilitator based on natural language processing. The team then tested its efficacy with a randomized controlled trail that included 53 middle school math teachers and 1,727 students in grades 6 and 7. The results showed that AI is a promising tool for creating effective, scalable teacher PD.

In fact, the students of the teachers who completed the online PD program had higher test scores after their teachers completed the program. This shows that the program not only “improves teachers’ knowledge and instruction, but also improves student learning. Students learned more when their teachers completed the program,” Copur-Gencturk says. The findings are particularly profound because only a limited number of in-person PD programs have been found to improve student learning, according to Copur-Gencturk.

Copur-Gencturk was a high school math teacher early in her career in her native Turkey. Her classroom experiences exposed her to the challenges that math teachers encounter. She went on to study how teachers understand proportional reasoning, a foundational concept that involves understanding multiplicative relationships between different quantities, such as understanding how to adjust ingredients in a recipe to make different amounts with the same taste. It’s a concept that some struggle with, leaving them unable to understand many concepts that involve multiplicative proportional relationships.

Copur-Gencturk and her team found that the students of teachers who completed the online PD program powered by AI had higher test scores after their teachers completed the program.

Copur-Gencturk has felt that the way math has been taught is mainly procedural, leaving little room for students to see how concepts are connected across grade levels. How teachers make sense of concepts relates to how they teach. When teachers can see how the concepts build on one another, they are more likely to have success in teaching their students.

“This research comes out of my desire to find ways to improve the instruction of mathematics,” says Copur- Gencturk. “Computer-based programs have been widely used with school-age kids, but they haven’t been used as much with teachers. This was part of the motivation for me to find a way to scale up effective programs so that teachers can complete their training anytime and anywhere.”

HOW DOES IT WORK?

The program that Copur-Gencturk and her colleagues designed was intended to be scalable and accessible to any middle school math teacher—the virtual agent guided teachers nationwide through the program’s modules. But how exactly does a virtual agent work?

One way to demystify AI technology in PD, and in general, is to think of the virtual agent as a math equation that’s guessing the probability of the next answer, Orrill says. The program developers anticipated likely responses that teachers would enter in the online PD program. The virtual agent determines a likely response and then looks at the actual response entered by the teacher. The tool finds out whether there are enough words that match what was entered to the likely response. At its core, AI is a tool that works off probability. If people understand how probability works, they can start to understand how AI operates.

As the teachers moved through various training modules, the virtual agent analyzed the teachers’ interactions with various learning activities, such as solving a math problem or watching and analyzing mathematics instruction. The virtual agent would then provide personalized, timely feedback to the teachers. If a teacher’s response did not indicate mastery of the key ideas covered in an activity, the virtual agent followed up by asking the teacher to complete another task or to probe their thinking until they met the expectations set for the activity. Once they passed a module, they would then be prompted to move on to the next module.

The program included two modules and eight submodules, with more than 50 activities, and took teachers on average about 11 hours to complete. The number of hours varied depending on the teacher, as some needed to complete additional tasks to master the targeted skills and concepts. By the end, Li felt that participants enjoyed interacting with the virtual agent and came to view it as a helpful study partner that encouraged them to think through their actions and responses.

COVID-19 PANDEMIC PUTS ANYTIME-AND-ANYWHERE TEACHER PD TO THE TEST

Data collection for middle school math teachers who were in the AI-driven professional development study occurred in the summer of 2021, while the COVID-19 pandemic protocols were largely in place. This was a time when many in-person events were limited, making remote online professional development a safe option.

“Yasemin wants to improve how teachers understand mathematics. She’s a very hard worker,” says Orrill. “[She] went up for tenure at the time when COVID was happening, and [was working on] multiple projects. … The world shut down, and she insisted that we get this research done.”

A global pandemic was not going to slow down Copur-Gencturk’s research on the effectiveness of AI-driven professional development, and the only impact was some attrition of teacher and student participants. Some teachers who were in the program dropped out of the study because of health-related issues or because their schools had put restrictions on their students’ taking the survey. That Copur- Gencturk and her team were able to forge ahead without delay, during that uncertain time, is a testamentto her leadership as a researcher.

THE PROMISE OF AI

The success that Copur-Gencturk and her team had with the AI-driven virtual agent shows that the technology can be applied to improve teaching and learning of other math concepts, as well as disciplines outside of mathematics. This program used one of the simplest models for intelligent tutoring systems, which provides adaptive learning opportunities to meet the needs of participants by asking questions, giving tasks and providing feedback to help the participant gain knowledge and to build skills.

“What allowed us to make a difference for these teachers is how this project was grounded in our research on how teachers understand proportional reasoning and other scholars’ research on how students stumble when learning proportional reasoning. We could anticipate where the students make mistakes and where teachers stumble to help make this more impactful,” Orrill explains. “We could create instruction that pushes teachers into those spaces where they may have difficulty before they have a real child sitting in front of them.”

Copur-Gencturk’s research is important because there’s a teacher shortage in the U.S. When indicators of teacher quality are accounted for, the shortage is more acute. Additionally, the COVID-19 pandemic exacerbated the teacher shortage crisis. Math teachers fall into the STEM subject pool of instructors that is already strained.

Finding qualified math teachers in rural communities is also a challenge. Teachers may not be able to travel long distances to attend in-person PD. Having access to quality, online professional development can make a difference in under-resourced school districts. This project helps fill in that gap, providing useful math teacher training for those who may not be able to attend in-person training.

WHAT’S NEXT?

The program Copur-Gencturk and her team developed provided proof of concept that AI-based tools can be used effectively to analyze teacher performance, provide individualized feedback and improve test scores for students of teachers who completed the program. In addition, the research could lead away from less-effective PD programs that are typically not tailored to individual needs.

With the emergence of ChatGPT, AI-driven PD may be applied to reading and literacy. The program that Copur- Gencturk and her team created will be used as a model for others in math and other subjects. A paper has been submitted for publication and is under review.

“Ultimately, this is about finding and creating effective resources for teachers to improve math instruction and math learning at scale.”

—Yasemin Copur-Gencturk, associate professor of education

Copur-Gencturk recently received a $2 million National Science Foundation grant to create an AI-driven program for preservice math teachers to improve knowledge about content and pedagogy through an interactive program with personalized feedback. While Copur-Gencturk and her colleagues will use the program that was previously developed as a prototype, they will incorporate new AI approaches and develop additional materials specifically designed to meet the needs of preservice teachers. She will be the principal investigator, working with co-principal investigators Jiliang Tang from Michigan State University and Allan Cohen and Shiyu Wang from the University of Georgia.

“I’m excited about this new funding because I’m really interested in scaling up teacher training so that when teachers need something, there will be a program that they can complete anywhere to address their needs,” Copur-Gencturk says. “Ultimately, this is about finding and creating effective resources for teachers to improve math instruction and math learning at scale.”