Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents
arXiv preprint arXiv:2608.08852 , 2026
Abstract
AI agents can now solve problems, answer like subject experts, and generate long-form multimodal content. However, whether they can adapt a lesson to fit a specified learner, which education calls Pedagogical Content Knowledge (PCK), has not been benchmarked. To measure it, we introduce the Teaching Monster Challenge, the first instructional video generation benchmark to treat the learner persona as an explicit evaluation criterion. Each system is given a topic and a learner persona and must generate a complete instructional video. Every video is screened by an LLM-judge, ranked by crowd pairwise voting, and finalized by an expert panel. The first edition shows that today’s systems handle the content well but are far weaker at presenting it and adapting it to the learner. The same process exposes a limit of automatic judging. The LLM-judge separates a clear low-performing tail but ranks the strongest systems poorly. The strongest systems receive nearly identical scores from the judge, so its ranking of them does not match human preference. Progress therefore requires not only better teaching systems but also better automatic judges, and we release the benchmark, rubric, and human judgments as a testbed for both.
BibTeX
@article{lin2026findings,
title = {Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents},
author = {Lin, Yi-Cheng and Guo, Yu-Kai and Chen, Szu-Chi and Feng, Bo-Han and Hsu, Yun-Man and Hsieh, Hsiang and Lin, Yu-Jung and Wu, Yue-Ling and Dong, Jia-Kai and Yu, Cheng An and Huang, Yu-Han and Ieong, Lok-Lam and Chen, Kuan-Yu and Tchouang, Ming-Douo and Sun, Shao-Hua and Lin, Che and Ding, Jian-Jiun and Lee, Hung-yi},
journal = {arXiv preprint arXiv:2608.08852},
year = {2026},
}