How Large Language Models are Transforming Teachers' Assessment of Student Competency: A Case Study on LLM-Based Report Writing

Evaluating student competencies is critical to fostering academic success, but the traditional process of assessment and report generation is time-consuming and labor-intensive for teachers. The emergence of Large Language Models (LLMs) presents an opportunity to automate this process, potentially reducing the workload for educators. However, concerns such as hallucinations in LLM outputs and the lack of usability studies involving teachers raise questions about their reliability and practical application. In response, we developed an LLM-based report writing system specifically designed for real-time competency evaluation. To assess its effectiveness, we conducted a case study involving five educational experts, each with over 15 years of experience. These experts used the system to generate student competency reports and found them to be both sensible and specific. While they expressed concerns about issues like insufficient detail, they recognized the system's significant time-saving benefits. Our findings demonstrate that LLMs can positively impact teachers' competency assessments by streamlining the reporting process, offering valuable support for educators.