|
Modern ubiquitous sensing produces immense information collections that offer unprecedented amounts of data for knowledge extraction, inference, and learning. Consequently, the significance of harnessing available artificial intelligence tools to boost human learning capabilities and accelerate the learning process is growing exponentially. Human learning relies on the activation of brain regions containing multi-level trees of knowledge that can be effectively built into human pretrained libraries through asking key questions at each level. In this pursuit, Multiple Choice Questions (MCQs) are frequently used due to their efficiency in grading and providing feedback. In particular, well-designed MCQs can assess knowledge across different levels of Bloom's Taxonomy, a framework that classifies different levels of cognitive skills and abilities that students use to learn. Thus, by asking these MCQs, we help learners to activate neural pathways involved in perception, cognition, and high-level functions such as meta-cognition, analysis, evaluation, and synthesis, as well as those related to information encoding, retrieval, and long-term memory formation. This study explores an AI-driven approach to creating and evaluating Multiple Choice Questions (MCQs) in domain-independent scenarios. The methodology involves generating Bloom's Taxonomy-aligned questions through zero-shot prompting with GPT-3.5, validating question alignment with Bloom’s Taxonomy with RoBERTa–a language model grounded in transformer architecture–, evaluating question quality using Item Writing Flaws (IWF)--issues that can arise in the creation of test items or questions--, and validating questions using subject matter experts. Our research demonstrates GPT-3.5's capacity to produce higher-order thinking questions, particularly at the "evaluation" level. We observe alignment between GPT-generated questions and human-assessed complexity, albeit with occasional disparities. Question quality assessment reveals differences between human and machine evaluations, correlating inversely with Bloom's Taxonomy levels. These findings shed light on automated question generation and assessment, presenting the potential for advancements in AI-driven human-learning enhancement approaches.
|
