details
Article (journal) accessible via
URN: urn:nbn:de:0111-pedocs-346189
DOI: 10.25656/01:34618; 10.1111/jcal.70105
URN: urn:nbn:de:0111-pedocs-346189
DOI: 10.25656/01:34618; 10.1111/jcal.70105
| Title |
ChatGPT in education. An effect in search of a cause. |
|---|---|
| Authors |
Weidlich, Joshua   ;
Gašević, Dragan ;
Drachsler, Hendrik ;
Kirschner, Paul |
| Source | Journal of computer assisted learning 41 (2025) 5, 10 S.  |
| Document | full text (284 KB) |
| License of the document |
|
| Keywords (German) | Artificial Intelligence |
| sub-discipline | Empirical Educational Research |
| Document type | Article (journal) |
| ISSN | 1549-6953; 15496953 |
| Language | English |
| Year of creation | 2025 |
| review status | Peer-Reviewed |
| Abstract (English): | Background: As researchers rush to investigate the potential of AI tools like ChatGPT to enhance learning, well-documented pitfalls threaten the validity of this emerging research. Issues of media comparison research, where the confounding of instructional methods and technological affordances is unrecognised, may render effects uninterpretable. Objectives: Using a recent meta-analysis by Deng et al. (Computers & Education, 227, 105224) as an example, we revisit key insights from the media/methods debate to highlight recurring conceptual challenges in ChatGPT efficacy studies. Methods: This conceptual article contrasts nascent ChatGPT research with the more established literature on Intelligent Tutoring Systems to identify three non-negotiable considerations for interpretable effects: (1) descriptions of the precise nature of the experimental treatment and (2) the activities of the control group, as well as (3) outcome measures as valid indicators of learning. To provide some initial evidence, we audited a subset of primary experiments included in Deng et al.'s meta-analysis, demonstrating that only a small minority of studies satisfied all three non-negotiable considerations. Results and Conclusions: Loosely defined treatments, mismatched or opaque controls, and outcome measures with unclear links to durable learning obscure causal claims of this emerging literature. Observed gains cannot, at this time, be confidently attributed to ChatGPT, and meta-analytics effect sizes may over- or understate its benefits. Progress, we argue, will require rigorous designs, transparent reporting, and a critical stance toward "fast science." (DIPF/Orig.) |
| Statistics |  Number of document requests
|
| Checksums | checksum comparison as proof of integrity |
| Date of publication | 25.03.2026 |
| Citation | Weidlich, Joshua; Gašević, Dragan; Drachsler, Hendrik; Kirschner, Paul: ChatGPT in education. An effect in search of a cause. - In: Journal of computer assisted learning 41 (2025) 5, 10 S. - URN: urn:nbn:de:0111-pedocs-346189 - DOI: 10.25656/01:34618; 10.1111/jcal.70105 |
share content at social platforms
Teile diese Seite:
