Loading...
ChatGPT vs. the State University of New York Purchase College Mathematics/Computer Science Majors: Can These Students Defeat the AI Chatbot?
Journal Title
Readers/Advisors
Shablinsky, Irina R.
Journal Title
Term and Year
Spring 2024
Publication Date
2024
Book Title
Publication Volume
Publication Issue
Publication Begin
Publication End
Number of pages
Collections
Files
7870_Khadijah_Boothe.pdf
Adobe PDF, 1.94 MB
- Embargoed until 2029-05-14
Research Projects
Organizational Units
Journal Issue
Abstract
This study aims to determine if the Artificial Intelligence chatbot, ChatGPT, can perform better on assignments, tests, and labs than Mathematics/Computer Science students at the State University of New York, Purchase College. Student grades and these materials are from Calculus II, Computer Science I, and Number Theory classes. The different versions of ChatGPT that were asked questions from the materials students completed are: GPT-3.5, GPT-4, and Wolfram GPT. The answers and grades that these chatbots generated were compared to each other and the students' answers and average grades. For each of the assignments, labs, and exams, the comparisons are based on which grade is the highest, out of the three different versions of ChatGPT and the students, as well as whether or not their answers utilize the most efficient strategy and is concise. GPT-4 outdid GPT-3.5 on Computer Science I problems and Wolfram GPT on Calculus II materials. Wolfram GPT eclipsed all versions of ChatGPT on the Number Theory exam. Yet, overall, students did better than all versions of ChatGPT. However, there are caveats to this finding because of the limitations this study encountered. On average one student from each class participated, thus the results are not totally conclusive. However, they do indicate that ChatGPT cannot be trusted to do math and computer science related tasks without human oversight. Therefore, corporations and students cannot wholly rely on ChatGPT. Based on ChatGPT's performance so far, it is likely in the future that it will learn enough to become as formidable as some fear.
Citation
DOI
Description
Accessibility Statement
Purchase College - State University of New York (PC) is committed to ensuring that people with disabilities have an opportunity equal to that of their nondisabled peers to participate in the College's programs, benefits, and services, including those delivered through electronic and information technology. If you encounter an access barrier with a specific item and have a remediation request, please contact lib.ir@purchase.edu.