Assessing ChatGPT's Effectiveness in Pharmacy Education: Impact on Student Performance using User Studies

Author : LARA ALHAJJ

Abstract :By introducing ChatGPT, OpenAI revolutionised the AI industry. Several studies suggest that ChatGPT can be utilized in the education field across various disciplines, ranging from medicine to history. This study examines the effectiveness of ChatGPT in answering exam questions for a pharmacy course called Cosmeticology. Particularly, we arrange user studies with Cosmeticology students of Cyprus International University. Same students took two similar exams: Group A employed their knowledge, while Group B used ChatGPT’s assistance while answering questions. To eliminate learning effect, the two groups were swapped. Pre-questionnaires were given to assess search experiences of students and post-questionnaires were given to evaluate students’ opinions on the use of ChatGPT in exams. Multiple choice and classical questions were carefully developed for both exams. Results analysis uses exam and questionnaire results: Correct responses (scores), post questionnaire feedback (participant opinions) and statistical paired t-tests are used. On average, ChatGPT solved classical questions better than manual replies (95.84% vs. 91.68% respectively). Notably, ChatGPT’s accuracy for open-ended questions was substantially higher. Manual responses outscored ChatGPT on multiple-choice issues (83.01% vs. 67.5%), which was the only statistically significant finding from the paired t-test. ChatGPT and manual approaches perform similarly on the majority of tests. The relevance and enjoyment scores of the post-questionnaire results show that ChatGPT does not improve or interfere with the learning process. This study suggests that whereas ChatGPT provides distinct advantages in particular situations (e.g., classical enquiries), its overall influence on the wider learning experience may be contingent upon nature of the task. The results show that ChatGPT may be useful for open-ended and classical queries. Another observation is that ChatGPT is useful for searching tasks, but pharmacy students did not have the prompt engineering skills to perform detailed enquiries

Keywords :ChatGPT, user study, Artificial Intelligence, Cosmeticology, Pharmacy education.

Conference Name :International Conference on Dermatology and Cosmetology Conference (ICDECOCO-25)

Conference Place Istanbul, Turkey

Conference Date 6th Mar 2025