image/svg+xml
Introduction
Personalized support and formative feedback are instrumental in the instruction and development of students. As already large classroom sizes increase, demands for limited resources also increase. In online e-learning environments, students often lack the support they need to fill gaps in their understanding.The Immediate Feedback System (IFS) is a robust, software system that aims to mitigate this problem by providing an interactive environment for students to submit work and receive continuous formative assessment feedback.
Methodology
a) Core tool developmentNatural and artificial language processing scripts were developed using open-source libraries. These scripts are used to parse student submissions and match error patterns.b) Feedback consolidation and presentationPattern matches, feedback, and suggestions for error correction are presented to the student providing the opportunity to improve their work with contextually relevant, on-demand information.c) Educational data-miningInteractions on the website are recorded to determine which errors are most prevalent, and which feedback is most useful to students.d) Survey data collection and course outcome assessmentStudents are asked to rank their self-efficacy and achievement goal orientation through a series of short surveys. Students consent to share grade outcomes in participating courses for this study. These data will be paired to assess how usage of the IFS affects assignment and course outcomes.
Discussion
Improving online educationThe IFS provides a modern, user-friendly environment for students to submit their work and receive feedback consisting of errors, warnings, and suggestions. Students can view statistics about their work, and relative standing when compared to peers within their classes. It is hoped that students will be able to use this system to iteratively improve their work and reduce the frequency of errors in writing and programming.Where students previously may not have had access to explanations of common or minor technical errors in their work, there now exists a platform and method for accessing this information.Student behaviour in an e-learning environment can be modeled through applications of the Social Cognitive Theory, which claims there is a correlation between environment, personal engagement, and social interaction [1]. In the IFS, this is done by recording interactions with the system, trends in errors over time, and surveying students.Interdisciplinary effortsBuilding the IFS required domain knowledge from computer science, statistics, psychology, and education. The IFS is designed with multiple disciplines in mind, providing tools for language and writing, APA formatting, code auditing, and more. It is also easy to add additional tools to the system, spanning multiple fields of study.Future Worka) Improving Error Reporting Methods using the IFS:The IFS could be used to improve tailored error reporting through collecting data on when feedback is most useful in certain contexts, aggregating that data, and building 'smarter' models, though human-computer interaction [3].b) Investigation of the Effects of Gamified Learning:Gamification has shown potential [2] to increase student motivation. An extension to the IFS which gamifies the learning experience may be built to investigate its effect on the learning experience.
Acknowledgments
Thanks go to Thomas Kosciuch and Alexander Parent for assistance in software testing and design suggestions throughout the development process.Thanks also go to the University of Guelph Central Computing Services for providing the physical hardware resources that were needed to develop and test this software.The IFS was made possible through the use of open-source software. For a complete list of included software libraries, see https://github.com/ian-james/IFS
References
[1] Bandura, A., and Walters, R. H. (1977). Social learning theory. Prentice-Hall Englewood Clis, NJ.[2] Prince, J. D. (2013). Gamification. Journal of Electronic Resources in Medical Libraries, 10(3), 162-169.[3] Tang, T. Y., and McCalla, G. (2005). Smart recommendation for an evolving e-learning system: Architecture and experiment. International Journal on elearning, 4(1), 105.
Figure Two.
Screenshot of the IFS Feedback System Interface. Detected errors and warnings within the text are presented as the highlighted sections. Clicking on an error yields more information, as well as suggestions for improvement.
Figure One.
An architectural diagram adapted from work by James Fraser, and updated to reflect the current state of the Immediate Feedback System v0.0.1. The primary interface to the system is a website front end, which communicates with a) the Assessment Core (comprising of various configurable tools and file processing), b) the Personalization System (which adapts the system to stored student preferences), and c) the Feedback System. Feedback and recommendations that are generated by the server back-end are consolidated and presented as a marked-up, interactive preview of the original file.
b) Personalization System
Adaptive ModelingSystem
Student Preferences
Student Model
Assessment Feedback
Student
IFS Website
a) Assessment Core
Tool Selection
File Upload
File Assessment
c) Feedback System
Consolidation andPresentation
RecommendationSystem
FeedbackGeneration
Preference Data
Copyright © 2017. University of Guelph, School of Computer Science.
This poster is also available online at <http://krourke.org/projects/pres/CUPS17>
Keefer Rourke, B.CS. student. <krourke@uoguelph.ca>
Keefer Rourke, James Fraser, Judi McCuaig, Daniel Gillis University of Guelph, School of Computer Science
Investigating the impact of automated feedback in e-learning environments