Journal article in Educational Psychology, in press.
Access [paywall]: Glass and Kang (2020) via doi.org
Scope of Paper
The study compares students performance on homework questions to student performance on related exam questions.
In total, 2433 US-based college students were assessed over an 11 year period.
Key Findings
- Through the course of the study, the percentage of students who did not benefit from correctly answering homework questions increased from 14% in 2008 to 55% in 2017.
- During the most recent two years of the study, when students were asked how they did their homework, students who benefitted from homework reported generating their own answers.
- Students who reported copying the answers from another source did not benefit from homework.
More Detail
Background to the study
The strength of a memory is related to the process of acquiring the memory. We can create memories via both doing things, and by observing things.
It was known that students were more likely to remember a pair of unrelated words, e.g. car – tree, if they were asked to create a sentence linking the two words, e.g. the car crashed into the tree, than if they were asked to remember the word pair (Bower & Clark, 1969). This result raised the issue of whether the sentence, the car crashed into the tree, was intrinsically more memor- able than the pair of unrelated words, car – tree, or whether it was the act of creating the sentence that made it more memorable.
Subsequently, it has been shown that doing things is significantly more likely to result in a memory being formed than observing things. This is known as the ‘generation effect’: generated responses are remembered better than responses that are copied.
A classroom-based example of the generation effect is the ‘testing effect’, related to retrieval practice, stating that answering a question about a recently read text is more likely to result in a memory being formed than extended study of the text.
The prevalence of smartphones has presented a modern challenge to teachers setting homework for students. Increasingly, when answering homework questions students have the option of using a smartphone to look up the answer. This is in contrast to the ‘old fashioned’ method of generating an answer from memory, or guessing. As suggested by the generation effect, these two different approaches have dramatically different impacts on the long-term retention of the answer.
The current study
Answering homework questions ultimately increases exam performance because the retrieval of the answer has a mnemonic effect that increases long-term retention of the answer, hence the probability of retrieving it on the exam. Furthermore, answering a question provides the opportunity for elaborative retrieval, the association of the question with other relevant information.
Data was collected on the homework, quiz, and exam performance of 2,433 students over an 11 year period. During the teaching of the courses, the instructor used distributed questioning techniques (see spaced practice and interleaving) designed to result in incrementally improved performance, culminating in the best performance during the exam.
For analysis, students were separated into two groups: those who performed better in exams than on pre-lesson questions were assumed to be generating their own answers to homework questions, and were labelled homework-generators.
Those who performed better in pre-lesson questions than in exams were assumed to be looking up the answers to homework questions, and were labelled homework-copiers.
These assumptions are based on the theory that peak performance should be reached in the exam, thanks to distributed questioning.
To provide converging evidence for the -generators vs -copiers hypothesis, students at the end of the 2017 course were asked which of the two strategies they had predominantly used during the semester. To provide further evidence for the hypothesis, in 2018, after finishing each homework assignment, students were asked the following question:
When you just did this online homework assignment, which of the alternatives below best describes how you answered each question?
- I always either answered the question from memory or guessed at the answer.
- I looked the answer up in some way including searching the textbook and course materials, using google, and checking with someone else who has already answered the question.
- Sometimes I looked the answer up and sometimes I did not look it up.
Results
Two kinds of data were analysed. The primary data was compiled of homework, quiz, and exam results for the 11 years covered by the study. The secondary data was compiled of the answers to the questions on the approaches to answering homework.
Figure 1 shows the performance of the generators vs. the copiers over time, and Figure 2 shows how the percentage of each cohort identified as copiers changed over time.
^ Figure 1, the performance of the generators vs. the copiers over time.
^ Figure 2, the percentage of each cohort identified as copiers.
As shown by Figure 2, generally, the percentage of homework-copiers has been increasing year on year, and correlates to increased accessibility to the internet via smartphones.
Discussion
The predominant pattern of student performance was a monotonic increase in the probability of answering each successive question in the sequence correctly, indicating that the online pre-lesson homework question, classroom post-lesson question, and online homework review question, all produced learning that ultimately increased the probability of answering the exam question correctly.
The study adds support to the wealth of research supporting the generation effect theory. Generated answers are remembered best, and aid learning, even if they’re wrong. Conversely, copied answers are not well-remembered and therefore do not form part of long term learning.
The increase in smartphone usage over the last decade explains an otherwise counter-intuitive trend: high homework scores predicting lower exam scores. In addition, the most recent portion of this study’s data shows that students who score well in homework are also more likely to report copying strategies after completing homework.
Interestingly, the observed long-term trend is not always matched in post-lesson, short-term performance, showing that the generation effect highlights the difference between ‘performance’ and ‘learning’.
It’s important to note that looking up the answers to questions can still form a useful part of the learning process:
The difference in the long-term retention between self-generated answers and answers retrieved from another source does not mean that a student must abstain from information retrieval technology when doing homework. What it means is that the student must first self-generate an answer before using information retrieval technology to check whether the self-generated answer is correct. In this case, the retrieved answer serves as self-generated feedback to the student response and feedback, especially delayed feedback, increases retention (Sinha & Glass, 2015).