A LIVELY ELECTRONIC COMPENDIUM OF RESEARCH,NEWS,RESOURCES,AND OPINION Astronomy Education review 2012.AER,11,010108-1,10.3847/AER2012011 Digital Devices,Distraction,and Student Performance: Does In-Class Cell Phone Use Reduce Learning? Douglas K.Duncan University of Colorado,Boulder,Colorado,80309 Angel R.Hoekstra University of Colorado,Boulder,Colorado,80309 Bethany R.Wilcox University of Colorado.Boulder.Colorado,80309 Received:04/5/12,Accepted:06/12/12.Published:07/31/12 2012 The American Astronomical Society.All rights reserved Abstract The recent increase in use of digital devices such as laptop computers,iPads,and web-enabled cell phones has generated concern about how technologies affect student performance.Combining observation,survey,and interview data,this research assesses the effects of technology use on student attitudes and learning.Data were gathered in eight introductory science courses at a major university.Results show a significant negative correlation between in-class phone use and final grades,with use of cell phones corresponding to a drop of 0.36+0.08 on a 4-point scale where 4.0=A.These findings are consistent with research (Ophir,Nass,and Wagner 2009,Proceedings of the National Academy of Sciences,106,15583)suggesting students cannot multitask nearly as effectively as they think they can.While 75%of students reported regular cell phone use, observation suggests undergraduates typically underreport the frequency of their in-class use of digital devices. 1.INTRODUCTION With the advent of affordable digital devices,use of technology by students and instructors is increasing in college classrooms(Hoekstra 2009).The present work was motivated by an unpublished study of one large engineering class,which found that students who took notes on laptops earned a full letter grade lower than those who took notes with pen and paper(D.Sieber,personal communication).In that study,students were told about these results after each of three tests given during the semester,and when the laptop note-takers ceased taking notes by means of the technology,their test scores rose to match those of their peers. While these data are suggestive of the negative effects of digital distractions for learning,this course used traditional lecture instruction,and recent research suggests active learning strategies may produce more effective learning overall(Crossgrove and Curran 2008;Smith et al.2009).Many large science classes are now taught with Peer Instruction(Mazur 1997)and clickers(wireless student response systems),and the result is often much higher student engagement(Hoekstra 2008;Gauci et al.2009;Deslauriers,Schelew,and Wieman,2011)and the potential for greater learning(Smith et al.2009).We undertook the present study to determine if the results from the engineering class would be replicated in a larger sample,particularly when clickers and Peer Instruction are used to stimulate engagement in the learning process. 2.METHODS AND RESULTS Research was conducted over two semesters at a large state university in the western U.S.In the first semester. digital-device use was investigated in three large introductory courses for nonscience majors(N-318;
A LIVE LY E LE C T RO NI C COMPE NDI UM O F RESEARCH, NEWS, RESOURCES, AND OPINION Astronomy Education Review 2012, AER, 11, 010108-1, 10.3847/AER2012011 Digital Devices, Distraction, and Student Performance: Does In-Class Cell Phone Use Reduce Learning? Douglas K. Duncan University of Colorado, Boulder, Colorado, 80309 Angel R. Hoekstra University of Colorado, Boulder, Colorado, 80309 Bethany R. Wilcox University of Colorado, Boulder, Colorado, 80309 Received: 04/5/12, Accepted: 06/12/12, Published: 07/31/12 VC 2012 The American Astronomical Society. All rights reserved. Abstract The recent increase in use of digital devices such as laptop computers, iPads, and web-enabled cell phones has generated concern about how technologies affect student performance. Combining observation, survey, and interview data, this research assesses the effects of technology use on student attitudes and learning. Data were gathered in eight introductory science courses at a major university. Results show a significant negative correlation between in-class phone use and final grades, with use of cell phones corresponding to a drop of 0.36 6 0.08 on a 4-point scale where 4.0 ¼ A. These findings are consistent with research (Ophir, Nass, and Wagner 2009, Proceedings of the National Academy of Sciences, 106, 15583) suggesting students cannot multitask nearly as effectively as they think they can. While 75% of students reported regular cell phone use, observation suggests undergraduates typically underreport the frequency of their in-class use of digital devices. 1. INTRODUCTION With the advent of affordable digital devices, use of technology by students and instructors is increasing in college classrooms (Hoekstra 2009). The present work was motivated by an unpublished study of one large engineering class, which found that students who took notes on laptops earned a full letter grade lower than those who took notes with pen and paper (D. Sieber, personal communication). In that study, students were told about these results after each of three tests given during the semester, and when the laptop note-takers ceased taking notes by means of the technology, their test scores rose to match those of their peers. While these data are suggestive of the negative effects of digital distractions for learning, this course used traditional lecture instruction, and recent research suggests active learning strategies may produce more effective learning overall (Crossgrove and Curran 2008; Smith et al. 2009). Many large science classes are now taught with Peer Instruction (Mazur 1997) and clickers (wireless student response systems), and the result is often much higher student engagement (Hoekstra 2008; Gauci et al. 2009; Deslauriers, Schelew, and Wieman, 2011) and the potential for greater learning (Smith et al. 2009). We undertook the present study to determine if the results from the engineering class would be replicated in a larger sample, particularly when clickers and Peer Instruction are used to stimulate engagement in the learning process. 2. METHODS AND RESULTS Research was conducted over two semesters at a large state university in the western U.S. In the first semester, digital-device use was investigated in three large introductory courses for nonscience majors (N ¼ 318;
Astronomy,Geology),and data gathering focused on students'in-class use of laptop computers.The following semester,the study focused on in-class use of cellular phones in five additional courses.All eight courses used iClickersTM and Peer Instruction(sometimes called Think-Pair-Share)exercises to engage students.The instructors teaching the courses were asked to refrain from verbally discussing technology use and note-taking behavior with their students,and all aspects of the research were approved by the internal Human Subjects Review Board. Utilizing a mixed-methods approach consisting of in-class observations,survey responses,and semi-structured interviews,we examined the effects of digital devices on student performance.Survey questions targeted demographics,student attitudes,and self-reported levels of technology use.The data were collected via a clicker polling system and then anonymously correlated with final course grades.A clicker system separate from that used by the instructor was brought into the classroom to collect our data,and students were assured of anonymity.Semi-structured interviews(N=24)provided insight into student attitudes as related to behavior and the use of digital devices.In-class observations (N=31 days)recorded the behavior of students and the instructors during lecture;this data permitted verification of students'self-reports in interviews and survey responses. In two of the three courses observed in semester 1,lecture notes were posted for student reference:one before class and in the other,after.Survey responses indicate 60%of the sample (N=316:91.6%response rate)took notes with pen and paper,12%took notes on a laptop computer or some other electronic device,and 28% reported no note taking.In the third course,the instructor did not post notes.Students'final grades were statistically indistinguishable across the three note-taking methods-we did not replicate the experience in the Engineering class that motivated this study.Our initial research therefore suggests that peer instruction and student engagement may play an important role in mitigating"digital distraction." It is interesting to note that when we presented the results of the initial Sieber study(personal communication)to faculty members,many interpreted the results as suggesting that writing on paper,drawing arrows or diagrams. and underlining topics engage the brain in a more complete way than typing.When we presented the results to students,the response was uniformly."the laptop users are getting distracted by Facebook or other things they can do during class."Our observations confirm that the student interpretation is more likely the correct one. However,we also learned that the laptop computer is no longer the device of choice for most students to bring to class. Given recent national data confirming high rates of computer ownership(EDUCAUSE 2010)among undergraduates,the number of students who took notes on laptop computers was much lower than we expected. At the same time,our observation and survey data indicate very high rates of in-class cell phone use:75%of the sample (anonymously)reported regular phone use during lecture.These data motivated an extension of the study into the subsequent semester,where we sought to isolate the effects of cell phone use on learning outcomes. Students in five additional classes (N=392,all introductory astronomy)reported their frequency of in-class cell phone use,and these data were then correlated with their final course grades. All survey data were kept anonymous and participation in this study was voluntary.Response rates across the five courses ranged from 96%to 98%of those in attendance on the day the data were collected,and the number of responses ranged from 44 in the smallest class to 111 in the largest.Results from the five courses (combined) are presented in Table 1 and Figure 1. The average grade difference between students who use their phones at all and those who do not is 0.36+0.08. Cell phone use is significantly correlated with reduced learning outcomes:students who reported no cell phone use earned significantly higher grades than those who used their phones during class. Interestingly,those who use their phones reported an average frequency of three times per class period,yet the observation data suggest this frequency may in fact be much higher.Observation notes suggest the average frequency of cell phone use is closer to seven times per class period(N=32 observations,average=6.85 incidents of use per user,per class period),indicating that undergraduates typically underreport their frequency of in-class cell phone use.These data are drawn from a relatively small sample;as we do not have more comprehensive data on rates of cell phone use across all five courses,we focus on the differences between phone users and nonusers in the ensuing discussion
Astronomy, Geology), and data gathering focused on students’ in-class use of laptop computers. The following semester, the study focused on in-class use of cellular phones in five additional courses. All eight courses used iClickersTM and Peer Instruction (sometimes called Think-Pair-Share) exercises to engage students. The instructors teaching the courses were asked to refrain from verbally discussing technology use and note-taking behavior with their students, and all aspects of the research were approved by the internal Human Subjects Review Board. Utilizing a mixed-methods approach consisting of in-class observations, survey responses, and semi-structured interviews, we examined the effects of digital devices on student performance. Survey questions targeted demographics, student attitudes, and self-reported levels of technology use. The data were collected via a clicker polling system and then anonymously correlated with final course grades. A clicker system separate from that used by the instructor was brought into the classroom to collect our data, and students were assured of anonymity. Semi-structured interviews (N ¼ 24) provided insight into student attitudes as related to behavior and the use of digital devices. In-class observations (N ¼ 31 days) recorded the behavior of students and the instructors during lecture; this data permitted verification of students’ self-reports in interviews and survey responses. In two of the three courses observed in semester 1, lecture notes were posted for student reference: one before class and in the other, after. Survey responses indicate 60% of the sample (N ¼ 316; 91.6% response rate) took notes with pen and paper, 12% took notes on a laptop computer or some other electronic device, and 28% reported no note taking. In the third course, the instructor did not post notes. Students’ final grades were statistically indistinguishable across the three note-taking methods—we did not replicate the experience in the Engineering class that motivated this study. Our initial research therefore suggests that peer instruction and student engagement may play an important role in mitigating “digital distraction.” It is interesting to note that when we presented the results of the initial Sieber study (personal communication) to faculty members, many interpreted the results as suggesting that writing on paper, drawing arrows or diagrams, and underlining topics engage the brain in a more complete way than typing. When we presented the results to students, the response was uniformly, “the laptop users are getting distracted by Facebook or other things they can do during class.” Our observations confirm that the student interpretation is more likely the correct one. However, we also learned that the laptop computer is no longer the device of choice for most students to bring to class. Given recent national data confirming high rates of computer ownership (EDUCAUSE 2010) among undergraduates, the number of students who took notes on laptop computers was much lower than we expected. At the same time, our observation and survey data indicate very high rates of in-class cell phone use: 75% of the sample (anonymously) reported regular phone use during lecture. These data motivated an extension of the study into the subsequent semester, where we sought to isolate the effects of cell phone use on learning outcomes. Students in five additional classes (N ¼ 392, all introductory astronomy) reported their frequency of in-class cell phone use, and these data were then correlated with their final course grades. All survey data were kept anonymous and participation in this study was voluntary. Response rates across the five courses ranged from 96% to 98% of those in attendance on the day the data were collected, and the number of responses ranged from 44 in the smallest class to 111 in the largest. Results from the five courses (combined) are presented in Table 1 and Figure 1. The average grade difference between students who use their phones at all and those who do not is 0.36 6 0.08. Cell phone use is significantly correlated with reduced learning outcomes: students who reported no cell phone use earned significantly higher grades than those who used their phones during class. Interestingly, those who use their phones reported an average frequency of three times per class period, yet the observation data suggest this frequency may in fact be much higher. Observation notes suggest the average frequency of cell phone use is closer to seven times per class period (N ¼ 32 observations, average ¼ 6.85 incidents of use per user, per class period), indicating that undergraduates typically underreport their frequency of in-class cell phone use. These data are drawn from a relatively small sample; as we do not have more comprehensive data on rates of cell phone use across all five courses, we focus on the differences between phone users and nonusers in the ensuing discussion
Table 1.Frequency of cell phone use vs class grade. Self-reported frequency of cell phone use per 1 h class,across five Astronomy courses Never 1-2 times/class 3-5 >5 times All phone users Number of students 93 175 66 58 299 Mean-all five courses 3.26 2.96 2.86 2.81 2.91 Standard deviation 0.68 0.80 0.84 0.66 0.79 Standard deviation of mean 0.07 0.06 0.10 0.09 0.05 The difference in final grades between phone users and nonusers is highly statistically significant,but causality cannot be determined from this data.It could be that students who are already more disciplined are more likely to turn their phones off during class or that students who are busier outside of class are more likely to use their phones in class.Still,the observation and interview data suggest many undergraduates likely overestimate their ability to multitask.If students miss some instruction while engaging with digital devices,they may end up earning lower grades overall. This conclusion is consistent with a recent finding that Stanford undergraduates consistently overestimate their ability to multitask(Ophir,Nass,and Wagner 2009).In our interviews,students argued both for and against the ability to multitask effectively:"I could probably take notes on a computer,but I don't [because]it's really easy to be doing other things in class than taking notes.""I feel phones can definitely be a distraction if you let them, but if you're using your phone when you have a minute between taking notes or while the professor is going back and discussing something...I don't find my phone distracting [then]." This issue will become increasingly important for teachers as more and more students come to class with Internet-capable handheld devices (EDUCAUSE 2010).On the bright side,our observation data suggest the use of clickers and peer instruction may mitigate distractions due to digital devices.We observed several occasions in which students stopped using their computers and cellular phones(for noncourse related purposes)when clicker questions were used.Yet the drive to multitask is strong,as confirmed by our interviewees:"I think I'll still bring [my laptop]to class,but I'll probably have to close all the applications except for [Microsoft]Word, that way I won't be tempted...If they're open,it's just so easy to go over there and come back.""It's just staying connected.I know if I look at my phone,there's a good chance someone's gonna want to know if I want to [meet up]after class...there's an email,I can check my horoscope.I could do something else right now." 3.DISCUSSION This research is limited by a single institutional context;further investigation is needed to determine whether these findings apply to learning behavior in other disciplines and across educational contexts.In particular. research might address how in-class use of digital devices is detrimental to the learning of others,in addition to 3.60 Average of 5 Astronomy Classes(392 students) 3.40 3.20 3.00 2.80 20 2.20 2.00 Never 1-2 times 3-5 times Over 5 Self-reported Frequency of Cell Phone Use(per class) Figure 1.Self-reported frequency of cell phone use vs final grade
The difference in final grades between phone users and nonusers is highly statistically significant, but causality cannot be determined from this data. It could be that students who are already more disciplined are more likely to turn their phones off during class or that students who are busier outside of class are more likely to use their phones in class. Still, the observation and interview data suggest many undergraduates likely overestimate their ability to multitask. If students miss some instruction while engaging with digital devices, they may end up earning lower grades overall. This conclusion is consistent with a recent finding that Stanford undergraduates consistently overestimate their ability to multitask (Ophir, Nass, and Wagner 2009). In our interviews, students argued both for and against the ability to multitask effectively: “I could probably take notes on a computer, but I don’t [because] it’s really easy to be doing other things in class than taking notes.” “I feel phones can definitely be a distraction if you let them, but if you’re using your phone when you have a minute between taking notes or while the professor is going back and discussing something… I don’t find my phone distracting [then].” This issue will become increasingly important for teachers as more and more students come to class with Internet-capable handheld devices (EDUCAUSE 2010). On the bright side, our observation data suggest the use of clickers and peer instruction may mitigate distractions due to digital devices. We observed several occasions in which students stopped using their computers and cellular phones (for noncourse related purposes) when clicker questions were used. Yet the drive to multitask is strong, as confirmed by our interviewees: “I think I’ll still bring [my laptop] to class, but I’ll probably have to close all the applications except for [Microsoft] Word, that way I won’t be tempted… If they’re open, it’s just so easy to go over there and come back.” “It’s just staying connected. I know if I look at my phone, there’s a good chance someone’s gonna want to know if I want to [meet up] after class…there’s an email, I can check my horoscope. I could do something else right now.” 3. DISCUSSION This research is limited by a single institutional context; further investigation is needed to determine whether these findings apply to learning behavior in other disciplines and across educational contexts. In particular, research might address how in-class use of digital devices is detrimental to the learning of others, in addition to Table 1. Frequency of cell phone use vs class grade. Self-reported frequency of cell phone use per 1 h class, across five Astronomy courses Never 1–2 times/class 3–5 >5 times All phone users Number of students 93 175 66 58 299 Mean—all five courses 3.26 2.96 2.86 2.81 2.91 Standard deviation 0.68 0.80 0.84 0.66 0.79 Standard deviation of mean 0.07 0.06 0.10 0.09 0.05 Figure 1. Self-reported frequency of cell phone use vs final grade
affecting individual students.When asked on the survey,just 32%of this sample found cell phone use by other students distracting,but this number increased to 46%when asked about the distraction-causing potential of other students'use of laptop computers.While it seems reasonable to assume that laptops would be more likely to distract other students than cell phones(given the larger size of laptop display screens),observation confirms many more students use cell phones regularly during class than they do laptop computers. Digital devices may be more likely to distract students in large,nonmajor courses such as the ones studied here. Future studies might seek to replicate these findings in major-and upper-level courses where class sizes are smaller and students may be more committed to doing well.Also important for teaching faculty,the survey data confirm students do not necessarily believe in-class cell phone use is disrespectful.Across the five courses,47% of students agreed that in-class cell phone use is"somewhat disrespectful,but it depends on the instructor's policy."Many students in this generation have their cell phones with them 24h a day,and some claimed in interviews that if a professor does not openly ban cell phone use,it is the student's right to use their phone.These data imply that in today's classrooms,there is be an even greater need than in the past for instructors to clearly state their policies concerning student use of digital devices. Acknowledgments We gratefully acknowledge Professor Diane Sieber for sharing the unpublished data that inspired this study.This research was funded by the Center for the Integration of Research,Teaching,and Learning(CIRTL)and the President's Teaching and Learning Collaborative at the university where this study was conducted. References Crossgrove,K.,and Curran,K.L.2008,"Using Clickers in Nonmajors-and Majors-Level Biology Courses: Student Opinion,Learning,and Long-Term Retention of Course Material,"CBE Life Sciences Education,7, 146. Deslauriers,L.,Schelew,E.,and Wieman,C.2011,"Improved Learning in a Large-Enrollment Physics Class," Science,332,862. EDUCAUSE Center for Applied Research 2010,"Students and Information Technology,"ECAR Research Study 6,37. Gauci,S.A.,Dantas,A.M.,Williams,D.A.,and Kemm,R.E.2009,"Promoting Student-Centered Active Learning in Lectures with a Personal Response System,"Advances in Physiology Education,33,30. Hoekstra,A.2008."Vibrant Student Voices:Exploring Effects of the Use of Clickers in Large College Classrooms,"Learning,Media,and Technology,33(4),329. Hoekstra,A.2009,"A Socio-Cultural Analysis of the Use of Clickers in Higher Education,"doctoral disserta- tion,University of Colorado,available in Digital Dissertations. Mazur,E.1997,Peer Instruction:A User's Manuel,Upper Saddle River,NJ:Prentice Hall. Ophir,E.,Nass,C.,and Wagner,A.D.2009,"Cognitive Control in Media Multi-Taskers,"Proceedings of the National Academy of Sciences,106,15583. Smith.M.K.,Wood,W.B..Adams.W.K..Wieman.C.K..Knight.J.K..Guild,N.,and Su,T.T.2009,"Why Peer Discussion Improves Student Performance on In-Class Concept Questions,"Science,323.122. ER 010108-1-010108-4
affecting individual students. When asked on the survey, just 32% of this sample found cell phone use by other students distracting, but this number increased to 46% when asked about the distraction-causing potential of other students’ use of laptop computers. While it seems reasonable to assume that laptops would be more likely to distract other students than cell phones (given the larger size of laptop display screens), observation confirms many more students use cell phones regularly during class than they do laptop computers. Digital devices may be more likely to distract students in large, nonmajor courses such as the ones studied here. Future studies might seek to replicate these findings in major- and upper-level courses where class sizes are smaller and students may be more committed to doing well. Also important for teaching faculty, the survey data confirm students do not necessarily believe in-class cell phone use is disrespectful. Across the five courses, 47% of students agreed that in-class cell phone use is “somewhat disrespectful, but it depends on the instructor’s policy.” Many students in this generation have their cell phones with them 24 h a day, and some claimed in interviews that if a professor does not openly ban cell phone use, it is the student’s right to use their phone. These data imply that in today’s classrooms, there is be an even greater need than in the past for instructors to clearly state their policies concerning student use of digital devices. Acknowledgments We gratefully acknowledge Professor Diane Sieber for sharing the unpublished data that inspired this study. This research was funded by the Center for the Integration of Research, Teaching, and Learning (CIRTL) and the President’s Teaching and Learning Collaborative at the university where this study was conducted. References Crossgrove, K., and Curran, K. L. 2008, “Using Clickers in Nonmajors- and Majors-Level Biology Courses: Student Opinion, Learning, and Long-Term Retention of Course Material,” CBE Life Sciences Education, 7, 146. Deslauriers, L., Schelew, E., and Wieman, C. 2011, “Improved Learning in a Large-Enrollment Physics Class,” Science, 332, 862. EDUCAUSE Center for Applied Research 2010, “Students and Information Technology,” ECAR Research Study 6, 37. Gauci, S. A., Dantas, A. M., Williams, D. A., and Kemm, R. E. 2009, “Promoting Student-Centered Active Learning in Lectures with a Personal Response System,” Advances in Physiology Education, 33, 30. Hoekstra, A. 2008, “Vibrant Student Voices: Exploring Effects of the Use of Clickers in Large College Classrooms,” Learning, Media, and Technology, 33(4), 329. Hoekstra, A. 2009, “A Socio-Cultural Analysis of the Use of Clickers in Higher Education,” doctoral dissertation, University of Colorado, available in Digital Dissertations. Mazur, E. 1997, Peer Instruction: A User’s Manuel, Upper Saddle River, NJ: Prentice Hall. Ophir, E., Nass, C., and Wagner, A. D. 2009, “Cognitive Control in Media Multi-Taskers,” Proceedings of the National Academy of Sciences, 106, 15583. Smith, M. K., Wood, W. B., Adams, W. K., Wieman, C. K., Knight, J. K., Guild, N., and Su, T. T. 2009, “Why Peer Discussion Improves Student Performance on In-Class Concept Questions,” Science, 323, 122. ÆR 010108-1–010108-4