Research Windows How Does Technology Influence Student Learning? This month's Research Windows highlights research findings for frequently asked questions regarding technology's effects on student learning as determined by the Center for Applied Research in Educational Technology (CARET) By John Cradler;Mary McNabb,Molly Freeman and Richard Burchett Subject:Research on academic performance and technology Audience:Teachers,technology coordinators.library/media special- ists,teacher educators ade Level:K-12 (Ages hnology:All ndards:NETS-T I .a v.iste.org/standards) Supplement:www.iste.orq/L&L Leamning&Leading with Technology Volume9 Number8 CopyrigtISTE (ntemational Socety for Technology in Education)0191(UCanada)or1.5413023777()istesArighs reserved
46 Learning & Leading with Technology Volume 29 Number 8 Subject: Research on academic performance and technology Audience: Teachers, technology coordinators, library/media specialists, teacher educators Grade Level: K–12 (Ages 5–18) Technology: All Standards: NETS•T II; NETS•A I (www.iste.org/standards) Supplement: www.iste.org/L&L How Does Technology Influence Student Learning? This month’s Research Windows highlights research findings for frequently asked questions regarding technology’s effects on student learning as determined by the Center for Applied Research in Educational Technology (CARET). By John Cradler, Mary McNabb, Molly Freeman, and Richard Burchett Research Windows 46 Learning & Leading with Technology Volume 29 Number 8 Copyright © ISTE (International Society for Technology in Education), 1.800.336.5191 (U.S. & Canada) or 1.541.302.3777 (Int’l), iste@iste.org. All rights reserved
Research Windows vidence is mounting to support Content Area Achievement Kottkamp.1999).This curriculum was technology advocatesclaims First and foremost.research remind reinforced with teacher instruction and us that techn uppo um sta hools)appe an positive oth attr the alignm a andards o the stud are,te (CARET)has der t un ling of m nd cu ter ently asked ners to revisit curricular standards as cos student they select technol achievement and academic pe A review of studies o mance in relation to three primary cur- CEO Forum (2001)emphasizes ricular goals: "technology can have the greatest impact when integrated into the cu achievement for elementary middl 1.Achievement in content area riculum to achieve clear,measurable and high school students when teach earning 2.Highe educational objecti ers are skilled in guiding student activi order thinking and problem- A recent study ates how ties (Hillel.kieran.Gurtner 1989 solving skill development McCoy.1996:Simmons cope 3.Workforce preparation 1990,1993). The research findings also emphasize tly increas In English language arts and social the importance of using technology es neight-yea studies,teachers report observing sig in conjunction with collaborative I nificant change in student skills and ing methods and leadership aimed at D knowledge acquired after technology planning for school im- 1999).studen their students n the tochnol first multimedia project.After student purpose For access to completion of the first multimedia findings applicable project,teachers reported increased age increases of 94 on,planning.procure student knowledge in points in combined SAT I performance ogy in sc over students who participated in the raditional school and vist at http. form efforts included a pioneer laptop to re ons program.where all students and faculty carry portable computers and have stn the tent (Cradle dy access to a campus network Along with technology implementa tion,Brewster's exter Higher-Order Skills Devel e school reform Higher- order thinking and proble curriculum,ar ving skills (e.g.inf lua h.2000 A152 anal and co ntr and evaluating) A West Virg enable eamers to apply their content knowledge in a va nderstand. and ing of content domains.Though some uctional so technology applications are designed ware (Mann,Shakeshaft,Becker., for use in specific content areas educa. May 00 Leamning&Leading with Technology
May 2002 Learning & Leading with Technology 47 Research Windows Evidence is mounting to support technology advocates’ claims that 21st-century information and communication tools as well as more traditional computer-assisted instructional applications can positively influence student learning processes and outcomes. The Center for Applied Research in Educational Technology (CARET) has gathered compelling research and evaluation findings to answer frequently asked questions about how technology influences student achievement and academic performance in relation to three primary curricular goals: 1. Achievement in content area learning 2. Higher-order thinking and problemsolving skill development 3. Workforce preparation The research findings also emphasize the importance of using technology in conjunction with collaborative learning methods and leadership aimed at technology planning for school improvement purposes. For access to additional research findings applicable to collaboration, planning, procurement, and implementation of technology in schools, read the supplement online at www.iste.org/L&L and visit the CARET Web site at http:// caret.iste.org. Content Area Achievement First and foremost, research reminds us that technology generally improves performance when the application directly supports the curriculum standards being assessed. In other words, making standards and learning objectives explicit to the students is part of effective technology implementation. Technology integration activities often require teachers and curriculum planners to revisit curricular standards as they select technology applications. A review of studies conducted by the CEO Forum (2001) emphasizes: “technology can have the greatest impact when integrated into the curriculum to achieve clear, measurable educational objectives.” A recent study illustrates how alignment between content-area learning standards and carefully selected technology uses can significantly increase test scores. In an eight-year longitudinal study of SAT-I performance at New Hampshire’s Brewster Academy (Bain & Ross, 1999), students participating in the technology-integrated schoolreform efforts (School Design Model) demonstrated average increases of 94 points in combined SAT I performance over students who participated in the traditional school experience. The reform efforts included a pioneer laptop program, where all students and faculty carry portable computers and have ready access to a campus network. Along with technology implementation, Brewster’s extensive school reform efforts involved “rethinking the way we teach, how we build curriculum, and the way we support and evaluate faculty” (Bain & Smith, 2000, p. 152). A West Virginia study shows an increase in test scores resulting from integrating curriculum objectives for basic skills development in reading and mathematics with instructional software (Mann, Shakeshaft, Becker, & Kottkamp, 1999). This curriculum was reinforced with teacher instruction and student achievement tests. Gains in student test scores on the SAT-9 (for 950 fifth graders in 18 schools) appeared attributable to the alignment of the targeted curriculum standards with the software, teacher instruction, and tests. Numerous studies document student understanding of mathematics concepts from using computer-based and -assisted software. Logo programming, computer-assisted instruction (CAI) microworlds, and algebra and geometry software are among those effective in facilitating mathematics achievement for elementary, middle, and high school students when teachers are skilled in guiding student activities (Hillel, Kieran, & Gurtner, 1989; McCoy, 1996; Simmons & Cope, 1990, 1993). In English language arts and social studies, teachers report observing significant change in student skills and knowledge acquired after their students’ first multimedia project. After student completion of the first multimedia project, teachers reported increased student knowledge in: • research skills, • ability to apply learning to real-world situations, • organizational skills, and • interest in the content (Cradler & Cradler, 1999). Higher-Order Skills Development Higher-order thinking and problemsolving skills (e.g., information research, comparing and contrasting, synthesizing, analyzing, and evaluating) enable learners to apply their content knowledge in a variety of ways leading to innovation and deeper understanding of content domains. Though some technology applications are designed for use in specific content areas, educaMay 2002 Learning & Leading with Technology 47 Copyright © ISTE (International Society for Technology in Education), 1.800.336.5191 (U.S. & Canada) or 1.541.302.3777 (Int’l), iste@iste.org. All rights reserved
Research Windows Research and evaluation shows that technology tools for constructing ar tifacts and electronic information and communication resources support the development of higher-order thinking skills. tors have also found valuable thinking note however that students may ma and the Internet.The control group of toosamong the technology applications nipulate simulation and presentation 38 students did use the computer but available for educational purposes.Re- software to create a visual artifact with- did not use the online resources with search and evaluation shows that tech- out really understanding or applying the curriculum.Center for Applied nology tools for constructing artifacts sound conceptual thinking.The role of Special Technology(CAST)researchers and electronic information and com- teachers is paramount in guiding the assessed the effect of Internet use on munication resources support the de- development of studentshigher-order student performance by looking at the velopment of higher-order thinking thinking skills during learning activities benefits it had on student projects.Ac skills.The findings hold true when stu- involving technology tools cording to the CAST(1996)research lents are taught to apply the processes n a landmark study analyzing a na of problems a Da 98)d ent test Intemet ents withou t ec a posit al. ect o able to skil His s hth-grad signif ntly highe omineusasrey on NAEP riehts (who.what stratedto atics inst where.why.how) h adjusting for cass size,teacher togethe different pointsof 53 niece for mathematics instructio eepo hree to four weeks were with students who did not The c om Research and evaluation shows that ar ative research demonstrated that the technoloay can enable the develonment tudents in classrooms who used the where computers were used for simula of critical thinking skills when students Jasper video programs were better able tions and applications to enhance use technology presentation and com to complete complex problem-solving higher-order thinking skills.the stu munication tools to present.publish tasks(Cognition and Technology dents performed better on the NAEP and share results of projects.The Group,1992) than did students whose teachers used CAST study also found that when n Pittsburgh.Pennsylvania.an the technology for drill and practice. students used the Internet to research intelligent-tutor software program "He found that fourth-grade stuc dents topics,share information.and complet as part of the regular curriculum fo who used computers primarily for a final project within the context of a nth-grade algebra,supports a curcu math/learning games scored highe semi-structured lesson,they became .fourth critical thinkers (Coley Ohee 0 ces in Cradler, ngel,1997) imulations and thinki gy tools to build exper e ing th e zet al 100 ix 1。 the dis e par o right Mark.1999. in a civil with 1)It ic ng c Learning Leading with Technology Volume 29 Number 8
48 Learning & Leading with Technology Volume 29 Number 8 tors have also found valuable thinking tools among the technology applications available for educational purposes. Research and evaluation shows that technology tools for constructing artifacts and electronic information and communication resources support the development of higher-order thinking skills. The findings hold true when students are taught to apply the processes of problem solving and then are allowed opportunities to apply technology tools to develop solutions. Powerful technologies are now available to significantly augment the skills necessary to convert data into information and transform information into knowledge. For example, interactive video programs have been demonstrated to increase problem-solving skills. Students across nine states who used Jasper video software as a centerpiece for mathematics instruction for three to four weeks were compared with students who did not. The comparative research demonstrated that the students in classrooms who used the Jasper video programs were better able to complete complex problem-solving tasks (Cognition and Technology Group, 1992). In Pittsburgh, Pennsylvania, an intelligent-tutor software program, as part of the regular curriculum for ninth-grade algebra, supports a curriculum focusing on mathematical analysis of real-world situations and the use of computational tools. “On average, the 470 students in the experimental classes using the software outperformed students in comparison classes by 15% on standardized tests and 100% on tests targeting the curriculum-focused objectives” (Koedinger, Anderson, Hadley, & Mark, 1999, p. 1). It is important to note, however, that students may manipulate simulation and presentation software to create a visual artifact without really understanding or applying sound conceptual thinking. The role of teachers is paramount in guiding the development of students’ higher-order thinking skills during learning activities involving technology tools. In a landmark study analyzing a national database of student test scores, Wenglinsky (1998) determined that technology can have a positive effect on students’ mathematics scores. His study used data of fourth- and eighth-grade students who took the math section of the 1996 National Assessment of Educational Progress (NAEP). That NAEP included questions about how computers are used in mathematics instruction. After adjusting for class size, teacher qualifications, and socioeconomics, Wenglinsky found that technology had more of an impact in middle schools than it did in elementary schools (Valdez et al., 1999). In eighth grade, where computers were used for simulations and applications to enhance higher-order thinking skills, the students performed better on the NAEP than did students whose teachers used the technology for drill and practice. “He found that fourth-grade students who used computers primarily for ‘math/learning games’ scored higher than students who did not. … fourth graders did not show differences in test score gains for either simulations and applications or drill and practice” (Valdez et al. 1999, p. 24). Another study of 22 fourth- and sixth-grade classes in seven urban school districts involved 66 of the participating students in a civil rights curriculum using online communication and the Internet. The control group of 38 students did use the computer but did not use the online resources with the curriculum. Center for Applied Special Technology (CAST) researchers assessed the effect of Internet use on student performance by looking at the benefits it had on student projects. According to the CAST (1996) researchers, “students with access to Scholastic Network and the Internet produced better projects than students without online access.” Of the nine measures of performance, the online users received significantly higher scores relative to: • presenting their work, • stating a civil rights issue, • presenting a full picture (who, what, when, where, why, how), • bringing together different points of view, and • producing a complete project (CAST, Table 2). Research and evaluation shows that technology can enable the development of critical thinking skills when students use technology presentation and communication tools to present, publish, and share results of projects. The CAST study also found that when students used the Internet to research topics, share information, and complete a final project within the context of a semi-structured lesson, they became independent, critical thinkers (Coley, Cradler, & Engel, 1997). Using technology tools to build thinking skills is not just for the best and brightest students. The Higher Order Thinking Skills (HOTS) pull-out program, developed in the early 1980s to build the thinking skills of students, combined technology with drama and Socratic dialogue. Through this combiResearch Windows Research and evaluation shows that technology tools for constructing artifacts and electronic information and communication resources support the development of higher-order thinking skills. Copyright © ISTE (International Society for Technology in Education), 1.800.336.5191 (U.S. & Canada) or 1.541.302.3777 (Int’l), iste@iste.org. All rights reserved
Research Windows nation,diatantagedsthudentbn Crades 4-7 achie ed twice the gains on readin isan esential part of thes effecti also is achieved honor roll status in 1994.sug teachers integrate a health care theme chnolo sesting a transfer of the students'cogni into acadamic als Th tive development to learning specific rch indica content.The students who used HOTS mbined effor also increased performance on measures ages students to analvze forces and of reading comprehension.metacog- angles in physical therapy.design a dants'a2 nce (Fo nition writing components of I building to house a health clinic transfer to novel tasks,and grade and determine the amount of money play.se point average (Coley et al.1997: a medical assistant must save in five Pogrow.1996) years to pay for college tuition. wwwiste.org/L&L) Technology can be useful in linking Workforce Preparation work experiences with academic sub- Preparing students for the workforce is jects.In a nationwide review of school- K (1999)School re a third area where technology plays a to-work programs.Olson (1998)found 2 pivotal role in helping school commu 148-153 or bas hat w ng sch ogy to com .90. ork The ing in 13961 Online gram nd the In ment pro of theprereq ing 20011. www.electronic. nd on the state school.com/200 anted education sta reases mastery of vorational and the benefits of usine information tech workforce skills and helns epare stu e world of work ofnchoredinstnucdiorT dents for work (Cradler 1994) om (lohnson 1996) Integration of technology with the matic and interdisciplinary roiects car Con nhance career prepa aration.A study of The research and evaluation studies four health career programs in Califor- cited in this article represent highlights nia (Stern rahn 1995)demon- from a larger body of evidence reviewed strated the effectiveness of work-based by CARET and available online.In San Mateo.CA:Educational Support learning models such as Tech Prep and sum,research is providing more and career academies that integrate student more clarity about how to use technol SyCraer() work experience with academic subjects ogy effectively within our school com- such as math.English.science,and so- munities to support and enhance the N 55 Sar CA:Educatio al studies.I hese programs allow higl mic performance of today's youth. school students to gain valuable knowl Collaborative activities and formative Technology continued on page 56 Research and evaluation shows that technology can enable the development of critical thinking skills when 网 students use technology presentation and communicatior tools to present,publish,and share results of projects. May 2002 Leamning&Leading with Technology
May 2002 Learning & Leading with Technology 49 nation, disadvantaged students in Grades 4–7 achieved twice the national average gains on reading and math test scores. Ten to 15% of the students also achieved honor roll status in 1994, suggesting a transfer of the students’ cognitive development to learning specific content. The students who used HOTS also increased performance on measures of reading comprehension, metacognition, writing, components of IQ, transfer to novel tasks, and grade point average (Coley et al., 1997; Pogrow, 1996). Workforce Preparation Preparing students for the workforce is a third area where technology plays a pivotal role in helping school communities reach their educational goals. Research shows that when students learn to use and apply applications used in the world of work, such as word processors, spreadsheets, computer-aided drawing, Web site development programs, and the Internet, they acquire some of the prerequisite skills for workforce preparedness. When content and problem-solving strategies meet accepted education standards, technology increases mastery of vocational and workforce skills and helps prepare students for work (Cradler, 1994). Integration of technology with thematic and interdisciplinary projects can enhance career preparation. A study of four health career programs in California (Stern & Rahn, 1995) demonstrated the effectiveness of work-based learning models such as Tech Prep and career academies that integrate students’ work experience with academic subjects such as math, English, science, and social studies. These programs allow high school students to gain valuable knowledge about how to conduct themselves in actual workplace environments. Reflection is an essential part of these work-based learning programs where teachers integrate a health care theme into academic assignments or interdisciplinary projects. For example, the math teacher in one program encourages students to analyze forces and angles in physical therapy, design a building to house a health clinic, and determine the amount of money a medical assistant must save in five years to pay for college tuition. Technology can be useful in linking work experiences with academic subjects. In a nationwide review of schoolto-work programs, Olson (1998) found programs where students were learning the new basics or basics plus skills. These skills include the ability to use technology to communicate ideas and information orally, as well as in writing. The new basics also include working in groups, solving problems when answers aren’t always self-evident, understanding how systems work, and collecting, analyzing, and organizing data. In a report on the state of technology integration in Minnesota, schools document the benefits of using information technologies to bring the world of work into the classroom (Johnson, 1996). Conclusion The research and evaluation studies cited in this article represent highlights from a larger body of evidence reviewed by CARET and available online. In sum, research is providing more and more clarity about how to use technology effectively within our school communities to support and enhance the academic performance of today’s youth. Collaborative activities and formative feedback are key components of instructional strategies that accompany effective technology implementation. Leadership also is pivotal in aligning available technology resources with systemic school improvement goals. The research indicates the need for understanding the combined efforts necessary for technology to positively influence students’ academic performance. (For more on the roles collaboration, leadership, and technology planning play, see the article supplement online at www.iste.org/L&L.) References Bain, A., & Ross, K. (1999). School reengineering and SAT-I performance: A case study. International Journal of Education Reform, 9(2), 148–153. Bain, A., & Smith, D. (2000). Technology enabling school reform. T.H.E. Journal, 28(3), 90. Center for Applied Special Technology. (1996). The role of online communications in schools: A national study [Online]. Available: www.cast.org/udl/RoleofOnline CommunicationsinSchools121.cfm. CEO Forum. (2001). Year 4 STaR Report [Online]. Available: www.electronicschool.com/2001/09/0901ewire.html#forum. Cognition and Technology Group at Vanderbilt. (1992). The Jasper series as an example of anchored instruction: Theory, program description, and assessment data. Educational Psychologist, 27, 291–315. Coley, R., Cradler, J., & Engel, P. (1997). Computers and classrooms: The status of technology in U.S. schools. Princeton, NJ: Policy Information Center, Educational Testing Service. Cradler, J. (1994). Summary of research and evaluation findings relating to technology in education. San Mateo, CA: Educational Support Systems. Cradler, R., & Cradler, J. (1999). Just in time: Technology innovation challenge grant year 2 evaluation report for Blackfoot School District No. 55. San Mateo, CA: Educational Support Systems. Research Windows Research and evaluation shows that technology can enable the development of critical thinking skills when students use technology presentation and communication tools to present, publish, and share results of projects. Technology continued on page 56. Copyright © ISTE (International Society for Technology in Education), 1.800.336.5191 (U.S. & Canada) or 1.541.302.3777 (Int’l), iste@iste.org. All rights reserved
Research Windows net,1.(1989 0s0 hnson.B.H.(6).Minnesota con UPDATE eh元 of state and fed 6防 sburgh.PA EXTRA!EXTRA! programs products and initiatives htm Mary menabh edd ( Don'tmissan issueof Update! Your source for what's happening ineducational. anta Monica ().Computer-based sity of Den e%n24s846 L()The puters and e p (16).Usingc NETS for 7 her CA with Educat M.&Co P (1990) Simmons,M.&Cope.P(1993).Angl PD血 nd D dies in ork-bas ISTE Update the key to CARET Raack, obtained his PhD in Cog g and expe k C0me的5Ebcha1526 sky.H (18).Doesit com steD四 Find research-based answers to your critical technology-planning questions. deatoraltectnoegy www.iste.org/publishing/update 3Leamning&Leading with Technology Volume9 Number8
50 Learning & Leading with Technology Volume 29 Number 8 Hillel, J., Kieran, C., & Gurtner, J. (1989). Solving structured geometry tasks on the computer: The role of feedback in generating strategies. Educational Studies in Mathematics, 20, 1–39. Johnson, B. H. (1996). Minnesota committed to providing technology to all students. Research/Practice [Online], 4(2). Available: http://education.umn.edu/carei/Reports/ Rpractice/Summer96/committed.htm. Koedinger, K., Anderson, J., Hadley, W., & Mark, M. (1999). Intelligent tutoring goes to school in the big city. [Online]. Pittsburgh, PA: Carnegie Mellon University. Available: http:// act.psy.cmu.edu/awpt/AlgebraPacket/kenPaper/ paper.html. Mann, D., Shakeshaft, C., Becker, J., & Kottkamp, R. (1999). West Virginia story: Achievement gains from a statewide comprehensive instructional technology program. Santa Monica, CA: Milken Exchange on Educational Technology. McCoy, L. P. (1996). Computer-based mathematics learning. Journal of Research on Computing in Education, 28(4), 438–460. Olson, L. (1998). The new basics in schoolto-work. Educational Leadership, 55(6), 50–54. Pogrow, S. (1996). Using computers and other visual technology to combine process and content. In A. Costa & R. Liebman (Eds.), When process is content: Toward renaissance learning (pp. 98–116). Thousand Oaks, CA: Corwin Press. Simmons, M., & Cope, P., (1990). Fragile knowledge of angle in turtle geometry. Educational Studies in Mathematics, 21(4), 375–382. Simmons, M., & Cope, P. (1993). Angle and rotation: Effects of different types of feedback on the quality of response. Educational Studies in Mathematics, 24(2), 163–176. Stern, D., & Rahn, M. (1995). How health career academies provide work-based learning. Educational Leadership, 52(8), 37–40. Valdez, G., McNabb, M., Foertsch, M., Anderson, M., Hawkes, M., & Raack, L. (1999). Computer-based technology and learning: Evolving uses and expectations. Oak Brook, IL: North Central Regional Educational Laboratory. Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics. Princeton, NJ: Policy Information Center, Educational Testing Services. Research Windows John Cradler (cradler@ earthlink.net) is the co-director of the CARET project and president of Educational Support Systems. During the past 15 years, he has made significant contributions to the development of state and federal legislation and policies related to educational technology. In his previous roles as director of technology for WestED, CCSSO, and Educational Support Systems, he has served as an advisor and evaluator for a wide variety of state, federal, and private sector educational technology programs, products, and initiatives. Mary McNabb, EdD (mlmcnabb@msn.com), works as a consultant focusing on investigating the nature of teaching, learning, and assessment in online cultures. Previously, she was a research scientist at the University of Denver Research Institute (DRI). She has also served as director of Research and Technology for the North Central Regional Education Laboratory (Oak Brooks, Illinois) and was on a national committee coordinating evaluation efforts for the Preparing Tomorrow’s Teachers to Use Technology (PT3 ) Program. She served on the leadership committee that developed ISTE’s NETS for Teachers. Molly Freeman (mollyfreeman@telis.org) currently conducts research with Educational Support Systems and since 1996 has consulted with the Internet Institute of Santa Clara County Office of Education to design staff development for K–12 teachers learning to use technology in the classroom. Her PhD is in Complex Systems and Distance Learning from The Union Institute, and her master’s degree is in sociology from the University of California at Davis. Richard Burchett (rburchett@iste.org) is a CARET reviewer and a research associate in ISTE’s Research and Evaluation Department. In 1994, he obtained his PhD in Cognitive Psychology from the University of California, Riverside. He has taught psychology, statistics, and research methodology courses at numerous West Coast universities. Richard has served as an associate professor of psychology at the American University in Cairo (Egypt) and the American University of Sharjah (United Arab Emirates). 56 Technology continued from page 49. Copyright © ISTE (International Society for Technology in Education), 1.800.336.5191 (U.S. & Canada) or 1.541.302.3777 (Int’l), iste@iste.org. All rights reserved