What is the Value of Learning Goal #1 Course-Specific Learning Goals? Learning Goal 1#3 By Beth Simon and Jared Taylor Lear: The authors examined student and faculty opinions regarding the use of detailed learning goals in three courses.Students reported the use of learning goals ☑ to be very positive,aiding them with studying,in lectures,and in determining the important material Learning Goal #5 to learn.Likewise,faculty indicated that using learning goals was a positive experience,especially for communicating course material to students and other faculty and for creating course assessments. xtensive research has demon- Holliday 2006;Kiewra 1985).We specific,course-level learning goals strated that experts inasubject hypothesized that these problems in improved the student's interaction have a mental organizational teaching might be addressed by hav- with the course,we looked specifi- structure for knowledge that ing explicit learning goals or objec- cally at the following questions: novices lack (Ross 2006;Ericsson tives for lectures.Such goals would 2006).Experts recognize patterns and provide learners(students)with orga- 。 Did students perceive learning goals classify material on the basis of this nizational scaffolding,and thus help as being valuable in the course? organizational system,but the orga- students understand the perspective What did students report about nizational system is deeply ingrained being taken by the instructor. how they used learning goals and and rarely consciously recognized. A range of work has discussed how was this different across sev- The subconscious nature of this orga- both the results and impact of learn- eral instructors and courses? nization impacts the ability of instruc- ing goals (as we will refer to them) Did the instructors perceive the tors to recognize that an explanation for programmatic assessment(Marsh value of learning goals both for they find compelling may be incoher- 2007;Adam2004,Harden2002). themselves and their students? ent to students.Many studies have However,in this work,we explored provided examples of instructors and the impact of learning goals on the As discussed in this article,our re- students perceiving an explanation individual student in a course and, sults indicate that explicit learning profoundly differently,with resulting to a lesser degree,on the instructors goals provide a valuable aid to guide negative consequences for student teaching the courses.We studied three students in their learning.These re- learning.Examples have been shown courses in which instructors had de- sults give instructors a glimpse into in lecture-scenario studies (Hrepic, tailed,course-specific learning goals how students use learning goals and Zollman,and Rebello 2007)and in that were integrated into their classes. suggest best practices for the use of studies of note taking (Bonner and To explore the general hypothesis that learning goals. 52 Journal College Science Teaching
52 Journal of College Science Teaching The authors examined student and faculty opinions regarding the use of detailed learning goals in three courses. Students reported the use of learning goals to be very positive, aiding them with studying, in lectures, and in determining the important material to learn. Likewise, faculty indicated that using learning goals was a positive experience, especially for communicating course material to students and other faculty and for creating course assessments. By Beth Simon and Jared Taylor What is the Value of Course-Specific Learning Goals? E xtensive research has demonstrated that experts in a subject have a mental organizational structure for knowledge that novices lack (Ross 2006; Ericsson 2006). Experts recognize patterns and classify material on the basis of this organizational system, but the organizational system is deeply ingrained and rarely consciously recognized. The subconscious nature of this organization impacts the ability of instructors to recognize that an explanation they find compelling may be incoherent to students. Many studies have provided examples of instructors and students perceiving an explanation profoundly differently, with resulting negative consequences for student learning. Examples have been shown in lecture-scenario studies (Hrepic, Zollman, and Rebello 2007) and in studies of note taking (Bonner and specific, course-level learning goals improved the student’s interaction with the course, we looked specifically at the following questions: • Did students perceive learning goals as being valuable in the course? • What did students report about how they used learning goals and how was this different across several instructors and courses? • Did the instructors perceive the value of learning goals both for themselves and their students? As discussed in this article, our results indicate that explicit learning goals provide a valuable aid to guide students in their learning. These results give instructors a glimpse into how students use learning goals and suggest best practices for the use of learning goals. Holliday 2006; Kiewra 1985). We hypothesized that these problems in teaching might be addressed by having explicit learning goals or objectives for lectures. Such goals would provide learners (students) with organizational scaffolding, and thus help students understand the perspective being taken by the instructor. A range of work has discussed both the results and impact of learning goals (as we will refer to them) for programmatic assessment (Marsh 2007; Adam 2004; Harden 2002). However, in this work, we explored the impact of learning goals on the individual student in a course and, to a lesser degree, on the instructors teaching the courses. We studied three courses in which instructors had detailed, course-specific learning goals that were integrated into their classes. To explore the general hypothesis that
The setting for this study was three although the spring computing course 44 were not included in this analysis courses in which the three instructors inherited and adopted some of the because they were clearly describing used learning goals on a daily basis learning goals from the fall course the value of the class.not the value in their classrooms.These courses (39 of 75 spring learning goals were oflearning goals.A content analysis- are part of the computer science and repeated from fall).All three instruc- based coding process was developed microbiology and immunology de- tors presented learning goals as part for the responses from Course A.We partments at the University of British oftheir lectures,two always presented organized the codes into a number of Columbia (UBC).The three classes them at the beginning of the lecture categories such as Study,Exams,Lec- were the fall 2007(Course A)and (Course A and Course C),and one ture,Focus,Guide,General Positive, spring 2008(Course B)offerings presented them at the beginning of and Negative (see Table 2).We also of a computing literacy course and each unit (Course B). grouped some related code categories an upper-level microbiology course into larger groupings:Knowing What (Course C).Both departments have Study methods I Need to Know and Comprehension. been granted funding through the In the last week of the term,we asked All positive comments fit within one UBC Carl Wieman Science Educa- students (during class)to complete or more of the first six categories.We tion Initiative to (1)establish what up to five copies of the sentence, independently coded the data in an students should learn,(2)scientifi- "For me,the use of learning goals in iterative process by which categories cally measure what students are actu- this course is..."Students were giv- were refined.We achieved an aver- ally learning,(3)adapt instructional en examples to help them feel more age intercoder reliability of 95%(29 methods and pedagogical research to comfortable about what they were out of 553 student comments lacked achieve desired learning outcomes, being asked to do.These examples complete coding agreement;Course A and (4)disseminate and adopt what were“helpful because..,”and“not =97%,Course B=93%,Course C= works.Two of the three professors something I've really noticed."We 95%).The three instructors were not involved in the study had attended performed interviews with the three involved in the coding or the analysis workshops at UBC on creating learn- instructors to characterize the differ- of the comments. ing goals,and all three were indepen- ences in development,intent,and use dently interested in considering the of learning goals for the courses. Results and discussion impacts of learning goals on student We collected 597 responses(225 We analyzed responses across the learning.The instructors all consid- from Course A,252 from Course B, classes first by the total number of ered the development of the learning 120 from Course C)from 186 stu- comments in each of the coding cat- goals to be an ongoing process,and dents(59 in Course A,76 in Course B, egories and then by the percentage of they continue to invest time and effort 51 in Course C).Of the 597 responses, students who made a comment that to improve and refine them The three 13-week courses each TABLE1 had between 57 and 75 learning goals that reflected single-lecture or Examples of learning goals. multiple-lecture content.Examples Class Learning goals of learning goals from these three classes are shown in Table 1.In gen- After the web unit/HTML lab you will be able to: eral,each of the goals completed the Briefly describe the parts of a web search engine. sentence,"At the end of this lecture/ Explain how a search engine finds and indexes web pages. topic students will be able to Predict how and whether a page will be found by a web Course A and B The goals are primarily written in crawler,given the link structure around the page and this form so that students are able to which pages the crawler already knows. identify whether they can accomplish Construct HTML to present the structure and visual ap- what is described,even if they have pearance you intend for an HTML web page. not yet mastered the terminology Comment on the strengths and weaknesses of classifi- of the subject.Vague terms such as cation systems used to organize diversity in the world understand are (usually)absent. around us. Instead,specifics on how a student Course C Describe the relationships between point mutation and would operationally demonstrate a gene transfer in the context of evolution. requisite level of understanding are Evaluate the concept of microbial species using compara- used.The instructors began to use tive genomics. learning goals of their own accord, November/December 2009 53
November/December 2009 53 The setting for this study was three courses in which the three instructors used learning goals on a daily basis in their classrooms. These courses are part of the computer science and microbiology and immunology departments at the University of British Columbia (UBC). The three classes were the fall 2007 (Course A) and spring 2008 (Course B) offerings of a computing literacy course and an upper-level microbiology course (Course C). Both departments have been granted funding through the UBC Carl Wieman Science Education Initiative to (1) establish what students should learn, (2) scientifically measure what students are actually learning, (3) adapt instructional methods and pedagogical research to achieve desired learning outcomes, and (4) disseminate and adopt what works. Two of the three professors involved in the study had attended workshops at UBC on creating learning goals, and all three were independently interested in considering the impacts of learning goals on student learning. The instructors all considered the development of the learning goals to be an ongoing process, and they continue to invest time and effort to improve and refine them. The three 13-week courses each had between 57 and 75 learning goals that reflected single-lecture or multiple-lecture content. Examples of learning goals from these three classes are shown in Table 1. In general, each of the goals completed the sentence, “At the end of this lecture/ topic students will be able to . . .” The goals are primarily written in this form so that students are able to identify whether they can accomplish what is described, even if they have not yet mastered the terminology of the subject. Vague terms such as understand are (usually) absent. Instead, specifics on how a student would operationally demonstrate a requisite level of understanding are used. The instructors began to use learning goals of their own accord, although the spring computing course inherited and adopted some of the learning goals from the fall course (39 of 75 spring learning goals were repeated from fall). All three instructors presented learning goals as part of their lectures, two always presented them at the beginning of the lecture (Course A and Course C), and one presented them at the beginning of each unit (Course B). Study methods In the last week of the term, we asked students (during class) to complete up to five copies of the sentence, “For me, the use of learning goals in this course is . . .” Students were given examples to help them feel more comfortable about what they were being asked to do. These examples were “helpful because . . .” and “not something I’ve really noticed.” We performed interviews with the three instructors to characterize the differences in development, intent, and use of learning goals for the courses. We collected 597 responses (225 from Course A, 252 from Course B, 120 from Course C) from 186 students (59 in Course A, 76 in Course B, 51 in Course C). Of the 597 responses, 44 were not included in this analysis because they were clearly describing the value of the class, not the value of learning goals. A content analysis– based coding process was developed for the responses from Course A. We organized the codes into a number of categories such as Study, Exams, Lecture, Focus, Guide, General Positive, and Negative (see Table 2). We also grouped some related code categories into larger groupings: Knowing What I Need to Know and Comprehension. All positive comments fit within one or more of the first six categories. We independently coded the data in an iterative process by which categories were refined. We achieved an average intercoder reliability of 95% (29 out of 553 student comments lacked complete coding agreement; Course A = 97%, Course B = 93%, Course C = 95%).The three instructors were not involved in the coding or the analysis of the comments. Results and discussion We analyzed responses across the classes first by the total number of comments in each of the coding categories and then by the percentage of students who made a comment that Table 1 Examples of learning goals. Class Learning goals Course A and B After the web unit/HTML lab you will be able to: • Briefly describe the parts of a web search engine. • Explain how a search engine finds and indexes web pages. • Predict how and whether a page will be found by a web crawler, given the link structure around the page and which pages the crawler already knows. • Construct HTML to present the structure and visual appearance you intend for an HTML web page. Course C • Comment on the strengths and weaknesses of classification systems used to organize diversity in the world around us. • Describe the relationships between point mutation and gene transfer in the context of evolution. • Evaluate the concept of microbial species using comparative genomics
fell into a category.Learning goals FIGURE1 were seen as very valuable by nearly Percentages of students and valuations. all students in total and consistently in the three classes.Of the 553 total 70% comments,471 were positive(Course A,85%;Course B,84%;Course C, 60% ■Course A■Course B■Course C 88%).Figure 1 shows that less than 10%of the students in each class 50% made only negative comments,and 409% nearly two-thirds made only positive comments.The fact that 85%of the 30% comments were positive also indi- cated that those students who made 20% mixed comments gave significantly more positive than negative com- 109% ments.Of the negative comments, the majority fell into the categories 0% of finding the learning goals unhelp- Students with only Students with both Students with only positive valuations positive and negative negative valuations ful or used unclear wording(Figure valuations 2).Students also provided some recommendations to make learning TABLE2 Examples of codes. Groupings Codes Example Study,Prepare,Review "improves my studying habits"(Course A) Exam,Test,Quiz "important for exam review"(Course B) Lecture,Class,Course "helpful to connect lectures"(Course C) Focus "helpful because it tells me what I need to focus on"(Course A) Implied focus "a good thing to highlight important concepts"(Course B) Summary,Outline "a good outline of the topics"(Course C) Knowing WhatI "useful because it guides me through the progression of Guide the class throughout the term"(Course A) Need to Know "helpful for organizing my notes with similar topics" Organize (Course B) Track "good for keeping me on track"(Course A) Preview,Introduction,Expectation 'gives an idea what the lecture will be about"(Course C) 'well-organized and helpful in understanding the material Understand during lectures"(Course A) Comprehension "nice because they are a list of all the new things I can say Learn I've learned in this class"(Course B) General positive "a good idea"(Course C) "sometimes confusing when I don't know what the learn- Negative ing goal(s)is/are referring to"(Course A) 54 Journal a College Science Teaching
54 Journal of College Science Teaching fell into a category. Learning goals were seen as very valuable by nearly all students in total and consistently in the three classes. Of the 553 total comments, 471 were positive (Course A, 85%; Course B, 84%; Course C, 88%). Figure 1 shows that less than 10% of the students in each class made only negative comments, and nearly two-thirds made only positive comments. The fact that 85% of the comments were positive also indicated that those students who made mixed comments gave significantly more positive than negative comments. Of the negative comments, the majority fell into the categories of finding the learning goals unhelpful or used unclear wording (Figure 2). Students also provided some recommendations to make learning Table 2 Examples of codes. Groupings Codes Example Study, Prepare, Review “improves my studying habits”(Course A) Exam,Test,Quiz “important for exam review”(Course B) Lecture,Class,Course “helpful to connect lectures”(Course C) Knowing What I Need to Know Focus “helpful because ittells me whatI need to focus on”(Course A) Implied focus “a good thing to highlight important concepts”(Course B) Summary,Outline “a good outline of the topics”(Course C) Guide “useful because it guides me through the progression of the class throughout the term”(Course A) Organize “helpful for organizing my notes with similar topics” (Course B) Track “good for keeping me on track”(Course A) Preview, Introduction, Expectation “gives an idea what the lecture will be about”(Course C) Comprehension Understand “well-organized and helpful in understanding the material during lectures”(Course A) Learn “nice because they are a list of all the new things I can say I’ve learned in this class”(Course B) General positive “a good idea”(Course C) Negative “sometimes confusing when I don’t know what the learning goal(s) is/are referring to”(Course A) FIGURE 1 Percentages of students and valuations. n Course A n Course B n Course C Students with only positive valuations Students with both positive and negative valuations Students with only negative valuations 70% 60% 50% 40% 30% 20% 10% 0% Percentage of students
What is the Value of Course-Specific Learning Goals? goals more effective (by improving wording,for example). FIGURE 2 Breakdown of valuation of learning goals,by percentage of total What is the perceived value of relevant comments.Totals for each course can be greater than 100% learning goals? due to cross-coding of comments. We identified the most commonly list- 50% ed values of goals by examining the distribution of comments across the 45% six categories (Figure 2).There was ■Course A■Course B■Course C striking similarity among the three 40% courses.The most frequently men- tioned value for learning goals was 35% that it helped students"know what I need to know"for a course.The next 30% most common valuations referred to lecture or study.Notably,valuation 25% 6 for exams is not terribly prevalent (though reference to“study”implies 20% preparation for exams).The exam 15% category was the only category that was noticeably different between the 10% courses.We believe this is due to the Course A instructor making an ex- 5% plicit and repeated promise that all exam questions would come from 0% learning goals,and that the Course C instructor had not yet given an exam at the time of the survey. Knowing What I Need to Know 5 The dominant value identified was clearly Knowing What I Need to Positive comments Negative comments Know.Students were very specific as to the ways that learning goals helped them:“focus,.”“guided me,” learning goals help them recognize of every lecture period,whereas in kept me on track,”“summarized,” this structure and approach their Course B learning goals were shown and“outlined the course.”Overall, learning more effectively. at the beginning of a unit or topic we found that these comments indi- Many comments were about how area.There is no obvious impact of cated that students valued additional students valued applying Knowing this difference. information or structure,which al- What I Need to Know.Of the com- lowed them to organize the informa- ments in that category,53%were Instructor impressions tion more effectively and be more also coded in the Lecture,Exam,or We interviewed the instructors to ex- expertlike in their approach to the Study categories.The percentages of amine their views of learning goals, class.The comments indicated that the cross-coded comments for each both for themselves and for their students appreciated that instructors course are shown in Figure 3.This students.By comparing notes and were being explicit about what was shows that students most commonly interview recordings,we identified of the most importance in the class, mentioned learning goals as helping several broad themes,most notably and that learning goals helped them them know what they needed to know communication with students and narrow down and carefully apply in the lecture setting.Learning goals other instructors,structured learning, their efforts to those topics,skills,or are often seen by students as helping and assessment development. concepts.This supports our hypoth- them get more out of a lecture by fo- The most common point made esis that students struggle to see the cusing and guiding their attention.In by the instructors was that learning organization and relative importance Course A and Course C,learning goals goals enhanced communication, of material in courses,and explicit were always shown at the beginning both with students and other fac- November/December 2009 55
November/December 2009 55 What is the Value of Course-Specific Learning Goals? FIGURE 2 Breakdown of valuation of learning goals, by percentage of total relevant comments. Totals for each course can be greater than 100% due to cross-coding of comments. n Course A n Course B n Course C 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Percent of Relevant Students Knowing what I need to know Study Lecture Comprehension Exams General positive Unhelpful Unclear wording Recommendations Positive comments Negative comments goals more effective (by improving wording, for example). What is the perceived value of learning goals? We identified the most commonly listed values of goals by examining the distribution of comments across the six categories (Figure 2). There was striking similarity among the three courses. The most frequently mentioned value for learning goals was that it helped students “know what I need to know” for a course. The next most common valuations referred to lecture or study. Notably, valuation for exams is not terribly prevalent (though reference to “study” implies preparation for exams). The exam category was the only category that was noticeably different between the courses. We believe this is due to the Course A instructor making an explicit and repeated promise that all exam questions would come from learning goals, and that the Course C instructor had not yet given an exam at the time of the survey. Knowing What I Need to Know The dominant value identified was clearly Knowing What I Need to Know. Students were very specific as to the ways that learning goals helped them: “focus,” “guided me,” “kept me on track,” “summarized,” and “outlined the course.” Overall, we found that these comments indicated that students valued additional information or structure, which allowed them to organize the information more effectively and be more expertlike in their approach to the class. The comments indicated that students appreciated that instructors were being explicit about what was of the most importance in the class, and that learning goals helped them narrow down and carefully apply their efforts to those topics, skills, or concepts. This supports our hypothesis that students struggle to see the organization and relative importance of material in courses, and explicit learning goals help them recognize this structure and approach their learning more effectively. Many comments were about how students valued applying Knowing What I Need to Know. Of the comments in that category, 53% were also coded in the Lecture, Exam, or Study categories. The percentages of the cross-coded comments for each course are shown in Figure 3. This shows that students most commonly mentioned learning goals as helping them know what they needed to know in the lecture setting. Learning goals are often seen by students as helping them get more out of a lecture by focusing and guiding their attention. In Course A and Course C, learning goals were always shown at the beginning of every lecture period, whereas in Course B learning goals were shown at the beginning of a unit or topic area. There is no obvious impact of this difference. Instructor impressions We interviewed the instructors to examine their views of learning goals, both for themselves and for their students. By comparing notes and interview recordings, we identified several broad themes, most notably communication with students and other instructors, structured learning, and assessment development. The most common point made by the instructors was that learning goals enhanced communication, both with students and other fac-
ulty members.The instructors felt that the learning goals provided a FIGURE 3 method to very clearly outline for Comments coded both as Knowing What I Need to Know and either students the important concepts and Lecture,Study,or Exams. material for the class.The learning 45% goals also provided a very easy way for instructors to communicate with 40% ■Course A■Course B■Course C other instructors about what is cov- ered in a course,something that is 35% needed when organizing material in a sequence of classes.For example, 30% Course A instructor commented: 25% "The learning goals are kind of a 堂 20% contract between me and the stu- dents so we can all know what 15% the foundation is that we should be building a class on and they 10% can call me out and say does this have anything to do with learn- 5% ing goals?...It's a shared struc- ture by which I and students and 0% other instructors can know what Lecture Study Exams the course is about.” Course B instructor commented: The instructors also commented Course C instructor commented: on what they perceived to be the "When learning goals weren't value of using learning goals from "A learning goal primes [the a primary focus,it was hard to a student's point of view:provid- students]to think about the in- establish expectations...even ing structure for learning.This was formation and organize it... when I really emphasize things closely related to the ideas they ex- We can tell students that these verbally,[the students]miss pressed about how communication learning goals are important these sorts of things.But to was enhanced with students.The in structuring your knowledge have something written down instructors observed that students Students are hungry for that you can point to and say were able to both use the learning anything that structures their that here are my expectations goals to determine the important learning.” and here is what we are intend- material in the course and use the ing to give you as a skill set learning goals as a measuring stick Finally.the instructors mentioned when you walk out of the class to gauge their own progress.This that the learning goals streamlined ..it gives instructors greater seems very similar to the students" the process of writing exam ques- confidence in interacting with comments coded as“on track.” tions and improved assessment. students and other faculty Course A instructor commented: members..” Course A instructor commented: "[Learning goals]will save you Course C instructor commented: "It's a question of organiza- time in the end.because instruc- tion as in knowing what I am tors will want to put together a "I would emphasize the need to doing and the students know- high-quality exam,and you share our learning goals with ing what I am doing as well... will put together a high-quality one another [other faculty]and They had a goal for what to exam much more easily with to communicate what is being learn.They didn't just learn the learning goals.Everyone I taught.Learning goals are a as much as they could in this have talked to that uses learn- parsed way to look at what a course,but they had some- ing goals has talked about this. course is about and what the thing to work towards and Your exam writes itself...I can core concepts are.” measure themselves against." check to see if each question ad- 56 Journal College Science Teaching
56 Journal of College Science Teaching ulty members. The instructors felt that the learning goals provided a method to very clearly outline for students the important concepts and material for the class. The learning goals also provided a very easy way for instructors to communicate with other instructors about what is covered in a course, something that is needed when organizing material in a sequence of classes. For example, Course A instructor commented: “The learning goals are kind of a contract between me and the students so we can all know what the foundation is that we should be building a class on and they can call me out and say does this have anything to do with learning goals? . . . It’s a shared structure by which I and students and other instructors can know what the course is about.” Course B instructor commented: “When learning goals weren’t a primary focus, it was hard to establish expectations . . . even when I really emphasize things verbally, [the students] miss these sorts of things. But to have something written down that you can point to and say that here are my expectations and here is what we are intending to give you as a skill set when you walk out of the class . . . it gives instructors greater confidence in interacting with students and other faculty members.” Course C instructor commented: “I would emphasize the need to share our learning goals with one another [other faculty] and to communicate what is being taught. Learning goals are a parsed way to look at what a course is about and what the core concepts are.” FIGURE 3 Comments coded both as Knowing What I Need to Know and either Lecture, Study, or Exams. n Course A n Course B n Course C Lecture Study Exams 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Percentage of students The instructors also commented on what they perceived to be the value of using learning goals from a student’s point of view: providing structure for learning. This was closely related to the ideas they expressed about how communication was enhanced with students. The instructors observed that students were able to both use the learning goals to determine the important material in the course and use the learning goals as a measuring stick to gauge their own progress. This seems very similar to the students’ comments coded as “on track.” Course A instructor commented: “It’s a question of organization as in knowing what I am doing and the students knowing what I am doing as well . . . They had a goal for what to learn. They didn’t just learn as much as they could in this course, but they had something to work towards and measure themselves against.” Course C instructor commented: “A learning goal primes [the students] to think about the information and organize it . . . We can tell students that these learning goals are important in structuring your knowledge . . . Students are hungry for anything that structures their learning.” Finally, the instructors mentioned that the learning goals streamlined the process of writing exam questions and improved assessment. Course A instructor commented: “[Learning goals] will save you time in the end, because instructors will want to put together a high-quality exam, and you will put together a high-quality exam much more easily with the learning goals. Everyone I have talked to that uses learning goals has talked about this. Your exam writes itself . . . I can check to see if each question ad-
What is the Value of Course-Specific Learning Goals? dresses a learning goal and if it orientation of their goals,which were Herriot-Watt University.Edinburgh, doesn't I will throw it out." similar across the three instructors. Scotland. Students of a discipline may lack Bonner,J.M.,and W.G.Holliday.2006. Course B instructor commented: the intellectual scaffolding and orga- How college science students engage nizational structure taken for granted in note-taking strategies.Journal of "The learning goals were not by experts instructing them.The Research in Science Teaching 43 (8). the primary source for exam primary value students expressed 786-818. questions,but one notch down. regarding learning goals in this study Ericsson.K.A.2006.The influence of They would drive the initial is that they provided them with struc- experience and deliberate practice on structure of the exam,and then ture in their efforts to know what it the development of superior expert I would fill in the remainder." is they need to know.The fact that performance.In The Cambridge many students indicated that learn- Handbook of Expertise and Expert Summary ing goals helped them scaffold their Performance,eds.K.A.Ericsson,N lecture experience provides hope that Charness.P.J.Feltovich.and R.R. Returning to our original research learning goals reduce some of the Hoffman,683-703.Cambridge,UK: questions,we found that students well-known difficulties students have Cambridge University Press. overwhelmingly found value in in understanding lecture and in taking Harden R.M.2002.Developments in the use of detailed,course-specific outcome-based education.Medical learning goals.Specifically,students notes and therefore support more ef- fective learning. Teacher24(2y:117-120. found that the learning goals helped We urge others to consider devel- Hrepic,Z.,D.A.Zollman,and N.S them determine what they needed to oping learning goals at the course Rebello.2007.Comparing students know.Student responses were con- level that complete the sentence,"By and experts'understanding of the sistent across all three courses in two the end of today.students will be able content of a lecture.Journal of Sci- very different academic disciplines. Upon repeated reading and coding to..."and to report both on the con- ence Education and Technology 16 tent of those goals and their valuation (3y213-224. of the comments,a general sentiment by students.We are encouraged by Kiewra,K.A.1985.Providing the of the students emerged.Students expressed relief and gratitude at be- the positive,shared valuation by stu- instructor's notes:An effective addi- dents across one computing and one tion to student notetaking.Educa- ing given clear direction as to how to microbiology class and are interested tional Psychologist 20(1):33-39 focus their efforts.most notably in the to see if others (and their students) Marsh,P.2007.What is known about stu- lectures,and also in organizing their find similar value.Exemplar learning dent learning outcomes and how does it studying,reviewing,and preparing goals of this style can be found at the relate to the scholarship of teaching and for exams.Interviews indicated that University of British Columbia's Carl learning?International Journal for the instructors found learning goals help- Wieman Science Education Initiative Scholarship of Teaching and Learning ful in keeping their lecture prepara- website (http://cwsei.ubc.ca). 1 (2).http://academics.georgiasouth- tions on track.This corresponded to ern.edw/ijsotl. student responses that indicated that Acknowledgments daily,explicit presentation oflearning Ross,P.E.2006.The expert mind.Sci- We thank Steve Wolfiman,Kim Voll,and entific American 295(2):64-71. goals made the instructor's organiza- Steven Hallam for their cooperation and tion and focus more clear.Students participation in this work,as well as Carl Beth Simon (bsimon@cs.ucsd.edu)is found learning goals to be a clear, Wieman for his invaluable input and feed- a faculty member in the Computer Sci- valuable form of communication from back.This work was funded by the Carl ence and Engineering Department at the the instructor that was missing in their Wieman Science Education Initiative at University of California,San Diego,and a other courses.As one student stated, the University of British Columbia. former Science Teaching and Learning Fel- learning goals are"useful in that they low in the Carl Wieman Science Education show me what the prof wants us to References Initiative and the Department of Computer learn”(Course A). Adam,S.2004.Using learning Science at the University of British Colum- Although the instructors were not outcomes:A consideration of the bia in Vancouver,British Columbia.Jared explicitly directed in their develop- nature,role,application and impli- Taylor (jtaylor@zoology.ubc.ca)is a Sci- ment of goals for this study,we found cations for European education of ence Teaching and Learning Fellow in the that their respective student popula- employing learning outcomes at the Carl Wieman Science Education Initiative tions valued learning goals similarly. local.national and international and the Department of Microbiology and This suggests that what is important levels.Report on the United King- Immunology at the University of British Co- is the detailed nature and student- dom Bologna Seminar,July 2004,at lumbia in Vancouver,British Columbia. November/December 2009 57
November/December 2009 57 What is the Value of Course-Specific Learning Goals? dresses a learning goal and if it doesn’t I will throw it out.” Course B instructor commented: “The learning goals were not the primary source for exam questions, but one notch down. They would drive the initial structure of the exam, and then I would fill in the remainder.” Summary Returning to our original research questions, we found that students overwhelmingly found value in the use of detailed, course-specific learning goals. Specifically, students found that the learning goals helped them determine what they needed to know. Student responses were consistent across all three courses in two very different academic disciplines. Upon repeated reading and coding of the comments, a general sentiment of the students emerged. Students expressed relief and gratitude at being given clear direction as to how to focus their efforts, most notably in the lectures, and also in organizing their studying, reviewing, and preparing for exams. Interviews indicated that instructors found learning goals helpful in keeping their lecture preparations on track. This corresponded to student responses that indicated that daily, explicit presentation of learning goals made the instructor’s organization and focus more clear. Students found learning goals to be a clear, valuable form of communication from the instructor that was missing in their other courses. As one student stated, learning goals are “useful in that they show me what the prof wants us to learn” (Course A). Although the instructors were not explicitly directed in their development of goals for this study, we found that their respective student populations valued learning goals similarly. This suggests that what is important is the detailed nature and studentHerriot-Watt University. Edinburgh, Scotland. Bonner, J.M., and W.G. Holliday. 2006. How college science students engage in note-taking strategies. Journal of Research in Science Teaching 43 (8): 786–818. Ericsson, K.A. 2006. The influence of experience and deliberate practice on the development of superior expert performance. In The Cambridge Handbook of Expertise and Expert Performance, eds. K.A. Ericsson, N. Charness, P.J. Feltovich, and R.R. Hoffman, 683–703. Cambridge, UK: Cambridge University Press. Harden R.M. 2002. Developments in outcome-based education. Medical Teacher 24 (2): 117–120. Hrepic, Z., D.A. Zollman, and N.S. Rebello. 2007. Comparing students’ and experts’ understanding of the content of a lecture. Journal of Science Education and Technology 16 (3): 213–224. Kiewra, K.A. 1985. Providing the instructor’s notes: An effective addition to student notetaking. Educational Psychologist 20 (1): 33–39. Marsh, P. 2007. What is known about student learning outcomes and how does it relate to the scholarship of teaching and learning? International Journal for the Scholarship of Teaching and Learning 1 (2). http://academics.georgiasouthern.edu/ijsotl. Ross, P.E. 2006. The expert mind. Scientific American 295 (2): 64–71. orientation of their goals, which were similar across the three instructors. Students of a discipline may lack the intellectual scaffolding and organizational structure taken for granted by experts instructing them. The primary value students expressed regarding learning goals in this study is that they provided them with structure in their efforts to know what it is they need to know. The fact that many students indicated that learning goals helped them scaffold their lecture experience provides hope that learning goals reduce some of the well-known difficulties students have in understanding lecture and in taking notes and therefore support more effective learning. We urge others to consider developing learning goals at the course level that complete the sentence, “By the end of today, students will be able to . . .” and to report both on the content of those goals and their valuation by students. We are encouraged by the positive, shared valuation by students across one computing and one microbiology class and are interested to see if others (and their students) find similar value. Exemplar learning goals of this style can be found at the University of British Columbia’s Carl Wieman Science Education Initiative website (http://cwsei.ubc.ca). n Acknowledgments We thank Steve Wolfman, Kim Voll, and Steven Hallam for their cooperation and participation in this work, as well as Carl Wieman for his invaluable input and feedback. This work was funded by the Carl Wieman Science Education Initiative at the University of British Columbia. References Adam, S. 2004. Using learning outcomes: A consideration of the nature, role, application and implications for European education of employing learning outcomes at the local, national and international levels. Report on the United Kingdom Bologna Seminar, July 2004, at Beth Simon (bsimon@cs.ucsd.edu) is a faculty member in the Computer Science and Engineering Department at the University of California, San Diego, and a former Science Teaching and Learning Fellow in the Carl Wieman Science Education Initiative and the Department of Computer Science at the University of British Columbia in Vancouver, British Columbia. Jared Taylor (jtaylor@zoology.ubc.ca) is a Science Teaching and Learning Fellow in the Carl Wieman Science Education Initiative and the Department of Microbiology and Immunology at the University of British Columbia in Vancouver, British Columbia