American Journal of Pharmaceutical Education 2007;71 (5)Article 85 Table 1.Description of Significant Revisions to Active Learning Inventory Tool Area Modified Modification(s)Made Expert Reviewers'Comments During Validation .Changed 'Risk Level'to'Complexity Level" .Need to revise and clarify active-leaming activity list .RemovedDirected Reading ·Added'Free Write .'Cold Calling'was renamed to'Question&Answer'by either Problems Ientified/Incurred During Reliability Testin .Need to revise and clarify active leaming activity lis ·Clarified descriptor .Would like to capture mographic information and demographic and cassroom information to .Need to clarifyQuestions&Answer'activity .Removed voluntary answer vs.cold calling and included Al denotes students responded to the question;A2 denotes 1t0-A1 question/A2 denotes students were asked to respond AND given time but did not respond-will track Al and A2 for mbers but n time as conducted in I ss than I minute provided time(min)to process then respond. active-learning techniqu e was cited as the most commor (14%).Most instructors believed that any increase in the reason for its use.The reported amount of time time devoted to active-learning came at the expense of completing active-learning activities varied widely lecture content(86%). among instructors (range 10%-50%of total classroom Results showed general congruence between instruc- time).Most instructors(71%)reported that the use of tor perception and the Active-Learning Inventory Tool active learning required more preparation time.particu- observations with respect to the time spent completing larly if the technique was new to them,if they were un- active-learning activities.With the exception of instruc tors underreporting their use of student questioning,in- e use of active learning inc out the types remove milar to tha siz sing th Active-L number of active range.58-100%):number of different ypes of active learning used (78%:ran ge time 8654 ■Number of active DISCUSSION The Active-Learning Inventory Tool is the first tool that utilizes qualitative and quantitative information to tive learning be ev alid ability. Lecture Number engagem ith Figure 1.Percent agreement among ob sfor each lecture orme reflectio active-learning technique was cited as the most common reason for its use. The reported amount of time spent completing active-learning activities varied widely among instructors (range 10%-50% of total classroom time). Most instructors (71%) reported that the use of active learning required more preparation time, particu￾larly if the technique was new to them, if they were un￾familiar with active learning overall, or if this was a new lecture. Barriers to the use of active learning included lack of time (86%), need to remove lecture content (43%), lack of technology in the classroom (14%), and large class size (14%). Most instructors believed that any increase in the time devoted to active-learning came at the expense of lecture content (86%). Results showed general congruence between instruc￾tor perception and the Active-Learning Inventory Tool observations with respect to the time spent completing active-learning activities. With the exception of instruc￾tors underreporting their use of student questioning, in￾structor perceptions about the types and quantity of active learning used in lectures was similar to that observed using the Active-Learning Inventory Tool. Agreement among observers and instructors was good for all out￾comes: number of active-learning episodes used (82%; range, 58%-100%); number of different types of active learning used (78%; range, 62%-100%), and average time per active-learning episode (68%; range, 50%-82%). DISCUSSION The Active-Learning Inventory Tool is the first tool that utilizes qualitative and quantitative information to capture the amount and type of active learning in the classroom that has been evaluated for validity and reli￾ability. Through an extensive review of the literature on active engagement, consultation with experts in the field, and continual and thoughtful informed reflection, Table 1. Description of Significant Revisions to Active Learning Inventory Tool Area Modified Modification(s) Made Expert Reviewers’ Comments During Validation d Categorizing active learning based on ‘Risk Level’ is not the appropriate descriptor d Changed ‘Risk Level’ to ‘Complexity Level’ d Need to revise and clarify active-learning activity list d Removed ‘Directed Reading’ d Added ‘Free Write’ d ‘Cold Calling’ was renamed to ‘Question & Answer’ by either u voluntary or u cold calling d Clarified facilitator for Small Group Presentations Problems Identified/Incurred During Reliability Testing d Need to revise and clarify active learning activity list d Clarified descriptors d Removed ‘Pause Procedure’ d Would like to capture demographic information and classroom environment d Added instructor demographic and classroom information to The active-learning inventory tool d Need to clarify ‘Questions & Answer’ activity d Removed voluntary answer vs. cold calling and included A1 denotes students responded to the question; A2 denotes students were asked to respond and given time, but did not respond. This does not include rhetorical questions. d Further changed to: A1 denotes students responded to question/A2 denotes students were asked to respond AND given time but did not respond – will track A1 and A2 for numbers but not time as conducted in less than 1 minute. A3 denote a higher-order question, where students are provided time (.1 min) to process then respond. Figure 1. Percent agreement among observers for each lecture. American Journal of Pharmaceutical Education 2007; 71 (5) Article 85. 4