L Chen P Pu Table 2 Questions to measure subjective perceptions Measured subjective variables Questions each responded on a 5-point Likert scale from Perceived decision accuracy I am confident that the product I just"purchased"is really the best choice for me Perceived effort I easily found the in nI Looking for a product using this interface required too much effort(reverse scale Intention to purchase I would purchase the product I just chose if given the opportunity like this was available, I would be very likely to usethinterface If I had to search for a product online in the future and an interface I dont like this interface. so I would choice for me")on a 5-point Likert scale ranging from"strongly disagree"to"strongly agree see Table 2 4.2 Decision effort According to the accuracy-effort framework(Payne et al. 1993), another important criterion is the amount of decision effort users expended in making their choice with the system. Similar to decision accuracy, we not only measured how much objective effort users actually consumed, but also their perceived cognitive effort which we hope would indicate the amount of subjective effort people exerted The objective effort further includes two dimensions: task time and interaction effort The task time is the total time a subject spent from she started using the system till she made her final choice. The interaction effort mainly considers the amount of interaction cycles(e.g, critiquing cycles)that a user was involved. The two variables have been widely used as main measurements in related work to evaluate their recommender systems(McCarthy et al. 2005b. c). Perceived effort Perceived effort refers to the psychological cognitive cost of information-processing It represents the ease with which the subject can perform the task of obtaining and processing the relevant information in order to arrive at her decision. Since it is a subjective variable, two unified scale items(e. g, "I easily found the information I was looking for") were used to quantify its value(see Table 2 of concrete questions 4.3 Accuracy and effort The objective and subjective assessments of both decision accuracy and decision effort can not only show their respecti ve value es. but also allow us to understand how the concepts are interrelated.180 L. Chen, P. Pu Table 2 Questions to measure subjective perceptions Measured subjective variables Questions each responded on a 5-point Likert scale from “strongly disagree” to “strongly agree” Perceived decision accuracy I am confident that the product I just “purchased” is really the best choice for me. Perceived effort I easily found the information I was looking for. Looking for a product using this interface required too much effort (reverse scale). Intention to purchase I would purchase the product I just chose if given the opportunity. Intention to return If I had to search for a product online in the future and an interface like this was available, I would be very likely to use it. I don’t like this interface, so I would not use it again (reverse scale). choice for me”) on a 5-point Likert scale ranging from “strongly disagree” to “strongly agree” see Table 2. 4.2 Decision effort According to the accuracy-effort framework (Payne et al. 1993), another important criterion is the amount of decision effort users expended in making their choice with the system. Similar to decision accuracy, we not only measured how much objective effort users actually consumed, but also their perceived cognitive effort which we hope would indicate the amount of subjective effort people exerted. Objective effort The objective effort further includes two dimensions: task time and interaction effort. The task time is the total time a subject spent from she started using the system till she made her final choice. The interaction effort mainly considers the amount of interaction cycles (e.g., critiquing cycles) that a user was involved. The two variables have been widely used as main measurements in related work to evaluate their recommender systems (McCarthy et al. 2005b,c). Perceived effort Perceived effort refers to the psychological cognitive cost of information-processing. It represents the ease with which the subject can perform the task of obtaining and processing the relevant information in order to arrive at her decision. Since it is a subjective variable, two unified scale items (e.g., “I easily found the information I was looking for”) were used to quantify its value (see Table 2 of concrete questions). 4.3 Accuracy and effort The objective and subjective assessments of both decision accuracy and decision effort can not only show their respective values, but also allow us to understand how the concepts are interrelated. 123