178 L Chen P Pu Table 1 Summary of control variables in a critiquing-based recommender system and the main differences between DynamicCritiquing and Example Critiquing in respect of these aspects Critiquing coverage Number of Number of Compound litian recom- recommended items critiquing after each (UC) antiquing(NCR) Single item User-initiated System-suggested (McCarthy et al. 2005c) k items(k=7) k items(k=7) User-initiated User-initiated ( Chen and Pu 2006) as a comparative user study of the two typical applications: Dynamic Critiquing and Example Critiquing, with the purpose of identifying which one would perform more effectively. In the second trial, we made some changes on the two systems to make them different only on one dimension, the critiquing aid, in order to observe the single elements influence. The third trial measured users' performance in a hybrid critiquing system where the two types of critiquing aids: system-suggested and user-initiated were combined on the same screen Combining the results from these three trials, we expected to reveal the effects of different independent variables on users'decision performance and quality Therefore, before carrying out these experiments, it was necessary to first define concrete dependent variables that we were to measure. We have established an evalua tion framework aimed to contain all of key standards. In fact, identifying the appropri- ate criteria for evaluating the true benefits of a recommender system is a challenging issue Related work has primarily focused on users'objective interaction effort, such as their interaction sessions(McCarthy et al. 2005a, b, c)and task completion time, while placing less emphasis on what actual decision accuracy In fact, the accuracy-effort model has long been studied in the domain of classical decision theories( Payne et al 1993: Spiekermann and Parachiv 2002), and it has been broadly accepted that both important to determine the fundamental user benefits of a decision suppor the system's ideal goal should be to enable its users to obtain high level of de ccuracy with low amount of effort(Haubl and Trifts 2000) In addition, a recommender systems ability in increasing user trust and convincing them of its recommendations, such as which camera to purchase, is also a crucial factor, particularly meaningful when the system is applied in the e-commerce envi- ronment. Two main trust-inspired behavioral intentions(called trusting intentions) include intention to purchase indi whether the system could stimulate its users to purchase a product, and intention to return referring whether the system could prompt users to return to it for future use so that a long-term relationship is estab lished( Grabner-Krauter and Kaluscha 2003) Therefore, motivated by these requirements, we have classified them into three categories of dependent variables in our evaluation framework: decision accuracy decision effort and trusting intentions(see Fig 4178 L. Chen, P. Pu Table 1 Summary of control variables in a critiquing-based recommender system and the main differences between DynamicCritiquing and ExampleCritiquing in respect of these aspects Critiquing coverage Critiquing aid Number of initial recom￾mendations (NIR) Number of recommended items after each critiquing (NCR) Unit critiquing (UC) Compound critiquing (CC) DynamicCritiquing (McCarthy et al. 2005c) Single item Single item User-initiated System-suggested ExampleCritiquing (Chen and Pu 2006) k items (k = 7) k items (k = 7) User-initiated User-initiated was a comparative user study of the two typical applications: DynamicCritiquing and ExampleCritiquing, with the purpose of identifying which one would perform more effectively. In the second trial, we made some changes on the two systems to make them different only on one dimension, the critiquing aid, in order to observe the single element’s influence. The third trial measured users’ performance in a hybrid critiquing system where the two types of critiquing aids: system-suggested and user-initiated, were combined on the same screen. Combining the results from these three trials, we expected to reveal the effects of different independent variables on users’ decision performance and quality. Therefore, before carrying out these experiments, it was necessary to first define concrete dependent variables that we were to measure. We have established an evalua￾tion framework aimed to contain all of key standards. In fact, identifying the appropri￾ate criteria for evaluating the true benefits of a recommender system is a challenging issue. Related work has primarily focused on users’ objective interaction effort, such as their interaction sessions (McCarthy et al. 2005a,b,c) and task completion time, while placing less emphasis on what actual decision accuracy users can eventually achieve and how much cognitive effort users perceive to exert. In fact, the accuracy-effort model has long been studied in the domain of classical decision theories (Payne et al. 1993; Spiekermann and Parachiv 2002), and it has been broadly accepted that they are both important to determine the fundamental user benefits of a decision support, since the system’s ideal goal should be to enable its users to obtain high level of decision accuracy with low amount of effort (Häubl and Trifts 2000). In addition, a recommender system’s ability in increasing user trust and convincing them of its recommendations, such as which camera to purchase, is also a crucial factor, particularly meaningful when the system is applied in the e-commerce envi￾ronment. Two main trust-inspired behavioral intentions (called trusting intentions) include intention to purchase indicating whether the system could stimulate its users to purchase a product, and intention to return referring whether the system could prompt users to return to it for future use so that a long-term relationship is estab￾lished (Grabner-Kräuter and Kaluscha 2003). Therefore, motivated by these requirements, we have classified them into three categories of dependent variables in our evaluation framework: decision accuracy, decision effort and trusting intentions (see Fig. 4). 123