Table 2: Accuracy and Precision considered ontology-based recommenders. Nevertheless, the Method Accuracy Precision recall is lower than some of the implemented ontology-based TF-IDF The knowledge base that is used, is partly created by a Concept Equivalence44% 22% domain expert and takes a lot of effort. Future research 23 should focus on automatically creating and maintaining such a knowledge base to support ontology-based recommenda- Related tion methods. Besides the improvement of the knowledge Ranked base, the algorithm can be improved as well. In our approach we have focused on a limited number of relations betwee concepts, for instance only the direct relations. However Table 3: Sensitivity and specificity concepts might be related to each other on different levels Method Recall Specificity I i.e., concepts might not be directly related to each other TF-IDF but there might exist a relation with one or more concepts Concept Equivalence etween them. Additionally, we would like, in the future, to take into account the importance of a concept in a news item Semantic Relatedness 927047% 7. REFERENCES [1J. Ahn, P. Brusilovsky, J. Grad Ie and S. Y Syn. Open User Profiles for Adaptive News Systems 6. CONCLUSION Help or Harm? In 16th International Conference on This paper describes Athena, an extension to the Hermes World wide web, pages 11-20. ACM, 2007 framework that provides several methods for news item rec- 2 D Billsus and M. J Pazzani. A Personal News Agent emendation based on the user's interests. The system uses that Talks, Learns and Explains In The Third Annua a user profile, news items, and several similarity measures Conference on Autonomous Agents, pages 268-275 At the heart of Athena is the ontology provided by the ACM, May 1999 Hermes framework. This ontology contains the domain con- 3 T Bogers and A van den Bosch. Comparing and cepts and the relationships between the concepts. with Evaluating Information Retrieval Algorithms for News these relationships, more information about each concept Recommendation In ACM Conference On is available than only the concept itself. This allows Athena Recommender Systems, pages 141-144. ACM, 2007. o consider different articles interesting than by using ex- 4 H. Cunningham. GATE, a General Architecture for sting technologies that employ content-based methods, like Text en Computers and the Humanities, TF-IDF, because it does not only consider the concepts that 6:223-254,2002 appear in the article, but also the ones that are related to [5]P. De Bra, A.T.M. Aerts, GJ.Houben, and H. Wu Making General-Purpose Adaptive Hypermedia Work We have described different methods to employ ontologies In WebNet 2000 Conference, pages 117-123. AACE comparing the user profile with a new article. We started 2000. with a content-based method that employs TF-IDF and the [6 C. Fellbaum, editor. WordNet: An ElectronicLerical cosine similarity measure, followed by three basic semantic- Database. MIT Press, Cambridge, MA, 1998 based methods. Concept equivalence is a simple, intuitive method that looks for articles that contain at 7 F. Frasincar, J. Borsje, and L. Levering. A Semantic Web-Based Approach for Building Personalized News oncepts from the profile. This method does account the number of concepts found in th es. International Journal of E-B Research,5(3):35-53,2009 In order to take into account these concepts, we have used binary cosine and Jaccard. Those methods compute the [8 F. Getahun, J. Tekli, C. Richard, M. Viviani, and similarity between the article and the profile Yetongnon. Relating RSS News/Items. In 9th A more advanced method also takes into account the se. International Conference on Web Engineering, pages mantic relatedness between different concepts, which are 442-452. Springer,2009 provided by the underlying ontology. a weight is assigned to 9S. Guzman-Lara KStem Java Implementation ach concept based on its neighborhood and the enclosure University of Massachusetts Amherst. 2007 imilarity. This method, referred as semantic relatedness. http://ciir.cs.umassedu/cgi-bin/ is based on linguistic relationships. Finally, we presented a downloads/downloads. cgi. lew method. called ranked recommender, which also uses [10 S. E Middleton, N. R Shadbolt, and D. C D. Roure he ontology relationships between the concepts. It takes Ontological User Profiling in Recommender Syster the concepts from the user profile and combines these with ACM Transactions on Information Systems, the related concepts to create the extended user profile. 22(1)5488,2004 In this paper, we have shown that the ranked recom- [11 G Salton and C. Buckley. Term Weighting mender, our ontology-based recommender, performs better Approaches in Automatic Text Retrieval. Information than a traditional recommender systems based on TF-IDF Processing and Management, 24 (5 ) 513-523, 1988 for accuracy, precision, and recall, and equally good for [12] A Singhal, G Salton, M. Mitra, and C. Buckley specificity. It also performs better, or equally good, with Document Length Normalization. Information espect to accuracy, precision, and specificity than the other Processing and Management, 32(5 ): 619-633, 1996Table 2: Accuracy and Precision Method Accuracy Precision TF-IDF 90% 90% Concept Equivalence 44% 22% Binary Cosine 47% 23% Jaccard 93% 92% Semantic Relatedness 57% 26% Ranked 94% 93% Table 3: Sensitivity and Specificity Method Recall Specificity TF-IDF 45% 99% Concept Equivalence 98% 32% Binary Cosine 95% 36% Jaccard 58% 99% Semantic Relatedness 92% 47% Ranked 62% 99% 6. CONCLUSION This paper describes Athena, an extension to the Hermes framework that provides several methods for news item rec￾ommendation based on the user’s interests. The system uses a user profile, news items, and several similarity measures. At the heart of Athena is the ontology provided by the Hermes framework. This ontology contains the domain con￾cepts and the relationships between the concepts. With these relationships, more information about each concept is available than only the concept itself. This allows Athena to consider different articles interesting than by using ex￾isting technologies that employ content-based methods, like TF-IDF, because it does not only consider the concepts that appear in the article, but also the ones that are related to them. We have described different methods to employ ontologies in comparing the user profile with a new article. We started with a content-based method that employs TF-IDF and the cosine similarity measure, followed by three basic semantic￾based methods. Concept equivalence is a simple, intuitive method that looks for articles that contain at least one of the concepts from the profile. This method does not take into account the number of concepts found in the news article. In order to take into account these concepts, we have used binary cosine and Jaccard. Those methods compute the similarity between the article and the profile. A more advanced method also takes into account the se￾mantic relatedness between different concepts, which are provided by the underlying ontology. A weight is assigned to each concept based on its neighborhood and the enclosure similarity. This method, referred as semantic relatedness, is based on linguistic relationships. Finally, we presented a new method, called ranked recommender, which also uses the ontology relationships between the concepts. It takes the concepts from the user profile and combines these with the related concepts to create the extended user profile. In this paper, we have shown that the ranked recom￾mender, our ontology-based recommender, performs better than a traditional recommender systems based on TF-IDF for accuracy, precision, and recall, and equally good for specificity. It also performs better, or equally good, with respect to accuracy, precision, and specificity than the other considered ontology-based recommenders. Nevertheless, the recall is lower than some of the implemented ontology-based recommenders. The knowledge base that is used, is partly created by a domain expert and takes a lot of effort. Future research should focus on automatically creating and maintaining such a knowledge base to support ontology-based recommenda￾tion methods. Besides the improvement of the knowledge base, the algorithm can be improved as well. In our approach we have focused on a limited number of relations between concepts, for instance only the direct relations. However, concepts might be related to each other on different levels, i.e., concepts might not be directly related to each other but there might exist a relation with one or more concepts between them. Additionally, we would like, in the future, to take into account the importance of a concept in a news item. 7. REFERENCES [1] J. Ahn, P. Brusilovsky, J. Grady, D. He, and S. Y. Syn. Open User Profiles for Adaptive News Systems: Help or Harm? In 16th International Conference on World Wide Web, pages 11–20. ACM, 2007. [2] D. Billsus and M. J. Pazzani. A Personal News Agent that Talks, Learns and Explains. In The Third Annual Conference on Autonomous Agents, pages 268–275. ACM, May 1999. [3] T. Bogers and A. van den Bosch. Comparing and Evaluating Information Retrieval Algorithms for News Recommendation. In ACM Conference On Recommender Systems, pages 141–144. ACM, 2007. [4] H. Cunningham. GATE, a General Architecture for Text Engineering. Computers and the Humanities, 36:223–254, 2002. [5] P. De Bra, A. T. M. Aerts, G. J. Houben, and H. Wu. Making General-Purpose Adaptive Hypermedia Work. In WebNet 2000 Conference, pages 117–123. AACE, 2000. [6] C. Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA, 1998. [7] F. Frasincar, J. Borsje, and L. Levering. A Semantic Web-Based Approach for Building Personalized News Services. International Journal of E-Business Research, 5(3):35–53, 2009. [8] F. Getahun, J. Tekli, C. Richard, M. Viviani, and K. Yetongnon. Relating RSS News/Items. In 9th International Conference on Web Engineering, pages 442–452. Springer, 2009. [9] S. Guzman-Lara. KStem Java Implementation. University of Massachusetts Amherst, 2007. http://ciir.cs.umass.edu/cgi-bin/ downloads/downloads.cgi. [10] S. E. Middleton, N. R. Shadbolt, and D. C. D. Roure. Ontological User Profiling in Recommender Systems. ACM Transactions on Information Systems, 22(1):54–88, 2004. [11] G. Salton and C. Buckley. Term Weighting Approaches in Automatic Text Retrieval. Information Processing and Management, 24(5):513–523, 1988. [12] A. Singhal, G. Salton, M. Mitra, and C. Buckley. Document Length Normalization. Information Processing and Management, 32(5):619–633, 1996