needs to be classified. The new article consists of a set of We also have included a concept list(similar to the well- nown tag cloud), which displays all the concepts that have been stored in the user profile. When a concept is read in er. Als a feature which highlights the concepts and related concepts For this article we define a vector containing the ranks. This found in the article in different colors vector is defined as VA (s (21) c, Additionally, Athena provides a testing environment for Rank(ei) 5. EVALUATION ∈ if ei A (22) Our research goal was to find whether ontology-based rec- ommenders perform better than a classic recommender like Each concept from the extended user profile that appears in TF-IDF. To evaluate our approach, we have developed a test he article is assigned the same rank as the one in vu. The method and built a test environment emaining concepts are assigned zero. Concepts appearing in The testing method we have chosen, is based on super- he article but not in the profile are ignored. In the current vised learning. First the user is shown a set of 300 news ork we assume that all concepts found in a news item are articles, assembled by the designer of the test. For each ar- equally important ticle the user has to read the title and the summary. Based To compare the article with the user profile we propose on this. he should decide whether the article is interesting to compute the extent to which the article fits the profile by or not. For the experiments we have used 5 users, each user dividing the sum of the ranks of concepts in the article by having different news interests than the other ones the sun of the ranks of the concepts in the user profile ly, this set of articles, with the corresponding ratings by the user, is split randomly into two different sets Similarity(VA v)= .u the training set(60%)and the validation set(40%). The two sets are filled with a relatively equal number of interest- ing items. The training set is used to create a user profile The article with the highest similarity measure fits best each item that is marked as interesting will be added to this th the user profile. The cut-off value for news item inter profile. The validation set is used by each recommender to estingness was fixed to 0.5, after experimenting with values determine for each news item the similarity with the user anging from 0 to l with a step of 0.1 profile. An article is considered to be interesting if the simi- 4. ATHENA IMPLEMENTATION larity to the user profile is higher than the predefined cut-off value. otherwise it is classified as not interesting As Athena is an extension to the hermes framework. it o determine the performance of a recommende has been implemented as a plug-in to the existing imple- sures like accuracy, precision, recall(sensitivity), an mentation of the Hermes framework, the Hermes News Por- ficity are used. These measures are calculated by tal(HNP). The implementation of Athena is done in the confusion matrix, which stores the number of true positives, coe language as the HNP, Java. As a stemmer, for the false positives, false negatives, and true negatives, for each content-based method. we have used the Krovetz Stemmer of the analyzed recommender systems. Based on these mea- sures, in the rest of this section, we compare the performance The user interface of Athena consists of 3 tabs: a browser of the ranked recommender with respect to the performance for all news items, a tab for the recommendations, and a of the other considered recommender systems ab for evaluation purposes. The browser contains the news The results in table 2 and table 3 show that the ranked items sorted by date. Here, the user can browse through recommender scores better than TF-IDF for accuracy(94% the news items instead of browsing through query results as vs. 90%), precision(93% vS. 90%), and recall(62%vs n the HNP. Each item is presented with a title, summary, 45%), and has the same high score for specificity(99%) an image which is related to the news item. and the date or accuracy and precision, from all implemented methods published the ranked recommender scores best, closely followed(differ- The user profile is created from the articles the user has ence of 1%)by the Jaccard recommender. The recall of the read. We define reading an article as opening it into the ranked recommender(62%)is nevertheless lower than the Web browser. After reading several articles, the user can se- recall of concept equivalence(98%), binary cosine(95%) choose a type of recommender, and get the recommended is for the ranked recommender, Jaccard, and TF-De(99% lect the recommendations tab in Athena. Here the user can and semantic relatedness(92%). The best specificity articles based on the user profile. Only one recommender The ranked recommender is able to propose interesting can be chosen at a time. By clicking the refresh button, stories for the user, eliminating most uninteresting stories he recommender starts analyzing the user profile. After a Nevertheless, during the news filtering, news items deemed short period of time, the recommender presents a list of news interesting by the user are also wrongly eliminated. How items that the user may find interesting. This list consists of ever, the ranked recommender provides the user with mor the news items that the recommender ranked highest. Each interesting news items relative to the total number of recor news item is presented with its corresponding ranks. The mended new items than a traditional recommender syster ser can browse through the results, and by double-clicking The ranked recommender also suggests more interesting sto- t a news item, it is registered in the user profile, whereafter ries relative to the total number of recommended new items the user's Web browser shows the concerning news article than the other considered semantic-based recommenderneeds to be classified. The new article consists of a set of concepts, specified as A: A = {a1, a2, · · · , at} . (20) For this article we define a vector containing the ranks. This vector is defined as VA: VA = (s1, s2, · · · , st) , (21) si =  Rank(ei) if ei ∈ A 0 if ei ∈/ A . (22) Each concept from the extended user profile that appears in the article is assigned the same rank as the one in VU . The remaining concepts are assigned zero. Concepts appearing in the article but not in the profile are ignored. In the current work we assume that all concepts found in a news item are equally important. To compare the article with the user profile we propose to compute the extent to which the article fits the profile by dividing the sum of the ranks of concepts in the article by the sum of the ranks of the concepts in the user profile: Similarity(VA, VU ) = P va∈VA va P vu∈VU vu . (23) The article with the highest similarity measure fits best with the user profile. The cut-off value for news item inter￾estingness was fixed to 0.5, after experimenting with values ranging from 0 to 1 with a step of 0.1. 4. ATHENA IMPLEMENTATION As Athena is an extension to the Hermes framework, it has been implemented as a plug-in to the existing imple￾mentation of the Hermes framework, the Hermes News Por￾tal (HNP). The implementation of Athena is done in the same language as the HNP, Java. As a stemmer, for the content-based method, we have used the Krovetz Stemmer [9]. The user interface of Athena consists of 3 tabs: a browser for all news items, a tab for the recommendations, and a tab for evaluation purposes. The browser contains the news items sorted by date. Here, the user can browse through the news items instead of browsing through query results as in the HNP. Each item is presented with a title, summary, an image which is related to the news item, and the date published. The user profile is created from the articles the user has read. We define reading an article as opening it into the Web browser. After reading several articles, the user can se￾lect the recommendations tab in Athena. Here the user can choose a type of recommender, and get the recommended articles based on the user profile. Only one recommender can be chosen at a time. By clicking the refresh button, the recommender starts analyzing the user profile. After a short period of time, the recommender presents a list of news items that the user may find interesting. This list consists of the news items that the recommender ranked highest. Each news item is presented with its corresponding ranks. The user can browse through the results, and by double-clicking at a news item, it is registered in the user profile, whereafter the user’s Web browser shows the concerning news article. We also have included a concept list (similar to the well￾known tag cloud), which displays all the concepts that have been stored in the user profile. When a concept is read in multiple articles, the font gets larger. Also, we have included a feature which highlights the concepts and related concepts found in the article in different colors. Additionally, Athena provides a testing environment for evaluation purposes which will be discussed in section 5. 5. EVALUATION Our research goal was to find whether ontology-based rec￾ommenders perform better than a classic recommender like TF-IDF. To evaluate our approach, we have developed a test method and built a test environment. The testing method we have chosen, is based on super￾vised learning. First the user is shown a set of 300 news articles, assembled by the designer of the test. For each ar￾ticle the user has to read the title and the summary. Based on this, he should decide whether the article is interesting or not. For the experiments we have used 5 users, each user having different news interests than the other ones. Subsequently, this set of articles, with the corresponding ratings by the user, is split randomly into two different sets, the training set (60%) and the validation set (40%). The two sets are filled with a relatively equal number of interest￾ing items. The training set is used to create a user profile. Each item that is marked as interesting will be added to this profile. The validation set is used by each recommender to determine for each news item the similarity with the user profile. An article is considered to be interesting if the simi￾larity to the user profile is higher than the predefined cut-off value, otherwise it is classified as not interesting. To determine the performance of a recommender, mea￾sures like accuracy, precision, recall (sensitivity), and speci- ficity are used. These measures are calculated by using a confusion matrix, which stores the number of true positives, false positives, false negatives, and true negatives, for each of the analyzed recommender systems. Based on these mea￾sures, in the rest of this section, we compare the performance of the ranked recommender with respect to the performance of the other considered recommender systems. The results in Table 2 and Table 3 show that the ranked recommender scores better than TF-IDF for accuracy (94% vs. 90%), precision (93% vs. 90%), and recall (62% vs. 45%), and has the same high score for specificity (99%). For accuracy and precision, from all implemented methods, the ranked recommender scores best, closely followed (differ￾ence of 1%) by the Jaccard recommender. The recall of the ranked recommender (62%) is nevertheless lower than the recall of concept equivalence (98%), binary cosine (95%), and semantic relatedness (92%). The best specificity (99%) is for the ranked recommender, Jaccard, and TF-IDF. The ranked recommender is able to propose interesting stories for the user, eliminating most uninteresting stories. Nevertheless, during the news filtering, news items deemed interesting by the user are also wrongly eliminated. How￾ever, the ranked recommender provides the user with more interesting news items relative to the total number of recom￾mended new items than a traditional recommender system. The ranked recommender also suggests more interesting sto￾ries relative to the total number of recommended new items than the other considered semantic-based recommenders