subject was given three questions to find related research and TTA is the "Tag, Title with Abstract" indexing method papers. They formulated their own queries according to the as shown in Table I given questions. They were asked to use same query for each search engine. Then, they were asked to rate the TABLE I TOP 10 RANKS AVERAGE NDCG FOR THREE DIFFERENT relevancy of the search result set on a five-point scale SEARCH ENGINES Score o is not relevant at all Average NDCG Score I is probably not relevant. TTA Score 2 is less relevant 0.4476 0.6106 Score 3 is probably relevant Score 4 is extremely relevant 0.4733 0.6305 0.6748 0490906240 0.517 0.5282 0.6344 0.6225 0.6367 0.522 0.6218 0.5214 0.6225 0.6189 0.6222 0.5331 0.6149 0.6267 Fig. 4 compares the average of NDCG score on three different search engines. The x-axis denotes the first 10 ranks of the search results, whereas the y-axis represents adage the average ndcg score Fig 3. Example of search results It suggests that the"Tag, Title with Abstract" indexing method provide a better set of search results compared with The top 20 search results of each search en "Tag only "indexing method and the"Title with Abstract displayed for relevancy judgment. The subject indexing method experiment were considered experts in the 0. science field; their relevancy ratings for each query are considered to be perfect 中Tag TItle and Astac E. Evaluation metric NDCG(Normalized Discounted Cumulative Gain)as originally proposed by Jarvelin and Kekalainen [9], was so used to evaluate the performance of each search engine This metric is a retrieval measurement devised specifically for web search evaluation. The NDCG is computed as in the equation(4) ADC=M.12(+ Where k is a truncation or threshold level, r() is an integer representing the relevancy given by the subject, and Fig 4 Comparison of the average NDCG for three indexing methods. M is a normalization constant calculated so that the perfect ordering would obtain a ndcg of 1. ndcg rewards Table ll. result of ftest relevant documents appearing in the top ranked search Rank sults and punishes irrelevant documents by reducing their F-test contributions to ndcg 0.008 IV RESULT AND DISCUSSION 0.000 We present the top ten ranks of average NDCG scores 12165 0.000 where k is level of the rank. T is"Tag only"indexing 13.971 0.000 method. TA is the "Title with abstract" indexing method 15.533 0.000subject was given three questions to find related research papers. They formulated their own queries according to the given questions. They were asked to use same query for each search engine. Then, they were asked to rate the relevancy of the search result set on a five-point scale: Score 0 is not relevant at all. Score 1 is probably not relevant. Score 2 is less relevant. Score 3 is probably relevant. Score 4 is extremely relevant. Fig.3. Example of search results The top 20 search results of each search engine were displayed for relevancy judgment. The subjects in this experiment were considered experts in the computer science field; their relevancy ratings for each query are considered to be perfect. E. Evaluation Metric NDCG (Normalized Discounted Cumulative Gain) as originally proposed by Jarvelin and Kekalainen [9], was used to evaluate the performance of each search engine. This metric is a retrieval measurement devised specifically for web search evaluation. The NDCG is computed as in the equation (4).       ¦   k j r j q q j NDCG M 1 log 1 2 1 Where k is a truncation or threshold level, r(j) is an integer representing the relevancy given by the subject, and Mq is a normalization constant calculated so that the perfect ordering would obtain a NDCG of 1. NDCG rewards relevant documents appearing in the top ranked search results and punishes irrelevant documents by reducing their contributions to NDCG IV. RESULT AND DISCUSSION We present the top ten ranks of average NDCG scores where k is level of the rank. T is “Tag only” indexing method, TA is the “Title with Abstract” indexing method, and TTA is the “Tag, Title with Abstract” indexing method as shown in Table I. TABLE I. TOP 10 RANKS AVERAGE NDCG FOR THREE DIFFERENT SEARCH ENGINES K Average NDCG T TA TTA 1 0.4476 0.6106 0.7187 2 0.4733 0.6305 0.6748 3 0.4909 0.6240 0.6515 4 0.5171 0.6334 0.6329 5 0.5282 0.6274 0.6344 6 0.5211 0.6225 0.6367 7 0.5227 0.6218 0.6281 8 0.5214 0.6197 0.6225 9 0.5279 0.6189 0.6222 10 0.5331 0.6149 0.6267 Fig. 4 compares the average of NDCG score on three different search engines. The x-axis denotes the first 10 ranks of the search results, whereas the y-axis represents the average NDCG score. It suggests that the “Tag, Title with Abstract” indexing method provide a better set of search results compared with “Tag only” indexing method and the “Title with Abstract” indexing method. Fig.4 Comparison of the average NDCG for three indexing methods. Table II. Result of F-test Rank (K) N F-test sig (2-tailed) 1 45 5.071 0.008 1-2 90 9.155 0.000 1-3 135 12.165 0.000 1-4 180 13.971 0.000 1-5 225 15.533 0.000 (4) 154