We applied One Way ANOVA on NDCG at K=l, 1-2, the"Tag only" indexing approach. In order to confirm the 1-3, 1-4 and 1-5 respectively to test whether there is a result of the experiment, additional experiments should be difference among the mean NDCG from three different conducted. Improving research paper indexing approaches dexing approaches. As shown in Table Il, we found that not only enhance the performance of academic paper there was evidence that not all of the means of NDCG of searches, but also all document searches in general the three indexing approaches were equal at a=0.05 levels We plan to integrate social tagging information to of significance. In other words, the difference in the set of improve a ranking of search results. Future research in search results returned from three different indexing the area consists of extending the scale of experiments approaches were statistically significant. comparing better index with CiteULike, der ranking as well as optimizing the parameters Table Ill Result of mul of top five ranks Rank Difference Error(2-tailed) The authors would like to thank suan sunandha 62936 Rajabhat University for scholarship support. The study TA TTA not possible without the data from CiteULike 0857099 TTA 2711ll 005 10814 REfERENCES 160087 [1]T. Budura, S. Michel, P. Cudre-Mauroux, and K. Aberer, "To .076224 236311 .0563720 Systems', SIGIS08, Singapore, 20-24 July 2008 TTA [2] A Capocci and G. Caldarelli, "Folksonomies and Clustering 076224 in the Collaborative System CiteULike, eprint ar Xiv: 0710.2835 151072 TA 3] w. Choochaiwattana, and M. B. Spring, Applying Social TTA 059981 annotations to Retrieve 0440961 211053 Proceedings of 2009 International Conference on Information TTA Management and Engineering(ICIME 2009), Kuala Lumpur TA 059981 Malaysia 3-5 April 2009 14 U. Farooq, C H. Ganoe, J M. Carroll, and C TA "Supporting distributed scientific collaboration: Im 036986 fordesigning the CiteSeer collaborator" Proceedings T 187231 awaii Int'/ Conference on System Sciences), IEEE c TTA Society, Waikoloa, Hawaii, 3-6 January 2007, 26c. 5]U. Farooq, T.G. Kannampallil, Y Song, C H. Ganoe John M. Carroll 133654 nd C. Lee Giles, "Evalating Tagging Behavior in Social Bookman TA Systems: Metrics and design heuristics", Proceedings of the 2007 037276 0322510 T 170930 (GROUP"07Sanibel Island, Florida, USA, 4-7 November 2007, pp. 351-360 TTA [6]J. Gelernter,"A Quantitative Analysis of Collaborative Tags 037276 ternational Conference on Collaborative Computing: Nenvorking applications and worksharing. 2007. Collaborate Com 2007., 12-15 Nov We then preformed a multiple comparisons to find the 2007. New York. ny. Dp 376-38 differences among the three indexing approaches table Im 7 C L. Giles, K. Bollacker, and S. Lawrence, CiteSeer: An shows the result of the multiple comparisons of the three Automatic Citation Indexing System", Proceedings of the difference indexing approaches. Although Fig. 4 suggests Conference on Digital Libraries, ACM Press, Pittsburg, that the " Tag, Title with Abstract" indexing approach Pennsylvania, 23-26 June 1998, pp. 89-98 provides a better set of search results compared with the 8]K. Jarvelin, and J. Kekalainen, "IR evaluat for retrieving highly relevant documents, Proceeding of the orld Wide other two approaches, the results from the multiple Web Conference(ww2006), May 2006 comparisons show that a set of search results provided by 9R. aschke, L. B. Marinho, A. Hotho, L. Schmidt-Thieme, and G. the"Tag, Title with Abstract" indexing approach is not Tag Recommendations in Folksonomies", In Proceedings of statistically different from a set of search results provided 4702 of Lecture Notes in Computer Science, by the"Title with Abstract indexing approach Verlag,2007,pp.506-514 [10B. Ll, and Q. Zhu, " The Determination of Semantic Dimension in V. CONCLUSION AND FUTURE WORK 2008(,08),20-22 Dec2008, Shanghai,pp909913 Appdioonal [11]S. Noel, and R. Beale, " Sharing vocabularies: Tag Usage in This paper proposes to use an interesting method of CiteULike", Proceedings of the 22 Anmual Conference of Interaction,a indexing of research papers on CiteULike. The experiment Specialist group of the British Computer Societ(HCl shows that the"Title with Abstract"indexing approach and 12]F. M. Suchanek, M. Vojnovic, and D Gunawardena, ""Social Tags the Tag, Title with Abstract" indexing approach provides Meaning and Suggestions", CIKM08, Napa Valley, California, USA. 26 a better research paper search result set as compared with 0 October 2008We applied One Way ANOVA on NDCG at K=1, 1-2, 1-3, 1-4 and 1-5 respectively to test whether there is a difference among the mean NDCG from three different indexing approaches. As shown in Table II, we found that there was evidence that not all of the means of NDCG of the three indexing approaches were equal at Į=0.05 levels of significance. In other words, the difference in the set of search results returned from three different indexing approaches were statistically significant. Table III Result of Multiple Comparisons of top five ranks Rank (K) Indexing Mean Difference (I-J) Std. Error sig (I) (J) (2-tailed) 1 TA T .162936 .0857099 .142 TTA -.108148 .419 TTA T .271111 .005 TA .108148 .419 1-2 TA T .160087 .0563720 0.13 TTA -.076224 .038 TTA T .236311 .000 TA .076224 .368 1-3 TA T .151072 .0440961 .002 TTA -.059981 .363 TTA T .211053 .000 TA .059981 .363 1-4 TA T .142373 .0369865 .000 TTA -.044857 .446 TTA T .187231 .000 TA .044857 .446 1-5 TA T .133654 .0322510 .000 TTA -.037276 .480 TTA T .170930 .000 TA .037276 .480 We then preformed a multiple comparisons to find the differences among the three indexing approaches. Table III shows the result of the multiple comparisons of the three difference indexing approaches. Although Fig. 4 suggests that the “Tag, Title with Abstract” indexing approach provides a better set of search results compared with the other two approaches, the results from the multiple comparisons show that a set of search results provided by the “Tag, Title with Abstract” indexing approach is not statistically different from a set of search results provided by the “Title with Abstract” indexing approach. V. CONCLUSION AND FUTURE WORK This paper proposes to use an interesting method of indexing of research papers on CiteULike. The experiment shows that the “Title with Abstract” indexing approach and the “Tag, Title with Abstract” indexing approach provides a better research paper search result set as compared with the “Tag only” indexing approach. In order to confirm the result of the experiment, additional experiments should be conducted. Improving research paper indexing approaches not only enhance the performance of academic paper searches, but also all document searches in general. We plan to integrate social tagging information to improve a ranking of search results. Future research in the area consists of extending the scale of experiments, comparing better index with CiteULike, developing ranking as well as optimizing the parameters. ACKNOWLEDGMENT The authors would like to thank Suan Sunandha Rajabhat University for scholarship support. The study is not possible without the data from CiteULike. REFERENCES [1] T. Budura, S. Michel, P. Cudre-Mauroux, and K. Aberer, “To Tag or Not to tag-Harvesting Adjacent Metadata in Large-Scale Tagging Systems”, SIGIS’08, Singapore, 20-24 July 2008. [2] A. Capocci and G. Caldarelli, “Folksonomies and Clustering in the Collaborative System CiteULike”, eprint arXiv: 0710.2835, 2007. [3] W. Choochaiwattana, and M.B. Spring, “Applying Social Annotations to Retrieve and Re-rank Web Resources”, Proceedings of 2009 International Conference on Information Management and Engineering (ICIME 2009), Kuala Lumpur, Malaysia 3 – 5 April 2009. [4] U. Farooq, C.H. Ganoe, J.M. Carroll, and C.L. Giles, “Supporting distributed scientific collaboration: Implications fordesigning the CiteSeer collaborator” Proceedings of the Hawaii Int’l Conference on System Sciences), IEEE Compute Society,Waikoloa, Hawaii, 3-6 January 2007, 26c. [5] U. Farooq, T.G. Kannampallil, Y. Song ,C.H. Ganoe ,John M. Carroll ,and C. Lee Giles, “Evalating Tagging Behavior in Social Bookmarking Systems: Metrics and design heuristics”, Proceedings of the 2007 international ACM conference on Supporting group work (GROUP’07)Sanibel Island,Florida, USA,4-7 November 2007, pp.351-360. [6] J. Gelernter, “A Quantitative Analysis of Collaborative Tags: Evaluation for Information Retrieval—a Preliminary Study”, International Conference on Collaborative Computing: Networking, Applications and Worksharing, 2007. CollaborateCom 2007. ,12-15 Nov. 2007, New York, NY. pp.376-381 [7] C.L. Giles, K. Bollacker, and S. Lawrence, “CiteSeer: An Automatic Citation Indexing System”, Proceedings of the Conference on Digital Libraries,ACM Press, Pittsburg, Pennsylvania, 23-26 June 1998, pp. 89-98. [8] K. Jarvelin, and J. Kekalainen, “IR evaluation methods for retrieving highly relevant documents”, Proceeding of the International World Wide Web Conference (WWW2006), May 2006. [9] R. J¨aschke, L. B. Marinho, A. Hotho, L. Schmidt-Thieme, and G. Stumme, “Tag Recommendations in Folksonomies”, In Proceedings of PKDD 2007, volume 4702 of Lecture Notes in Computer Science, Springer Verlag, 2007, pp. 506–514. [10] B. LI, and Q. Zhu, “The Determination of Semantic Dimension in Social Tagging System Based on SOM Model”, Second International Symposium on Intelligent Information Technology Application 2008(IITA’08), 20-22 Dec. 2008, Shanghai, pp. 909-913 [11] S. Noel, and R. Beale, “Sharing vocabularies : Tag Usage in CiteULike”, Proceedings of the 22nd Annual Conference of Interaction, a Specialist group of the British Computer Society(HCI2008), Liverpool, UK, 1-5 September. [12] F. M. Suchanek, M. Vojnovi´c, and D. Gunawardena, “Social Tags: Meaning and Suggestions”, CIKM’08, Napa Valley, California, USA. 26– 30 October 2008. 155