Table 1: Two topics discovered by RBM-CS from the NIPS data. Topic 12: Markov Words probability 0.057 global optimization of a neural network hidden Markov model hybrid 00169 eld 0.018 neural network classifiers estimate kermel 0.0S3 the nature of statistical learning training algorithm for optimal margin classifiers a tutorial on support vector machines for pattern recognition statistical learning theory machine 0.069 01511 Evaluation Measure and Baseline Methods We used P@l, P@3, P@5, P@10, Rpec, MAP, Bpref, and MRR as the evaluation measures. For the details of the measures, please refer to [1][2]. We conducted the evaluation on both paper-level (without con- sidering the citation position)and sentence-level(considering the citation position We defined two baseline methods. One is based on language model (LM). Given a according to this score and recommended the top K ranked网个 citation context c, we computed the score of each paper d by p(cd)=llnsp(uld) where p(ald) is the maximum likelihood of word w in document d The other baseline is based on RBM, which learns a generative model for papers and the citation context. Then we use KL-divergence to calculate a score for each paper (by a similar equation to Equation(7). For both RBM and RBM-CS, we set the number of topic as T'=200 and the number of recommended references as the average number of the data set, i.e. K= 7 for NIPS and K= ll for Citeseer. The weights were update using a learning rate of 0.01/batch-size, momentum of 0.9, and a weight decay of 0.001 Estimated Topics Table I shows two example topics discovered by RBM-CS from the NIPS data. We can see that our model can capture the topic distribution very well. Performance of Citation recommendation Table 2 shows the result of citation rec- ommendation. We see that our proposed model clearly outperforms the two baseline models. The language model suffers from the fact that it is based on only keyword matching. The RBM uses a hidden topic layer to alleviate the problem. However, it is aimed at optimize p(w), which might be inappropriate for citation recommendation. In addition, RBM cannot capture the dependencies between paper contents and citation relationships. Our proposed RBM-CS can be advantageous to optimize p(ll) directly and to model the dependencies between paper contents and citation relationships We can also see from Table 2 that the recommendation performance is much better on the Citeseer data than that on the NiPs data. This means that on the sparse data, the recommendation tasks would be more difficult. How to improve the recommendation performance on the sparse data is also one of our ongoing work.6 Jie Tang and Jing Zhang Table 1: Two topics discovered by RBM-CS from the NIPS data. “Topic 12: Markov Model” Words Citation hmm 0.091 state 0.063 markov 0.058 probability 0.057 field 0.018 links between Markov models and multilayer perceptrons 0.0347 a tutorial on hidden Markov models and selected applications in speech recognition 0.0221 connectionist speech recognition a hybrid approach 0.0169 global optimization of a neural network hidden Markov model hybrid 0.0169 neural network classifiers estimate Bayesian a posteriori probabilities 0.0169 “Topic 97: Support Vector Machines” Words Citation kernel 0.083 margin 0.079 support 0.075 svm 0.075 machine 0.069 the nature of statistical learning 0.036363 a training algorithm for optimal margin classifiers 0.026984 a tutorial on support vector machines for pattern recognition 0.026763 statistical learning theory 0.020220 support vector networks 0.015117 Evaluation Measure and Baseline Methods We used P@1, P@3, P@5, P@10, Rpec, MAP, Bpref, and MRR as the evaluation measures. For the details of the measures, please refer to [1] [2]. We conducted the evaluation on both paper-level (without con￾sidering the citation position) and sentence-level (considering the citation position). We defined two baseline methods. One is based on language model (LM). Given a citation context c, we computed the score of each paper d by p(c|d) = Q w∈c p(w|d), where p(w|d) is the maximum likelihood of word w in document d. We ranked papers according to this score and recommended the top K ranked papers. The other baseline is based on RBM, which learns a generative model for papers and the citation context. Then we use KL-divergence to calculate a score for each paper (by a similar equation to Equation (7)). For both RBM and RBM-CS, we set the number of topic as T = 200 and the number of recommended references as the average number of the data set, i.e. K = 7 for NIPS and K = 11 for Citeseer. The weights were updated using a learning rate of 0.01/batch-size, momentum of 0.9, and a weight decay of 0.001. 4.2 Experimental Results Estimated Topics Table 1 shows two example topics discovered by RBM-CS from the NIPS data. We can see that our model can capture the topic distribution very well. Performance of Citation recommendation Table 2 shows the result of citation rec￾ommendation. We see that our proposed model clearly outperforms the two baseline models. The language model suffers from the fact that it is based on only keyword matching. The RBM uses a hidden topic layer to alleviate the problem. However, it is aimed at optimize p(w), which might be inappropriate for citation recommendation. In addition, RBM cannot capture the dependencies between paper contents and citation relationships. Our proposed RBM-CS can be advantageous to optimize p(l|w) directly and to model the dependencies between paper contents and citation relationships. We can also see from Table 2 that the recommendation performance is much better on the Citeseer data than that on the NIPS data. This means that on the sparse data, the recommendation tasks would be more difficult. How to improve the recommendation performance on the sparse data is also one of our ongoing work