Latent Wishart Processes for Relational Kernel Learning Wu-Jun Li Zhihua Zhang Dit-Yan Yeung Dept.of Comp.Sci.and Eng. College of Comp.Sci.and Tech. Dept.of Comp.Sci.and Eng. Hong Kong Univ.of Sci.and Tech. Zhejiang University Hong Kong Univ.of Sci.and Tech. Hong Kong,China Zhejiang 310027,China Hong Kong,China liwujun@cse.ust.hk zhzhang@cs.zju.edu.cn dyyeung@cse.ust.hk Abstract can impair the performance significantly.Hence,ker- nel learning (Lanckriet et al.,2004;Zhang et al.,2006), One main concern towards kernel classifiers which tries to find a good kernel matrix for the train- is on their sensitivity to the choice of kernel ing data,is very important for kernel-based classifier function or kernel matrix which characterizes design. the similarity between instances.Many real- In many real-world applications,relationships or world data,such as web pages and protein- "links"between (some)instances may also be avail- protein interaction data,are relational in na- able in the data in addition to the input attributes. ture in the sense that different instances are Data of this sort,referred to as relational data(Getoor correlated (linked)with each other.The re- and Taskar,2007),can be found in such diverse appli- lational information available in such data cation areas as web mining,social network analysis, often provides strong hints on the correla- bioinformatics,marketing,and so on.In relational tion (or similarity)between instances.In this data,the attributes of connected (linked)instances paper,we propose a novel relational kernel are often correlated and the class label of one instance learning model based on latent Wishart pro- may have an influence on that of a linked instance. cesses(LWP)to learn the kernel function for This means that the relationships (or links)between relational data.This is done by seamlessly in- instances are very informative for instance classifica- tegrating the relational information and the tion (Getoor and Taskar,2007),sometimes even much input attributes into the kernel learning pro- more informative than input attributes.For exam- cess.Through extensive experiments on real- ple,two hyperlinked web pages are very likely to be world applications,we demonstrate that our related to the same topic,even when their attributes LWP model can give very promising perfor- may look quite different when represented as bags of mance in practice. words.In biology,interacting proteins are more likely to have the same biological function than those with- out interaction.In marketing,knowing one's shopping 1 Introduction habit will provide useful information about his/her close friends'shopping inclinations.Hence,in such Kernel methods,such as support vector machines data,relational information often provides very strong (SVM)and Gaussian processes(GP)(Rasmussen and hints to refine the correlation (or similarity)between Williams,2006),have been widely used in many ap- instances.This motivates our work on relational ker- plications giving very promising performance.In ker- nel learning (RKL),which refers to learning a kernel nel methods,the similarity between instances is rep- matrix (or a kernel function)for relational data by resented by a kernel function defined over the input incorporating relational information into the learning attributes.In general,the choice of an appropriate process. kernel function and its corresponding parameters is Thanks to its promising potential in many applica- difficult in practice.Poorly chosen kernel functions tion areas,relational learning (Getoor and Taskar, 2007),which tries to model relational data,has be- Appearing in Proceedings of the 12th International Confe- come a hot topic in the machine learning community. rence on Artificial Intelligence and Statistics (AISTATS) 2009,Clearwater Beach,Florida,USA.Volume 5 of JMLR: Many methods have been proposed over the past few W&CP 5.Copyright 2009 by the authors. years.For example,probabilistic relational modelsLatent Wishart Processes for Relational Kernel Learning Wu-Jun Li Dept. of Comp. Sci. and Eng. Hong Kong Univ. of Sci. and Tech. Hong Kong, China liwujun@cse.ust.hk Zhihua Zhang College of Comp. Sci. and Tech. Zhejiang University Zhejiang 310027, China zhzhang@cs.zju.edu.cn Dit-Yan Yeung Dept. of Comp. Sci. and Eng. Hong Kong Univ. of Sci. and Tech. Hong Kong, China dyyeung@cse.ust.hk Abstract One main concern towards kernel classifiers is on their sensitivity to the choice of kernel function or kernel matrix which characterizes the similarity between instances. Many real￾world data, such as web pages and protein￾protein interaction data, are relational in na￾ture in the sense that different instances are correlated (linked) with each other. The re￾lational information available in such data often provides strong hints on the correla￾tion (or similarity) between instances. In this paper, we propose a novel relational kernel learning model based on latent Wishart pro￾cesses (LWP) to learn the kernel function for relational data. This is done by seamlessly in￾tegrating the relational information and the input attributes into the kernel learning pro￾cess. Through extensive experiments on real￾world applications, we demonstrate that our LWP model can give very promising perfor￾mance in practice. 1 Introduction Kernel methods, such as support vector machines (SVM) and Gaussian processes (GP) (Rasmussen and Williams, 2006), have been widely used in many ap￾plications giving very promising performance. In ker￾nel methods, the similarity between instances is rep￾resented by a kernel function defined over the input attributes. In general, the choice of an appropriate kernel function and its corresponding parameters is difficult in practice. Poorly chosen kernel functions Appearing in Proceedings of the 12th International Confe￾rence on Artificial Intelligence and Statistics (AISTATS) 2009, Clearwater Beach, Florida, USA. Volume 5 of JMLR: W&CP 5. Copyright 2009 by the authors. can impair the performance significantly. Hence, ker￾nel learning (Lanckriet et al., 2004; Zhang et al., 2006), which tries to find a good kernel matrix for the train￾ing data, is very important for kernel-based classifier design. In many real-world applications, relationships or “links” between (some) instances may also be avail￾able in the data in addition to the input attributes. Data of this sort, referred to as relational data (Getoor and Taskar, 2007), can be found in such diverse appli￾cation areas as web mining, social network analysis, bioinformatics, marketing, and so on. In relational data, the attributes of connected (linked) instances are often correlated and the class label of one instance may have an influence on that of a linked instance. This means that the relationships (or links) between instances are very informative for instance classifica￾tion (Getoor and Taskar, 2007), sometimes even much more informative than input attributes. For exam￾ple, two hyperlinked web pages are very likely to be related to the same topic, even when their attributes may look quite different when represented as bags of words. In biology, interacting proteins are more likely to have the same biological function than those with￾out interaction. In marketing, knowing one’s shopping habit will provide useful information about his/her close friends’ shopping inclinations. Hence, in such data, relational information often provides very strong hints to refine the correlation (or similarity) between instances. This motivates our work on relational ker￾nel learning (RKL), which refers to learning a kernel matrix (or a kernel function) for relational data by incorporating relational information into the learning process. Thanks to its promising potential in many applica￾tion areas, relational learning (Getoor and Taskar, 2007), which tries to model relational data, has be￾come a hot topic in the machine learning community. Many methods have been proposed over the past few years. For example, probabilistic relational models