document 6865 Neural correlates 4. Exploring complex 353537. Computational 101.Netwo of decision variables in networks. Strogatz Nature roles for dopamine in imple building blocks arietal cortex Platt and 410, 8 (2001 behavioural control. Mon- of complex networks Glimcher Nature 400.15 tague et al. Nature 431 Milo et al. Science 298 14(2004) 324(2002) decision making compl assigned decisionmaking complexity euroscience combinatorics complex networks reinforcement learning complexity CiteULike monkey graph review motifs networks Idiosyncratic: network Idiosyncratic: brain small world action selection, attention, sub graph pattern social networks behavior. behavioral con- decision, trol, cognitive control electrophysiology, eye- learning, network, rein- datamining, data min- ovements, limitations, Idiosyncratic: 2001, adap- forcementlearning, re- onkeys, neuroecono tive systems, bistability ward. td model is,modularity, net lICs, neurons, neuro- pled oscilla ce,other, ppc, qua aphs, exploring, science,sysbio, web reinforcementlearning network biological, neu- characterization. web rons, strogatz grap Tags cortex complex networks complex networks assigned decision networks network euroscience motifs synchronization review gene visual graph reward Table 6. Tags assigned by CiteULike taggers and Maui to four sample documents these taggers ranges from 11.5% to 56%, with an Brooks and Montanez(2006) extract terms average of 35%. This places it only 2.6 percent- with the highest TFXIDF values as tags for posts age points behind the average performance of the on technorati. com. They do not report precision best CiteULike taggers. In fact, it outperforms 17 and recall values for their system, but our re- of them(cf. Table 1) implementation resulted in precision of 16.8% and recall of 17. 3% for the top five assigned 4.5 Examples tags, compared to those agreed to by at least two Table 6 compares Maui with some of CiteULike users on 180 documents. Adding CiteULike's best human taggers on four ran- eight additional features and combining them domly chosen test documents. Boldface in the using machine learning gives a clear improve taggers row indicates a tag that has been chosen ment-Maui achieves 45.7% and 48. 7% preci by at least two other human taggers; the remain- sion and recall respectively ng tags have been chosen by just one human Mishne(2006)uses TFXIDF-weighted terms Boldface in Maui's row shows tags that match as full-text queries to retrieve posts similar to the human tags. For each document Maui extracts one being analyzed. Tags assigned to these posts several tags assigned by at least two humans. are analyzed to retrieve the best ones using clus- The other tags it chooses are generally chosen by tering and heuristic ranking, tags assigned by the at least one human tagger, and even if not, they given user receive extra weight. Mishne per- are still related to the main theme of the docu- forms manual evaluation on 30 short articles and reports precision and recall for the top ten tags of 38% and 47% respectively. We matched Maui's 5 Discussion and related work op ten terms to all tags assigned to 180 docu ments automatically and obtained precision and It is possible to indirectly compare the results of recall of 44% and 29% respectively(We believe several previously published automatic tagging that manual rather than automatic evaluation approaches with Maui's. For each paper, we would be likely to give a far more favorable as compute Maui's results in settings closest to the sessment of our system. reported ones Chirita et al.(2007) gs. Given a web they first retrievethese taggers ranges from 11.5% to 56%, with an average of 35%. This places it only 2.6 percent￾age points behind the average performance of the best CiteULike taggers. In fact, it outperforms 17 of them (cf. Table 1). 4.5 Examples Table 6 compares Maui with some of CiteULike’s best human taggers on four ran￾domly chosen test documents. Boldface in the taggers’ row indicates a tag that has been chosen by at least two other human taggers; the remain￾ing tags have been chosen by just one human. Boldface in Maui’s row shows tags that match human tags. For each document Maui extracts several tags assigned by at least two humans. The other tags it chooses are generally chosen by at least one human tagger, and even if not, they are still related to the main theme of the docu￾ment. 5 Discussion and related work It is possible to indirectly compare the results of several previously published automatic tagging approaches with Maui’s. For each paper, we compute Maui’s results in settings closest to the reported ones. Brooks and Montanez (2006) extract terms with the highest TF×IDF values as tags for posts on technorati.com. They do not report precision and recall values for their system, but our re￾implementation resulted in precision of 16.8% and recall of 17.3% for the top five assigned tags, compared to those agreed to by at least two CiteULike users on 180 documents. Adding eight additional features and combining them using machine learning gives a clear improve￾ment—Maui achieves 45.7% and 48.7% preci￾sion and recall respectively. Mishne (2006) uses TF×IDF-weighted terms as full-text queries to retrieve posts similar to the one being analyzed. Tags assigned to these posts are analyzed to retrieve the best ones using clus￾tering and heuristic ranking; tags assigned by the given user receive extra weight. Mishne per￾forms manual evaluation on 30 short articles and reports precision and recall for the top ten tags of 38% and 47% respectively. We matched Maui’s top ten terms to all tags assigned to 180 docu￾ments automatically and obtained precision and recall of 44% and 29% respectively. (We believe that manual rather than automatic evaluation would be likely to give a far more favorable as￾sessment of our system.) Chirita et al. (2007) aim to extract personal￾ized tags. Given a web page, they first retrieve Document 86865. Neural correlates of decision variables in parietal cortex. Platt and Glimcher. Nature 400,15 (1999) 44. Exploring complex networks. Strogatz. Nature 410, 8 (2001) 353537. Computational roles for dopamine in behavioural control. Mon￾tague et al. Nature 431, 14 (2004) 101. Network motifs: simple building blocks of complex networks. Milo et al. Science 298, 824 (2002) Tags assigned by CiteULike taggers decision making decisionmaking lip monkey neurophysiology reward Idiosyncratic: brain, choice, cortex, decision, electrophysiology, eye￾movements, limitations, monkeys, neuroecono￾mics, neurons, neuro￾science, other, ppc, quals, reinforcementlearning complex complexity complex networks graph networks review small world social networks survey Idiosyncratic: 2001, adap￾tive systems, bistability, coupled oscillator, graph mining, graphs, explorig, network biological, neu￾rons, strogatz dopamine neuroscience reinforcement learning review Idiosyncratic: action selection, attention, behavior, behavioral con￾trol, cognitive control, learning, network, rein￾forcementlearning, re￾ward, td model applied math combinatorics complexity motifs network original sub graph pattern Idiosyncratic: 2002, datamining, data min￾ing, graphs, link analy￾sis, modularity, net paper, patterns, protein, science, sysbio, web characterization, web graph Tags assigned by Maui cortex decision lip monkey visual complex networks networks review synchronization graph dopamine learning neuroscience review reward complex networks network motifs gene complex Table 6. Tags assigned by CiteULike taggers and Maui to four sample documents 1325