Y. Xu et al/ Expert Systems with Applications 37(2010)6948-6956 6949 assignment of candidate reviewers for each proposal, using either the aforementioned decision process, including the matching de- one or both of these approaches(Papagelis, Plexousakis, Niko- gree calculation model, the proposal grouping model and assignment model to assign reviewers to proposal groups, etc. According to our literature review, the majority of existing ap- The remainder of this paper is organized as follows. The re- proaches either from information retrieval stream or from optimi- search background is introduced in Section 2. Section 3 proposes ation stream are based on the same strategy which seeks an approach for solving the assignment problem. Section 4 pre earch appropriate reviewers for each individual proposal (as sents a prototype system based on the proposed approach. Section shown in Part A of Fig. 1). This strategy is in an attempt to ensure 5 validates the proposed approach and discusses the potential that every reviewer has sufficient expertise to evaluate the merits application in government funding agencies. Section 6 concludes of the proposals assigned to him/her. This strategy has several lim- the paper. itations: first, selecting appropriate reviewers for every individual proposal is time-consuming. For large funding agencies, the num ber of proposals received and the number of qualified reviewers 2. Backgro can be very large. Besides, the review process must be usually com- pleted with a tight deadline(Wang et al, 2008). Thus, partitioning 2.1. Background of NSFC proposals into groups and searching reviewers for each proposal up can be an efficient alternative. The time spent on reviewer one of the largest and most reputable research funding agen- assignment can be reduced by assigning reviewers to proposal cies in China, NSFC (National Natural Science Foundation of China) groups instead of individual proposals. Second, assigning individ- aims to fund research projects that have great potential of ual proposals to reviewers can hardly meet the specific require- tific and social impacts. NSFC has one general office, five bureaus, nents of the funding agencies, e.g. balancing the backgrounds of and seven scientific departments(Tian, Ma, Liang, Kwok, &Liu, plicants and affiliations of proposals assigned to reviewers. The 2005). The general office and bure mainly in charge of pol limitations of this strategy call for another approach in this con- icy making, operational management, administrative work and re- text. In this research, a new assignment strategy is proposed which lated affaires. The scientific departments are the key parts of NSFC. is based on grouping proposals first and then search qualified and they are responsible for the selection and management of re- reviewers for each proposal group(shown in Fig. 1B). To the best search projects. The scientific departments include Department of of our knowledge, it has never been addressed in the literature. Mathematicaland Physical Sciences, Department of Chemical Sciences, Furthermore, previous research usually used exact algorithms Department of Life Sciences, Department of Earth Sciences, Depart to solve the reviewer assignment problem. But it may be difficult ment of Engineering and Materials Sciences, Department of Informa- to solve the assignment problem using exact algorithms when tion Sciences and Department of Management Sciences. These seven the numbers of both proposals and reviewers are very large. Genet- scientific departments are further divided into divisions focusing various assignment and combinational applications where it dem- Management Sciences is further divided into three divisions: Man- onstrated satisfactory performances(Deep Das, 2008: Harper, de agement Science and Engineering, Macro Management and Policy Senna, vieira, Shahani, 2005: Huang Lim, 2006). While genetic and Business Administration algorithms in their elementary forms can be designed to tackle a here are various categories of programs in NSFC. The general real-world problem, the incorporation of domain knowledge and Program(including Project for young scientists'fund and Project local search techniques may improve the computational perfor- for developing regions)is the major one. The number of proposals mance significantly submitted to NSFC for the General Program increased dramati- This paper proposes an integrated approach assisting in assign- cally from 23, 636 in year 2001 to 73, 785 in year 2008(see ing proposals to reviewers where proposals need to be partitioned Fig. 2). Note that the average funded rate( funded over submit- into groups. The proposed approach facilitates the reviewer assign- ted )in 2005-2008 is only about 18%. In order to support and fi- nent through the following steps: identify valid proposals and nance the most promising proposals a limited budget, a viewers, classify proposals and reviewers according to their dis- fair and unbiased project selection pi ciplines, partition proposals into groups and assign reviewers to one of the most important tasks is to approprlate review proposal groups. Knowledge rules and models are used to support ers to proposals Proposals Reviewers Proposals Grouping Reviewers (A)Individual proposal assignment (B)Proposal assignment based on grouping them firstassignment of candidate reviewers for each proposal, using either one or both of these approaches (Papagelis, Plexousakis, & Niko￾laou, 2005; Sun et al., 2008). According to our literature review, the majority of existing ap￾proaches either from information retrieval stream or from optimi￾zation stream are based on the same strategy which seeks to search appropriate reviewers for each individual proposal (as shown in Part A of Fig. 1). This strategy is in an attempt to ensure that every reviewer has sufficient expertise to evaluate the merits of the proposals assigned to him/her. This strategy has several lim￾itations: first, selecting appropriate reviewers for every individual proposal is time-consuming. For large funding agencies, the num￾ber of proposals received and the number of qualified reviewers can be very large. Besides, the review process must be usually com￾pleted with a tight deadline (Wang et al., 2008). Thus, partitioning proposals into groups and searching reviewers for each proposal group can be an efficient alternative. The time spent on reviewer assignment can be reduced by assigning reviewers to proposal groups instead of individual proposals. Second, assigning individ￾ual proposals to reviewers can hardly meet the specific require￾ments of the funding agencies, e.g. balancing the backgrounds of applicants and affiliations of proposals assigned to reviewers. The limitations of this strategy call for another approach in this con￾text. In this research, a new assignment strategy is proposed which is based on grouping proposals first and then search qualified reviewers for each proposal group (shown in Fig. 1B). To the best of our knowledge, it has never been addressed in the literature. Furthermore, previous research usually used exact algorithms to solve the reviewer assignment problem. But it may be difficult to solve the assignment problem using exact algorithms when the numbers of both proposals and reviewers are very large. Genet￾ic algorithm (GA) has been widely used as a search algorithm in various assignment and combinational applications where it dem￾onstrated satisfactory performances (Deep & Das, 2008; Harper, de Senna, Vieira, & Shahani, 2005; Huang & Lim, 2006). While genetic algorithms in their elementary forms can be designed to tackle a real-world problem, the incorporation of domain knowledge and local search techniques may improve the computational perfor￾mance significantly. This paper proposes an integrated approach assisting in assign￾ing proposals to reviewers where proposals need to be partitioned into groups. The proposed approach facilitates the reviewer assign￾ment through the following steps: identify valid proposals and reviewers, classify proposals and reviewers according to their dis￾ciplines, partition proposals into groups and assign reviewers to proposal groups. Knowledge rules and models are used to support the aforementioned decision process, including the matching de￾gree calculation model, the proposal grouping model and the assignment model to assign reviewers to proposal groups, etc. The remainder of this paper is organized as follows. The re￾search background is introduced in Section 2. Section 3 proposes an approach for solving the assignment problem. Section 4 pre￾sents a prototype system based on the proposed approach. Section 5 validates the proposed approach and discusses the potential application in government funding agencies. Section 6 concludes the paper. 2. Background 2.1. Background of NSFC As one of the largest and most reputable research funding agen￾cies in China, NSFC (National Natural Science Foundation of China) aims to fund research projects that have great potential of scien￾tific and social impacts. NSFC has one general office, five bureaus, and seven scientific departments (Tian, Ma, Liang, Kwok, & Liu, 2005). The general office and bureaus are mainly in charge of pol￾icy making, operational management, administrative work and re￾lated affaires. The scientific departments are the key parts of NSFC, and they are responsible for the selection and management of re￾search projects. The scientific departments include Department of Mathematical and Physical Sciences, Department of Chemical Sciences, Department of Life Sciences, Department of Earth Sciences, Depart￾ment of Engineering and Materials Sciences, Department of Informa￾tion Sciences and Department of Management Sciences. These seven scientific departments are further divided into divisions focusing on more specific research areas. For example, the Department of Management Sciences is further divided into three divisions: Man￾agement Science and Engineering, Macro Management and Policy and Business Administration. There are various categories of programs in NSFC. The General Program (including Project for young scientists’ fund and Project for developing regions) is the major one. The number of proposals submitted to NSFC for the General Program increased dramati￾cally from 23,636 in year 2001 to 73,785 in year 2008 (see Fig. 2). Note that the average funded rate (funded over submit￾ted) in 2005–2008 is only about 18%. In order to support and fi- nance the most promising proposals within a limited budget, a fair and unbiased project selection process is necessary where one of the most important tasks is to assign appropriate review￾ers to proposals. (A) Individual proposal assignment (B) Proposal assignment based on grouping them first Proposals Reviewers Proposals’ Grouping Reviewers Fig. 1. Two strategies for assigning proposals. Y. Xu et al. / Expert Systems with Applications 37 (2010) 6948–6956 6949