International Evaluation Procedure for Supergrants

Abstract

Sergey Ivanets, director of the department of international integration at the Ministry of Education and Science, talks about the specifics of the evaluation procedures for inviting leading scientists (Decree of the Government of the Russian Federation dated April 9, 2010, N 220: "On Measures for Attracting Leading Scientists into Russian Higher Professional Educational Institutions")

Full Text

Yes, of course. Of the 900 selected experts, more than half are foreigners recommended to us by the American Association of Universities, the Association of European Universities, the Association of Universities and Institutes of Higher Education of Germany, the Association of Universities of Flanders, the International Bureau of the Federal Ministry of Education of the Federal Republic of Germany, the American Charitable Foundation for Informational Support of Education and Science, the United States National Science Foundation, and others. Russian experts were mostly provided by the Russian Foundation for Basic Research. Each application is sent to two Russian experts and two foreign experts. If the opinions of the experts turn out to be completely opposite and cannot be reconciled, then that application is assessed by additional experts. The experts are paid, as that is the accepted practice, but I cannot say that this is a major expenditure. H as the evaluation procedure been launched? What is the timeline for this procedure? We plan on completing the expert evaluation procedure by the end of October. Most of the applications have already been sent out. Some of the experts have asked for more time, because of their personal schedules. But judging by the current progress, most experts need only a few days to complete the task. Therefore, by the beginning of November we will have our winners, who will be selected by the grant council. What is the role of formal criteria, including those that characterize publication activity and number of citations? Formal parameters that characterize scientific expertise were required in the competition documentation. These parameters are very important for the evaluation procedure. In general, the expert evaluates each application for a group of issues. These can be divided into three parts: one characterizes the leading scientist, his standing in the scientific community; the second characterizes the application project, its quality, and the predicted results of the project; the third characterizes the higher education institution and its potential and ability to undertake and complete the project. Of course, both formal and informal criteria are important for evaluating all of these issues. However, there are criteria that cannot be disputed, such as citation indexes. Will all the winners in natural and precise sciences have a high Hirsch index and citation index? Here I must explain the role of the governmental grant council, which will be making the final decision on the winners. The results of R&D competitions all over the world are decided based on the results of expert evaluation; however, the final decision is made by special committees or the heads of the organization which provides the funding. It was precisely for these reasons that our council included renowned scientists with the highest bibliometrical indices. They posses wide knowledge, which allows them to asses any application, and they can take various issues into account. However, the results of the expert evaluation are the basis for their decisions. Were applications concerning public and humanitarian fields also sent to international experts? Is there any difference in the treatment of the various scientific fields? There is none. There are no quotas and no discrimination concerning the themes of the applications. But we can be sure that priority fields of research will yield much stronger applications. Won’t this mean that some lines of scientific research will be left with no winners? The council has selected 21 lines of research. There were no limitations for applications. It is natural that they will be unevenly distributed between the various scientific fields. However, we do not think that any quotas are required. Using them would make inviting the best leading scientists pointless. Sergey Ivanets On the other hand, it is no good if all of the winners belong to the same scientific field. We would like to see some variety. But resolving this issue is not up to the ministry; it will be decided by the council, which was created exactly for this reason. Among the applicants, some of which are renowned and highly cited scientists, there are some very elderly people. Is age a factor in evaluating the application? There are no formal limits on age. We have no reason to incorporate them. Of course we would like to see more leading scientists who are young and talented people, since the task of creating a scientific group requires an energetic approach. However, I think many would agree with me that some scientists who are no longer young will give any young researcher a run for their money. Of course the experts will factor age into their evaluation. Their task is to evaluate the expertise of the applicant, taking activity and age into account. The experts can asses this by analyzing publications in recent years. If a 90-year-old person publishes several articles every year, then there are no reasons to doubt his or her working ability. There is a certain group of applicants who are heads and deputy heads of major institutes and higher educational institutions. Sometimes they apply together with an educational institution in an adjacent city, let us say from Novosibirsk to Tomsk or from Moscow to Tver. What can you say about the participation of such people in the competition? We have a formal requirement: the scientist must apply into a different Federation constituent entity. This requirement must be fulfilled. However there is another requirement: starting in 2011 the scientist must spend at least 4 months in the year working in the higher educational institution. In other words, the work of the leading scientist cannot be limited to “Skype supervision.” If you are a director or deputy director of an organization, then in order to leave your place of employment for several months you must resolve formal issues. We have a number of such applications. I think the council members (if they decide to grant such applications) will require a formal confirmation, which will acknowledge the possibility that the head of the organization will be absent from his post for the allotted time. This will be strictly enforced even before awarding the grants. How many “stars” will there be among the winners? Are there enough real world-class leading scientists among the applicants? My answer is this: the average Hirsch index is not that high, it is 17.5. However, more than 70 applicants have an index in excess of 30. In other words, a considerable number of applicants are indeed world-class researchers. The average citation index of the 10 best papers of each applicant is 559. However, more than a 100 people have a citation index in excess of 1,000 and several tens of people have more than 2,000. Thus, we have a considerable number of world-renowned and acknowledged scientists among our applicants. In other words, we have our choice of specialists.