UCount: a Community-Driven Approach for Measuring Scientific Reputation

Assessment of research has proven to be a complex issue. Scientific excellence has different meaning across communities [Lamont2009], and it is not easy to determine what are the more relevant characteristics that influence it. The most common approach to simplify the evaluation process is the use of bibliometric indicators such as the h-index [Hirsch2005]. Scientific impact, however, is a multi-dimensional construct that can not be adequately measured by any single indicator [Bollen2009] [Lehmann2006] [Martin1996] and the advent of the web era brought a whole new set of challenges that support this argument. While nothing is essentially wrong with using bibliometrics, we have come to understand that is not enough to measure the full scope of scientific impact [Adler 2009]. The social web opened new ways for disseminating scientific knowledge, thus suggesting that the social dimension can be an important component of scientific reputation. The altmetrics initiative is on the right track to lead this development by analyzing research impact in terms of web and social attention alternative metrics. One use of social metrics is as an extension to bibliometric impact measures (e.g. article download statistics, number of bookmarks on Connotea or Mendeley). However, social impact can involve many other factors, including participation in events or communities or providing comments or reviews of others’ work. Our intuition is that both social and bibliometric dimensions are important and we propose to combine them for measuring impact. In order to determine which metrics are important for which research community, we propose to analyse the subjective opinions of researchers, and then to combine both social and traditional metrics to approximate this subjective information. Therefore, we propose UCount, a community-driven approach for evaluation of researchers that will give the power of evaluation to the community and to researchers themselves. UCount will do this by enabling researchers to provide feedback, keep track and affect their own community- driven reputation.