Skip to Main Content

Scholarly Publishing and Communication

Measuring Research Impact

Introduction

This guide provides an introduction to the tools and indicators used for measuring research impact. Some benefits of research impact, and the role it plays in the research evaluation process, include:

  • Discovering how research is being used and built upon by others
  • Adding additional value to grant, job, and tenure and promotion applications
  • Identifying potential collaborators in your field
  • Comparing research output between groups for strategic planning
  • Analyzing trends and shifts in research over time

Library Support

HSLS provides educational and advisory services for University of Pittsburgh's health sciences community to support research impact projects and inquires. Here are some examples of how the library can help:

  • Provide training on tools that utilize research indicators and publication reports
  • Suggest appropriate indicators for your project
  • Identify impactful journals for publication
  • Suggest ways to increase the discoverability and visibility of your research

Bibliometrics

What are Bibliometrics?

Bibliometrics consists of different quantitative indicators that analyze the impact of scholarly works. Most of these indicators are citation-based with a focus on journal articles. This page covers some of the most commonly used indicators for authors, articles, and journals.

Author Indicators: h-index

The h-index measures the productivity and citation impact of an author. Its calculation is based on the number of papers (h) that have received at least h citations. Example: an author's h-index is 20. This means that they've published 20 papers that have been cited at least 20 times each.

Limitations include:

  • No context for citations or publication types.
  • First and corresponding authorship is not considered.
  • Early career researchers will likely have a low h-index.
  • Cannot be used to compare authors from different disciplines.

Look up the h-index in:

Article Indicators

Citation counts are the main way of determining the impact of an individual article. In theory, highly cited articles are interpreted as impactful publications. Keep in mind that it takes time to accrue citations and citation patterns are different across disciplines. Google Scholar and Web of Science all keep track of citation counts and are displayed on the search results page.

Other citation-based article indicators include:

Journal Indicators

Journal indicators, which are based on citation patterns of their articles, can be helpful for finding publications that will give your research greater visibility. However, it's important to understand that these indicators are only one part of journal evaluation process. Use of these indicators for measuring the quality of a journal, as well as the quality of individual authors who publish in them, has been argued against. Read the DORA declaration for more information.

Resource: Essential Science Indicators

Essential Science Indicators from Clarivate provides citation analysis on authors, journals, institutions, and countries. This can be helpful for adding additional support to CVs, biosketches, grant applications, and more to show:

  • publications in "Top Papers" (top 1% of papers for the past ten years)
  • publications in the top 10%, 25%, .01% (can be broken down by year range)
  • top 1% of authors and institutions
  • top percentage of countries and journals in a 10-year period

To learn more about ESI, see Clarivate's training resources page or contact the library.

Altmetrics

What are altmetrics?

Altmetrics, short for alternative metrics, are indicators that reflect methods of online interaction and engagement with scholarship. They can compliment bibliometrics by providing a qualitative insight to research impact.

While altmetrics can be used to discover the impact of articles, it can also be used to track other forms of digital scholarship, such as software, datasets, code, and more. Remember to cite data so it can be tracked.

Here are a few other examples of altmetric indicators:

  • Shares on social media (Twitter, Facebook, etc.)
  • Comments on websites (YouTube, Reddit, etc.)
  • References on blogs and wikis
  • Mentions on major news outlets
  • Downloads and views from databases

Altmetrics Tools

Limitations and Responsible Use

Keep in mind the following limitations when using indicators for research evaluation:

  • Indicators only provide a glimpse at impact since no database covers all time periods, publications, and citation data. Results will vary across databases (i.e., the h-index in Web of Science might be different from Google Scholar).
  • No single indicator can tell us the entire research impact story. All indicators have strengths and weaknesses, so it's important to use multiple, relevant indicators as appropriate.
  • Not all indicators can be used to compare different subject fields and groups of researchers. Indicators that cannot be used to compare different groups include citation count, h-index, and journal impact factor.
  • Bibliometric indicators only provide a number, which makes it difficult to determine context and reasons why works were cited. Using altmetrics with bibliometrics can help give a more balanced analysis.

For more information on using indicators responsibly, see the Leiden Manifesto, DORA, and The Metric Tide.