Skip to Main Content

Scholarly Publishing and Communication

Measuring Research Impact

Introduction

This guide provides an introduction to the tools and indicators used for measuring research impact. Some benefits of research impact, and the role it plays in the research evaluation process, include:

  • Discovering how research is being used and built upon by others
  • Adding additional value to grant, job, and tenure and promotion applications
  • Identifying potential collaborators in your field
  • Comparing research output between groups for strategic planning
  • Analyzing trends and shifts in research over time

Library Support

HSLS provides educational and advisory services for University of Pittsburgh's health sciences community to support research impact projects and inquires. Here are some examples of how the library can help:

  • Provide training on tools that utilize research indicators and publication reports
  • Suggest appropriate indicators for your project
  • Identify impactful journals for publication
  • Suggest ways to increase the discoverability and visibility of your research

Open Scholarship and Research Impact Challenge

Bibliometrics

What are Bibliometrics?

Bibliometrics consists of different quantitative indicators that analyze the impact of scholarly works. Most of these indicators are citation-based with a focus on journal articles. This page covers some of the most commonly used indicators for authors, articles, and journals.

Author Indicators: h-index

The h-index measures the productivity and citation impact of an author. Its calculation is based on the number of papers (h) that have received at least h citations. Example: an author's h-index is 20. This means that they've published 20 papers that have been cited at least 20 times each.

Limitations include:

  • No context for citations or publication types.
  • First and corresponding authorship is not considered.
  • Early career researchers will likely have a low h-index.
  • Cannot be used to compare authors from different disciplines.

Look up the h-index in:

Article Indicators

Citation counts are the main way of determining the impact of an individual article. In theory, highly cited articles are interpreted as impactful publications. Keep in mind that it takes time to accrue citations and citation patterns are different across disciplines. Google Scholar and Web of Science all keep track of citation counts and are displayed on the search results page.

Other citation-based article indicators include:

Journal Indicators

Journal indicators, which are based on citation patterns of their articles, can be helpful for finding publications that will give your research greater visibility. However, it's important to understand that these indicators are only one part of journal evaluation process. Use of these indicators for measuring the quality of a journal, as well as the quality of individual authors who publish in them, has been argued against. Read the DORA declaration for more information.

Advanced Resources

Essential Science Indicators (ESI)

ESI provides citation analysis on authors, journals, institutions, and countries. This can be helpful for adding additional support to CVs, biosketches, grant applications, and more to show:

  • publications in "Top Papers" (top 1% of papers for the past ten years)
  • publications in the top 10%, 25%, .01% (can be broken down by year range)
  • top 1% of authors and institutions
  • top percentage of countries and journals in a 10-year period

Publication data from the Web of Science is used on a rolling 10-year period.

InCites Benchmarking & Analytics

InCites helps analyze the research impact and productivity of authors, journals, institutions, and research areas. Use InCites to:

  • View citation metrics, such as times cited, h-index, and field-normalized metrics such as the Category Normalized Citation Impact
  • View other types of metrics such as authorship position and number of open access publications
  • Discover influential researchers, journals, and institutions in different subject areas
  • Find collaborators in your field
  • Generate visualizations and save them as custom reports

Publication data from the Web of Science is used from 1980 and is updated on a monthly basis.

Altmetrics

What are altmetrics?

Altmetrics are a collection of indicators that complement bibliometrics by capturing the online engagement with scholarship, such as (but not limited to) mentions from:

  • Social media (X, Facebook, Reddit, etc.)
  • News media
  • Policy documents and patents
  • Blogs
  • Wikipedia

Nearly any type of research output can be tracked with altmetrics, as long as it has a DOI number. Examples include:

  • Book chapters
  • Datasets
  • Preprints
  • Protocol papers
  • And more!

Why use altmetrics?

Altmetrics are a great way to see the immediate impact of your work, in comparison to other research metrics such as article citation count, which can take years to accrue meaningfully. They also complement bibliometrics because not only can you see the number of times someone has mentioned your work, you can also read the Tweet or news article to understand how your work was discussed. This might also provide a glimpse at the societal impact of your work.

Altmetric Resources

Limitations and Responsible Use

Keep in mind the following limitations when using indicators for research evaluation:

  • Indicators only provide a glimpse at impact since no database covers all time periods, publications, and citation data. Results will vary across databases (i.e., the h-index in Web of Science might be different from Google Scholar).
  • No single indicator can tell us the entire research impact story. All indicators have strengths and weaknesses, so it's important to use multiple, relevant indicators as appropriate.
  • Not all indicators can be used to compare different subject fields and groups of researchers. Indicators that cannot be used to compare different groups include citation count, h-index, and journal impact factor.
  • Bibliometric indicators only provide a number, which makes it difficult to determine context and reasons why works were cited. Using altmetrics with bibliometrics can help give a more balanced analysis.

For more information on using indicators responsibly, see the Leiden Manifesto, DORA, and The Metric Tide.