Research Assessment and Metrics Discussion Group

 View Only
last person joined: 4 days ago 

Charge: To provide a forum for discussion of trends and developments in the field of research metrics and scholarly impact, and the many ways academic librarians can support their faculty, administrators and students to understand, measure and extend the contributions of their research to their fields and the world beyond academia.

Community members can post as a new Discussion or email

Before you post: please note job postings are prohibited on ALA Connect. Please see the Code of Conduct for more information.
  • 1.  Anyone here attend this meeting?

    Posted Apr 03, 2024 11:47 AM
    This was a meeting I hadn't seen, and wondering if any on the list attended and what some takeaways might've been for librarians working in this space. I'm also wondering if anyone has done anything with COARA on their campuses.
    Laura Mullen

    Dear INORMS REG Supporters,

    The Responsible Research Evaluation Forum 2024 focusing specifically on the SCOPE framework took place three weeks ago. See below for a brief report on the event!


    A logo with orange circles and a black background  Description automatically generatedOn March 13-14, 2024, more than 70 individuals from across the United States and Canada convened in Albuquerque, New Mexico for the first U.S.-based Click or tap if you trust this link." data-linkindex="0" data-loopstyle="linkonly">workshop using the SCOPE framework for Responsible Research Evaluation. The two-day, in-person event was supported through a $149,535 grant from the Click or tap if you trust this link." data-linkindex="1" data-loopstyle="linkonly">Institute of Museum and Library Services. Additional support was provided by Click or tap if you trust this link." data-linkindex="2" data-loopstyle="linkonly">OpenAlex. The goal of the forum was to create a community of practice of North American-based practitioners and librarians that are trained in the use and application of not only responsible research assessment practices-but also in the SCOPE framework, which was originally developed by the INORMS Research Evaluation Group (REG). 

    The grant and forum was led by Dr. Baron Wolf from the University of Kentucky and supported through the efforts of Dr. Elizabeth (Lizzie) Gadd, Loughborough University, Dr. Erica Conte, Unity Health Toronto, Ms. Rachel Miles from Virgina Tech University, Ms. Tanja Strom from Oslo Metropolitan University, and Ms. Laura Beaupre from the University of Guelph.  

    Attendees included individuals from more than 60 institutions and organizations and included individuals working as research impact librarians, sponsored project managers, assessment and planning staff, and research development staff. The keynote speaker for the event was Dr. Nicky Agate, Associate Dean for Academic Engagement at Carnegie Mellon University. 

    Attendees spent the two days networking together and learning about the SCOPE framework and using it in practical applications. Workshop participants spend time using the framework in three specific uses cases: 1) How should promotion and tenure be evaluated; 2) Evaluation of organizational success: What does it mean to be a high-quality institution?; and 3) How do we evaluate research excellence for a department within arts & humanities.

    Findings of each of these topics will be developed into white papers and published over the next few months. In addition, session materials will be archived in an open-access repository and follow-up online webinars will be held to create future communities of practice. To learn more about the forum, please visit: Click or tap if you trust this link." data-linkindex="3">

        A group of people sitting at tables in a room        A person standing at a podium with a large projector screen  Description automatically generated


    Research Evaluation Group is regularly invited to facilitate workshops on the SCOPE process to support planning and conducting an evaluation responsibly. If you'd like to talk to the SCOPE team about bespoke workshops for your evaluation setting, please contact us at INORMS@ARMA.AC.UK

    Best regards,

    Laura Himanen
    Project Manager
    +358 50 3310103

    CSC – For Brilliant Minds |

    Laura Bowering Mullen (she/her)

    Behavioral Sciences Librarian; Open Access Specialist

    Rutgers University-New Brunswick

    Library of Science and Medicine

    165 Bevier Road, Piscataway, NJ  08854-8009

  • 2.  RE: Anyone here attend this meeting?

    Posted Apr 03, 2024 08:33 PM

    I ran into this one kind of by accident, but when I asked about going, I learned that another of our Associate Deans was giving the keynote.  So, for me it was a completely missed opportunity. 


    Matthew Marsteller
    Principal Librarian, Engineering & Science
    Carnegie Mellon University

  • 3.  RE: Anyone here attend this meeting?

    Posted Apr 07, 2024 04:17 PM

    Hi Matt and Laura,

    Perhaps I can respond as both the Chair of the INORMS (International Network of Research Management Societies) Research Evaluation Group and one of the Vice-Chairs of CoARA (Coalition on Advancing Research Assessment).  

    One of the INORMS REG members, Dr Baron Wolf of the University of Kentucky, led on applying for funding from the IMLS to run a two-day US-based workshop on the INORMS SCOPE framework for responsible research evaluation. SCOPE was developed by practitioners for practitioners who want a better way of designing responsible evaluations that don't fall foul of the poor evaluation practice that is the focus of principles such as DORA and the CoARA Agreement on Reforming Research Assessment. Some of the workshop attendees have kindly agreed to write papers outlining how they've applied their learnings in various evaluation settings, so that's something to look out for. We are also running another NSF-funded SCOPE workshop later this year - more details to follow. You can follow the work of INORMS REG by subscribing to our discussion list here:

    With regards to CoARA, the coalition has been running for almost 18 months now and has over 600 signatories. Signatories are invited to develop their own Action Plans to address assessment issues of concern to them, and can propose and join Working Groups to consider how we might together improve various aspects of research assessment. Our first US-based member was the Center for Open Science, and we are in regular discussions with other US-based organisations with similar ambitions. Happy to answer any questions list members have about CoARA!

    All best


    Elizabeth Gadd
    Research Policy Manager
    Loughborough University