Go to:
Online Doc
Meeting Request
Ellen Keith's picture

Discussion Forum at ALA Annual 2012

ERUS Discussion Forum at ALA Annual
Saturday, June 23, 2012, 4-5:30

The How, What, and Why of Reference Evaluation and Assessment



  • ·         Defining WOREP
    • o   In-person only
    • o   Used to do results for you
  • ·         Defining READ
    • o   Incorporated into LibAnalytics
    • o   1-6 rating on difficulty of question


Question thrown out to attendees: Who is doing assessment?

  • ·         Reference
  • ·         Directors and coordinators
    • o   Project management for reference (and other services) to identify level of staff/help needed
    • o   Create assessment reports
  • ·         Assessment and Planning Director
  • ·         Assessment Team/Committee
  • ·         Assessment Librarian
  • ·         Director of Assessment on campus
    • o   Are student’s learning?/Learning outcomes


What tools are being used

  • ·         LibQual – Customer satisfaction
  • ·         Surveys
  • ·         Secret Shopper
    • o   Virtual and in-person
    • o   Used LIS students as the secret shoppers
    • o   All service points
    • o   Used a rubric
  • ·         Customer satisfaction surveys
  • ·         Google Analytics
  • ·         Desk tracker
    • o   User query week
    • o   More detailed level of questioning coded with the READ scale
  • ·         Assessing consultation services
    • o   Rubric of undergraduate papers
    • o   Students who had consultations versus those who did not
  • ·         Customer relationship management software
    • o   Homegrown
    • o   Determining how people are finding out about services
  • ·         Credit course
    • o   Interim tests and measures
    • o   Worked with student affairs office on campus
    • o   Measurement of GPA and class versus no class
  • ·         Reasons for assessment
    • o   Proving value outside of the library
    • o   Student outcomes


How are people paying for assessment?

  • ·         Strategic initiatives fund
    • o   Anyone can propose a plan
  • ·         Office of Institutional Assessment (there is an institutional commitment to assessment)


What do you want to find out?

  • ·         Students returning to the reference desk
    • o   Is this a positive or a system failure?
  • ·         Information literacy measurement
  • ·         Rethinking reference model
  • ·         Why are the patrons using the services?
    • o   Market segmentation (Faculty, Grad, Undergrad)
    • o   Non-users and how to assess this group


What decisions are being made with assessment data?

  • ·         Staffing
    • o   Identifying the busiest hours for staffing purposes
    • o   Reinforce the need for more staff
    • o   Ramping up of training because of a shift in available assistance
    • o   Budget cuts driving assessment to address staffing
    • o   Assessing how having librarians off the reference desk is working or not working
  • ·         Services
    • o   Assessment results used to create a whole new consultation system that are like office hours
    • o   Appointment services with librarians
    • o   Geographically located services (e.g. dorms)
  • ·         Time use
    • o   How long work takes, FTE needed
    • o   Better use of time for reference librarians
  • ·         Information given to deans, architects for space renovations


Statistics Fatigue

  • ·         How to avoid or correct
    • o   Sampling rather than everyday stat taking
    • o   Transparency, showing statistics results
    • o   Show the connection to decisions being made with the results
    • o   Capture qualitative with the quantitative