Assessment for the Rest of Us: Informal Techniques You Can Use
Program conducted by LLAMA MAES at ALA Annual 2010.
More than 250 people packed Washington Convention Center 145A on Monday morning to listen to presenters from ten libraries describe informal assessment techniques they used to quickly evaluate and improve services throughout their libraries. Here are summaries of the ten presentations. The PowerPoint from the session and most of the speakers’ notes are available in the LLAMA MAES group area on ALA Connect http://connect.ala.org/node/107288 and on Slideshare.
See attached presentation and handouts.
Short summaries of the 10 presentations follow.
Karen Neurohr and Jennifer Paustenbaugh, Oklahoma State University presented a combination survey and focus-group-lunch technique aimed at student scholars. Scholars were identified by the university. Since the focus groups were informal they were termed "listening sessions." Together the survey and listening sessions served to uncover the most important topics for the library to address related to this population. The activity will be repeated next year.
Louise Lowe and Judith Brook, Mercer University presented on their use of product demonstrations involving students, and the impact of the student feedback on purchasing. One trial was for a coffee vendor, with taste tests being quite popular with the students (a log was kept next to the coffee for students to weigh in on their opinions). The pattern with products was that students preferred value over bells and whistles. Decisions were based on the feedback, and a To Do list was put on a poster in the library, with items checked off when completed.
Kornelia Tancheva, Cornell University, discussed their use of unobtrusive user observations. They performed both day and night observations, simply walking through and noting student activities. Since security walks through at night as well, they felt the students did not perceive they were being studied. She cautioned against over-reliance on observations. Their follow up will be to find out why students were doing what the staff observed them doing.
Sharon Naylor and Bruce Stoffel, Illinois State University, reported on their investigation of chat reference through focus groups. They identified a campus faculty member who became their advisor. He suggested that they continue doing focus groups until they heard nothing new, which is what they did. They found that their students saw web 2.0 modalities to be social, not for academic purposes. They also found that the students prefer a personal approach.
Jeff Gatten, California Institute of the Arts spoke on the use and value of poster surveys for a distinct population, one that is right-brained and less linear. The poster approach was interactive -- students grouped around the poster and answered three questions together. He also felt that the library got feedback they would not have in a traditional survey, since the students could report their immediate frustrations, which might fade. In fact they did try an online survey and got no participants.
Kirsten Kinsley and Rachel Besara, Florida State University, discussed what they learned from interviewing students in their natural environments on campus. As part of a project to renovate student study spaces in the library, librarians and staff asked students about their study habits. Most of the interviews were done by the library’s undergraduate student services staff, many of whom were only slightly older than the students. Interviews were recorded using a digital voice recorder, and were later transcribed. The results challenged many of their assumptions about students, for example: what “quiet” means to students, the nuances of group study, and when students like to study and for how long. The results identified a need for more night-time services, more software for curriculum support, and more technical support.
Wanda Dole and J.B. Hill, University of Arkansas at Little Rock (UALR), presented the results of a combined method assessment of use of UALR by community users. Quantitative measures included an examination of data from the library’s integrated library system, print management system, and donor list. Qualitative measures included data collected from a survey of community user needs and expectations. They found that community users broke out into two groups: borrowers and computer users. Borrowers were more likely to be UALR graduates, use the library for school work, and encourage others to use the library. Computer users were more likely to use the library for Internet access and to live nearby. While there have been few if any financial gifts from community users, the cost of providing access is also low. Despite having no borrower’s fees and no overdue fines, the library has lost only $4,000 in non-returned materials over a two-year period. Dole and Hill concluded that offering library services to unaffiliated users has been an important contribution to the local community, even though “good will” benefits are hard to quantify.
Ameet Doshi, Georgia Tech, described the “Flip the Library” assessment project. Georgia Tech Library’s 20-person student library advisory group was tasked with looking at four areas of the library (entrance, signage, study areas, and website) and suggesting improvements. The four groups took 15-30 minutes to record each area with a flip camera, and then met as a group to debrief. “Flip the Library” allowed library staff to view the library from the student perspective to help assess completed renovations, inform new signage and way-finding efforts, and identify a graffiti problem.
Lisa Horowitz, MIT, described a five-question Zoomerang survey used to determine if the benefits of the Humanities Library’s bookmobile service - used to promote recreational reading, DVDs and music CDs before long weekends and breaks – outweighed its costs. Staff members involved with the bookmobile were surveyed to understand better the impact of staffing the bookmobile on their regular workload. She concluded that the increased visibility for the library was worth the staff time invested and that the bookmobile was valuable to those who used it, as well as to staff. The informal assessment of 51 users provided the information needed to move forward with a decision to continue the service, but with fewer outings.
Kathy Ray, American University of Sharjah, observed undergraduate students in the library during the busiest hours to determine why students chose to use particular spaces in the library. Fifteen observations were conducted over a period of five weeks. She marked on a photocopy of the floor map where people were sitting and what they were doing. The study helped pinpoint one particularly noisy area where people liked to socialize in an area that was intended for quiet study. Furnishings in the area were reconfigured and a browsing area expanded into the space, resulting in a decrease in the noise level.