RUSA MARS Local Systems and Services (Machine-Assisted Reference Section) Committee
RUSA MARS LSS Committee Online Open Meeting on "The Learning Commons and the Academic Library: Using Emerging Technologies and New Partnerships to Build Student Success"by Chanitra Bishop on Mon, Mar 18, 2013 at 01:38 pm
Many new library spaces, such as learning or information commons, lend themselves to providing both library and non-library services to students, a situation that provides its own challenges and opportunities. If you have such a space or are planning to develop one, the RUSA MARS LSS Committee invites you to join us for a free online open meeting on the Learning Commons on Monday, March 25 from 1-2:30PM EST.
While the event is free we ask that you register online at: http://learningcommons.eventbrite.com. After you register, you will receive a link to join the meeting. Space is limited so be sure to register in advance.
What: RUSA MARS LSS Committee Online Open Meeting on "The Learning Commons and the Academic Library: Using Emerging Technologies and New Partnerships to Build Student Success"
When: Monday, March 25 1-2:30PM EST
Where: Online - You will receive a link after you register
Title: The Learning Commons and the Academic Library: Using Emerging Technologies and New Partnerships to Build Student Success.
Description: Has your library partnered with academic support services to offer a “learning commons” in the library, or are you discussing/planning such a partnership? If so, please join us for a follow-up to the discussion forum “The Learning Commons and the Academic Library: Using Emerging Technologies and New Partnerships to Build Student Success” held at Midwinter 2012. Attend the online open meeting and learn how these libraries have developed partnerships and designed spaces to support student learning.
Ameet Doshi - Head, User Experience Department Georgia Tech University
Doshi will describe the role of the student advisory board in the design of the Library and the learning commons at Georgia Tech, known as the Clough Commons. The Clough Commons provides group study spaces, academics support services and a wide variety of technology for students to use.
Michelle Paon - Reference & Instruction Librarian, Dalhouise University
Marc Comeau - IT Manager, Dalhouise University
Eleven stakeholders currently share space in Dalhousie University's Killam Learning Commons, with services ranging from GIS and statistical computing consultations to traditional research assistance and the IT help desk. Paon and Comeau will identify the service areas located in the facility as well as touch on the innovative technologies that havebeen deployed in the commons.
Lori Mestre - Head Undergraduate Library, University of Illinois, Urbana-Champaign
Mestre will share UIUC's process for creating the initial plans for the Learning Commons and how student need for multimodal projects has resulted in the Learning Commons becoming a Media Commons (including floor layout, services, furniture, technology, class support, use patterns, partnerships, loanable technology acquisition). She will discuss how UIUC incorporated technology and designed the layout based on assessment of student needs.
Notes from the MARS Local Systems & Services Midwinter 2012 Discussion Forum:
The Learning Commons and the Academic Library: Using Emerging Technologies and New Partnerships to Build Student Success
Sunday, January 22, 2012, 4-5:30pm
featured speaker: Carrie Donovan, Head, Teaching & Learning, Indiana University Libraries
handout: Carrie's handout, with bibliography, is attached to this online document
In her opening remarks, Carrie talked about the idea of form following function in the Learning Commons and reminded attendees that “libraries put the ‘fun’ in function.” It doesn’t matter what the space is called (some libraries are using terms like “sandbox” and “information arcade”). Rather, Carrie encouraged us to think first about what kinds of user behavior we wanted to see happen and then create a learning commons that makes students think, “This is a place I can learn” (rather than what one student at Indiana University thought when, while looking at that library’s information commons, he asked her, “Is that a bank?”).
Over forty attendees got into four groups to discuss the following themes:
- Form & Function
- Direction & Discovery
- Collaboration & Motivation
- Collections & Connections
After the small group discussions, the large group reconnected to share notable points from each small group’s discussion.
- Multimedia labs w/video editing software
- Digital text analysis programs
- Scanning equipment for the public
- iPads and mobile technology (checkoutable)
- staffing models that mirror consultant approach
- 20 hour/week student workers in the library (for tuition credits)
- Committed, ongoing student advisory board is a good way to get student input (making sure the students have a means of communicating directly/regularly to library dean). Ameet Doshi from Georgia Tech (email@example.com) can answer any questions about student advisory groups.
- Different, evolving roles for librarians
- Peer model of research assistance (librarians as trainers, rather than direct service providers)
- IT, writing, research – staff working together
- Shared administrative structures would eliminate silos
- Info Commons staff are “learning support professionals” EVEN LIBRARIANS
- Shared service points – referrals, one-desk, multi-desk solution?
- Technology! iPads, mobile reference with iPhones, media production room, online tutoring, software availability in special rooms
- Signage – must be the best! But it’s difficult to agree upon.
- More general signage/labels (e.g. “ask” “ask here”) is best.
- Different types of signs – touchscreen, digital, etc. This allows students to get help based on the type of question/need they have.
- Mall signage – “YOU ARE HERE”
- Students don’t read signs, but perhaps they would pay attention to mall signage.
- Service points: what is better - Many desks? Shared service points? At least make sure the desks are nearby.
- With revised library space, parners will emerge. Libraries should maintain focus on mission and student learning, while embracing new partners/models.
- Partners include: writing centers, digital humanities labs, teaching centers, adaptive technology centers, language labs, etc.
- Newly designed space cannot change culture, but it can demonstrate expected behavior
- Information commons are not intended to be quiet, but sometimes people expect them to be.
- Recommended furniture = mediascape pods from Steelcase\
Collaboration & Motivation – additional notes
David Nutty (University of Southern Maine) and Judy Solberg (Seattle University), among others, made valuable contributions to the “Collaboration and Motivation” discussion group.
Nutty shared the exciting fact that his university had just opened its learning commons the Monday prior to the conference. Tutoring, writing center assistance, group study room management and reference were being offered at a combined service desk.
They were offering virtual tutoring using AskOnline and web-based group study room scheduling. They also maintain a small group of computers with enhanced, specialized software in addition to a larger number of computers with the typical suite of software.
Judy Solberg informed the group that Seattle University’s academic support services and public services librarians had been working together for four or five years and hold regular meetings. They maintain one service desk for all services, though it is not regularly staffed by a librarian – instead, the reference librarian on duty carries an iPhone and can be called to come out and meet with students wherever their questions take them. Other than a media support specialist, they did not have funding for additional staffing when they opened their learning commons, but they did have money to hire eight graduate students. Solberg suggested that new professional roles might be developing for reference librarians because our current model – an expert available at all times to provide one-on-one assistance – is perhaps not sustainable. Instead, the learning commons model might cause us to move more towards providing peer services for all areas (writing assistance, tutoring, and research assistance) with full-time librarians and learning support professionals supervising, training, and managing these services.
The conversation among this group revealed real curiosity about service models (is it better to have one service point or multiple service points?), staffing (how do we break down silos and territorial tendencies and provide for co-management and cross-communication between information technology, academic support and reference services?), and the use of technology in the learning commons. Lots of the participants’ libraries offered iPad and laptop checkout, media production spaces, and online question tracking, though virtual learning and tutoring services and spaces that complement the developing physical spaces of our learning commonses are still, perhaps, to come. The group also returned to the idea that it is necessary to begin with a student perspective.
ALA RUSA MARS Local Systems & Services Committee
Meeting, ALA Annual 2011
June 26, 2011
Members present: Amy Fry (chair), Geoffrey Morse, Carole Pilkinton
Review of membership
The committee welcomed its newest member, Geoffrey Morse.
tabled for now
Midwinter discussion forum
The group returned to our idea of hosting a discussion forum at Midwinter 2012 about collaborations between Learning Commons and libraries. Amy agreed to write up a proposal and present it at the MARS Executive Committee meeting on June 27. We also discussed following up the forum with a webinar in February as well as recording the forum itself. Members agreed to see what options were available for software to use to host the webinar.
2012 Annual program
Amy reviewed the status of our program proposal for Annual 2012. It has been submitted to the RUSA Executive Committee, which will vote at conference.
Notes from MW11 Discussion Forum: "Designing Around a Single Search: How Discovery Layers Are Changing Library Websites"by Amy Fry on Wed, Jan 26, 2011 at 12:48 pm
Designing Around a Single Search: How Discovery Layers Are Changing Library Websites
ALA 2011 Midwinter Meeting, San Diego
Hosted by MARS Local Systems & Services
Sunday, January 9th, from 10:30 a.m. to 12:00 noon
Discussion Forum Notes
MARS Local Systems & Services committee chair Amy Fry welcomed the audience of around 130 attendees and introduced the three panelists.
Josh Boyer, North Carolina State University, Summon and locally-developed QuickSearch
Beth Bernhardt, University of North Carolina, Greensboro, EBSCO Discovery Service
Johan Oberg, Macalester College, WorldCat Local
Josh Boyer from NCSU (http://www.lib.ncsu.edu/) led off the discussion.
The Libraries of North Carolina State University were awarded a prestigious ALA OITP “Cutting Edge Technology Service Award” following a website redesign project between January and August of 2010. That redesign incorporated a locally-created tool called QuickSearch, which displays article, catalog, library website, and journal list results on a single search result page, along with a “Best Bets” category of locally-relevant resources. Article results are compiled from multiple library databases using ProQuest Summon.
This search and display interface features tabbed searching, which NCSU decided upon based on their hesitation to simply place a single search box on their website. A review of ARL libraries also indicated tabbed searching as a popular method. NCSU wondered whether searchers would choose a tab or use the default and whether a tab, if chosen, would be the most appropriate one.
User testing was completed in a “guerilla” fashion, grabbing students in the library for quick testing on the spot, rather than using a more formal approach. Over half of testers selected a tab (which surprised the researchers), and of those that did select a tab, ¾ selected the most appropriate one for the search at hand. However, users rarely switched tabs from the results page. Searches that tended to work in testing included finding a book by title, finding a recent issue of a journal, and finding the cost for making copies at the library. Problem areas included distinguishing between journals and articles and databases, finding journal articles by subject, scrolling on the results page to all results, and spelling inconsistencies.
Usage tracking of the site finds that 43% of use is given to article exploration, 36% to catalog records, 2% to library website research, and 8% to the Best Bets category. Between August 2009 and August 2010, the library website saw an increase of 17% in traffic overall.
Future usability testing will separate testing for students and faculty. The faculty testing will be more open ended interviewing as opposed to testing a list of tasks.
Beth Bernhardt from UNC Greensboro (http://library.uncg.edu/) was the next speaker.
Historically, students at UNC Greensboro have avoided using the library catalog because of its unappealing interface; students also expressed a preference for a ‘search’ option over use of the A-Z database list. EBSCO Discovery Service provides a much improved look and feel for the catalog. The decision to implement EBSCO Discovery Service was made by a Systems Advisory Committee made up of library public services, technical services, and systems members. The committee trialed both EBSCO Discovery and ProQuest Summon, but chose EBSCO in part because North Carolina has numerous EBSCO databases already in use.
The EBSCO Discovery Service search box is included on the library home page, database page, and on the library’s mobile site, as well as within numerous LibGuides. This takes advantage of the service’s ability to create topic-focused search boxes, highlighting business resources, for example, or health information.
A literature review substantiated the move to a discovery system. Discovery systems tend to be popular with all groups of library users (though graduate students tend to like them less than undergrads and users in general like them more than librarians do); they tend to appear to be easy to use; users like the single search box; and discovery systems seem to increase resource usage.
At UNC Greensboro, about 50% of librarians report using EBSCO Discovery Service with students at the reference desk. The library has seen an increase in the use of collections and databases, and is also tracking an increase in patron-driven acquisitions.
Johan Oberg from Macalester College (http://www.macalester.edu/library/) was the third speaker.
Macalester College uses a shared catalog with other institutions in a consortium, but determined that WorldCat Local would provide a better search experience for their students. WorldCat Local offers a simple search interface that provides access to a broad research experience with content beyond what is owned by a single library. From the WorldCat Local search box, a student may select between local, state, and world content.
Unlike NCSU, Macalester made a decision to reject the tabbed interface model in favor of a single search box, prominently placed on the library home page. This prominent placement has resulted in increased usage (usage far outpacing that at a research university 3x the size of Macalester that places the search box on a secondary page). Placed at the top of the page, the simple search box allows for the devotion of home page real estate to library events and other items of interest.
The library is still unsure what to call WorldCat Local, though students refer to it simply as “Library Search” and have quickly adopted it. The word ‘catalog’ is seldom used by anyone. Faculty have also responded positively, citing the time-saving advantages of the service. Usage has increased 30% since initial implementation in 2008, as has interlibrary loan and overall circulation. The takeaway lesson was ‘Be bold!’ and put the new search front and center!
In the future, Macalester looks toward implementation of a faceted database page, increasing mobile access, highlighting digital content, exploring web scale management, and taking advantage of the WorldCat API to reach students where they are.
A question and answer period ensued.
Q: How did you decide to “brand” your discovery service? Using the vendor name vs. not?
A: Boyer (NCSU) Discovery systems tend not to require vendor branding and instead allow an institution to label by material type – for example “books” or “articles.”
Q: Any thoughts for tracking and recording the “value” of discovery systems beyond full text article views?
A: Bernhardt (UNCG) and Boyer (NCSU) Systems allow for tracking, but comparing data from database to database is like opening a can of worms. More work needs to be done in this area.
Q: Speak to databases that are not included in the indexes of discovery systems? What happens to those, which are often specialized and topic-focused?
A: Bernhardt, Boyer, and Oberg (Macalester) Faculty likely already are aware of specialized databases in their fields. Use LibGuides to drive students to specific databases. Discovery systems perhaps offer most value to undergrads, looking for x number of articles on a topic. Perhaps use discovery system as aggregator of content from several large databases, then trying to push students to database pages. Perhaps use discovery system as starting place – the default place to begin research.
Q: What enhancements are you looking forward to from discovery system vendors?
A: Oberg and Boyer Web-scale for OCLC, APIs to use on home page for better display and visual browsing of material. Search is currently in a primitive state when compared to Google – Google knows that someone searching for a zip code likely wants a map, or that someone searching for “Huckleberry Finn” probably wants the book rather than a book review – discovery search not at that level yet. Search relevancy rankings are not good enough – must use result facets to take advantage of databases and discovery systems – do students use those facets???
Q: What are “Do”s and “Don’t”s for discovery systems and website design?
A: Oberg, Boyer, and Bernhardt Placement on the site is key to driving usage. Get as many people involved with the process to improve “buy-in.” When considering customization of discovery services, be aware of technical time requirements. Oberg suggested looking at the University of Michigan site as an example of using the Summon API with other tools (eg Drupal) to build an interface that is completely customized. Programming time for these custom looks can be significant.
Discovery systems promise to simplify search by providing access to multiple resources through a single index and interface. This promise has numerous implications for library website design. If you’ve grappled with questions of how to integrate a discovery system (such as Summon, Encore, EBSCO Discovery Service, Primo, or Primo Central) into your library’s website, wondered about pitfalls in that process, or find yourself curious about what may be the next new design standard for information discovery and delivery, join us at Midwinter for an exciting and informative discussion forum.
Designing Around a Single Search: How Discovery Layers are Changing Library Websites
Sunday, January 9th, from 10:30 a.m. to 12:00 noon
Location: SDCC Room 10
Josh Boyer and Susan Teague Rector, who will discuss North Carolina State University’s experience with Summon and the locally-developed QuickSearch
Beth Bernhardt, who will discuss the University of North Carolina Greensboro’s experience with EBSCO Discovery Service
Johan Oberg, who will discuss Macalester College’s experience with WorldCat Local
This forum is presented by the RUSA/MARS Local Systems & Services committee. Please join us!
ALA RUSA MAR Local Systems & Services Committee
Meeting Minutes, Midwinter 2011
January 9, 2011
- Review of the minutes of the June 2010 meeting
- These were approved.
- Last-minute things for 1/9/11 discussion forum
- Chanitra will be timekeeper and Matt will take notes at today’s forum
- Possible questions to get discussion going:
- Tips on managing stakeholders’ views in website changes so there is no uproar
- How to present databases not included in discovery system
- Do you still have federated search? Do you think you’ll continue to have it?
- Impact on usage statistics for databases? Have you identified under-used dbs and seen any rise?
- What advice would you give other places about re-designing home page when you put in a discovery layer?
- Managing authentication
- Feedback from users on the site
- So much content – but not all. How do you clarify what’s missing and what’s in the layer/what’s not in it. Is this different than how you did this with a federated search system?
- We discussed whether committee members are required to rotate off after their second term, and thought that perhaps it was not required by MARS Executive Committee. The current chair would like to continue for one more year as chair, especially to continue the planning process for the Annual 2012 Program Proposal, and other members present did not express any opposition; she will check with Dianna McKellar (MARS Vice Chair). We also discussed recruiting new people; Carole said that since she will definitely not be continuing past her current term (which ends at Annual 2011) she did have someone she could recommend for membership on the committee.
- 2012 Program Proposal
- publishers’ power to kill the value of the discovery system as a tool (if they refuse to allow access to their content)effect on interlibrary loan/resource sharing, possible impact on e-books
- We discussed possible themes for panelists on the 2012 Program if the proposal is accepted. Concerns related to discovery systems include:
- We discussed recording the speakers and putting the recording on a website together with their slides.
- We decided to ask Marshall Breeding to talk about discovery systems’ promise/big picture and to try to recruit another big name to talk about the issues as co-keynotes, but to let both Marshall and the other keynote speaker talk to each other and decide between themselves the issues they’d like to address; also so they know they’re not covering the same ground.
- Still need to extend a formal invitation to Mark Dehmlow.
- Carole suggested Peter Morville as the other co-keynote (University of Michigan, author of Ambient Findability) and the others liked her suggestion. She said she would follow up with people she knows at Michigan.
The committee also discussed proposing a discussion forum for Midwinter 2012, something not related to discovery systems. Ideas proposed included:
- library guides (LibGuides, Library A La Carte, statewide systems)
- working within Course Management Systems (Blackboard)
- getting students to use them
- usability testing on them
- with portals (like the U of M’s)
- How do you make people aware of a subject guide?
- working with faculty
- library instruction
- learning commons
- not as just physical space but reference desk and innovative learning environment. Partnerships with tutoring center, writing center. Libraries’ role. Technology and teaching centers. OIT. CTL. Is this where reference is going? Or can this be a place to revitalize reference? Ask-an-expert? Local services.
- Have a focus on technology here. Leveraging partnerships on campus, like with technology. What is successful? What can we be aiming at for reference?
- Partner with another committee? Virtual reference? Education, training & support.
The committee liked the learning commons idea and decided to move forward with it.
Submitted by Amy Fry, January 18, 2011.
- Marshall Breeding, who provided an "Overview of Discovery Systems."
- Scott R. Anderson, who shared Millersville University Ganser Library's experience with EBSCO's Discovery Service.
- Dale Poulter, who shared Vanderbilt University's experience with DiscoverLibrary (Primo Central).
- Ellen Safley, who shared the University of Texas at Dallas McDermott Library's DISCOVER search (Innovative Interfaces Encore).
Our panelists highlighted the experiences of libraries that have implemented "next generation discovery tools" that provide access to library collections from a single search box. Panelists' comments focused on user response and subsequent assessment of local implementations. A summary of the Midwinter 2010 discussion forum on the same topic can be read at
Panelist: Marshall Breeding, Vanderbilt University, Overview of Discovery Systems
Breeding began with an overview of the current discovery landscape by asking and answering the following two questions: from what has this technology evolved, and where is it heading? A typical library website may currently offer a menu of disjointed things requiring students/researchers to attempt different strategies and tactics to complete research. The job of the library, Breeding asserted, is to make the experience of accessing both content and services a more unified one for users.
Discovery services move toward accomplishing this mission by broadening the scope of a traditional ILS to include library content from multiple collections, including print, local digital, and vendor-supplied. Discovery services also offer a modernized interface, incorporating features familiar to web users, such as a single search box (and unambiguous starting point), intuitive search assistance (did you mean?, type-ahead suggestions), relevancy ranking, faceted search refinements, visual display features (book covers, user reviews), and community recommendations. In contrast, the typical library website at this time features a disjointed approach to information and service delivery, where silos of content prevail and distinctions between content are confusing to users.
Breeding stressed the point that non-library sites are often more unified, using Amazon as an example of a complex site which can be used successfully without instruction or apparent confusion.
While current trends have focused on discovery interface improvement of products that are mostly locally installed, the new phase of development will move to web-scale discovery, looking to integrate access to local content together with article content managed in a remote, single infrastructure. Examples of this trend include Serials Solutions' Summon, ExLibris' Primo Central, the EBSCO Discovery Service, WorldCat Local, and III Encore Synergy.
The near future of discovery services lies in continued focus on deep indexing (searching beyond the MARC record to the full text of ALL included content) and incorporation of social discovery features such as user tags and recommendations. Breeding suggested that the ideal interface will highlight a single point of entry along with very sophisticated tools to allow the user to drill down to more nuanced results. A final point was recognition of the huge business partnership and technology challenges entailed in pulling such a broad scope of content together in a centralized infrastructure for improved deep indexing and access. This achievement is well beyond what single libraries or even most consortia can accomplish on their own.
Panelist: Scott R. Anderson, Millersville University, EBSCO Discovery Service
As EBSCO Discovery Service’s (EDS) first customer, Millersville softly rolled out EDS to the Millersville community in February 2010. In so doing, they combined access to multiple content sources (multiple EBSCO databases, local collections housed in ContentDM, etc.) and library services (chat widgets, LibGuides widgets) into one place. EDS has increased the use of topic-specific databases. Because Millersville faces a two year period of being a virtual library due to a building project, Anderson needed to find a solution that could be quickly implemented without significant management overhead. He needed a solution that could integrate and present as much library content as possible, along with library services and contacts. Millersville conducted user testing and found that students actually seem to prefer the Advanced Search screen and found the interface to be user-friendly on the whole.
Panelist: Dale Poulter, Vanderbilt University, Primo Central
Vanderbilt’s instance of Primo Central has only been live for a short time, but has successfully provided access to licensed content from EBSCO, LexisNexis, and other providers, alongside open access repository content into a single search and retrieve interface. Students at Vanderbilt appreciate the “Google-like” search box, and database usage seems to be increasing. Poulter reported that the new deep indexing allows users to discover content they hadn’t found in the past. Examples include items from the Global Music Archive, Naxos recordings (through an ‘include results from music’ feature), ArtSTOR, and the Vanderbilt Television News Archive. In many cases the new aggregated discovery systems reveal content previously ‘buried’ in individual database containers. The move to Primo Central as an integrated discovery system saw a performance boost when compared to the earlier federated search model. Primo’s e-shelf content folder option is particularly popular.
Panelist: Ellen Safley, University of Texas at Dallas, Innovative Interfaces Encore
UT Dallas’ implementation of Encore sprung largely from dissatisfaction with its ILS. Usability testing on that ILS uncovered multiple points of failure; Encore was adopted to provide a more intuitive interface. The Encore discovery system currently lives right next to the traditional ILS search box on UT Dallas’ home page and supports about 30% of library searches. Informal testing has garnered mostly positive feedback.
MARS Local Systems & Services held a Discussion Forum on Sunday, January 17, 2010 at the Midwinter Meeting.
(1) Marshall Breeding, Director for Innovative Technology and Research,Vanderbilt University, "Overview of Discovery Systems."
(2) Cody Hanson, Technology Library, University of Minnesota, "User testing and feedback on Primo at the University of Minnesota."
(3) Frances McNamara, Director, Integrated Library Systems and Administrative and Desktop Systems,University of Chicago, "LENS Discovery System, based on AquaBrowser."
(4) Barbara DeFelice, Director Digital Resources Program, Dartmouth "Summon @ Dartmouth College: the User View."
Try the discovery tools first hand:
- Dartmouth College's implementation of Summon
- University of Chicago's Lens
- University of Minnesota's MNCATPlus
Major points made by the panelists:
First, Marshall Breeding (Director for Innovative Technology and Research, Vanderbilt University) gave an "Overview of Discovery Systems." He discussed the “crowded landscape of information providers and pointed out that libraries should look to e-commerce sites for usability practices: Amazon.com, for example, is self-explanatory.
Breeding also discussed the shift from print to electronic in libraries. We are delivering different kinds of content (electronic) and serving a different kind of user (millennials), which must change the way that we expect to deliver content to users. Our websites now tend to be menus of disjointed silos: books, articles, local digital collections – all searched separately. Libraries also tend to lack a unified web presence within their websites – in other sites, the search function is part of the website, not something separate from the website.
Users have been saying they want a "single point of entry" to library content, while librarians continue to say we don’t want this. That’s part of our problem – we are reluctant to embrace these new systems.
Library websites are evolving and must move towards modernization, which includes features like type-ahead, "do you mean,"relevance ranking, one search box, visual components, and enriched content. This is the target, and such a target goes beyond the content that is in the OPAC. Therefore it must be decoupled from the OPAC. This is the future.
Breeding also talked about the possibilities afforded by social discovery: tags, ratings, etc., are all fodder for the infosearch process. Many systems no longer search merely thin metadata (like what is in the OPAC) but the whole item (the full text). Thus the item itself is its indexing, which can also inform search results.
Technologically, he says, the problem of the single point of entry has been solved by searching pre-populated indexes instead of using federated search technology. Now, he says, the problem is going to be convincing the information providers to let their information be used this way.
Next, Cody Hanson, Technology Librarian at the University of Minnesota, talked about "Primo User Testing: Feedback and Metrics." He began by giving an introduction to Primo.
The University of Minnesota was a Primo development partner with Ex Libris, which included a usability testing component. They conducted tests in 2007-2008 in their usability lab, which is set up to capture screen activity, do eye tracking, film the user, record sound, etc.
The tests involved users thinking aloud while completing scenarios and tasks, followed by a debriefing in which users also filled out a desirability matrix. Their goals were to see if users could be aware of and use the variety of information in Primo, find its availability, use facets, and more. The tasks included things like:
a) find this article from this citation
b) find books about this
c) save the items you find
d) find this book and tag it
e) find this book and read its user review
They found that users had trouble finding the tabs and using the advanced search (though Hanson didn’t think this was a Primo problem).
Overall, University of Minnesota users liked Primo. On the desirability matrix, which included negative and positive adjectives, of which users were asked to circle 5, the most common chosen were organized, clean and straightforward.
Their use stats reflect most students finding something useful when they do a search, because most click through to view more about something in their results lists. They have also learned that most searches are for books, not many users have been adding tags, and not many are saving items. The advanced features in Primo were not heavily used at first.
Hanson also discussed user comments, which can sometimes be very negative, because feedback is rarely motivated by a positive experience. Staff, especially, had negative comments, but it’s important to remember that staff are not our audience.
Frances McNamara (Director, Integrated Library Systems and Administrative and Desktop Systems at the University of Chicago) then discussed the "LENS Discovery System, based on AquaBrowser." She gave us an overview of LENS, pointing out how it contains more information than had been in the OPAC, like tables of contents, covers, and Google book links.
They tested the facets in 2006 on 12 PhD candidates (a primary audience for them) which demonstrated users’ difficulties with the old catalog, and in 2008 did usability testing on LENS followed by other specific pathfinding studies.
In the 2008 study they asked 8 graduate students to do 10 tasks. The study compared the old catalog with the new (LENS). They found that the advanced search needed to be easier to find, and that users wanted clarification on what LENS was searching (transparency). Users weren’t 100% confident when using LENS that they were finding everything and some mistrusted the relevancy ranking. McNamara also said they discovered it was important to pay attention to what people liked about the old system as they continued to design the new.
One of their later studies was on finding journal articles; users found this was difficult. Another further study was on music researchers. At the University of Chicago, the developers use personae when talking about the design of their system in order to stay focused on their different user types.
Finally, Barbara DeFelice (Director, Digital Resources Program at Dartmouth College) talked about "Summon @ Dartmouth College: the User View."
They implemented Summon (Serials Solutions) at Dartmouth in April of 2009. They still have the catalog, federated search, and other silos, in addition to their Summon Beta.
They did usability testing on the beta before making it public by doing six one-on-one sessions with undergraduates and two focus groups. They used TechSmith UserVue software for their one-on-one sessions. They did not use tasks, but asked the students to work on real assignments while using the system.
They found that the students liked the look of the system. Students also liked limiting to known databases/sources – DeFelice pointed out that they learned from this that students bring prior knowledge to a new system and want a way to work that knowledge into their use of it. Users also have a low tolerance for not getting smoothly to full text: they have found that their users hate the openURL results page.
Dartmouth also talked to faculty one-on-one. They found out their faculty really like Google Scholar and use it, and also that they recognize the continued value of known subject indexes. They felt the new system was good for finding known items.
During some time for Q & A, DeFelice and Hanson both recommended getting as many people to observe your tests as you can because it can be very eye-opening for the observers, especially if those people are library staff. It’s a good way to create buy-in for a new system.
Also during the Q & A, one of our panelists (Breeding?) made the point that no ILS vendor is updating its OPAC: the OPAC is dead, he said. Precision will be the thing to work on next in these systems. Also, keeping both (a discovery system alongside an OPAC and separate article databases) won’t be feasible for very long.
Notes taken by committee member Amy Fry during the panelists discussion.
Table Discussion Comments: