2015 Emerging Leaders Committee
Emerging Leaders Team B Project: Development of a Digital Collaboratory for the ACRL Science and Technology Sectionby Joy McGehee on Sun, Jun 22, 2014 at 08:28 pm
EL Team B Members
- Katelyn Angell (Long Island University)
- Jonathan DaSo (Oxford College of Emory University)
- Joy Keller (American Orthopaedic Foot and Ankle Society)
- Ashley Rayner (Chicago Public Library)
- Erica Tyler (Charlotte School of Law)
EL Team B ACRL Liaison
- Megan Griffin, Program Officer, ACRL, American Library Association
EL Team B STS Liaison
- Melissa Gold, Science Librarian, Millersville University, & ALA Science & Technology Section, Research Agenda Task Force Member
The ACRL/STS Research Agenda Task Force was created to help guide research within the ACRL/STS division. Many of the members of the division have to publish as part of their work, therefore providing research help and a way to collaborate on research would be a valuable contribution. The Research Agenda Task Force charged us with developing a virtual collaboratory that will allow exploration of the research agenda as well as provide members a place to find like-minded research partners. At ALA Midwinter 2014, we met with the Research Agenda Task Force for guidance as to what they wanted from their collaboratory. We talked about budget, how we planned to communicate with the Task Force, the expected timeline, and the functionality that the Task Force wanted for the collaboratory. After some discussion, we determined that we needed more clarification on a few points, and talked to Melissa Gold about some of our questions. She communicated that while there were a few important functions that the Task Force required, they were very open to any options we could find.
The Task Force has been open to any and all suggestions about the type of functionality that they want in their collaboratory. They were not sure what was available, leaving this analysis up to us. We were provided with a wish-list of nice to have options. We also surveyed previous literature to determine best practices for developing and operating collaboratories. We determined that we wanted to look for open-source solutions with very little setup work. This led us to create a few sets of criteria. The most general set was an objective look at the basic features: did the site have chat capabilities, for example. Then we evaluated the ease of use and support, which are more subjective for each user.
Based on what we determined to be the basic features needed in the collaboratory, we selected three different products to evaluate: Wiggio, Freedcamp, and Producteev.
To evaluate the products, we developed a list of yes or no statements focusing on three categories: ease of use, accessibility, and quality of user support. We then used a Likert scale to quantify our agreement with each statement. A score of 1 represents strong disagreement, while a score of 5 represents strong agreement.
Each member spent time learning and using the products, and then rated them based on their experiences. The data was then evaluated for inter-rater reliability using Cronbach's alpha and the Intraclass Correlation Coefficient to ensure that the group was in agreement with their evaluations and scoring.
Sample Evaluation Rubric
To see the Sample Evaluation Rubric used to evaluate the collaboratory products, please see attachment entitled rubric.jpg.
The following evaluation scale was used:
1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree
Inter-Rater Reliability (IRR) in Assessing Three Products
For IRR scores, please see the attachment entitled irr.jpg
The 5 raters on Team B scored >.7 on two popular measures of inter-rater reliability (IRR), Cronbach’s alpha and the Intraclass Correlation Coefficient (ICC), on all 3 products. These high levels of IRR show that the team demonstrated strong agreement in evaluation of the 3 collaboration tools. Our decisions were evidence-based and reached by consensus.
Wiggio Interface Snapshots
Please see the attachments entitled wiggio_interface.jpg and wiggio_whiteboard.jpg for a glimpse of the Wiggio interface visual appearance.
Please see the attachment entitled wiggio_features.jpg for a summary of key features of the collaboratory.
Conclusion and Recommendations
Due to its ease of use, excellent support, and the fact that it’s cost-free, we recommend the use of Wiggio as the digital collaboratory for STS members.
Wiggio provides multiple ways to communicate among group members, unlimited file storage, the ability to create private spaces within groups, and various task management tools to facilitate the progress of projects. Each member will be able to create an individual account, and Wiggio facilitates this process by allowing individuals to create accounts from their already existing Facebook accounts. We're confident that Wiggio will meet the needs of STS and facilitate research among the section.
Moving forward, we recommend that STS create a Wiggio group and begin advertising it to their members. The Wiggio group will require little management, but it may be best to assign a member to moderate it. Management of the Wiggio group will be left to the discretion of the Task Force.
Further Reading - Annotated Bibliography
If you are interested in further reading, please see the following annotated bibliography.
Arinze, B. (2012). E-research Collaboration in academia and industry. International Journal of e-Collaboration, 8(2), 1-13.
This paper provides a description of the incorporation of collaborative tools into e-research as well as a framework for mapping research needs to appropriate virtual software. Both the corporate world and academia are taken into account, as the former has used virtual collaboration software for a greater period of time. E-research tools are beneficial to academics in several major ways, such as the creation of new communication channels, greater opportunities for collaboration, and increased dissemination of research findings. Despite these tangible assets, users must learn new skill sets for successful collaboration in these virtual spaces. These skills include gaining a better understanding of “how each member uses language, categories that are important to them, heuristics they employ, and the forms of verbal and nonverbal shorthand and codes they use” (p. 6). Additionally, there are multiple factors researchers need to take into account when deciding which virtual solution to employ. These include internally vs. externally hosted products (cloud computing), voluntary vs. mandatory incorporation into existing research processes, and closed vs. open architecture (i.e. restricted or freely available information). The article features an intuitive table of five common scenarios and solutions to situations experienced by those seeking optimal e-collaboration software.
Barua, A., Chellappa, R., & Whinston, A.B. (1996). The design and development of Internet- and Intranet-based collaboratories. International Journal of Electronic Commerce, 1(2), 32-58.
Baru, Chellappa, and Whinston devised a definition of “collaboratory” still relevant nearly twenty years later: “An open electronic platform for individuals, SIG or organizations to efficiently exchange, disseminate, and create issues, ideas, and knowledge, and and to accomplish ad-hoc tasks toward the fulfillment of shared objectives” (p. 33). Arguing for a push from proprietary toward open Internet platforms, the authors highlight limits of working with vendors’ products and advocate the flexibility and independence of web-based collaboratories. Special interest groups (SIG) are an ideal candidate for collaboratories, as they necessitate a structured environment centered around a clearly delineated topic. In order to succeed, collaboratories must meet numerous needs of the users, offering features such as asynchronous and synchronous communication channels, archival storage, and creating links to external documents. A particularly significant need of the collaboratory is to support information scalability, with the goal of not overloading a user with needless information. For example, collaboratories should offer users some control over quantity and content of communications they receive. The authors conducted a survey of a collaboratory they had designed in which nearly 300 people gave their feedback. Notable responses are that the two most important features of a collaboratory are the capabilities to generate comments automatically and to link to external documents.
Doyle, B. (2008). Work in groups. EContent, 31(7), 27.
This brief but essential article provides an overview of numerous software options targeted at virtual collaborators. Nearly ten tools are named and described, and most are free of charge. In addition to longtime group tools such as Yahoo Groups, the author also includes cutting-edge products such as AirSet and Basecamp. The latter site, while not free, offers such useful collaboration features as to-do lists, file sharing, and forum-style messaging. A particularly desirable product named in the article is Wiggio, a free productivity tool which boasts innovative functionalities. These include the ability to record and send voice notes, schedule meetings, and create polls. While directed at college students tasked to work in groups, Wiggio could easily be translated to the collaboration tool of choice for academic researchers.
Lunsford, K.J., & Bruce, B.C. (2001). Collaboratories: Working together on the Web. Journal of Adolescent and Adult Literacy, 45(1), 52-58.
Written in the early 2000s, this article provides a well-researched and philosophical depiction of the virtual collaboratory at present, as well as a brief history of its trajectory. Citing economics as the need for greater collaboration among professionals in numerous fields, the authors describe a solution to this problem, the virtual collaboratory. The collaboratory’s participants are those who share “not just common goals but a common set of problems or issues” (p. 55). Six features of a collaboratory are delineated: shared inquiry, intentionality, active participation and contribution, access to shared resources, technologies, and boundary-crossings. The latter term signifies the ability of collaboratories to transcend physical boundaries by providing users with the time, space, and tools to interact. Collaboratories are ideal tools to both record and create scientific knowledge, allowing people from all over the world to meet, pool their resources, and engage in levels of communication only recently available to humankind.
Nicholas, D., & Rowlands, I. (2011). Social media use in the research workflow. Information Services & Use, 31(1/2), 61-83.
The authors of this study interviewed nearly 2000 researchers in an attempt to explore how their research process is influenced by social media tools. A control group of nearly 500 researchers who don’t use social media was included as a means of comparison. In terms of the former group, the three most popular social media tools were collaborative authoring, conferencing, and scheduling/meeting tools. These are not surprising findings given the prevalence of geographically dispersed scholarly collaborators. Earth and environmental researchers were the most likely to use social media, with social sciences in the middle and business/management the least likely. Social networks were seen as most useful not only to collaborate and share results but also to locate new research projects and potential partners. Participants identified their choice tools as Skype, Wikipedia, and Google Docs, despite the availability of a plethora of free, multifaceted platforms such as Freedcamp or Wiggio.
Oakleaf, M. (2009). Using rubrics to assess information literacy: An examination of methodology and interrater reliability. Journal of the American Society for Information Science & Technology, 60(5), 969-983.
This article discusses the viability of using rubrics to assess the information literacy skill set of college students. Rubrics measure performance levels of a specific activity or service, and can provide a reliable and valid method of evaluating student skill levels. There are many benefits of rubric development, such as increased collaboration between colleagues, direct feedback to students and/or other stakeholders, and an objective method of judgment. Although the author focuses on using a rubric to assess student learning these tools can also be used to evaluate services and resources. When multiple people are using a rubric it is very important for them to be consistent in their grading processes. This concept is called inter-rater reliability (IRR), and can be measured with various statistical techniques. If used properly, IRR offers, “a statistical estimate of the extent to which two or more judges are applying their ratings in a manner that is predictable and reliable” (Stemler, 2004)” (p. 970). This study provides an excellent outline and methodology for academic librarians interested in calculating their IRR on any project involving a grading rubric and multiple raters.
Rebmann, K. (2012). Connecting distance learning communities to research via virtual collaboratories: a case study from library and information science. Open Learning, 27(3), 273-282.
This article details the process of converting a distance LIS class into a virtual collaboratory. A main goal of the collaboratory in higher education is to “‘extend cooperative learning’ to the virtual spaces made possible by new information and communication technologies” (p. 274). According to the authors, collaboratories have only been appearing in LIS literature since 2007. These systems can unite geographically dispersed students and faculty, providing an effective medium for learning, researching, and educating. The author designed a study in which 58 students and employees of a LIS graduate program participated in a virtual collaboratory with a theme of writing and researching. The main topics of conversation were scholarly communication and academic advising. Participants were eager to communicate both synchronously and asynchronously. A later version of the collaboratory included updates which streamlined communication and collaboration, such as assigned projects and a liaison to foster connections between students and alumni. This initiative offers much promise in both mentoring distance students and encouraging research partnerships between faculty and students.
Sonnenwald, D. H., Lassi, M., Olson, N., Ponti, M., & Axelsson, A. (2009). Exploring new ways of working using virtual research environments in library and information science. Library Hi Tech, 27(2), 191-204.
Until recently the majority of scholarship on collaboratories, or virtual research environments (VRE), has pertained to the natural sciences. However, the authors are eager to explore the potential of VREs to enhance research and professional practice in library and information science (LIS). The authors were involved in four distinct projects pertaining to the use of VREs within LIS, two of which are most relevant to the creation of a collaboratory for ACRL’s Science & Technology Section (STS). The focus of the first project was to create a VRE in which LIS researchers and professionals can work to bridge the gap between research and professional practice, resulting in greater understanding and collaboration. LIS VREs could serve as “decentralized peer-to-peer production networks in which the right to give away over the right of ownership is a key element” (p. 195). The networks could facilitate expansive professional development, encouraging participants to share research ideas and gain new knowledge from people outside of their daily environment. VREs need to continuously be improved to make certain features such as communication methods and data collection instruments are fully functional. The second project was an exploratory study in which ten LIS professionals were interviewed to ascertain their attitudes toward VREs and its possible incorporation into their workplace. A few aspects of VREs appeared particularly salient to the participants. For example, most participants expressed interest in sharing experiences and insight with other LIS professionals. Overall the most appealing part of a LIS VRE would be the ability to virtually interact with a geographically and professionally diverse group of LIS colleagues.
Sonnenwald, D. H., Whitton, M. C., & Maglaughlin, K. L. (2002). Scientific collaboratories: Evaluating their potential. Bulletin of the American Society for Information Science and Technology, 28(6), 12-15.
Prior to the publication of this article, there was a fair deal of scholarship on the development of virtual collaboratories but a lack of information detailing the evaluation of these tools. The authors conducted an experiment in which two groups of students completed scientific tasks in each of two conditions: face-to-face with the shared mode of a collaboratory and remotely with a private mode of the same system. In addition to grading these tasks, the authors also interviewed participants to better assess the positives and negatives of the collaboratory environment. The advantages of remote collaboratory use included greater productivity (i.e. no human distractions), and its impersonal nature spurred the development of explicit cues which better elucidated actions and motives to colleagues. In terms of disadvantages of working remotely with the collaboratory, participants reported a lack of implicit interpersonal cues which they would have easily discerned in a face-to-face setting. However, this issue could be remedied by consciously providing clearer and continual explanations of their thoughts. Overall, the participants determined that the minor disadvantages associated with the collaboratory did not lessen the quality of their work. Coping mechanisms were created to combat these disadvantages, which, in combination with the many assets, bodes well for the further use of collaboratories within scientific research.
Turban, E., Liang, T., Wu, S. P., & J. (2011). A framework for adopting collaboration 2.0 tools for virtual group decision making. Group Decision and Negotiation, 20(2), 137-154.
The main goal of this article is to explain how collaboratories, or collaboration 2.0 software, can be utilized to improve communication and decision-making of virtual groups. Unlike many other papers on this subject matter, this one focuses on business partnerships. Collaborative tools have greatly improved over the years, using Web 2.0 features such as wikis and group chat to increase efficiency and productivity. Collaboration 2.0 is contrasted with Collaboration 1.0, with the former hallmarked by the incorporation of social software tools and networks. Unlike its predecessor, 2.0 is usually inexpensive, highly flexible, and user created content. “Properly designing collaborative environments enable decision makers to discuss issues, brainstorm, evaluate their pros and cons, and agree on a final course of action using social software,” (p. 147) a point which could easily be extrapolated to a group of academics involved in virtual networking and research pursuits. In order to ensure a collaboratory is effective it must be assessed. The authors offer a framework called the “fit-viability model,” which can be used to determine if a collaborative tool is properly serving virtual teams. These assessment criteria include whether the tool satisfies decision tasks, financial needs and constraints, present IT infrastructure, and performance measurements of the group both before and after the tool’s implementation.
Voss, A., & Procter, R. (2009). Virtual research environments in scholarly work and communications. Library Hi Tech, 27(2), 174-190.
The influx of emerging technologies into LIS has greatly changed the face of traditional research practices. The term “e-research” is used to qualify this shift, one which prioritizes collaborative over individual research endeavors and exists at least partially within the virtual world. The authors posit that the scholarly communication process can be improved with the use of virtual research environments (VREs). One of the primary strengths of VRE are that they collocate formerly disparate tools for research and communication into one powerful system, and can support collaborators throughout the entire research cycle. The authors provide examples of several different VREs and descriptions of their mechanics and functionality. A list of desired functionalities of VREs is offered, including services and features such as transfer data, discover available resources, re-use and archive data, and task tracking (p. 179). The sheer amount of data production spurred by modern information and communication technologies is markedly changing the research process. This shift is not restricted to major scientific enterprises but also impacts researchers of a multitude of topics. There are a few salient challenges associated with VREs which must be addressed. For examples, users must be allowed greater flexibility to customize VREs to their own liking without the help of technical support employees. Additionally, researchers and technologists must engage in frequent communication in order to continually adapt and embed these tools into the research process.
The ALA 2014 class of Emerging Leaders will showcase their final projects at this poster session. It will be the culminating event for this class of Emerging Leaders. Since the Midwinter Meeting teams have been working virtually on projects related either to ALA units or other professional concerns. This poster session will allow each team to highlight its creative and innovative solutions for their project assignments. Emerging Leader sponsors, member guides, and all conference attendees are welcomed and encouraged to attend.