Evidence Synthesis Methods Interest Group

 View Only
last person joined: yesterday 

Charge: To promote and develop competencies around evidence synthesis including systematic reviews, meta-analyses, scoping reviews, and other related methods of research synthesis, through activities such as: Facilitating discussion and peer-support; Creating and managing a resource page; Encouraging programming and publications around systematic reviews through ACRL.
Community members can post as a new Discussion or email ALA-acrlesmig@ConnectedCommunity.org
Before you post: please note job postings are prohibited on ALA Connect. Please see the Code of Conduct for more information.
  • 1.  Journal credibility question

    Posted Jul 19, 2021 11:36 AM
    Hi all, 
    A colleague shared the following question from a graduate student in natural resources with me. My response was that limiting by journal/country of origin would introduce bias, so I wouldn't be comfortable with this and would rely on the Risk of Bias tool to assess quality. But, I wanted to find out if any of you have heard this before or have a different opinion.

    "Quick question about systematic reviews. Do you ever acknowledge or account for varying levels of credibility in journals? One person helping work through our articles (round 2!) mentioned that in her lab, they often don't cite work from many Chinese journals, unless they know the researchers in some way. Her work is in genetic engineering and mRNA, so that type of credibility might be more important/tricky in her line of research (as opposed to copper exposure), but I'm really not sure. What are your thoughts, or have you talked about this?"


    --
    Megan Kocher
    Science Librarian | University Libraries | lib.umn.edu
    University of Minnesota | umn.edu | 612-625-3605
    she, her, hers







  • 2.  RE: Journal credibility question

    Posted Jul 19, 2021 12:22 PM

    Hello Megan,

     

    This is an opinion I have shared with my colleagues:

     

    Yes, this does introduce bias if you eliminate a study published in a peer-reviewed journal by the investigator's country of origin.  However, clinical medical and biomedical sciences research are a global phenomenon. I am of the opinion, if the journal is indexed and vetted for inclusion in Medline, Scopus, or another authoritative bibliographic database that uses a committee to review the contents of the journal, or the paper is peer-reviewed in a journal that follows the guidelines of ICMJE, COPE, or other authoritative body for editors and academic publishers, it should be included.  Additionally, journals with a rigorous peer-review process, an international editorial body, and who also publish studies that are sound and reproducible have merit for inclusion.  

     

    I tend to discourage investigators from using papers published in potential or identified predatory journal titles without first reaching out to the authors or critically examining for reproducibility. Where is gets a bit murky is those journals where they do not follow the conventions of the ICMJE – it may be due to inexperience.  Several newer journal publishers are now being incubated by the larger STM presses, such as Wiley.  If not stated on the journal's website, you can usually do a Google search to find this information.  If still in question, I suggest to the investigator to review the DOAJ and Think, Check, Submit websites. 

     

    Last, another reason I do not support eliminating based on country of origin, as many of our school's PIs write with colleagues at other research centers outside of North America. Our school also trains several foreign-born investigators as well, who often write with colleagues abroad too.

     

    I hope you find this helpful.

    Sincerely,

    Liz Lorbeer

     

     

    Elizabeth Lorbeer, EdM, MLS, AHIP

    Chair and Associate Professor, Department of the Medical Library

    Library Director

    Western Michigan University Homer Stryker M.D. School of Medicine

     


    This electronic message is intended for the named recipient(s) only, and may contain information that is confidential or privileged. If you are not the named recipient, you are hereby notified that any disclosure, copying, distribution, or use of the contents of this message is strictly prohibited. If you have received this message in error, or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.





  • 3.  RE: Journal credibility question

    Posted Jul 20, 2021 09:35 AM
    Megan, eek, like you, I would not feel comfortable excluding certain journals. Maybe mention that western journals with high impact factors, decent reputations, etc. also have published flawed or fraudulent studies. This 2012 article's table comes to mind----

    Journals-with-most-retracted-articles.png
    Also, figure 1 (below) from (Retracted Science and the Retraction Index) shows the correlation between impact factor and retractions.
    figure 1 retraction relation.jpeg
    But... it's complicated because "journals with high impact factors-a measure of how often papers are cited-have taken the lead in policing their papers after publication. In 2004, just one-fourth of a sampling of high-impact biomedical journals reported having policies on publishing retractions, according to the Journal of the Medical Library Association (JMLA). Then, in 2009, the Committee on Publication Ethics (COPE), a nonprofit group in Eastleigh, U.K., that now advises more than 12,000 journal editors and publishers, released a model policy for how journals should handle retractions. By 2015, two-thirds of 147 high-impact journals, most of them biomedical titles, had adopted such policies, JMLA reported" (Brainard & You, 2018).

    TL;DR You could advise that they stop putting high impact (and western) journals (and labs) on a pedestal.  ¯\_(ツ)_/¯