Does Bradford's Law of Scattering predict the size of the literature in Cochrane Reviews?






Latest articles from "Journal of the Medical Library Association":

Martin M. Cummings, AHIP, FMLA, 1920-2011 (July 1, 2012)

Improving information retrieval using Medical Subject Headings Concepts: a test case on rare and chronic diseases (July 1, 2012)

The Internet and health information: differences in pet owners based on age, gender, and education (July 1, 2012)

Academic Librarianship (July 1, 2012)

Describing Electronic, Digital, and Other Media Using AACR2 and RDA: A How-To-Do-It Manual and CD-ROM for Librarians (July 1, 2012)

A case study: the evolution of a "facilitator model" liaison program in an academic medical library (July 1, 2012)

Building projects: redefining hospital libraries (July 1, 2012)

Other interesting articles:

Adult Vaccination Coverage - United States, 2010
MMWR. Morbidity and Mortality Weekly Report (February 3, 2012)

Issues in Correctional Care: Abuse of Club Drugs-Ketamine
American Jails (March 1, 2010)

Investigation of Viral Hepatitis Infections Possibly Associated with Health-Care Delivery - New York City, 2008-2011
MMWR. Morbidity and Mortality Weekly Report (May 18, 2012)

Cranial melioidosis with extradural extension after a fall in the bathroom
Australasian Medical Journal (Online) (August 1, 2012)

Incidence, severity and management of cancer chemotherapy related oral mucositis in Eastern Cape and Western Cape
Health S A (January 1, 2012)

Akut Ürtikerli Yatan Hastalarin Klinik Olarak Degerlendirilmesi/Clinical Evaluation of Inpatients with Acute Urticaria
Turkderm (October 1, 2011)

Challenges in the Accurate Identification of the Ideal Catheter Tip Location
Journal of the Association for Vascular Access (December 1, 2010)

Publication: Journal of the Medical Library Association
Author: Nash-Stewart, Charlotte E
Date published: April 1, 2012

INTRODUCTION

Bradford's Law of Scattering is a law of diminishing returns and scattering. Bradford formulated the law in 1948 and claimed that for a given subject area "there are a few very productive periodicals, a larger number of more moderate producers, and a still larger number of constantly diminishing productivity" [I]. For any single issue, or subject area, the top third (Zone 1 or core) represents the journals that are the most frequently cited in the literature of that subject and that are, therefore, likely to be of highest interest to researchers in the discipline. The middle third (Zone 2) includes the journals that have had an average amount of citations, and the bottom third (Zone 3 or tail) comprises the long tail of journals that are seldom cited and regarded as of marginal importance to the subject [2]. Researchers have defined a subject area in lexical, semantic, and subject scattering terms [3], and some argue that problems in defining "subject" may not matter, provided it is applied consistently [4].

Bradford's law predicts that the number of journals in the second and third zones will be n and n2 times larger than the first zone respectively [5, 6], and therefore, it should be possible to predict the total number of journals containing papers on a subject once the number in the core and middle zone of journals is known. Once the total number of journals is known, it should be possible to predict how much relevant information is missing from an incomplete search. Given the time-consuming and extensive effort required to identify sources on a subject for a systematic review, accurately predicting the size (and quality, if possible) of the literature from Bradford's law would be useful for such studies.

Empirical testing of Bradford's law requires a complete and large bibliography, a well-defined subject, and a limited time frame [5, 7]. Bradford's law has been applied successfully to measure the literature of many subjects, such as nursing [I], science [8], crystallography [9], and occupational therapy [2]. In addition, many librarians wiU be familiar with Eugene Garfield's Science Citation Index, which is based on Bradford's law [1O]. One analysis of all randomized controlled trials (RCTs) in the MEDLINE database found that the journal distribution varied from the standard Bradford's law [U]. However, there have not been any studies on the usefulness of Bradford's law to predict the size of literature in systematic reviews. Other methods to estimate the total number of articles when searching for systematic reviews, such as the Horizon Estimate, have been applied with varying success [12].

This study arose from work with the Acute Respiratory Infections Group, one of the fifty-two entities making up the Cochrane Collaboration. Cochrane Reviews are systematic literature reviews aiming at high quality and, therefore, explicitly striving for completeness. The validity of Bradford's law for systematic reviews has not been addressed in the literature. Accordingly, this study examined whether Bradford's law was valid for the Cochrane Review-identified literature on acute otitis media and pneumonia, conditions that are reported in a wide variety of clinical and health journals [13].

METHODS

In late 2006, a search in titles, abstracts, and keywords for the term, "acute otitis media," and the term "pneumonia," were undertaken in the Cochrane Library. The two conditions were chosen because they are the subject of many Cochrane Reviews and have a large literature. RCTs were extracted from the references in the chosen Cochrane Reviews that addressed only treatment of acute otitis media or pneumonia. Reviews and protocols concerning prevention rather than treatment were not selected for the study. Clinical subject experts were consulted to confirm the suitability of the treatment reviews for the study. Extracted RCTs were grouped by subject, marked by whether they were included or excluded by the Cochrane Review, and duplicates were removed. The included status was used as proxy for high-quality RCTs. These meet strict eligibility criteria to be included in Cochrane Reviews and are excluded if they fail to meet the criteria. For this study, RCTs were marked as included when at least one Cochrane Review article included it, even if it was excluded in other Cochrane Reviews. The purpose of categorizing included and excluded status was to identify whether higher-quality studies were more likely to be found in Zone 1 journals. RCTs were then sorted by journal to generate a cumulative distribution for the subject. Journals that had changed names or merged were listed under the most current journal title available from the MEDLINE journals database [14].

The following analyses were undertaken for each of the two subjects: (1) the journal distribution was analyzed (according to the number of RCTs each contained) to test Bradford's law; (2) the Zone 1 journals were identified; and (3) the correlation between inclusion status and publication in a Zone 1 journal was determined.

It was assumed that the Cochrane Reviews missed few references.

RESULTS

The search in the Cochrane Library on the subject acute otitis media found 22 Cochrane Reviews, 5 of which met inclusion criteria [15-19], yielding 135 articles from 73 journals. For pneumonia, the search in the Cochrane Library found 62 reviews, of which 9 met the inclusion criteria, yielding 251 articles from 110 journals [20-28].

Bradford's Law of Scattering

The journals publishing articles on each topic were divided into 3 regions, with approximately equal numbers of articles, detailed in Table 1 (online only). When journals were tied in terms of articles published, they were ranked according to convention "Cl" in Heine (1998) [4]. This convention gives journals that produce the same number of articles an arbitrary and sequential rank from the previously ranked journal, rather than assigning all the journals the same rank. The 3 zones were determined by closest fit to the l:n:n2 Bradford model, while keeping all journals that produced the same number of papers in the same zone. As a result, the articles were not divided into 3 equal parts. The core journals, Zone 1, were defined as those providing approximately one-third of all the RCTs. For acute otitis media, the core accounted for 41 of 135 papers (30%) from 5 journals, and for pneumonia, 69 (27%) of 251 papers from 6 journals. The middle journals, Zone 2, contained 15 journals (41 articles, 30%) for acute otitis media and 21 journals (87 articles, 35%) for pneumonia.

Journal rank (on a log scale) is plotted against cumulative articles in Figure 1 and gives similar appearances for both conditions. The figure does mirror broadly the shape of a Bradford-Zipf plot, albeit with wider horizontal axis to illustrate the zone information.

This represents a larger tail than predicted by Bradford's law (53 versus 45 predicted Zone 3 journals for acute otitis media and 83 versus 73 predicted Zone 3 journals for pneumonia) and is statistically significant for pneumonia (?2=5.09; ?=0.024) but not otitis media (?2=3.68; P=0.055 and pneumonia).

Quality and location of studies

For acute otitis media, 29 of the 41 articles in 5 journals (71%) from Zone 1 were included in Cochrane Reviews (proxy for high quality). This compares with 33 of 94 noncore (Zone 2 or 3) articles in 68 journals (35%) being included in Cochrane Reviews (?2 =14.6, P=0.00013). Overall, only 29 of the 62 included articles (47%) were found in the 5 journals from Zone 1 and 14 (23%) were from the journals in Zone 3.

For pneumonia, 30 of 69 articles in 6 journals (43%) from Zone 1 were included in Cochrane Reviews. This compared with 50 of 182 noncore articles in 104 journals (27%) being included in Cochrane Reviews (?2=4.96, P=0.026). Only 30 of the 80 included articles (38%) were found in the 6 journals from Zone 1. This number was exceeded by the 33 included articles (41%) from Zone 3.

Overall, 86% of all RCTs and 92% of included studies came from MEDLINE-indexed journals.

DISCUSSION

Bradford's law predicts that finding the core literature addressing a topic will enable an estimate of the total literature, but this has not been tested empirically for systematic reviews. The Cochrane Reviews examined did not conform sufficiently to Bradford's law to allow the total number of RCTs on a subject to be predicted from just the number found in core journals. Although the journal distribution for both conditions broadly matches Bradford's law, uncertainty in the tail size creates difficulties in prediction based on only a set of core journals. This suggests little use in estimating literature size for subjects from RCTs found in core journals alone (e.g., when planning a review for a given subject) or in identifying how much information would be missed if the search was incomplete. Other tests of the law have also been disappointing, including Bradford's own and other empirical testing [29, 3O]. This study's finding that Zone 3 size differed from that predicted by the classical Bradford model may reflect the highly interdisciplinary nature of acute respiratory infections (crossing the disciplines of primary care, paediatrics, and infectious diseases, as well as general internal medicine) [3] or uncertainties in classifying the "subject" [31].

MEDLINE indexes 94% of its medical literature [32]. Not all of the RCTs (86%) included in Cochrane Reviews that were identified in this study were indexed in MEDLINE, supporting the need for systematic reviewers to anticipate that perhaps 15% of available articles would be found in other places (e.g., databases such as Embase) and that, of these, about half could be valid for inclusion. The large proportion of high-quality RCTs that came from the large Zone 3 zone of journals in this study confirms previous opinions that searching the tail is necessary to capture relevant literature [32]. Finding studies in noncore journal publications is, therefore, necessary both for completeness and to avoid bias in the review.

CONCLUSION

In this case, Bradford's law was not useful for predicting the size of the literature on a subject from the number of articles appearing in core journals. However, the Zone 3 of the distribution in this study contained a large number of valid studies, essential to include in a complete systematic review. Further research is needed to enable accurate estimates of the anticipated literature size in systematic reviews of other Cochrane topics.

ACKNOWLEDGMENT

The authors thank Professor David Goldgar, The University of Utah, for his help with the biostatistics.

References:

1. Bradford SC, Egan ME, Shera JH. Documentation. 2nd ed. London, UK: Crossby Lockwood; 1953.

2. Potter J. Mapping the literature of occupational therapy: an update. J Med Lib Assoc. 2010 Jul,-98(3):235-42. DOI: http://dx.doi.0rg/10.3163/1536-5050.98.3.012.

3. Hj0rland B, Nicolaisen J. Bradford's Law of Scattering: ambiguities in the concept of "subject." In: Crestani F, Ruthven I, eds. Context: nature, impact, and role: 5th International Conference on Conceptions of Library and Information Sciences Springer; 2005. p. 96-106. (Lecture Notes in Computer Science, v.3507.)

4. Heine M. Bradford ranking conventions and their application to a growing literature. J Documentation. 1998 Jun;54(3):303-31.

5. Fairthorne R. Empirical hyperbolic distributions (Bradford-Zipf-Mandelbrot) for bibliometric description and prediction. J Documentation. 2005;61(2):171-93.

6. Garfield E. Bradford's law and related statistical patterns. Essays Inf Scientist. 1980 May;4:476-83.

7. Naranan S. Bradford's law of bibliography of science: an interpretation. Nature. 1970 Aug 8;227(5258):631-2.

8. Brookes BC. Bradford's law and the bibliography of science. Nature. 1969 Dec;224(5223):953-6.

9. Behrens H, Luksch P. A bibliometric study in crystallography. Acta Crystallogr B. 2006 Dec;62(6):993-L001.

10. Bensman SJ. Garfield and the impact factor. Ann Rev Inf Sci Technol. 2007;41(1):93-155.

11. Tsay MY, Yang YH. Bibliometric analysis of the literature of randomized controlled trials. J Med Lib Assoc. 2005 Oct;93(4):450-e.

12. Kastner M, Straus SE, McKibbon KA, Goldsmith CH. The capture-mark-recapture technique can be used as a stopping rule when searching in systematic reviews. J Clin Epidemiol. 2009 Feb;62(2):149-57.

13. White H. "Bradfordizing"search output: how it would help online users. Online Inf Rev. 1993;5(l):47-54.

14. National Library of Medicine. List of journals indexed for MEDLINE [Internet]. The Library; 2010 [cited 17 Oct 2011]. <http://www.nlm.nih.gov/tsd/serials/lji.html>.

15. Flynn C, Griffin G, Tudiver F. Decongestants and antihistamines for acute otitis media in children. Cochrane Database Syst Rev. 2001;(2):CD001727.

16. Foxlee R, Johansson A, Wejfalk J, Dawkins J, Dooley L, Del Mar C. Topical analgesia for acute otitis media. Cochrane Database Syst Rev. 2006 Jul;(3):CD005657.

17. Glasziou PP, Del Mar CB, Sanders SL, Hayem M. Antibiotics for acute otitis media in children. Cochrane Database Syst Rev. 2004;(1):CD000219.

18. Kozyrskyj A, Klassen TP, Moffatt M, Harvey K. Shortcourse antibiotics for acute otitis media. Cochrane Database Syst Rev. 2010 Sep;(9):CD001095.

19. Spurting GK, Del Mar CB, Dooley L, Foxlee R. Delayed antibiotics for symptoms and complications of respiratory infections. Cochrane Database Syst Rev. 2004 Oct;(4):CD004417.

20. Bjerre LM, Verheij Tf, Kochen MM. Antibiotics for community acquired pneumonia in adult outpatients. Cochrane Database Syst Rev. 2004 Oct;(2):CD002109.

21. Briel M, Bucher HC, Boscacci R, Furrer H. Adjunctive corticosteroids for pneumocystis jiroveci pneumonia in patients with HIV-infection. Cochrane Database Syst Rev. 2006 Jan;(3):CD006150.

22. Cheng AC, Stephens DP, Currie BJ. Granulocyte-colony stimulating factor (G-CSF) as an adjunct to antibiotics in the treatment of pneumonia in adults. Cochrane Database Syst Rev. 2007 Apr;(2):CD004400.

23. Gavranich JB, Chang AB. Antibiotics for community acquired lower respiratory tract infections (LRTI) secondary to mycoplasma pneumoniae in children. Cochrane Database Syst Rev. 2005 Jul;(3):CD004875.

24. Kabra SK, Lodha R, Pandey RM. Antibiotics for community acquired pneumonia in children. Cochrane Database Syst Rev. 2006 Jul;(3):CD004874.

25. Ni J, Wei J, Wu T. Vitamin A for non-measles pneumonia in children. Cochrane Database Syst Rev. 2005 Jul;(3):CD003700.

26. Panpanich R, Lerttrakarnnon P, Laopaiboon M. Azithromycin for acute lower respiratory tract infections. Cochrane Database Syst Rev. 2004 Oct;(4):CD001954.

27. Rojas MX, Granados C. Oral antibiotics versus parenteral antibiotics for severe pneumonia in children. Cochrane Database Syst Rev. 2006 Apr;(2):CD004979.

28. Shefet D, Robenshtock E, Paul M, Leibovici L. Empiric antibiotic coverage of atypical pathogens for community acquired pneumonia in hospitalized adults. Cochrane Database Syst Rev. 2005 Apr;(2):CD004418.

29. Bradford SC. Sources of information on specific subjects (reprinted from Engineering an Illustrated Weekly Journal, vol 137, pg 85-86, 1934). J Info Sci. 1985 Apr;10(4):173-80.

30. Drott MC, Griffith BC. An empirical examination of Bradford's law and the scattering of scientific literature. J Am Soc Inf Science. 1978 Sep;29(5):238-46.

31. Nicolaisen J, Hj0rland B. Practical potentials of Bradford's law: a critical examination of the received view. J Documentation. 2007;63(3):359-77.

32. Royle P, Bain L, Waugh N. Systematic reviews of epidemiology in diabetes: finding the evidence. BMC Med Res Methodol. 2005 Jan;5(l):2.

Author affiliation:

Charlotte E. Nash-Stewart, MBBS; Lisa M. Kruesi, MBIT; Chris B. Del Mar, MD

DOI: http://dx.doi.0rg/10.3163/1536-5050.100.2.013

Author affiliation:

AUTHORS' AFFILIATIONS

Charlotte E. Nash-Stewart, MBBS, ckn.stewart@ gmail.com, Freelance Technical Writer; Lisa M. Kruesi, MBIT (corresponding author), l.kruesi@library.uq.edu. au, Associate Director, Scholarly Publishing and Digitisation Service, The University of Queensland Library, The University of Queensland, St. Lucia Campus, QId 4029, Australia; Chris B. Del Mar, MD, cdehnar@bond.edu.au, Professor of Public Health, Bond University, Queensland 4299, Australia

Received June 2011; accepted October 2011

The use of this website is subject to the following Terms of Use