Evaluation: Never More ImportantThis first Information Resources column deals with evaluation '” the foremost consideration in using health-related resources on the Web. Librarians and nursing faculty have a long history of teaching students that not all published information is of the same quality or credibility and what criteria need to be used for making such determinations, such as the differences between a periodical and a scholarly journal. Now, in the networked environment, evaluation has become more challenging. Anyone with the will and some computer know-how can become their own publisher on the Web. Not only does this by-pass the traditional filters for quality that we have come to rely on for the traditional journal literature (e.g., journal reputation, refereed review, credentialed authors and editors), but the electronic medium seems to serve up information with a certain mystique that encourages many to accept what is offered without question. It also tends to obscure some of the distinguishing characteristics of resources that are often more obvious with print products.Without a healthy skepticism and set of evaluation criteria, the health consumer on the Web can be taken in by fraudulent claims and inaccurate information. Events have shown that the Web is a profitable haven for those promoting bogus and potentially harmful cures, drugs, and devices. The Federal Trade Commission, in cooperation with the Canadian and Mexican governments, held a "Health Claims Surf Day" to ferret out dubious sites related to six diseases: AIDS, arthritis, cancer, diabetes, heart disease, and multiple sclerosis. Promoters and advertisers providing insufficient documentation were then notified that claims must be supported with reliable scientific evidence (Skolnick, 1997a). The World Health Organization established an ad hoc working group in late 1997 to develop recommendations to help regulate health ads and sale of medical products, but the problems of these being adopted and enforced within a global computer network are daunting (Skolnick, 1997b). The British Medical Journal recently reported that enforcement agencies from 20 countries searched the Internet for "potentially misleading health claims and miracle cures" and also identified numerous bogus claims (Brooks, 1998). In addition to the problem of fraudulent claims is the danger of inaccurate or incomplete information from seemingly reliable sources. In one recent study, researchers surveyed the Web for parent-oriented health sites that provided information on the management of fever in a child (Impicciatore, Pandolfini, Casella, Bonati, 1997). Forty-one Web sites were identified '” 32 were commercial and the remainder from individual practitioners, clinics, and educational organizations. The information given at each site was compared with established treatment guidelines. Only one-tenth of the sites closely adhered to the guidelines. Several gave potentially harmful information (e.g., treating with aspirin). Many failed to provide complete information (e.g., how to take a temperature).A similar, well-publicized study on the extent to which recommendations found on the Web for the treatment of childhood diarrhea concurred with guidelines from the American Academy of Pediatrics (AAP) found significant misinformation as well (McClung, Murray, Heitlinger, 1998). Only 20% of the 60 traditional medical sites concurred with AAP guidelines. The authors concluded steps to rectify this situation and insure dissemination of accurate information must be taken by physicians, teaching institutions, professional societies, and patients themselves. With regard to the latter, the authors stated "Practitioners need to warn their patients about the need for a very critical review of all medical information obtained from the Web, even when it seems to be from a 'reliable' sources" (McClung et al, para.17).Web Evaluation SitesEvaluation, then, is very thorny, certainly for consumers. We can expect that in the future more organized, formal evaluation mechanisms will be put in place by the healthcare and library communities. At present, some Web directories provide "ratings" for sites, but a recent study of these found that less than one-third published their criteria (Jadad & Gagliardi, 1998). Of those that did, there was no indication of formalized evaluation addressing issues of interrater reliability or construct validity. Several of the more notable Web directories with a health-specific scope that you may wish to visit are:
- American Medical Association Library Choices (www.ama-assn.org/med_link/med_link.htm)
- Health on the Net Foundation (www.hon.ch)
- OncoLink's Editors' Choice Awards (http://oncolink.upenn.edu/ed_choice/)
- Physician's Choice (www.mdchoice.com/)
- Six Senses Seal of Approval (www.sixsenses.com/)
- National Council Against Health Fraud (www.ncahf.org): "... a non-profit, tax-exempt voluntary health agency that focuses its attention upon health fraud, misinformation and quackery as public health problems. It is private, nonpolitical and nonsectarian. The organization is comprised of health professionals, educators, researchers, attorneys and concerned citizens. Its officers and board members serve without compensation." Includes "Recommended AntiQuackery Web Sites."
- QuackWatch (www.quackwatch.com/): "... a member of Consumer Federation of America, is a nonprofit corporation whose purpose is to combat health-related frauds, myths, fads, and fallacies. ... Our activities include: Investigating questionable claims, Answering inquiries, Distributing reliable publications, Reporting illegal marketing, Improving the quality of health information on the Internet, Attacking misleading advertising on the Internet." Includes meritorious and "nonrecommended" sites, as well as "Signs of a 'Quacky' Web Site."
- what is their authority?
- do they have expertise or experience with the topic?
- what are their credentials, institutional affiliation?
- is an e-mail address provided for a contact person with responsibility for given content or the site?
- is is the site organization logical and easy to maneuver?
- does the URL suggest a reputable affiliation with regard to the topic '” personal or official site; type of Internet domain (i.e., .edu: educational institution; .org: non-profit organization; .com: commercial enterprise; .net: Internet Service Provider; .gov: governmental body; .mil: military body)?
- are the purpose and scope stated?
- who is the intended audience?
- is the information clearly presented as being factual or opinion, primary or secondary in origin?
- what criteria are used for inclusion of the information?
- is any sponsorship or underwriting fully disclosed?
- are the facts documented or well-researched?
- do the facts compare to related print or other online sources?
- are the Web resources for which links are provided quality sites?
- is the content current?
- are the pages date-stamped with last update?
- is the site organization logical and easy to manuever?
- is the content readable by the intended audience?
- has attention been paid to presenting the information as error-free (e.g., spelling, punctuation) as possible?
- is there a readily identifiable link back to the institutional or organizational home page?
- is the site readable by those with basic access capabilities (i.e., browsers and network speed)?
- are links provided when necessary to download needed browser plug-ins?
- is the site reliably accessible?
Director, Library Information Services
Libraries & Media Services
Kent State University
Kent, OH 44242
E-mail Address: firstname.lastname@example.orgKeywords: World Wide Web sites '” evaluation, health information '” evaluation, evaluation criteria Brooks, A. (1998). Miracle cures advertised on the Internet. British Medical Journal, International Edition, 317, 769. Retrieved December 28, 1998 from the World Wide Web: http://www.bmj.com/cgi/content/full/317/7161/769/aImpicciatore, P., Pandolfini, C., Casella N., & Bonati, M. (1997). Reliability of health information for the public on the World Wide Web: Systematic survey of advice on managing fever in children at home. British Medical Journal, Clinical Research Edition, 314, 1875-1879. Retrieved December 28, 1998 from the World Wide Web: http://www.bmj.com/cgi/content/full/314/7098/1875Jadad, A., & Gagliardi, A. (1998). Rating health information on the Internet: Navigating to knowledge or to Babel? JAMA, 279, 611-614. Retrieved December 28, 1998 from the World Wide Web: http://www.ama-assn.org/sci-pubs/journals/ archive/jama/vol_279/no_8/rv71042a.htmMcClung, H. J., Murray, R. D., & Heitlinger, L. A. (1998). The Internet as a source for current patient information. Pediatrics, 101(6), e2. Retrieved December 28, 1998 from the World Wide Web: http://www.pediatrics.org/cgi/ content/full/101/6/e2Silberg, W. M., Lundberg, G. D., & Musacchio, R. A. (1997). Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor-let the reader and viewer beware. JAMA, 277, 1244-1245. Retrieved December 28, 1998 from the World Wide Web: http://www.ama-assn.org/sci-pubs/journals/archive/jama/vol_277/ no_15/ed7016x.htmSkolnick, A. A. (1997a). Surf's up for health fraud investigators. JAMA, 278, 1725. Retrieved December 28, 1998 from the World Wide Web: http://www.ama-assn.org/sci-pubs/journals/archive/jama/ vol_278/no_21/jmn71156.htmSkolnick, A. A. (1997b). WHO considers regulating ads, sale of medical products on Internet. JAMA, 278, 1723-1724. Retrieved December 28, 1998 from the World Wide Web: http://www.ama-assn.org/sci-pubs/journals/archive/jama/ vol_278/no_21/jmn71155.htmSmith, A. (1997). Criteria for evaluation of Internet information resources. Retrieved November 20, 1998 from the World Wide Web: http://www.vuw.ac.nz/~agsmith/evalnDisclaimer: Mention of a Web site does not imply endorsement by the author, OJIN, or NursingWorld.