How big is the haystack?

Finding information on the Web can be time-consuming and unpredictable. Web growth is a cycle fueled by more users, more sites, more Web pages. A recent article in JAMA reported that Internet users in 1998 were estimated at 80 million in North America alone, up from 3 million worldwide in 1994 (Coelho, 1998). This influx of users has been accompanied by tremendous growth in the number of individuals and organizations publishing and offering services online.

Network Wizards, in their 1999 Internet Domain Survey found over 43 million named, Internet host addresses (Network Wizards, 1999). Lawrence and Giles (1998) estimated that publicly indexable Web sites minimally contain 320 million pages. A 1999 report by the same authors put the number at 800 million pages (Lawrence & Giles, 1999). Of these, 83% are commercial and only 6% have scientific/educational content.So it is little wonder that health professionals and consumers can easily be swamped and find that the time spent searching for credible, current information on the Web is not used efficiently. In fact, Coelho (1998) cited classic informatics theory which "shows that as information increases, the amount of irrelevant and inaccurate information (often referred to as '˜noise') also increases."This column will highlight some specific search tools for finding health-related Web sites '” directories, search engines, and megasearch engines. Some are specific to the health care environment. Others are generic, all-purpose tools. This is a selective listing.Possible Search Approaches

1. Following the links from one good, identified site: Similar to following the references found in a key journal article, this can yield nuggets. Given the size of the Internet, however, this is a marginal approach for serious inquiries.
2. Web directories: Also called subject catalogs, gateways, or guides, these sites provide browsing of hierarchical subject arrangements of selected Web-based information. Site selection is done by humans. Many directories are also searchable. One of the most commonly used general directories is Yahoo! (www.yahoo.com), which is the largest human-compiled guide to the Web and the oldest, dating from 1994.
3. Web search engines: These services utilize automatic indexing software to search the Web, locate sites, and index Web pages. All search engines vary as to how they build and maintain their databases, resulting in substantial variability in search results. Therefore, comprehensiveness improves by the use of multiple search engines. However, Lawrence and Giles (1999) report that no search engine provides coverage to more than 16% of the Web and that coverage overall has continued to decline since December 1997.
4. Megasearch engines: These services send a search query to multiple search engines simultaneously and then present the results. Again, each works in a unique manner and searches on a different combination of resources.

Specialized Web DirectoriesThe Web resources listed here are designed for health professionals, although health consumers may also find them useful. All provide free access. Some require registration and may also offer specialized, for-fee services.

Specialized Search Engines for Health InformationThese search engines restrict their retrieval to Web pages from sites selected for their quality and utility for health professionals. They search primarily on clinically related sites.

General Purpose Web Search EnginesSome of the most popular general search engines are listed below. They can provide very broad, unfiltered retrieval on health-related topics. A few general sites also provide reviews or ratings that may be of use in picking quality resources. A listing of some of these can be found at Internet Sleuth (www.isleuth.com/review.html).

Megasearch EnginesMegasearch engines offer great efficiency in that they quickly execute a search against multiple search engines. Their limitation lies in not accommodating the variation in search formats utilized by different search engines.

Other Resources for Searching the WebNumerous resources and tips on Web searching are available. This is a sampling:Finding Information on the Internet: A TUTORIAL (www.lib.berkeley.edu/TeachingLib/Guides/Internet/FindInfo.html)

Tutorial provides guidance for many aspects of successful Internet use, including searching. It is one of the Teaching Library Internet Workshops produced by the University of California, Berkeley, Library.

Learn the Net: Knowledge when you need it. (www.learnthenet.com/english/index.html).

Tutorial guide to using the Internet produced by Michael Lerner Productions. "Searching the Web" feature covers basic search engine use. Available in English, French, German, Spanish and Italian editions. Guidelines are given for reproducing this content.

Maze, S., Moxley, D., and Smith, D.J. (1997). Authoritative guide to Web search engines. Neal-Shuman Publishers: New York.

The authors describe how search engines work and the advantages and disadvantages of the various types. Includes comparisons of seven leading, general-purpose search engines.

Search Engine Watch (searchenginewatch.com/).

This is a key site for up-to-date information on search engines and includes reviews, information on special search assistance features, and search engine tutorials.

ConclusionThe tools described here provide valuable aids to Web searching. As always, analyze your topic and the possible terms and phrases that could be useful. To get the most out of any searching tool, learn its features and the recommended search formats. Remember that the great variability between the design of different search tools and how they work requires that a combination of services be used to get more complete results.

THE AUTHOR

Barbara F. Schloman, PhD, AHIP
Director, Library Information Services
Libraries & Media Services
Kent State University
Kent, OH 44242
E-mail Address: schloman@kent.edu