menu
William and Mary
search

General Guidelines to History Research on the World Wide Web

The World Wide Web has opened a new world of historical research for everyone; however, it can be a frustrating and even risky place to gather information. The sheer amount of information found on the web often tempts writers into simply listing information in their papers without analyzing it or determining its usefulness in supporting the papers' thesis. Just because you find an interesting fact or anecdote on the web does not mean it should appear in your final draft. The ease and excitement of researching on the web can also lure you into devoting too much time to exploration, which leaves you little time for the all-important tasks of writing and revising (and revising and revising) your paper. The best research in the world will not make up for a paper without a solid thesis. This handout offers suggestions on the most efficient ways to search the web without getting caught in an information overload.

There are millions of pages on the web, written by thousands of individuals, companies, organizations, and governments. Web researchers must remember that not all of the evidence they find on the web is equally reliable or even correct. Just as researchers learn not to believe everything they read in print, they must learn to properly evaluate sources of information on the web. Web sites owned by governments (federal, state, and local), universities, libraries, and museums are generally reliable sources of information. You must be wary of sites posted by organizations with an agenda or product to promote. For example, if you were researching the origins of the Second Amendment, you would want to be very careful relying on web pages posted by gun control or gun advocacy groups. You should be extremely cautious about relying on sites owned by individuals-as a general rule, you might want to avoid using these sites. As always, you should try to confirm a source's information by using a second source if possible.

Although the World Wide Web is a superb resource, it does not come close to having all of the information you can find in a library. While there are history sites which offer specialized collections of history sources, web research often lacks access to primary documents, the best evidence historians have to prove their arguments. In other words, do not limit your research to what is on the Internet-think of it as simply one tool you can use when researching your topic. For further information on how to use the web to research history, ask your professor, the HWRC, or the real information specialists-the reference librarians at Swem Library.

Conducting a Productive Web Search

There are two main types of searches when working on the web: the first utilizes a web index, the second a search or metasearch engine. Both types of searches have advantages and disadvantages.

I: Web Indexes

There are two main types of web indexes: 1) those that are hierarchical (i.e. that lead one from a general topic to a more specific one) and 2) those that list sources in some sort of order (most commonly alphabetical). The first type of index often contains a broad range of topics, while the second usually consists of sources designed to address a particular topic or concern. Probably the best known and most comprehensive web index is "Yahoo!"

Indexes are extremely valuable for web researchers who have an area on which they want to focus, but who do not yet have a specific topic. This is especially true for students without much background in a subject. An index can help students get general information or a "feel" for the topic and then gently lead them to more specific information. An index search can also assist the researcher by pointing out productive sub-topics that they might not have originally envisioned. For example, indexes are extremely helpful in narrowing down a topic area for a research paper in an introductory course such as Western Civilization.

The disadvantage to using index sites is that they are labor intensive. In addition, you must know enough about your topic to make choices about the direction you want to research at each step of your research journey.

An Example of Using a Web Index to Narrow a Research Topic

  •      Go to Yahoo! at http://www.yahoo.com
  •      Find the topic area that you must research (history)
  •      Follow it through to specifics (historyè French historyè French Revolutionè Terror)
  •      (French Revolution-Terror) is a specific topic that can be feasibly researched, either by following the links listed by the search index or by   using that phrase in a keyword search


II: Search and/or Metasearch Engines

A search engine is a device that sends out inquiries to sites on the web and catalogs any web site it encounters, without evaluating it. Users can then search those catalogs by keyword(s) or any number of other methods to come up with "links" or pathways to the information they are seeking. A metasearch engine is a specialized program which conducts individual searches on a number of the most popular search engines and reports the top ten "hits" or matches to your search term on each. Metasearch engines can save time by doing individual searches on a number of sites for you automatically. One reason this approach is helpful is that methods of searching differ from search engine to search engine, so the results reported by each one will also differ.

Search engines maintain an incredibly large number of sites in their archives, so you must limit your search terms in order to avoid becoming overwhelmed by an unmanageable number of responses. Thus, search engines are good for finding sources for well-defined topics. Typing in a general term such as (history) or (French History) will bring back far too many results to deal with, but by significantly narrowing your topic to (Terror and French and Revolution), you can often get the kind of information that you need. Always remember, search engines will put the most relevant sites at the top of their lists, but most engines determine relevancy by the number of key term matches. This means that the most repetitive site will be the most relevant in their list, and that may not turn up the best sites for your use.

In addition, keep in mind that recent studies indicate that search engines only index about 16% of the total content on the Web and that they are biased toward well-known information. (One study recently found that Northern Light has about 16% of the total content and Snap and Alta Vista each have about 15.5%; Lycos has about 2.5%.)

Some of the most powerful search engines include the following:
  • Excite (http://www.excite.com)
  • Google (http://www.google.com)
  • HotBot (http://www.hotbot.com)
The following are metasearch engines and will search other search engines:
Using a Search or Metasearch Engine

To be more productive in searching with search and metasearch engines, read the instructions and FAQs to learn how each particular site works. Each search engine is slightly different, and a few minutes learning how to use the site properly will save you large amounts of time and prevent useless searching. Each search engine has different advantages; for example, Alta Vista offers the option of selecting which language you want to search in, and HotBot permits you to specify date, location, media type, etc.

Be sure to select your terms carefully. If your terms are too broad or general, the search engine may not process them. Also, check the search engine to see if it has a list of stopwords, words the designers determined to be so general that a search would turn up hundreds of thousands of references. One stopword, for example, is (computers); you should try to avoid using such terms.

Most search engines allow you to combine multiple terms and words using connecting words known as Boolean operators. Knowing how to use these terms is very important for a successful search.

Important Boolean operators:

AND: AND is the most useful and most important term. It tells the search engine to find your first word AND your second word or term. AND can, however, cause problems, especially when you use it with words that are broad in scope or likely to appear together in contexts other than the one you are searching. For example, if you are working on a sports history topic and need information about the Chicago Bulls, you might type in (Chicago AND Bulls). You will get references to Chicago and to bulls; however, since Chicago is the center of a large meat-packing industry, many of the references the search engine will list will be about butchers rather than basketball, since it is likely that (Chicago) and (bull) will appear in many of the references relating to the meat-packing industry.

NEAR and "xxxx": NEAR and quotation marks, used by different search engines, operate basically the same way and are etremely useful tools when searching the web. They tell the search engine to find documents with both words but only when they appear near each other, usually within a few words. For example, typing (Monroe NEAR Doctrine) or "Monroe Doctrine" would pull up information on the Monroe Doctrine without pulling up sites about President Monroe that do not talk about the doctrine.

NOT: NOT tells the search engine to find a reference that contains one term but not the other. If your early searches turn up too many references, try searching some relevant ones to find more specific or exact terms. You can start combining these specific terms with NOT when you see which terms come up in references that are not relevant to your topic. In other words, keep refining your search as you learn more about the terms. For example, if you want information on James Monroe or the Monroe Doctrine, but not on Marilyn Monroe, try (Monroe NOT Marilyn).

OR: OR is not always a helpful term because you may find too many combinations with OR. For example, if you want information on American history and you type in (American OR history), you will get thousands of references to documents containing the word American and thousands of unrelated ones with the word history. Use OR when a key term may appear in two different ways. For example, if you want information on the New Deal's Works Progress Administration, try (Works Progress Administration OR WPA).

An Example of Using a Search Engine:

  • Go to AltaVista at http://www.altavista.com
  • Type in a keyword or phrase (French Revolution)
  • Add modifiers to further define and narrow your topic (Terror)
  • Be as specific as you can (Terror AND "French Revolution")
  • Submit your search and adjust it based upon the number of responses you receive (if you get too few
    responses, submit a more general search; if you get too many, add more modifiers).
Other Search Strategies

Do not limit your Internet searching to using search engines and indexes. Be creative and think about which Internet sites might have the information you are looking for. Governments, organizations, universities, museums, newspapers, libraries, and other information entities are good sources to look for information on your history topic.