Better searching on the Internet

The two most popular things people do on the Internet are: (1) search for things and (2) wait for pages to download. The latter is always and the former is often a huge waste of time.


Web usability expert Jakob Nielsen has estimated that around the world, billions of dollars are wasted every year as employees attempt to locate information on the web using search engines. Some of that waste is the fault of the search engines themselves, but some of it also comes from the fact that many of us don’t know how to search the web properly.
Here are three ways to search the web better:
1. Use the best search engine there is. I recently gave a talk to a trade union research department (not CWU’s) and asked people what search engine they used. No one raised their hands except for one man who said, “um … Lycos?”. I learned from this that even people who you’d think would have a lot of experience (if not expertise) working with search engines don’t know much about them.
Search engines are the most popular sites on the Internet — by far. They also differ widely in what they offer. Some specialize in attempting to be comprehensive (such as alltheweb.com). Others (such as AltaVista) have blown themselves up from simple search engines into full blown “portal” websites offering email, shopping, and so on. The best search engine is one you probably never heard of — Google (http://www.google.com). Google is different because it uses a complex algorithm to produce the most accurate results. It’s so accurate that if offers you two options — to search for a list of sites that meet your search criteria or, by pushing a button that says “I’m feeling lucky”, to directly access the site that would have topped such a list.
2. No search engine really works because none of them search within databases. Let’s say you want to look up a news story you once saw listed on LabourStart or an article that appeared in CWU Briefing. You won’t find either of them through conventional search engines because these are programmed to read through ordinary hypertext documents — and nothing else. They cannot search the databases that are increasingly important on websites, including CWU’s. Proper searching means going to the individual website and using its own internal search engine (which all good sites provide). This is a lot harder than simply telling Google to look for something across all the millions of websites it indexes, but it is the only way to get at the core of material available on specialized sites.
I should also add that no search engine is really comprehensive even when looking at ordinary pages, and probably none of them have managed to index the majority of pages on the web. In the time it takes you to read this article, 35,000 new pages have been added to the web — no search engine can keep up with that. (Actually, I made up that figure. But whatever number it is, there are a lot of new pages being added all the time.)
3. Use advanced search techniques. It doesn’t matter if you’re searching the entire web or just a single database — the same rules apply. Search on more than one word, for example. If you search for “union”, you’ll find a lot of material about rugby and about the EU. Add a single word (“trade”) before “union” and you’ll do a lot better. Studies have shown that most users still do searches on single words, which usually results in lots of irrelevant results. Also, most search engines offer “advanced” search, sometimes using what are called Boolean searches. This means searching using and/or criteria, which are often misunderstood. If you are interested in the CWU’s sister unions and search for “CWU and CWA and DPG” most search engines will tell you they can find nothing. If, however you typed “CWU or CWA or DPG” you’d have gotten results. Most search engines have pages explaining how to do advanced searches and it’s worth spending a few moments learning how to do this.