Search engine is a coordinated set of programs that allows searching specific information on the vast expanse of the Web. The first search engine known as Archie was created in 1990 by Alan Emtage, a student at McGill University in Montreal. Now as per available information there are some 370 search engines available for Internet users out of which ten are considered major ones. Then there are directories and indexes. Here in this article both directories and indices, for the sake of clarity, are considered as search engines though there are some fundamental differences in their anatomies such as directories are human powered and search engines have crawlers or robots to do the job. A number of Web portal sites also offer both the search engine and directory approaches to finding information.
The purpose of search engines is to locate selected range of Web pages that contain words matching search keyword(s) or phrase(s). Search engines automatically try to sort the results so that only handful of the most relevant matches is presented. Every search engine searches own pre indexed data base and works using own set of rules called algorithm – a closely guarded trade secret. Some sites search not only using one search engine but also give the results from simultaneous searches of other engines. One common thing is that all search engines follow while ranking the result listing is location (where in the document are keywords located – in the beginning, in the body or at the end) and frequency (number of time keywords are repeated on a page) method for returning results. But their data bases, relevancy criteria and other strategies differ from each other. Which is one reason why the same search on different search engines produces different results? Also, some search engines index more web pages than others. Search engines may exclude some pages from their data as well. Any major search engines may be a good choice to start with when searching for information: Google is all time favourite of users all over the world. It reigns “with its empire stretching into millions of computers” as IT analyst Bill Thompson puts it. According to OneStat Google's global usage share has increased to 56.1 percent whereas Ask Jeeves stands at 1.6 percent. Another search engine, AllTheWeb also provides both good coverage of the Web and close relevancy. If Google could not deliver, chances are that AllTheWeb probably will. (All those users who have been trying Google since it was launched five years ago may recall that presently the result page of AllTheWeb is as ad free as Google’s result page used to be). In addition to Web page results, AllTheWeb provides the ability to search for news stories, pictures, video clips, MP3s and FTP files. Alta Vista, Ask Jeeves, Excite, Hot Bot, Infoseek, Inktomi, Lycos, Overture (former Go To) and Teoma are some other popular search engines. Some others are specific for countries or regions. Many search engines get their results by turning to third party search providers to power their listings. Several major search engines provide search toolbars for the browsers as a shortcut to searching the Web from within the browser without going to a search engine first. Alexa, Google, Groowe, Ultra, and Teoma search bars are most common toolbars that put search results, some from multiple search engines, right in the browser. They are useful utilities though Walton Computer Lady Patty Furgosen thinks, “some toolbars contain spyware and adware which are harmful to users’ machines and invades their privacy. I have had several such cases (in her computer repair facility) in the past that really messed up the users’ computers.” Searching for what? The amount of information on the Web has exploded for the past few years, and is growing exponentially. Number of information seekers rummaging around the Web “for enlightenment and answers” is also growing. The problem of locating information is also growing with the Web. Users want to find exactly what they want on the Internet promptly, correctly, and with little effort. Tools which support the finding of relevant material within a few mouse clicks and key strokes are becoming increasingly critical. Search engines and how they index web pages have changed drastically in the recent past, getting more scientific and meticulous by the day. But the situation is far from adequate. A survey conducted by Bright Planet estimated that the web is some 500 times larger than indicated by the results provided by popular search engines. Greater part of the body of knowledge created in the world is still not on the Web; some of the knowledge is never ever likely to be included in the cyberspace. Dr. Norbert Pintsch, a computer dependent and computer literate German architect says, “The problem is that we consider a smaller part of knowledge available on the Web as a whole forgetting the larger part.” Some of the knowledge is off limits to search engines being behind the subscriptions or security firewalls. In today’s increasingly commercial information society, one more thing that any innocent user mining the Internet for information should understand is what goes on before search results appear on the result page of any search engine. The fact that, more than ever, companies, businesses, website owners and Webmasters alike continue to optimize their sites as much as they can. The ones that do continue to harvest the yields of their efforts and are rewarded in the search results. Marketers think that search results listings are much more effective than any other form of online advertisements, and businesses are more than willing to pay for better placement. Hence the paid inclusions -- marketing in which companies pay price to the search engines to get their Web site crawled more often and more thoroughly so that they figure out prominently in search results. Google and Overture have hundreds of thousands of advertisers paying them huge amounts for getting prominent places in result listings. Trend watchers estimate that about 30 percent of search on the Internet is already purely commercial. These attempts to attract people's attention by taking measures mislead automated search engines and they churn out skewed and biased results. Mergers, acquisitions, partnerships and “estimated growth of 10 to 30 percent” in the search industry also point out to the dynamism of the search phenomenon and points to major changes in near future. Search companies are spending huge sums to improve their service and to look different than other. They are also earning big revenues. That said; just because some particular information was not fond does not necessarily mean that it does not exist on the Web. Sometimes it takes a few searches in different search engines using inclusive key words to find required information. The best way to learn searching successfully is by reading the help files on the search engine sites and practice. Most Internet savvy users I talked to had never opened help files of the search engine and majority was found using Google only. “Why bother with the help files? Searching the Web is self explanatory and one can easily learn by trial” is the general attitude. Go for specialized search engine if there is one covering a particular topic of interest. Consider using a tool (like dogpile) that will use a number of search engines simultaneously. Use of Boolean operators (such as AND, OR and NOT) between wisely chosen search keywords and phrases can locate required results more efficiently. I wanted to remove remains of some spyware program from my computer and I typed "websavings" AND "removal". The name of the little spyware was Websavings and I wanted the removal instructions. The answer came up in the first item. Dr. Norbert says, “If search engines cannot make out the difference between academic and intellectual searches and searches for finding required product or service, searchers should.”Scouting of local users’ fraternity reveals that Web searching has not matured in Pakistan as some other Internet functions like emailing or Web surfing. Exception apart, a usual Web search starts with Google and ends up there. Besides all time Web wise top search themes, users here try to look for local contents and end up with such unfamiliar information that frustrate them. They forget that the scope of the Internet is international. “Internet naïve (but PC literate) users some time do not know the search basics and are intimidated by quantum of results their searches generate. Who really can read through dozens of pages of results produced? And if the huge quantum of documents is to be read then what is the search for,” says a University of the Punjab student Naeem Alvi? On the other hand, there are some inspiring examples of local users taking advantage of searching information on the Web: People have gone for higher studies after searching information about the desired institutions and managing admission online, have contacted specialist doctors for treatment of uncommon diseases. Many writers are conducting searches for their work on the subjects which have not made to the books and encyclopaedias – traditional sources of knowledge. Students are found searching relevant material for deeper understanding of the subjects and assignments. Users are turning to Web search for information that they can apply to problem solving on ground. Some search information to enrich their lives; soul searching. Brilliant as Web search engines are, the search technology is still evolving. It has not reached the point where humans and machines understand each other well enough to communicate clearly. People are already more perceptive and are demanding better results, and they are demanding a more powerful search experience. Given the developments taking place, importance being attached to the Web searches, one thing is certain: search technology has to go much beyond what it is today. We are likely to see some seismic changes in the field. My recommendation: try different search engines to locate the information on the Web and have enriching experience. (Appeard in Dawn, Saturday, Aug 19, 2006)
