Monday, June 26, 2006

What is a bot?

When searching the Web, bot is short for robot. It is a program that can be directed to automatically find information on the Internet. Bots are able to sort through large amounts of data and decipher the relevant information. A bot searches the Net in the same way a person does when he (or she) uses a Web browser like Internet Explorer or Netscape. But instead of a person directing a Web browser to go to certain pages, code directs a bot. A bot is programmed to go to different websites then follow the links on those sites. It can be instructed to make decisions about where to go next based on the information the bot finds at each site.

Is a bot the same thing as a spider? A bot is synonymous with spider, crawler and agent.
This sounds a lot like a search engine, what’s the difference? A bot is an essential component of a search engine. When a user enters a word or phrase into the little box of a search engine, like Google, the search engine makes a query, which means it searches an index. The index that the search engine searches is created in part by a bot.

The search engine gives the bot specific instructions to bring back information. The information is then put into an index that the search engine looks at in order to locate the user’s request.
So, the search engine tells a bot what to look for, and then with that information it makes an index. When I put the word “apricot” into the search engine, it searches the index that a bot helped create, and brings me back the relevant information.

Can a bot help me? It depends on what kind of help you want. When looking for general information a bot can help you indirectly by working as part of a search engine. Bot programs can be downloaded (in some cases for free), but the list of information they return with isn't useful to the average user.

What about a shopbot? A shopbot can be a useful tool for someone shopping on the Web because they can search the Web for the best prices on products. Shopbots can also compare features and different merchants.

For example, I’ve been thinking about buying a keyboard for my palm pilot. They cost about $100 in retail stores, but I think I can find a better deal so I turn to a shopbot like shopbot.com. At their website I enter “palm V handheld keyboard” and the shopbot brings back a list of keyboards and related items. I find that LandWare makes a keyboard for about $70. But it doesn’t look like it’s the style I want. So I check Buy.com, which is selling the one I am interested in—for a bargain price of about $90.

Can a website block bots? Yes, and some do, but most websites have directions for bots when they come to their sites. They tell them what parts of the site they should go to and where they can’t go.

Can bots be a nuisance? If a bot is badly designed then it can clog the servers of the websites it visits and slow down the sites. This is rare, however, and in most cases bots do not cause problems.

Are bots used in the wireless world? Bots are set to play a significant role in the wireless world. Imagine finding what you want in a store and then having the capability to find it cheaper on the Web while still standing in the store. Will the store match the price? Or will you choose to have the item shipped to your house? Right now the technology to do this isn’t that simple (you could have problems getting a wireless signal, or it could take forever to find the relevant information), but the future is full of potential.

No comments: