Week 1: Clear- Bots Within Search Engines

I use a search engine everyday at work, but I never stopped to think about how the search engines work and pull the information I am looking for. In order for these search engines to work, they rely on "bots" which may also be referred to as spiders, crawlers or many other names. These programs run through millions of pages searching for related information which is added into that search engines database. The bots repeat this process for the most up to date information available within the search engines index. Some search engines may have broader indexes than others.

Some search engines use multiple bots for different types of searches to ensure their indexes are up to date. One type of bot maybe designed for deep searching that may only happen once a month due to the time constraints. While another bot may search through the Web multiple times in one hour. As you may have noticed, some search engines yield more results than others. This is a factor of the bots and the depth of the search engines index or database.

If you have your own website, it is a positive thing to have bots "crawling" around your page. You want to welcome the bots which means your page is being indexed. This will hopefully increase your views and allow more people to access your page who may have not known it existed without the use of a search engine. Read "What Exactly is a Bot?" for a good article about bots within your web pages.
 
For my generation, it is hard to remember a time without a search engine, whether it was dial up AOL or another slower form. I use search engines daily now and without them I am unsure how I would find information. Information is much more accessible with t he help of bots. Of course, we are able to do good, old fashion library searching, but we have relied on having fast access to information for so long, we may not know what to do in a library! But when we think of future generations, do we want them to rely so heavily on search engines? I still remember card catalogs but even the libraries rely on a search engine. Technology is great, but if we rely on it to much we may regret it in the future.