Big Webmaster

Tips and Help for Webmasters

Changing a site to https (SSL)

2 Comments

Changing a site to https (SSL)

Aug 6, Google shows 7,092 pages indexed for the site. Aug 7 (10 pm), changed a site over to HTTPS and redirected all pages. Aug 9, large spike in Google-bot crawler activity. Aug 9 showing 2 pages indexed. Aug 11 showing 899 pages indexed.

Filed Under: bots

Cuil Shuts Down?

2 Comments

Cuil Shuts Down?

It appears the one time rising search engine Cuil is offline and not coming back online. The search engine was released on high hopes to overtake Google. But such an ambitious goal does not come by easily. Cuil came across to many as overly arrogant immediately claiming they there were better than Google and would […]

Filed Under: bots, misc Tagged With: cuil

Tasapspider/0.9

7 Comments

Just found this “bot” going crazy on a site that is on a dedicated server, took the entire server offline. This bot looks to be new and from China, so be on the lookout. Other reports are saying it is ignoring the robots.txt file, so might not be good enough to deny it there. It […]

Filed Under: bots Tagged With: Tasapspider/0.9

Spinn3r Bot

1 Comment

Spinn3r is a bot that indexes blog posts. Looks like this in your log files. Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1; aggregator:Spinn3r (Spinn3r 3.1); http://spinn3r.com/robot) Gecko/20021130

Filed Under: bots Tagged With: bot, Spinn3r

YebolBot

4 Comments

Anyone know anything about this bot? YebolBot (compatible; Mozilla/5.0; MSIE 7.0; Windows NT 6.0; rv:1.8.1.11; mailTo:[email protected] )

Filed Under: bots Tagged With: bot, Yebol, Yebolbot

Categories

  • bots
  • code
  • ecommerce
  • misc
  • software
  • tips
  • tools
  • wordpress

Do Follow!

Wordpress Podcast Theme

Pages

  • About

Copyright © 2022