or that the first update detection can achieve the correct update, but continue to click update failed again, continue to click it, or this again, let people feel a little less than the law, love Shanghai just repeatedly prompted 403 mistakes. The final is how to solve it? At the beginning title said: IP bug, lead to the spider can’t identify robots.txt, your site is also made such a mistake? If you are a shield of IP address, please check if it contains
? ?In addition to the above
has a lot of stationmaster encounters website attacks, used to shield the IP address, especially access to large amount of Web site, even more so. But you know that IP will bring to the site of the shield, the bitter? Today with a practical example.
IP network resources search, such as there is no perfect place, some friends can contact me to add, for the webmaster more.
website for shielding IP segment a large number of
121.14.89.* 203.208.60.* 210.72.225.* 125.90.88.* 220.181.108.* 123.125.71.* 59.226.50.* 25.90.88.* 61.135.162.*
the recent diagnosis of a qianwanji included site, in 3 days after diagnosis, included a sudden surge of 300W. However, these included the increase is really a good thing? Second days, the website traffic dropped by 2W-3W, what is the reason? Included links, all dynamic link robots.txt has been blocked.
love Shanghai spiders do not follow the robots.txt protocol?
IP, the other IP can shield? Tips: if you identify a specific IP in a sustained attack, suggestions for shielding, if is flood type, shielding has no effect, not careful, will shield more spiders, such as 360. Search:
often have said, love Shanghai spiders do not follow robots.txt, shielding the content or get to grab. In fact I do not rule out such examples, but basically to update robots.txt will be resolved. It will not solve, what is the cause? Today, I am here to reveal a little secret: the website of robots.txt, but the spider visited the robots.txt file, your letter
Such as: 123.125.68.* 220.181.68.* 220.181.7.* 123.125.66.*