Microsoft says "don't block our bots!" (1 Viewer)

[mjp]

Well, this is funny:

---

Hi,

I am contacting you from the Microsoft Corporation and its Internet search engine Live Search (http://search.live.com) in regards to your robots.txt https://web.archive.org/web/2020/https://bukowski.net/robots.txt file. Our customers have alerted us that your site was not visible in our results and we have discovered that you are hiding your site to our search engine crawler, named msnbot, via your robots.txt file.

We would be pleased if you edit your robots.txt file to allow our search engine to index your content which additionally will increase traffic to your site via our search results. The only step needed on your part is to remove the following restriction for msnbot within the robots.txt file.

User-agent: msnbot
Disallow: /

If you feel that msnbot is crawling your site too frequently, you can use the crawl delay directive in robots.txt
http://blogs.msdn.com/webmaster/archive/2008/04/18/ramping-up-msnbot.aspx

For more information about robots.txt http://blogs.msdn.com/webmaster/arc...together-to-provide-better-documentation.aspx

Our Webmaster Central website (http://webmaster.live.com/) can assist you further in obtaining the best results possible for your business or website.
If you have further questions please let me know.

Best regards,

Amy Wilcox
Web Analyst, Microsoft Live Search

---

"Our customers have alerted us that your site was not visible in our results..."

Yeeeaaah, sure, okay.

I do indeed block the msnbot (and limit the Yahoo! bot to 100 visits a day). What's really funny is I searched "Bukowski" on Microsoft's new Bing search engine (which replaced search.live.com), and blocked bots and all, we're right there in the second spot.

Sometimes I wonder how Microsoft manages to employ 90,000 people, then I see stuff like this and it all makes sense.

 

[Bukfan]

Right now buknet comes up first on Bing when you search for "bukowski", so bots or no bots, it does'nt seem to make much of a difference.
"Our costumers have alerted us..." - yeah, right! :rolleyes:

 

[Socratease]

Msn = kgb

 

[Johannes]

Yes, but you know: "Never play cards with a man called Doc. Never eat at a place called Mom's. Never sleep with a woman whose troubles are worse than your own. And better not fuck with the msnbot!"

 

[Gerard K H Love]

Yeah, right.

We would be pleased if you edit your robots.txt file to allow our search engine to index your content which additionally will increase traffic to your site via our search results.

And Bukowski.net would be pleased if you sent a box of some of that bank theft dough you are so easily racking in on an hourly basis.

How fucking arrogant.

 

[Erik]

Well, this is funny:
---
I do indeed block the msnbot (and limit the Yahoo! bot to 100 visits a day).

MJP = the one man Guardian Council of Los Angeles

awesome. :cool:

 

[bospress.net]

I think that Microsoft is used to people lining up around the block to lick their assholes. I bet that there are not many websites that don't play ball with the mighty MS and jump at their whims. Good job mjp. We are successful without their help. Without them and their bots your site is normally #1 in google.

People that GET US will find us. So will trolls, but they will find us without the help of Mr. Gates...

Bill