View Post

<< Back to Forum Post

What is Robot.txt File ?

I am learning SEO. I would like to know what is robot.txt file, what are its content and how it helps from SEO point of view for any website.

Thank you in Advance


  Feb 21,2014     Fobbme    2 Reply

Reply Vote


  • robots.txt file, as in robots exclusion protocol its a robots exclusion standard. It helps search engine crawlers to exclude your listed pages in conjunction with xml sitemap to guide them to crawl specified pages to be indexed in search engines. Also its our choice to decide to which crawlers we want to allow the permission to crawl our portal.

    I hope, I have answered your query.

    Thankx And Regards


    Keep posting ....Continue Sharing....Multiply your knowledge

  • Very interesting